Session 7: Assessment and Evaluation in Virtual Worlds

2nd Global Conference


Monday 12th March – Wednesday 14th March 2012
Prague, Czech Republic

Social Media Good, Video Games Bad: Negotiating the Dialectic Tensions of Digital Interaction Paradigms in Distance Learning
Jon Leon Torn
Northern Arizona University, USA

In developing online learning environments, two models of digital interact suggest themselves: Video games and social media. Views of these forms of digital media reflect a dialectical “tension between the progressive or utopian possibilities offered by new communications technologies” (Czitrom, 1982, p. 184) with social media assigned the progressive role. According to the current paradigm, social media are expansions of consciousness leading to increased issue awareness, outward-directed socialization, and civic responsibility, while video games promote isolated, self-directed, accumulative individualism. Although I share these preconceptions to a certain extent, I am also aware of the ways these paradigmatic understandings may be limited. In this paper I will survey a number of assessment techniques I have used in online classes that engage this dialectical tension. On the one hand, video games initiate users into the paradigm of discrete skill increase. They offer the satisfaction of self-monitoring increased mastery of an interface through the increase in levels, the gathering of tools, and the acquisition of skills. This paradigm is something that students are very familiar with, and designing tools with that sort of perspective in mind is helpful. On the other hand, the sort of collaborative, group intelligence model that Henry Jenkins sees as essential to social media’s promise in democracy can work pedagogically the strategic impulse is somewhat suppressed, otherwise information withholding might result. Assessment of collaborative work, and creation of incentives for student involvement is complicated by the prevalence of the strategic paradigm in the educational experience: subjective individuals confronting an environment (the university) and wanting discrete means of assessing progress. Conventional students can engage collective intelligence through social interaction, online learning needs to present this information virtually. Challenges in doing so will be a major are of focus.

User-centered Design and Evaluation of Virtual Worlds
Tomas Bouda
Brno, Czech Republic

Virtual worlds are objects of much research. Such research is needed for future wider application within education, science field or market domain. For instance, Hype cycle presents that virtual worlds are coming in productive phase in 5-10 years. Virtual world as we know today are not user friendly or effective as a seriously used communication tool. Of course, there is a lot of progress in this area. Let me consider Facebook plug-in for instance, which brings much easier login process compared to Second Life or Open Simulator installed client. Second thing could be release of SL Viewer 2.0. However, studies show that there are a lot of users demotivated or disappointed from the virtual world`s environment and after their first visit they never come back. That has to change.

Paper focuses at the problem from Information science point of view. Specifically, the author takes user-centered design as a core approach, which is needed to improve software usability. Usability as a broader term for experienced usability, apparent usability, user friendliness, quality of user interface, etc. must be measured and evaluated. The target of the paper is to present methodology of testing and evaluating virtual world environments. Paper summarizes core steps within user-center design, which is user task analysis, expert guidelines-based evaluation, formative user-centered evaluation, summative comparative evaluation. More closely, user task analysis contains pre-research, which is needed for understanding of users` needs, their work culture and their work goals, which they perform daily. There are work scenarios developed within expert guidelines-based evaluation and then users go through them within formative user-centered evaluation. After redesign of environment users go through two or more variations of developed user interfaces and analyze which one is better.

There is going to be a short overview of the virtual region VIAKISK within Open Simulator presented as well.

Download Draft Conference Paper (pdf)

Meaningful Assessment for Student Learning Activities That Are Conducted Within Virtual Worlds – What Should It Look Like?
Geoffrey Crisp
Office of Dean Learning and Teaching, RMIT University, Melbourne, Victoria, Australia

Elaborate learning activities have been designed for students in virtual worlds (de Freitas & Maharg, 2010). Virtual worlds can provide opportunities for authentic learning in 3D environments but for many discipline-based teachers the level of skill required to construct these environments means that professional expertise is required to build the virtual world. Since teachers are usually allocated the task of designing assessment activities and the associated marking of those assessments, the use of professionally designed learning environments often results in a disconnect between the design of the assessment activities and the design of the learning activities in virtual worlds.

Although significant effort has been placed on designing elaborate 3D environments that are engaging for learners, the corresponding effort associated with designing engaging assessment activities within virtual worlds has been minimal. Many teachers will use learning activities within virtual worlds and then assess their students outside of the 3D environment, using traditional methods such as essays or online tools such as wikis, blogs, discussion boards or e-portfolios. This means that the assessment is associated with reflections after the events within the virtual world, rather than timely assessments associated with in-world activities or the analysis of data accumulated as students perform tasks within the virtual world.

This paper examines some of the current attempts to provide assessment tools for teachers to use within virtual worlds and how assessment tasks can be designed to integrate numerous in-world and browser accessed activities that facilitate flow in learning and assessment. The use of the Sloodle set combined with browser based resources will be examined, as will the use of scripted prims and bots to engage students in scenario based activities where the students’ responses can be used for assessment purposes. Our island in Second Life can be found at

Download Draft Conference Paper (pdf)