Authenticity, Interactivity, and Collaboration in Virtual Reality Games: Best Practices and Lessons Learned

Wang, Annie; Thompson, Meredith; Uz-Bilgin, Cigdem; Klopfer, Eric

doi:10.3389/frvir.2021.734083

ORIGINAL RESEARCH article

Front. Virtual Real., 13 October 2021
Sec. Virtual Reality and Human Behaviour
Volume 2 - 2021 | https://doi.org/10.3389/frvir.2021.734083

Authenticity, Interactivity, and Collaboration in Virtual Reality Games: Best Practices and Lessons Learned

The Education Arcade, Comparative Media Studies & Writing, MIT, Cambridge, MA, United States

Virtual reality has become an increasingly important topic in the field of education research, going from a tool of interest to a tool of practice. In this paper, we document and summarize the studies associated with our 4-year design project, Collaborative Learning Environments in Virtual Reality (CLEVR). Our goal is to share the lessons we gleaned from the design and development of the game so that others may learn from our experiences as they are designing, developing, and testing VR for learning. We translate “lessons learned” from our user studies into “best practices” when developing authentic, interactive, and collaborative experiences in VR. We learned that authentic representations can enhance learning in virtual environments but come at a cost of increased time and resources in development. Interactive experiences can motivate learning and enable users to understand spatial relationships in ways that two dimensional representations cannot. Collaboration in VR can be used to alleviate some of the cognitive load inherent in VR environments, and VR can serve as a context for collaborative problem solving with the appropriate distribution of roles and resources. The paper concludes with a summation of best practices intended to inform future VR designers and researchers.

Introduction

Virtual reality can bring new perspectives to classroom learning. In the last 20 years, immersive VR has become an increasingly common topic in the field of education research (Hew and Cheung, 2010; Merchant et al., 2012; Ahn et al., 2017) as the technology becomes more viable for classroom use (Castaneda et al., 2017), prompting educators to explore how to leverage VR for educational purposes. The accessibility of VR has increased as the overall cost of VR has decreased in recent years (Korbey, 2017). However, more research is necessary to move beyond the “novelty” of VR (Merchant et al., 2014) and understand its full potential in K-12 learning. Increasing access has supported a growth in the number of studies of VR and learning; however, additional research is needed on longer term learning outcomes (Jensen & Konradsen, 2018; Pellas, Dengel & Christoupoulos, 2020), especially with projects that extend beyond one-time implementations of VR experiences (Merchant et al., 2014; Hamilton, McKechnie & Edgerton, 2020). This paper addresses the need for extended studies of VR projects by documenting a set of studies on one multi-year design project, Collaborative Learning Environments in Virtual Reality, or CLEVR. The CLEVR team designed, developed, and deployed Cellverse, a game designed to help introductory high school students learn cellular biology. In this article, we discuss lessons learned in our design, development, testing, and analysis so designers and educators can learn from them.

At the beginning of the CLEVR development process, we described our intentions for the game in an article titled “Authenticity, Interactivity, and Collaboration in VR Learning Games” (Thompson et al., 2018a). This article outlined our theoretical frameworks for the game, initial design, and planned trajectory. Since the start of the project (2017), Cellverse has developed significantly in breadth, depth, and focus. Moreover, we approach game development through a framework of Design-based Research, or DBR (Ameel & Reeves, 2008; Sandoval & Bell, 2004). Ongoing user testing, studies with various types of users, and reviews by subject matter experts have enabled us to collect valuable qualitative and quantitative data that have enhanced our understanding of how to incorporate authenticity, interactivity, and collaboration in VR learning games (Thompson et al., 2018b; Wang et al., 2019; Thompson et al., 2020; Uz Bilgin, Anteneh, & Thompson, 2020; Wang, 2020; Uz Bilgin, Anteneh, & Thompson, 2021; Uz Bilgin & Thompson, 2021).

We begin this manuscript by defining authenticity, interactivity, and collaboration in context to VR, followed by several theories and frameworks essential to understanding learning in VR, including the Cognitive Theory of Multimedia Learning and the Cognitive Affective Model of Immersive Learning (VR, in this article, should always be assumed to refer only to immersive VR). We then introduce the CLEVR project and Cellverse, the game that was ultimately produced from CLEVR research. This is followed with a critical analysis of our Cellverse studies between Summer 2017 - Spring 2020, describing both lessons learned and best practices of VR in learning. The recommendations made for best practices arise from both our research results and from our practical experiences creating, testing, facilitating, and studying the game. The manuscript concludes with a discussion of the advantages and challenges for authenticity, interactivity, and collaboration in educational VR.

Authenticity, Interactivity, and Collaboration

Authenticity

Our goal for an authentic game had three levels of authenticity: authenticity of narrative, authenticity of environment, and authenticity of action. Authenticity of narrative is critical for promoting interest and motivation, but is not inherently tied to VR; authenticity of action and environment, conversely are closely tied to VR’s affordances. We will briefly discuss all three types of authenticity in this section. Authenticity refers to the ability for VR to produce and render scenarios, experiences, and processes that closely resemble real life (Thompson et al., 2018a). Such an affordance is unique to the technology due to its multisensory qualities; VR stimulates the user’s sense of sight, sound, and can even include smell and touch. This sensory engagement allows the user to virtually experience environments that may be too distant, expensive, or dangerous to approach otherwise (Bailenson, 2018).

One dimension of authenticity is the authenticity of environment. Biology in particular makes for particularly fertile ground for depicting the authenticity of environment through VR. As mentioned above, individuals do not have accurate views of cells, in part because of how cells are depicted in biology education (Deniş Çeliker, 2015). This need for authentic virtual environments (VEs) may tap into a critical need in K-12 education, particularly within the sciences. Teaching realistic systems requires authenticity in order to ensure that students are able to gain accurate mental models of critical topics (Jacobsen, 2017). The authenticity of the environment also helps establish a sense of presence in the virtual environment. Slater and Sanchez-Vives (2016) suggest that presence is related to the user’s “place illusion” of the VE and the perception of “plausibility” of interactions. Makransky and Peterson (2021) discuss representational fidelity in realism, smoothness of interaction, and consistency felt by the user in the interactions with the VE, three variables originally proposed by Dalgarno and Lee (2010). In the case of an educational game, authenticity of environment was one of our learning goals. The virtual environment of Cellverse attempts to represent cells as they exist in nature: three-dimensional, active, and densely packed. Using feedback from subject matter experts, we iteratively redesigned the cellular environment to reflect cutting-edge research on cell internal structure. We also incorporated ongoing research using databases and resources designed for scientists -- for example, the biological quantitative database B1onumb3rs (Wang et al., 2019) in order to provide an accurate presentation of the relative size of organelles and the density of organelles and proteins within the cell.

Authenticity of action means that, as much as possible, the actions available to the player within the game reflect actual techniques available to scientists the player takes within the game resemble actions that individuals can do in real life. This closely ties to the definition of situated learning, as the player is able to join a community of practice by “doing what the experts do”. While this is not a requirement for VEs, we prioritized authenticity of action to ensure that our game introduced the capabilities of biologists and did not create or amplify misconceptions about biology.

Finally, authenticity of narrative contributes to the users degree of buy-in for the virtual environment. Johnson-Glenberg & Megowan-Romanowicz (2017) found that narrative increased users’ interest in the experience. Users can be primed for social interactions in the VE by watching an engaging conversation between two agents in the virtual world (Daher et al., 2017). Authenticity of narrative ties together the action and environment to create a more powerful learning experience in XR.

The three levels of authenticity complement each other in building a sense of presence and agency in educational games, as depicted in Figure 1.

FIGURE 1

FIGURE 1. The theory of authenticity in XR (TAX).

Interactivity

In the context of VR, interactivity is closely tied to immersion. Immersion is a function of VR hardware, creating the illusion of physical presence in a non-physical world (Slater and Sanchez-Vives, 2017). Liu et al. (2017) argue that immersion is essential to motivation and learning in VR. Whereas presence reflects psychological feeling, immersion is the technology or practical application that creates presence. Slater & Sanchez-Vives (2016) suggest that the more seamless the underlying technology, the more potential for immersion that exists–this is encapsulated in the various levels of technological sophistication, including but not limited to haptic feedback or in the degrees of freedom available to the user. The high level of interactivity possible with virtual reality is recognized as a key affordance that sets VR apart from other technologies, such as film and video (Lindgren & Johnson-Glenberg, 2013; Makransky & Petersen, 2020).

On the other hand, since a user both provides and is provided with information, interactivity requires a combination of hardware and careful design to be successfully implemented. Interactivity occurs when a user affects virtual objects or avatars, prompting changes in the VE. Interactivity embedded within a VR experience enables the user to communicate with the VE, by using buttons, manipulations, gestures, or other modalities to produce feedback from their virtual surroundings. Embodiment, such as gesture and movement, has been linked to positive learning outcomes in improved learning in physics (Johnson-Glenberg & Megowan-Romanowicz, 2017) developing a better understanding of electricity (Johnson-Glenberg, 2017), helping students learn laboratory skills (Lindgren et al., 2016); helping medical students learn anatomy (Jang et al., 2017), as well as helping scientists prepare samples for microscopy (Leinen et al., 2015) and testing compounds for new pharmaceutical drugs (Yuan, Chan & Hu, 2017).

Interaction may have multiple definitions depending on context, and is defined in this manuscript as the level of responsiveness the VE provides to a user. Johnson-Glenburg (2017) outlines three constructs that contribute to the degree of embodiment: sensori-motor engagement, gestural congruence, and sense of immersion. Sensorimotor engagement can offload cognition to enable the user to learn more complex topics (Weisberg & Newcombe, 2017). Gestures that match the learning objectives can reinforce learning and facilitate the initial uptake of ideas (Pouw et al., 2014). Immersion also supports embodiment; more sophisticated virtual environments can give the user more options for agency within the environment, which makes the user an active part of the virtual environment. From macroscopic interactions such as moving across the virtual space and controlling what is in their field of view to microscopic interactions such as waving your hand or looking in a mirror, the user’s actions prompt a virtual response. This action and response cycle draws the user into the virtual experience (Wang, 2020). As the type of manipulation and the type of response can vary, designers must consider how interaction can enhance learning goals (Johnson-Glenburg, 2017; 2018).

Collaboration

Virtual environments provide new venues for collaboration between individuals. Collaborative problem-solving is considered essential for the future of work and is deemed a vital “21st-century learning” skill (Fiore, 2017). Collaborative, goal-oriented activities create what Johnson and Johnson (1999) call “positive interdependence” among team members, wherein individuals in a group rely on each other’s strengths to achieve their goal (Laal, 2013). Previous research has identified principles of collaborative learning that may be integrated into VR experiences including interdependence, thoughtful group formation, individual accountability, and attention to social skill development (Cuseo, 1992; Lee, 2009).

Establishing rules and developing distinct roles for users are both useful ways of encouraging collaboration within VR environments (Uz-Bilgin et al., 2020). Earlier studies have described the benefits of establishing collaborative roles in VR. Jensen and Konradsen (2018) used games to create rules for social interaction and roles for individuals in virtual problem-based activities. Defined roles also helped visitors engage with a VR museum exhibit experience on an aircraft carrier (Zhou et al., 2016). Finally, middle school students in the EvoRoom VR environment benefited from clear roles in gathering and sharing information with their peers (Lui and Slotta, 2014). VR may also encourage individuals uncomfortable with leadership to be proactive and assume roles with more responsibility. Slater et al. (2000) found that users participating in a VR activity using a head-mounted display were more likely to willingly take on a leadership role than when they were involved in with the same activity within an in-person group.

Learning Theories and Theoretical Framework

Here, we will define the learning theories and frameworks we draw upon in this project and in this article. The learning theories that are particularly salient to virtual reality are situated learning, spatial learning, embodied learning, and embedded learning.

Learning Theories: Situated, Spatial, Embodied, Embedded

Situated learning theory suggests that optimal learning occurs when the learner is able to experience the activities and environment in as authentic of a context as possible. Understanding the context and activities of the area being studied allows students to experience “legitimate peripheral participation” and inducts the learner into a community of practice (Wenger, 1998). Part of this induction is in the expert making their thinking visible to the learner. For example, in an introductory research course for freshmen, students become cognitive apprentices when the professor makes implicit ideas about research explicit (Thompson et al., 2016). The ability to transfer skills learned in a VE to real-life experiences is a result of the similarities between the learning environment and the actual environment (Liu et al., 2017), or when students are able to “do what the experts do” by emulating real-life scientific techniques. VR experiences involving situated learning are popular within the sciences, particularly virtual laboratories that enable users to iteratively practice essential skills in “lab-like” VEs without requiring real-world resources (Chiu et al., 2015; Lindgren et al., 2016).

Spatial learning refers to both learning to navigate a real or artificial (VR-rendered) space. Spatial learning is helpful for individuals in navigating their everyday lives, but has also been identified as an important skill in learning STEM. For example, size and scale are important concepts to understand within STEM learning domains, but can be challenging to conceptualize for learners (Jones et al., 2003). Because VR allows the opportunity for users to directly manipulate virtual objects, it may be used to enhance learners’ perception of relative size and scale.

The theory of embodied learning states that connecting physical action to learning objectives creates deeper learning (Kiefer and Trumpp, 2012). Through embodied learning, knowledge is cemented as memory through the body’s repeated interactions with the physical environment (Lindgren & Johnson-Glenberg, 2013). Previous research suggests that the multimodal nature of VR may make it optimal for facilitating information retrieval in 3D spaces, thus strengthening users’ mental models (Dede et al., 2017; Johnson-Glenberg, 2018). Providing a 3D virtual environment for users to experience abstract concepts may produce more effective learning than in 2D models, in biology (Tan & Waugh, 2014), physics (Johnson-Glenberg, 2018) and chemistry (Chiu, DeJaehjer & Caho, 2015; Lindgren et al., 2016).

While embodied learning supports user cognition through physical movement, embedded learning supports user cognition through features that are part of the virtual environment. A signpost on a highway is a real-world example of embedding learning into an environment; instead of being forced to memorize highway numbers while navigating a road, a driver can simply recall their location by glancing at the words on a passing sign. Embedding cognitive activity within the environment frees up mental capacity by storing extraneous (non-essential) information into accessible actions or tools (Weisberg & Newcombe, 2017), instead of overloading limited mental resources (Pouw et al., 2014).

Cognitive Theory of Multimedia Learning

Introduced by Mayer (2020), the Cognitive Theory of Multimedia Learning (CTML) describes two main sensory channels for memory processing: visual (sight) and auditory (sound). These two input methods are processed separately by the mind and do not overlap with each other. According to CTML, the brain processes information through a series of steps: filtering information, organizing it, integrating it into previous knowledge or schema, and finally processing it into long-term memory storage. According to CTML, when cognitive processing exceeds a user’s mental capacity, “essential overload is experienced, inhibiting learning” (Meyer et al., 2019).

VR creates higher cognitive load among users compared to other forms of media, which may impede memory recall and memorization (Parmar et al., 2016; Makransky et al., 2019; Roettl and Terlutter 2018). Critics argue that VR produces comparatively poorer learning outcomes because the medium is overwhelming to users (Moreno & Mayer, 2002). These critics point to evidence that VR is a poor medium for imparting declarative or factual, static knowledge (Mayer, 2019). Findings from ongoing cognitive science research in VR learning seem congruent with CTML; when users doing VR-based tasks were compared with users working on the same task on a non-immersive platform, the VR users reported higher enjoyment but revealed lower levels of gained declarative knowledge (Parong and Mayer, 2018; Maransky, Terkildsen, and Mayer, 2019). Mayer et al. (2019) suggest that the heavier cognitive load inherent to VR prevents users from processing incoming facts into long-term memory, thus preventing effective learning.

The rich sensory experience afforded by VR comes at a cost. Designers should be aware that cognitive load informs all design principles of VR, so learning designers must temper their VR experience to avoid overwhelming users with excessive cognitive load. That being said, this manuscript argues against the suggestion that VR consistently makes for poor learning experiences simply because it produces high cognitive load. As Mayer, Omdahl, & Makransky (2019) have argued, VR may not be a good medium for transferring declarative (fact-based) knowledge -- however, the numerous examples listed in this document suggest that it is useful for other forms of learning. Declarative knowledge, while central to the American education system, is not the only type of knowledge essential for 21st-century learners. We must also note that not all cognitive load is “bad,” as cognitive load is inherent to any learning material (Paas et al., 2003). Cognitive load can be essential to the learning process, generative based on what the learner is learning, or extraneous and thus hinder learning. In designing with the information-rich and sensory-stimulating technology of VR, designers need to be purposeful in maximizing essential and generative cognitive load and minimizing extraneous cognitive load (Mayer, 2020).

The Cognitive Affective Model of Immersive Learning

More recently, Makransky & Petersen (2021) proposed a model of learning designed for immersive learning: the Cognitive Affective Model of Immersive Learning (CAMIL). Much of the early research on learning in VR focused on whether or not VR helped learning compared to other media. A central premise in CAMIL is that media and method interact; maximizing the effectiveness of an immersive learning experience requires an understanding of the affordances of that medium and how to tap into (manipulate) those affordances. Presence and agency are the two main psychological affordances of the medium of VR (Johnson-Glenberg, 2018; Makransky & Petersen, 2021) and so “instructional methods that enrich learning through higher presence or agency will specifically increase learning through immersive technology” (Makransky & Petersen, 2021, p. 6). Presence and agency are linked to the level of immersion, the degrees of interactivity (control factors), and the degree of representational fidelity of the experience. CAMIL states that presence and agency impact six factors that influence learning: interest, intrinsic motivation, self-efficacy, embodiment, cognitive load, and self-regulation. Calibrating presence and agency in VR environments impacts each of those learning outcomes.

In order to better understand how VR can best be harnessed for learning, we must understand how researchers can participate in the VR design process and how scholarly research on VR-based learning can inform ongoing development of educational games and simulations. Our suggestions are discussed through the lens of the CLEVR Project and its resulting game Cellverse.

Collaborative Learning in Virtual Reality and Cellverse

CLEVR is a research collaboration between the MIT Education Arcade and the MIT Game Lab. It is funded by Oculus Education and has been developed by an interdisciplinary team of researchers, game designers, programmers, and artists.

Cellverse, the game produced through the CLEVR Project, has been developed as both a single and two-player game that explores concepts of cell biology, particularly cell organelles and cell processes. Our team used the Next Generation Science Standards (NGSS, 2013) as a baseline for Cellverse’s educational content to orient learning goals for high school-age student users. The software was built using Unity 3D and is supported by the Oculus Rift system. We used a design-based research methodology (Collins et al., 2004), where we conducted tests and interviews with users and experts throughout the design process.

We sought a narrative that would focus on the DNA-to-RNA-to-protein process. We met with biologists to explore different diseases that could support the game narrative. We chose cystic fibrosis (CF) because it was the first genetic disease that can be treated through FDA-approved gene therapy. This helped support our goal of authenticity of action, as one of our initial goals for the game was to end by creating a specific gene sequence to fix the faulty sequence causing CF in the patient. CF is caused by disruptions at one of a few points in the process of protein synthesis; each of these disruptions is caused by different genetic sequences and is best addressed with a targeted treatment. In the single-player narrative of Cellverse, the player is a student intern using a remote-controlled microbot to navigate through a human lung cell. The cell, like its human host, has CF; the players must find clues in the cell structure, organelles, and processes to diagnose and recommend treatment that is suited to the class of CF that matches the clues. The player’s goal is to explore the cell’s internal structure and observe the cellular process of translation to figure out which form of CF is affecting the cell in order to provide the unnamed patient with the most effective medical treatment. In the game, players view the cell using a machine of microscopic size, a “microbot”, and an even smaller “nanobot”, rather than shrinking down to the cellular scale. Here we maintain authenticity of action as microscopic and nanoscopic robots are already being developed, and using those as a probe of the living cell is more realistic than making a person smaller (e.g., Venugopian, et al., 2020).

VR remains a novel experience for many people - to reduce the risk of extraneous cognitive load overwhelming users right away, players begin with a tutorial that places their microbot into a remote, sparsely populated area of the lung cell. This was an intentional decision on the part of the designers, as we wanted the players to focus on game mechanics during the initial part of the game rather than become distracted or overwhelmed by their surroundings. The structure of the cell that causes CF, an ionocyte, has projections that contain fewer organelles. The feature of this type of cell lends itself well to the goal of segmenting the introduction to the game while maintaining authenticity of environment. Players are immediately greeted by a non-player character (NPC) named FR3ND, a robot who teaches the player the basics of head movement (e.g. that they have a 360° view of the cell), selection (e.g. of organelles), and navigation (e.g. point to your destination and press “A”). The tutorial gradually guides the player from their starting location into the densely populated main “body” of the cell, where the tutorial ends and the game begins.

To accomplish the task of identifying forms of CF, the player is equipped with a number of tools and informational tips that allow them to shift between different levels of scale (microscopic and nanoscopic scale), read descriptions of selected organelles, collect virtual samples, review different classes of CF, and determine whether the clues meet the description of the class of CF. The “clipboard,” for example, is a tool that is attached to the player’s virtual left hand. Players can select organelles around them with their right hand, and a description of the organelle’s functions will instantly appear on the clipboard. They may also use the clipboard to sample organelles in order to collect clues using a “Sample” button at the bottom of the clipboard, as shown in Figure 2. Finally, players are capable of shifting their viewing robot between microscale (the microbot’s original scale) and nanoscale (a smaller nanobot) in order to view particles of different sizes. By approaching the rough endoplasmic reticulum (ER), they can activate a nanobot that enables them to “shrink” to nanoscale and observe macromolecules (e.g. RNA and amino acids) that would not be visible at the microscopic (or micro) scale.

FIGURE 2

FIGURE 2. Screenshot of clipboard tool showing a user sampling glutamine, an amino acid.

The virtual environment of the multiplayer narrative of CLEVR Cellverse is identical to the above description, but otherwise varies drastically. The multiplayer game is a cross-platform experience involving two players and offers greater challenge and complexity than the single-player version. One player, the “Explorer,” wears the HMD and is tasked with navigating through the cell’s virtual environment and viewing cell functions up close. Unlike the single-player experience, the Explorer is not provided with as much textual information on organelles or cellular disorders - access to this information is granted to the second player, or the “Navigator.” The Navigator is equipped with a touchscreen tablet interface that provides a limited “bird’s-eye” view of the same cellular environment. The Explorer and Navigator have to combine their complementary roles and resources in order to accomplish their task, creating positive interdependence between the users (Thompson et al., 2019).

Cellverse has been in development since Summer of 2017 and has undergone numerous iterations, which we have discussed in other publications (Wang et al., 2019; Thompson et al., 2020; Uz Bilgin, Anteneh, & Thompson, 2020; Uz Bilgin & Thompson, 2020; Wang, 2020). In this paper, we look across all of the studies and papers to synthesize our experiences as lessons learned and best practices,” in designing learning games that include authenticity, interactivity, and collaboration.

Studies and Methods

Our Cellverse user study encompasses 3 + years and many user tests, each with their own goals, target user groups, collaborators, and data collection methods. One aspect of each study design was to include a diverse body of participants, with a wide range of ages, backgrounds, and previous access to VR. Each of these articles we have published about Cellverse draws from four main studies we conducted during the project, which we describe below. We have organized the research questions and findings for each article into a table format and have indicated the data source for each article to one of the four examples below.

1. User testing (2017–2018)–Using a design-based research framework, we ran ongoing user testing with subject matter experts (11), adult volunteers (35), and teachers (8), 54 people in total, between 2017 and 2019. These user tests occurred once every 8 weeks and included individuals that were invited to test different games and simulations being developed for educators. During the user test, individuals answered pre and post surveys, created cell drawings before and after using Cellverse, and were interviewed at the end. Data were also gathered from observation notes gathered while the users used Cellverse.

2. Qualitative Studies (2018)–In the summer and fall of 2018 we conducted two qualitative studies of the collaborative version of the game. Participants completed pre and post surveys, created cell drawings before and after they played the game, and were interviewed at the end of the game. All participants were videotaped. Video recordings were transcribed and analyzed using qualitative coding and epistemic network analysis. These studies included a study of eight pairs of STEM teachers, and a study of four pairs of K-12 students (2 from middle school, two from high school) and four pairs of high school graduates in a biotechnology workforce development program.

3. Quantitative study (2019)–In the fall of 2019 we conducted a quantitative study at two urban high schools near the Boston area. One hundred and fifty-three students participated in the study. All students completed a pre and post survey about their knowledge of cellular biology and CF. The post survey questions also included scales about presence, mental workload, and spatial skills. All students drew pictures of a cell before and after they played the game. They were given 25 min in the VE, where they were told to figure out what was wrong with the cell. Data were analyzed using descriptive statistics and inferential statistics.

4. Quantitative study (2020)–In the spring of 2020, we conducted a quantitative study of adults. Sixty-one people participated in the study. Participants were randomly assigned to one of two interventions: playing Cellverse in the head-mounted display (HMD) with hand controllers or playing the game viewing the game on a flatscreen with hand controllers. All participants completed a pre and post survey about their knowledge of cellular biology. Post survey questions also included scales about presence, mental workload, and spatial skills. All participants drew pictures of a cell and of the process of translation before and after they experienced the game. After participants were set up and given 5 min to explore, they were asked to find three organelles in the cell, and the researcher timed how long it took them to find those organelles. They were given 25 min in the virtual environment, where they were told to figure out what was wrong with the cell. After they were finished, they were asked to find the same organelles, and the time it took to find them was recorded. Participants engaged in a short interview at the end of the session where they described their drawings of a cell and of the process of translation and also provided feedback about the game.

These four data collection activities are the foundation for the research studies and experiences described in this paper. We link the data collection activities, research questions, and findings in Table 1.

TABLE 1

TABLE 1. Summary of all studies of Cellverse from 2017-2019.

Lessons Learned and Best Practices

In designing and studying the game, we gained additional insight into the opportunities and challenges of creating an authentic, interactive, and collaborative game. Below we describe how we incorporated those three features into the design, what we learned, and the resulting best practices for design.