Toward Facilitating Team Formation and Communication Through Avatar Based Interaction in Desktop-Based Immersive Virtual Environments

1. Introduction

With the rise of the COVID-19 pandemic, online media platforms have garnered immense popularity as tools to facilitate communication and collaboration in personal, work, and educational activities. Estimates show that as many as 77% of US office employees have begun working remotely (USR, 2020), and globally over 1.2 billion students in 186 countries have adopted some form of online learning due to the pandemic (WEF, 2020). This scenario poses many challenges, especially to activities involving teamwork. Teamwork is an essential part of educational and work-related activities. The ability to work in teams is considered highly desirable by employers (Meehan and Thomas, 2006), and generic skills such as problem-solving, communication, and working effectively in a team have been identified as top skills looked for in college graduates (Bowden and Marton, 1998). Hence, it is of the utmost importance that there be further investigation into the tools and approaches that can provide the means for students and professionals to successfully meet and collaborate remotely in their work and educational activities even in times of isolation.

Online media platforms offer an alternative to real-world settings to conduct group activities and interact socially. Among online media platforms, video-mediated communication (VMC) platforms have gained much popularity. Zoom and Microsoft Teams (Onl, 2020; Vid, 2020) are two examples of video-conferencing platforms, a class of VMC systems. These platforms support several aspects of real-world face-to-face communication such as non-verbal feedback, attitude cues, and afford gestural modalities for emphasis and elaboration (Isaacs and Tang, 1994, 1997; Doherty-Sneddon et al., 1997). However, video conferencing has limitations. There is a lack of spatial interaction and mobility. Typically, users are automatically assigned a position in a mosaic of camera feeds, and repositioning one's camera feed in the grid is not supported. Turn-taking and floor management are also difficult in groups because it relies on judging exact gaze direction, which most video-conferencing systems don't support (Isaacs and Tang, 1994; Whittaker and O'Conaill, 1997). Judging a collaborator's center of attention when observing or helping with a task is difficult for the same reason (Neale et al., 1998). As a result of these limitations, research conducted to evaluate video conferencing's efficiency in education suggests that the participants' expectations are usually not met (Motamedi, 2001; Knipe and Lee, 2002; Delaney et al., 2004).

Several variations of online media platforms have been developed, each attempting to address some of the existing limitations. An innovative approach that has emerged is to bring participants into a 3D shared virtual environment in systems based on Avatar Mediated Communication (AMC), such as Mozilla Hubs (MH) (Hub, 2020). An avatar is a user's visual embodiment in a virtual environment (Vilhjálmsson, 2003). Because the avatars of all participants occupy the same virtual environment, participants can experience some of the spatial interaction and mobility that exists in real-world interactions. Therefore, AMC platforms can complement video-conferencing platforms by addressing one of their fundamental limitations, which is the lack of a 3D shared space (Vilhjálmsson, 2003). In AMC systems, small groups can work and communicate independently from each other while sharing the same 3D virtual space, which is often facilitated by spatial audio technology. The ability to personalize virtual rooms can help develop a collaborative atmosphere and increase engagement in work and educational settings, which has been shown to increase productivity and learning (Barrett et al., 2015). In Mozilla Hubs, collaboration and engagement are greatly enhanced by a focus on end-user content creation. Users can easily add multimedia content from all over the Internet by pasting links to content into the virtual rooms, creating opportunities for active learning sessions (Felder and Brent, 2009).

Others have investigated broadly how to develop AMC systems, including for education (Dickey, 1999; Scavarelli et al., 2019; Bredikhina et al., 2020; Holt et al., 2020). However, to our knowledge, there is little research on how to develop 3D virtual rooms and environments for avatar-mediated interaction aimed specifically at facilitating students' interaction and communication toward promoting team formation in remote teaching environments. This paper describes our semester-long iterative efforts in teaching the Virtual Reality for the Social Good (VR4SG) course at the University of Florida in the Fall 2020 semester. The VR4SG course is a project-based introductory virtual reality course traditionally taught face-to-face. Due to the isolation imposed by the Covid-19 pandemic, we sought to combine the use of a video conferencing platform (Zoom) and an avatar-based communication platform (Mozilla Hubs) in teaching the class remotely. We describe activities and rooms developed with Mozilla Hubs along the semester to facilitate students' interaction and communication, toward helping them form teams to work on the course's main project. We collected students' impressions and investigated their behavior and communication patterns in the MH platform through surveys and observations throughout the semester. We describe the development of crucial class activities that illustrate how students were prepared to use MH, learn about virtual reality, and form teams. We contribute to the field of remote teaching and learning by presenting design observations in creating 3D virtual social spaces that aim to promote student engagement and support team formation. Additionally, we contribute to the knowledge base by reflecting on students' group behavior and communication patterns in avatar-based virtual spaces in class-oriented environments.

2. Background

2.1. Computer-Mediated Communication

Technologies that mediate communication between two or more people have continued to evolve since the telegraph. Computer-Mediated Communication (CMC) is an extensive field of study that has gained importance and interest recently, especially with the rise of the Covid-19 pandemic. CMC platforms can be divided into synchronous and asynchronous (Herring, 1996). Synchronous applications include platforms such as chat rooms and video-conferencing, while asynchronous applications include emails and newsgroups (Vilhjálmsson, 2003). The first synchronous CMC platforms were based purely on text and allowed users to write messages onto each other's consoles (Nardi et al., 2000). Derivations of these early platforms are, for example, instant messengers such as Facebook Messenger (Messenger, 2020), and WhatsApp (Wha, 2020). Text-based CMCs have also been incorporated into specialized tools for teamwork and communication such as Slack (Wel, 2020) and Discord (Dis, 2020). However, several factors limit the ability of text-based synchronous CMC to replace face-to-face communication. These often include overlapping threads, limited turn negotiation mechanisms, no dedicated feedback channels, and no way of visually establishing referents or focus of attention (Vilhjálmsson, 2003). Video-mediated communication (VMC) platforms partially address these limitations.

2.2. Video Mediated Communication

Video conferencing technologies have gained immense popularity in recent years. When compared to text-based communications, some of the benefits of VMC include the exchange of non-verbal feedback and attitude cues, as well as gestural modalities for emphasis and elaboration (Isaacs and Tang, 1994; Doherty-Sneddon et al., 1997). For example, when there are interruptions in the audio channel, the visual channel can still provide important context for interpreting the pause (Isaacs and Tang, 1994). However, there are still many limitations in comparison to face-to-face communication. When communicating with groups, such as in classroom environments, turn-taking and floor management is difficult since it relies on judging exact gaze directions, something usually absent from VMC platforms (Doherty-Sneddon et al., 1997). Helping with a task is difficult for the same reason (Neale et al., 1998). Without additional channels, side conversations cannot take place, and pointing and manipulation of actual shared objects can be troublesome (Neale et al., 1998; Nardi and Whittaker, 2002). More recently, others have researched the effects of VMC systems such as Zoom, and its impacts on the user after prolonged use. Serhan investigated students perceptions toward the use of Zoom in remote learning and its effects on their learning and engagement in comparison to face-to-face learning (Serhan, 2020). The findings suggest that students had a negative attitude toward the use of Zoom and perceived it as having a negative effect on their learning experience and their motivation to learn. Students listed flexibility as a main disadvantage to using Zoom for learning, suggesting that more flexible tools, such as Mozilla Hubs, could complement Zoom in learning environments. Accessibility aspects, however, seem to still favor VMC platforms over Avatar mediated ones. Most VMC platforms support screen reading, and the inclusion of sign language interpreters, while accessibility aspects in MH still needs improvement (Usi, 2021).

2.3. Avatar Mediated Communication

Arguably, one of the most challenging problems faced by CMC platforms is to give the impression that participants are sharing the same virtual space (Vilhjálmsson, 2003). Avatar-mediated platforms have the ability to bring participants into a shared virtual environment (Vilhjálmsson, 2003). The first multi-user graphical online world was Habitat, developed circa 1985 (Damer et al., 1997). Habitat users could gather in a virtual town to chat, trade virtual props, play games, and solve quests. Users could move their avatars using cursor keys and communicate with other users only by typing short messages. No voice was supported. Avatar-based systems since Habitat have been many and varied, with applications ranging from casual chat and games to military training simulations and online classrooms. For instance, Second Life is an earlier avatar-based system in which players themselves design the world, its objects and their behaviors. Second Life incorporates three-dimensional modeling tools and a powerful scripting language (Delwiche, 2006). Mayrath et al. applied Second Life on a pilot case study integrating it into a two-semester English course at a large university (Mayrath et al., 2007). Similar to our work, the paper focuses on describing the process of implementing the avatar-based tool in the classroom and conducting a formative evaluation of the students' relevant experiences. However, it focuses on identifying the main instructional design aspects of the learning activities in the virtual environment, while little effort is spent regarding how to design the virtual spaces, the behavior and the pattern of communication of students in those environments.

2.4. Social VR Platforms in Education

Several research efforts have investigated shared online 3D environments and aspects of social interaction in VR. DIVE is a distributed interactive virtual environment platform designed to scale with a large number of simultaneous participants (Frécon and Stenius, 1998). In their work, the authors describe the network software architecture adopted and strategies to achieve scalability while maintaining high levels of interaction among users. Other research efforts have explored immersive group-to-group telepresence based on projection-based multi-user setups in which multiple users are provided with perspectively correct stereoscopic images (Beck et al., 2013), while others have focused on developing distributed multi-user and multi-platform system for collaborative visualization (Khadka et al., 2018). This approach shares characteristics with Mozilla Hubs. It supports system access from multiple platforms such as desktop, head-mounted display, or a smartphone. Others have explored aspects of social interaction in VR. Researchers have sought to interview industry experts of different social VR applications toward exploring how their platforms frame, support, shape, or constrain social interaction (McVeigh-Schultz et al., 2019). Others have also interviewed industry experts in an effort to probe into current design practices, and reflect on the design approaches that characterize avatars and avatar systems (Kolesnichenko et al., 2019). Our approach differs since it focuses on the observation and analysis of graduate and undergraduate students in a classroom environment. In a research effort closer to our own approach, researchers have developed a custom build of Mozilla Hubs to analyze several aspects such as group formation and behavior in the virtual rooms. They adopted mostly automated methods of data collection (e.g., for proxemics) and their work observed users during a shorter period of time, on a workshop setting (Williamson et al., 2021).

Online 3D environments can provide remote presence to in-class lectures and a medium for discussions for those unable to attend physically (Isaacs and Tang, 1997). Additionally, these environments facilitate various activities, experiments, and explorations in virtual worlds that would be too costly, dangerous, or simply impossible in the physical world (Eiris et al., 2020; Wen and Gheisari, 2020). The use of avatars in learning environments has been investigated in this context. The University of Colorado-Boulder conducted a Business Computing course online, relying on various CMC tools, including the Web, video-conferencing, and avatar-based shared online virtual environments. The environment essentially provided a virtual campus, where students could access resources located in various buildings. Walking paths and shared patios next to these virtual buildings naturally grouped students working on related topics and provided opportunities for discussion and unplanned encounters (Dickey, 1999). However, these platforms were resource-intensive, required specialized software, and relatively expensive machines to run at an acceptable performance. Developing for them was a task left to computer science specialists, and the existing tools did not afford easy user-centered content creation.

Researchers have investigated several aspects of social VR platforms for education (Pan et al., 2006; Scavarelli et al., 2019; Vergara et al., 2019; Bredikhina et al., 2020; Holt et al., 2020). Pan et al. analyzed virtual learning environments (VLE) both in regards to virtual reality and augmented reality. They show that VLEs can enhance, motivate and stimulate learners' understanding of certain events in learning environments (Pan et al., 2006). Scavarelli et al. proposed a framework for accessible and social VR environments in education (Scavarelli et al., 2019). This work is relevant since it brings elements of Computer-Supported Collaborative Learning into the development of socio-educational VR systems. To some extent, Scavarelli et al.'s work complements our own. While that work provides guidelines for building accessible Social VR platforms for learning, our work is concerned with providing guidelines for creating virtual environments in such platforms. For instance, Scavarelli et al. propose that accessible Social VR platforms should allow for multi-user collaboration and scale to variable group sizes, including various forms of communication, e.g., voice and gestural. They also suggest that avatars should be able to be customized by the user. However, these guidelines do not inform how the virtual spaces should be built to promote communication, collaboration, or team formation, which is the focus of this work. Nevertheless, it is noticeable that the Mozilla Hubs platform, adopted in our investigations, seems to follow the guidelines proposed by Scavarelli et al.'s work.

Others have begun investigating the use of avatars and Mozilla Hubs in educational settings. Bredikhina et al. (2020) described the design and procedures of a live workshop for children in Mozilla Hubs. They developed a step-by-step method called “Sugoroku,” for spatial design and audio calculations, which was found to be efficient during the kids' workshop. Others have explored Mozilla Hubs' application to scientific conferences and classroom poster sessions motivated by the limitations of isolation imposed by the Covid-19 pandemic. In their work, Holt et al. (2020) describe their efforts in presenting end-of-semester virtual poster sessions facilitated by Mozilla Hubs. They collected qualitative data and shared the benefits and challenges of their experience as felt by the students, the instructors, and external observers. To our knowledge, our research differs and is unique in its goals and length. In our efforts, we adopted Mozilla Hubs as an integral part of an undergraduate and graduate course, with students being repeatedly exposed to the virtual environments for an entire semester. Additionally, we aimed to investigate and build virtual rooms that facilitated interaction, communication, and team formation. We contribute with a set of guidelines for developing such environments, which we hope can generalize to other classes and topics beyond teaching virtual reality.

3. The VR4SG Course

The VR4SG (VRF, 2020) is a course that seeks to teach immersive technologies by applying them to social good topics. In this course, Solvers (students of any major and year) who want to learn virtual, augmented, and mixed reality are connected with Seekers (researchers, innovators, and entrepreneurs) who are seeking solutions to real-world social good problems, on a blend of theory and hands-on approaches. Some of the VR4SG course's main characteristics is that students apply Lean and Agile principles (Plenert, 2011; Williams, 2012), so teams learn VR concepts and develop their projects with the primary directive to deliver value efficiently by implementing short iterative improvement cycles of project deliverables. Another defining aspect of the course is that students self-form cross-functional teams to work on projects throughout the semester.

In previous years, course projects implemented fully-immersive virtual reality experiences built with the Unity3D game engine (Jerald et al., 2014). Most projects required virtual reality head-mounted displays (HMDs). A limited number of HMDs would be provided, and students would self-organize to develop their projects using the shared equipment. Learning how to develop with Unity3D took a considerable part of the students' efforts compared to the efforts dedicated to designing and conceptualizing the virtual environments. In this face-to-face format, during the last class of the semester all groups would present the virtual experiences they had developed along the semester to the Seekers and the general public.

3.1. Going Remote

During the Fall semester of 2020, the VR4SG course was offered once again. This time, due to the isolation requirements imposed by the pandemic, the course would be taught online. Converting the VR4SG course to an online format presented several challenges. One challenge was how to promote student collaboration and teamwork. We were also concerned with how to support students to self-form teams since they would have limited contact due to the online format. Other challenges were technical. Typically teams would develop fully immersive VR experiences based on HMDs using the Unity3D platform, and would resort to having group meetings to access shared spaces and equipment, none of which would be available for the semester. Additionally, Unity3D does not natively support remote collaborative work, which made it impractical for remote learning. New approaches to teach, develop, and collaborate had to be implemented to conduct the course under the imposed limitations successfully.

3.1.1. Defining a New Course Format

To teach the VR4SG lectures online, the Zoom video conferencing platform was chosen. Its mosaic structure showing the camera feeds of a large number of students (in our case 49 students) affords some aspects of real-world face-to-face communication such as non-verbal feedback and attitude cues that were well-suited to online lectures. However, while Zoom can efficiently support the instructors' presentations, its limitations significantly affect group activities and teamwork in teaching environments (Serhan, 2020). The lack of spatial interaction and the lack of mobility in Zoom's rooms made it clear that our typical group activities would not directly translate or be effective in the Zoom platform. In fact, the lack of direct interaction between the students meant that leading the students to self-form teams to work on the course's main projects was a major challenge to overcome.

3.1.2. Choosing Mozilla Hubs

Toward overcoming the limitations of Zoom, and video-conferencing platforms in general, we sought to employ an Avatar Mediated Communication (AMC) platform that would complement the classes taught on Zoom and serve as the primary development tool for students. AMC platforms can provide students with 3D virtual shared spaces where they can meet and work in groups, reproducing many real-world teamwork aspects. We looked for an avatar-based platform that would allow students to collaboratively work to design and build immersive environments. Additionally, we sought to find a platform that would provide students with the option to develop fully-immersive experiences (e.g., that required wearing HMDs) and experiences that could be experienced via desktop or other devices without the need for VR-specific equipment, and one that would not require users to install specific software on their devices, to reduce complexity. Moreover, we aimed for an application that would be centered on end-user content creation. After examining and testing several AMC platforms (Alt, 2020; Hub, 2020; VRC, 2020), we found that Mozilla Hubs (MH) fulfilled most, if not all of the requirements, while other platforms lacked in different aspects. AltSpace VR (Alt, 2020), for example, requires users to download an application, and is not focused on end-user content creation. VRChat (VRC, 2020) requires Unity3D for users to create content and environments.

Mozilla Hubs is an AMC platform designed for every headset and browser. It is open-source and has a focus on user-generated content creation. With MH, groups of up to 24 concurrent students can easily share virtual rooms, interact, and collaborate in group activities. Since the MH platform is web-based, no software installation is required (beyond an Internet browser), and the software runs on devices ranging from HMDs to laptops, desktops, and smartphones. Users embody and interact with avatars that can be easily created and customized. Users can choose to use their own names or adopt pseudonyms for their avatars. Content creation is achieved by dragging and dropping content such as 3D objects, PDF files, multimedia content, or web pages into the virtual environment or by selecting content from built-in libraries. MH is capable of spatial audio, providing a realistic feel to sound in the room, since it fades or gets louder according to the avatar's distance from the source. To enhance user interaction and provide visual feedback, the avatar's head move when the user controlling them speaks. Besides voice, users can also interact via a text, however this functionality is not present when MH is accessed with HMDs. There are, however, limitations to MH in comparison to Unity3D for developing immersive virtual environments. For instance, MH does not allow the implementation of scripts (code which automate actions), user defined graphical user interfaces, or character animation. Nevertheless, the ease of use, portability, and speed in which social virtual environments could be created and shared meant that students would potentially spend less time and effort dealing with development and technical issues (when compared to Unity3D). By spending less time on technical aspects, students could focus more of their efforts on the conceptualization and design of their projects, which could greatly transform and potentially enhance the quality of their creations. For these reasons, the MH platform was chosen to complement Zoom in teaching the VR4SG class remotely.

4. Preparing the Students

Once MH was chosen, the instructors followed best practices guidelines suggested by Mayrath et al. for training and supporting students using AMC platforms (Mayrath et al., 2007), and developed three tutorials on different aspects of MH. The goal was to introduce Mozilla Hubs to students concisely and effectively so that they could quickly begin to extract the most from the platform. To that end, three hands-on tutorial sessions (delivered through Zoom and Mozilla Hubs) were developed and presented to students. The first tutorial covered basic instructions on how to use MH. The second showed how to create advanced MH rooms. The third tutorial presented several approaches related to the creation and customization of avatars.

4.1. Basic Mozilla Hubs Tutorial

The first tutorial presented to the students combined Zoom presentations and hands-on activities in MH. After a quick review of MH functionalities in Zoom, students were instructed to join MH rooms. In the MH portion, students navigated a room with five stations to perform different activities. Students practiced how to access the platform, the basic controls for navigation, and how to add content to the rooms. Students practiced how to walk, run, fly, as well as how to customize the spaces by adding several types of content, including 3d models and persistent text messages. Students were also instructed on how to set their names and how to select avatars. Figure 1 illustrates the Basic MH tutorial room.

FIGURE 1

Figure 1. Mozilla Hubs Tutorial room. A tutorial room with five stages containing instructional information and activities was developed to provide students the basic skills to navigate and use the Mozilla Hubs platform.

The tutorial room was built as a large open environment with no walls and few details, and was created specifically for this purpose. To support all students, two identical rooms were created. Students were allocated evenly and randomly to each of the rooms (room link: https://hubs.mozilla.com/bRpoAXc/basic-tutorial-mozilla-hubs-a). By not having walls and adopting a minimalist design (with few 3D models and details), we aimed to develop a room that did not require too many computational resources from the students' hardware toward providing all students with the best possible experience.

To better understand the students' behaviors in MH, besides observing the students interact in the tutorial room, we also applied surveys after the activity was completed. With the surveys, we sought to probe the students on their impressions of MH and their hardware choices when using the platform.

4.1.1. Survey Responses

Nineteen out of the 49 students responded to the surveys. In this IRB approved study, students were offered 1% extra credit to participate. All participants who chose to participate were males, with ages between 20 and 48 (M = 24, SD = 6.07). Participants reported accessing the rooms with desktop or laptop machines as their primary devices. As a measure of participants' expertise and familiarity with virtual worlds or similar environments, participants were asked how often they played video games. This was deemed more effective than asking participants about their familiarity with fully-immersive VR since we were interested in learning about their levels of desktop-based experience. 26.3% of the students reported playing video games daily and 42.1% reported playing video games at least twice a week. Only 15% reported not playing video games. This suggests that over 60% of the students had familiarity with virtual environments similar to those presented in Mozilla Hubs. Regarding usability aspects, participants were asked to rate the performance of Mozilla Hubs on the primary device they used to access Mozilla Hubs. On a five-point Likert scale, 74% of the students judged the performance of MH in their primary devices as being either moderately or extremely fast while the rest of the participants judged it to be adequate or slightly fast. This suggested that more complex and detailed environments could be explored, possibly without compromising students' user experience in the rooms.

The majority of the students (89.5%) agreed or strongly agreed that MH is easy to use, and the same percentage felt very confident in using the platform after the tutorial. When asked about adding objects to the room, most students (63.1%) found it easy or extremely easy. However, some students (50%) noted that it was difficult to position and scale objects. One of the students noted “The addition of objects is extremely easy, but the placement of the objects is extremely difficult.” Another student mentioned, “.it is hard to position objects to the exact size and position you want them.” These comments may have been motivated by the fact that positioning and resizing objects in Mozilla Hubs is not parametric. Users cannot choose to move or scale objects by any unit of measure (e.g., by 1 inch). Also, there are no grids or guides to support these tasks. The free-form nature of positioning and sizing of objects in Mozilla Hubs may have contributed to these comments. However, further work is needed to fully understand students' motivations for these comments.

4.1.2. Researchers' Observations

In this research, we opted to perform our observations out into “the wild” toward understanding the impacts of the MH platform in the real world setting of classroom activities (Chamberlain et al., 2012). In this room, one researcher observed the students in each of the two identical rooms by using a split-screen approach. The researchers observed that navigating in MH seemed easy task for most students. This may be related to the fact that basic navigational controls involve using a combination of mouse and keyboard actions, which is similar to the controls adopted in most video game platforms, and 73% of the students reported playing video games at least once a week. Other functionalities such as precision positioning and scaling objects seemed challenging for students to master, which is in line with students' survey responses.

Regarding avatars, students were encouraged to choose from a large selection provided by MH. However, most of the existing avatars were very similar, with several of them representing a robot-like figure, with male or female overall shapes, and some color variations (see Figure 3A). As a result, several students chose identical avatars, which limited their ability to show their individuality in the virtual rooms. We also noticed that while most students followed the activities offered in each of the tutorial's stations sequentially, some students chose to wander around the room, ignoring the activities proposed or completing them out of sequence. Additionally, the tutorial's large open-space design did not seem conducive to promoting interaction among the students since students were rarely close to each other. It also made it harder for the instructors to manage students, especially those that chose to fly or freely explore the room. These initial observations guided the design of following 3D spaces we developed for the students.

4.2. Creating Customized Rooms

Our observations of the students interacting with the Basic Tutorial showed that they could not precisely position content in the rooms. This fact could potentially limit the complexity and quality of what the students would be able to create. To address these limitations, the second tutorial presented to students aimed at enabling them to develop elaborate MH rooms and precisely scale and position objects in the environment.

Mozilla Spoke (Spo, 2020) is a world-building tool for MH, meaning that its main focus is to facilitate building 3D virtual spaces for the Mozilla Hubs platform. Mozilla Spoke is fully integrated with MH, and environments created in Mozilla Spoke can automatically be published to MH. Its user interface shares many similarities with other existing 3D authoring tools (e.g., Unity3D), and it comes with a large set of predefined assets, such as floor elements, walls, doors, windows, etc. Importantly, it adopts a parametric approach to orienting, positioning, and scaling objects (see Figure 2), so users can numerically determine those properties, and precisely define them. Similarly to the previous tutorial, the Mozilla Spoke tutorial session was presented using a combination of Zoom and Mozilla Spoke. Users were invited to follow along, and complete activities with Mozilla Spoke during the tutorial.

FIGURE 2

Figure 2. Mozilla Spoke building tool. Mozilla Spoke facilitates the creation of virtual environments by allowing precise control over positioning, rotating, and scaling objects in Mozilla Hubs rooms. The image details some of the parametric controls.

4.3. Customizing Avatars

Another shortcoming observed during the Basic Tutorial was the students' limited ability to customize avatars when they relied on the existing MH avatar library. It was essential to provide students with additional tools to customize avatars since avatars can influence the user's experience in 3D virtual social spaces in several ways (Nowak and Fox, 2018). For instance, it has been demonstrated that avatar characteristics can augment or limit people's ability to self-present, as well as influence how people engage with each other in the digital environment (Nowak and Fox, 2018). Hence, a tutorial session reinforcing the importance of avatars and demonstrating different approaches to create and customize them, was presented to students. Like in previous tutorial sessions, this tutorial was presented on Zoom, with hands-on activities performed by the students in MH. The three strategies presented to students to create and customize Avatars were:

• Changing the avatar's skin with Quilt. Quilt (Qui, 2020) is a simple web-based avatar editor for MH that provides an easy way to design new skins for avatars. Quilt applies regular image files as avatar skins that can be previewed and exported to MH. While not capable of changing Avatars' shapes, the ability to re-skin them greatly increases the avatars' customizability. Figure 3B shows an avatar re-skinned with Quilt.

• Changing the avatar's shape. To further provide students with options to customize avatars, students were instructed on how to create avatars' 3D models. MH uses GLTF 2.0 (glT, 2020) file format to render avatars. Users were instructed on how to create 3D models for their avatars using open-source 3D modeling tools such as Blender (Ble, 2020) and were shown how to download 3D models from existing repositories (Pol, 2020; Ske, 2020). Figure 3C shows an avatar imported from Sketchfab.com (Ske, 2020).

• Creating avatars based on the user's picture. The most popular tool for avatar creation among students was readyplayer.me (Rea, 2020). Readyplayer.me automatically generates avatars based on images taken from the user's face. Users can customize several aspects of the new avatar's appearance. Customization involves facial features such as eye and hair color, as well as clothes' color and style. One limiting factor is that an avatar generated with this tool is composed only of the user's head and upper-torso, instead of their full body. However, the ability to automatically create an avatar similar to the user's face gives users great flexibility in creating personalized avatars. Figure 3D shows an avatar created with readyplayer.me.

FIGURE 3

Figure 3. Mozilla Hubs avatars. (A) Basic Mozilla Hubs avatar. (B) Avatar customized for the VR4SG course with the Quilt tool. (C) Custom model of an avatar imported from Sketchfab.com (Ske, 2020). (D) Avatar created from user's picture with readyplayer.me (Rea, 2020). (E,F) Other avatars chosen by students of the VR4SG course.

In the activities that followed this tutorial, students were encouraged to customize their avatars. However, avatar cutomization was not strictly required. Students were free to choose any avatars to navigate MH rooms. Figures 3E,F shows some of the avatars chosen by the students. We discuss the students' avatar choices in more detail in the following sections.

5. Developing Mozilla Hubs Rooms

Once students were trained on how to use the MH platform, three rooms were developed with activities that complemented Zoom lectures. These rooms were intended to further familiarize the students with the MH platform, as well as help the instructors understand how the students behaved in the virtual environments.

5.1. Room 1 - UNSDG

The Sustainable Development Goals (UNSDG) room was designed so that each student could create content in support of one of the SDGs of their choice. To support all students, five identical rooms were created. The SDGs are a set of 17 goals designed to be a blueprint for achieving a better and more sustainable future for all (ARE, 2020). The SDGs were set in 2015 by the United Nations and are intended to be achieved by 2030. Examples of UNSDGs are No Poverty, Zero Hunger, Good Health and Well-being, and Quality Education (ARE, 2020). These goals are aligned with the VR4SG course goals and provide relevant real-world topics for the students to work on. A link to one of the UNSDG rooms follows: https://hubs.mozilla.com/Rm7wCxq/group-e-activity-1-and-2.

5.1.1. Objectives and Tasks

Each student was instructed to pick one of the 17 UNSDGs and populate a space explaining that topic in the MH room. Their objective was to go beyond raising visitors' awareness about the chosen issue by leading visitors to donate to those causes. Students were oriented to find real-world entities that work toward the goals they chose so that real donations could be made. The goal was to achieve real-world impacts with this class activity. Students were given one week to develop content for their spaces. The instructors selected the five best projects, and invited students to vote. To that end, all students were given a set of 10 fictional coins that they could use to vote for their preferred projects. In the end, the room that received the most votes was recognized by the instructors and received extra-credit for their work. This served as an incentive for students to do their best in developing the projects.

With the UNSDG room, our goal was to observe and collect data toward understanding how to create virtual rooms that could facilitate collaboration among students, as well as provide students with a virtual space that would motivate them to interact and communicate. To that end, we created the UNSDG room shown in Figure 4.

FIGURE 4

Figure 4. United Nations Sustainable Development Goals (UNSDG) room. (a) A wall in the central area of the room displayed instructions and information regarding the UNSDGs. (b) Overall layout of the room. Each room contained 10 circular spaces for students to populate with content regarding the UNSDGs they chose to work on. (c) Space populated by one of the students. The orange square shows a website that is misaligned in the room, highlighting the difficulty faced by students to precisely position objects in MH.

5.1.2. Room Design

The UNSDG room design is based on 11 inter-connected open spaces delimited by circles on the floor. The room has one larger central area, with the other 10 circular spaces surrounding it. The central area shows information about each UNSDG and instructions for students, such as how to locate their own spaces. Each area around the central space is numbered, and each student was assigned a corresponding number before receiving access to the room. Similar to the Basic Tutorial room, we opted to design a room with few details, so that users would be able to add their own content (increasing the room's complexity and need for more computational resources) without compromising the user experience.

To promote collaboration and a sense of sharing the room with others, we positioned the individual spaces on a circular pattern. With this design, each student was able to see each other at all times. We expected that it would facilitate collaboration and interaction among students. To understand the students' behaviors and impressions of this room, besides observing the students, we also applied surveys after the activity was completed.

5.1.3. Students' Survey Responses

The students that agreed to participate in the initial survey (related to the Basic Tutorial described in the section 4.1) were also invited to participate in subsequent surveys. However, only 15 of the initial 19 students chose to respond to this survey.

All participants reported accessing the UNSDG room with desktop or laptop machines. Students were asked about the time they spent in the room, and their impressions about the room's size and level of detail. Over half of the participants (60%) reported spending between 10 and 20 min in the room, 3 participants (20%) reported spending between 20 and 30 min, and the other 3 participants over 40 min. When asked about the room's size, 11 participants (73.3 %) reported that the room was large or too large, while the rest of the participants responded that the room was about the right size. The room's size was also mentioned in response to what participants disliked the most about the room. One-third of the participants had comments about the room's size being too large. One participant mentioned, “Too much wasted space between each content area making it take too long to go from one area to another.” Another student simply mentioned that “It was too big.” Regarding the room's level of detail, students noted that the room was “too simple,” “too basic,” and felt “empty.”

5.1.4. Researchers' Observations

Two researchers from our group observed students into “the wild” (Chamberlain et al., 2012), in the rooms described in this manuscript. Researchers observed from a distance to avoid influencing the behavior of students. From observing the students, we noticed that they could easily understand the room's layout and locate their work areas. Although we adopted a simple open-space design (without walls), the circles on the floor delimited the different work areas, helping students understand and navigate the room.

One positive aspect of this room design is that we observed that some students walked around the room and looked at other students' work. We also observed moments in which students interacted with each other while working on their projects, which was another positive aspect of this room. However, rarely more than three students shared the same room, and during those moments, students would seldom talk to each other or interact in any way. We speculated that three factors were at play. First, we chose to create five identical UNSDG rooms, each with ten individual work areas to accommodate all 49 students of the VR4SG class. With this setup, at most ten students had the chance to see each other and interact while working on their projects. Second, since each student had to pick one out of 17 UNSDGs, few worked on the same topic, which did not promote collaboration or exchange of ideas. Finally, students had one week to develop their rooms, with no required schedule to work on the projects. While this offered great flexibility, it did not encourage students to work at the same time, so not many students encountered each other during that period. Additionally, the fact that the students' spaces had numbers but no names (or other identifying information) did not help students to contact each other.

To promote collaboration and interaction among students more effectively, for the next activity, we aimed to develop a task and a MH room that would follow four principles:

1. Sharing the same goal. All students should share the same goal to feel compelled to communicate and collaborate.

2. Participating simultaneously. All students should participate in the activity simultaneously, so they could all meet and interact.

3. Developing smaller rooms. The rooms developed should be designed to hold as many students as possible, while not being too large.

4. Developing more detailed rooms. Based on students' comments, a more detailed environment should be created. We next describe the room and activity developed following these principles.

5.2. Room 2 - Pioneers of VR

The VR4SG is an introductory virtual reality course. As such, one of the topics covered is the history and evolution of virtual reality (VR). The Pioneers of VR room was developed as a complement to a Zoom lecture covering several pieces of equipment, researchers, and inventors that have impacted the VR research field. In this MH room, seven short videos were made available to the students. The videos describe the history of influential early VR researchers and their contributions such as Ivan Sutherland's “The Ultimate Display” (Sutherland, 1965), and Morton Heitig's “Sensorama” (Heilig, 1962), as well as later work such as the “Data Glove,” developed circa 1987 (Foley, 1987). A link to access the room follows: https://hubs.mozilla.com/9kHLTMK/pioneers-of-vr-2021.

5.2.1. Objectives and Tasks

The objective of this room was to explore students' behavior and communication in a virtual room where students would primarily consume content. After a lecture by the instructors on the Zoom platform, the students were given 40 min to join one of two identical Pioneers of VR rooms and explore its content. Students were instructed to watch all the videos and interact in the virtual space.

5.2.2. Room Design

Based on the UNSDG room feedback, in this room all students had a shared goal (watch the videos), and the activity happened at a specific time, during a short period (40 min). The room's design is very different from the previous rooms (see Figure 4). The rooms developed for previous activities were minimalistic (had few details) and were based on large open spaces. For this activity, we created a small enclosed room, with walls, doors, and lighting, simulating the inside of a building; The virtual environment was more complex and had more details. Art galleries and museums inspired the room's design. Museums were found to be a useful metaphor since they are usually well-designed, well lit, rich social spaces. Also, the museum metaphor is aligned with the historical aspects of the videos presented and is suggestive of the students' task in the space, which was to watch the videos made available to them. We hypothesized that this design language would reduce the feeling of emptiness and simplicity that the students' expressed about the previous rooms, while encouraging communication and interaction among students in the room.

The room was composed of two adjacent areas. The first area was a small squared room with a couch, some VR equipment, and a large wall-mounted display. The display showed slides with task instructions and described how the room was organized. A door to the left of this room led to a larger L-shaped space where seven videos were positioned on the walls. Each video had a small poster to its left with the author's name, the title, and a number that helped students easily identify each video. Each of the videos was illuminated by one or more spotlights so that the student's attention would be drawn to them. To define the room size, the heuristic we adopted was that 24 regular-sized Mozilla Hubs avatars should fit in the room comfortably, meaning that in general avatars should be able to maintain at least a couple avatar-length distance from one another. The design of the room is depicted in Figure 5.

FIGURE 5

Figure 5. Pioneers of VR room. (a) Room with instructions to students. (b) L-shaped room layout with seven videos about influential work in the field of virtual reality. (c) Detail of video number one.

5.2.3. Students' Survey Responses

Similar to the previous rooms, all participants reported accessing the Pioneers of VR room with desktop or laptop machines. On average, the videos in the room were 3 min long. Since there were seven videos, a little over 21 min would be needed to watch all videos. Six out of 16 participants that answered the survey (37.5%) reported spending between 10 and 20 min in the room, nine participants (56%) reported spending between 20 and 30 min, and just one participant (6%) reported spending over 50 min in the room. The times reported by the students suggest that most students stayed long enough in the room to watch all videos.

One of the main goals for this room was to learn about the ideal size of a virtual space for classroom activities. When asked how they felt about the room's size in relation to its purpose, contrary to the previous room, no participant reported that the room was “large” or “too large.” Twelve students (75%) reported that the room was “about the right size.” Only one student (6%) reported that the room felt “small,” and three students (18%) reported that the room felt “slightly small.” Three students also mentioned the room's size in response to what they liked the most about the room. One participant mentioned “It was very nice to walk around, and it wasn't too big.” Another participant noted that the room was “Small and fast to navigate through the video materials.” Just a couple of students mentioned the size of the room as something that they disliked. For example, one student mentioned that the room “felt a little cramped in some areas.” Overall, these results were very positive, suggesting that the students preferred smaller rooms, over large spaces. It is also important to note that, contrary to the UNSDG room, no participant mentioned feelings of emptiness, or loneliness.

Another aspect highlighted by the students was the overall design and detail of the room. About one-third of the students praised the museum or art-exhibit style of the room. Students also mentioned the lighting, layout, and overall aesthetics in response to what they most liked in the room. One student mentioned “I liked that all the things looked and felt like I was at a museum watching different videos with people.” The overall positive feedback is reinforced by the fact that three participants mentioned not having anything that they disliked in the room. However, at least one aspect was noticeable. Four participants (25%) complained that when more than one video was played, their sound would interfere with each other. While this is directly related to the fact that the videos are positioned close together, we hoped that it could be fixed in future iterations by fine-tuning the environment's spatial-audio configuration. This could potentially allow the distance between the videos to be maintained (avoiding the need for a larger room), and the audio from each video to be adjusted so that they would not interfere with each other.

5.2.4. Researchers' Observations

The researchers observed the students in the room by joining the virtual space and walking among the students during the MH session. Our efforts to have more students sharing the same room were successful and created more opportunities for students to interact and communicate. For instance, one of the two identical rooms reached full capacity during the session. Having several students share the room simultaneously allowed us to observe students' behavior and communication patterns in the environment with more detail.

Regarding students' use of avatars, 75% of the participants chose not to customize them, instead embodying the default robot-like avatars. Among the students who chose to customize their avatars, one chose to be represented as a panda, while others chose avatars that did not have an animal or human-like features. For example, two participants chose drone-like avatars as their virtual representations (see Figure 3F).

In terms of students' behavior, we observed that over 50% of the students spent time reading the room's instructions when they first joined them. Having the instructions in a separate room (from the main room that showed the videos) seems to have kept the students from being distracted, and this may have contributed to their attention to the instructional slides.

We observed that groups of three to five students formed around the videos to watch them. The fact that students formed groups was relevant and vital to our goals of promoting collaboration and communication. By forming groups and coming close to each other, opportunities to meet and interact emerged, which is essential toward developing virtual environments that could support and facilitate team formation. We observed noteworthy behaviors as these groups formed. When students were already watching a video, a newcomer would either join the group or wait from a distance to begin watching the video from the beginning. These were exciting moments since some students communicated and interacted to judge if they would just wait or continue watching with the group. In some occasions, students would just join the group, and the group would agree to restart the video to include the new student.

When asking about a specific video, we noted that students would typically come closer to the group watching the video and use voice to ask about it. However, to make general comments about a video, the room, or ask a general question, students would type a message on the chat. For instance, students used the text chat to ask about the event's duration or comment about a given aspect of one of the videos. While communication among students was not frequent, this behavior suggested that students were not just using both communication channels (voice and text) available in Mozilla Hubs to interact, but that they were using them for different purposes. The voice channel for one-to-one conversations or with a small group, and the text channel to talk to the entire group, in this case, about more generic topics or comments.

The next room we developed would support forming teams for the students' major project. Based on the students' responses and our observations, we were confident in the Pioneers of VR room layout and overall appearance as a template for the new room. However, there were areas for improvement. One change requested by the students through the surveys was adjusting the audio of the videos so they would not interfere with each other. We also noticed that just walking around the room among the students limited our ability to fully understand how groups formed and how students communicated, and we recognize this could also influence students' behavior. In the next section, we describe the room developed toward promoting team formation and the strategies we implemented to analyze the result of our efforts.

6. Room 3 - Seekers Room

For the main group project of the VR4SG course, the students (Solvers) had to select one out of eight problems proposed by members of the University of Florida community (Seekers) and self-form teams of five to six students. The problem proposals were initially presented to students in a Zoom lecture. In that lecture, each Seeker presented their proposals to the entire class, and the students had the opportunity to interact with the Seekers and ask questions. The Seekers room was created to further assist the students in learning about the problem proposals and forming teams. Figure 6 shows the Seekers room. The room can be accessed with the following link: https://hubs.mozilla.com/SbF9uYS/call-to-action-room-1.

FIGURE 6

Figure 6. Seekers room. (a) Room 1, containing problem proposals one to four. The room is full of textual comments and other objects added by the students. (b) Detail of the problem proposed by the Athletic Association, with problem number, textual summary, and video, surrounded by several comments left by students. (c) Room 2. Only a few students visited room 2, and this is reflected by the small number of comments left there.

6.1. Objectives and Tasks

While the UNSDG room focused on content creation and the Pioneers of VR room focused on content visualization, we adopted a mixed approach in this room. Students' task was to review the problem proposals by watching the videos in the room and leave written messages to each other with comments and their reasons for wanting to form a team to work on a given problem proposal. Besides leaving textual messages, students were also encouraged to add other content to the virtual environment (such as 3D objects, documents, or web pages) that could motivate others to work on the same project and form teams. While students were encouraged to interact, the instructors made sure not to make any comments on how the students should behave or communicate while in the room to avoid influencing students' behavior and communication patterns.

6.2. Room Design

The Seekers room design is based on the Pioneers of VR room. We chose to maintain the overall layout due to the similarity of the tasks and the students' positive feedback. We adopted the same L-shaped room design for the main area, in addition to the smaller room where students begin the interaction by reading a set of instructions. The level of detail and lighting of the Seekers room is similar to those of the Pioneers of VR room. Each video was positioned as if they were hanging from the walls, and a textual description with a summary of the problem proposal was added to one of the sides of the video.

To further advance our understanding of the students' behavior and communication patterns and address the issues raised by students in the previous room, some changes were adopted. The spatial sound was adjusted so that the sound of one video would not interfere with another. The audible distance (the distance from which the sound of a video can be heard) was reduced, creating areas in the room where a minimal sound from the videos could be perceived. We speculated that this could motivate more people to interact verbally with each other since those were “quiet” spaces. Regarding size, while most students mentioned that the previous room was “about the right size,” some suggested the room as “slightly small.” Since the Seekers room had one more video than the Pioneers of VR room, and students were invited to add their comments about each of the videos, we sought to leave more space between them. Hence, instead of creating two identical rooms with all videos, we chose to create two rooms (room 1 and room 2) displaying four videos in each of them. We linked the rooms so that students would be able to navigate between them freely. With this approach, while the number of rooms and their sizes remained the same (compared to the Pioneers of VR room), each problem proposal had more space to be presented, and students had more space to contribute with their comments and other content.

Regarding observing the students, instead of just walking around the students (as was done in the previous rooms), we opted to observe them from a higher viewpoint (MacKenzie, 2012). A higher position would allow the researchers to observe better how the students navigated the room and formed groups while in that space – an into “the wild” approach (Chamberlain et al., 2012). To that end, the room's ceiling was removed, and a blue sky was added to the scene to make it seem more natural. By removing the ceiling, the researchers were able to position themselves at one corner of the room and stand at the top of the walls, observing the entire room. Figures 7a,b depict the researchers' view of the rooms. We also hypothesized that the removal of the ceiling would make participants perceive the room as larger.

FIGURE 7

Figure 7. Students Interacting in the Seekers rooms while embodying several avatars. (a) Students in room 1 and (b) room 2 in min 15 of the session. Fourteen students are visiting room 1 while just two in room 2. (c) Students arranged in a circle interacting as a group. Names have been blurred for anonymity.

6.3. Students' Survey Responses

Similar to the other rooms, all participants reported accessing the Seekers room with desktop or laptop machines. On average, the videos in rooms 1 and 2 were about 6.5 min long (SD = 2.23 min). With four projects displayed in each room, it would take a little more than 52 min to watch all videos. Five out of 15 participants that answered the survey (33.3 %) reported spending up to 20 min; Seven participants (46.6 %) reported spending between 20 and 40 min in the room, while three participants reported spending over 40 min. These reported times show that most students did not watch all the videos while in the MH room, focusing instead on leaving comments and interacting with other students. This behavior may be partially explained by the textual summary that was added to each of the videos, which allowed students to recall details of the problem proposals faster, without the need to watch the videos to do so.

Similar to their choices in the previous rooms, the selection of avatars did not seem to reflect their self-images in the real-world. Only a couple of students chose to create avatars based on their faces (using readyplayer.me, as described in section 4.3). All other participants chose robots, animals, or other cartoon-like avatars. Two students mentioned that they changed avatars every time they entered the rooms. One participant that chose a drone-like avatar mentioned that he chose that because it was “fun,” while another participant mentioned that he chose a standard avatar because he did not want to “stand out.” Figure 7 shows rooms 1 and 2 populated with students and several of their avatar choices.

When asked how they felt about the rooms' size in relation to their purpose, most participants (9 participants, 60%) answered that they thought “about the right size,” while the other 40% reported that they felt the room was “small” or “slightly small.” Several survey participants commented that they thought the rooms felt too “cluttered” because of so many messages and objects other students left in the rooms. This may have contributed to more students feeling that the Seekers room was small compared to their perception of the Pioneers of VR room. These answers and comments suggest that slightly larger rooms with more space for students to leave their comments would have been a better choice. It is unclear if removing the ceiling helped the students perceive the rooms as larger since no comments were made about it. More research is needed to ascertain that aspect. Regarding sound conflicts between the videos, only one participant mentioned the sound as an issue for this room, suggesting that the issue was resolved by tweaking the spatial-audio settings of Mozilla Hubs.

Regarding having the videos be accompanied by summary texts, three students mentioned that they liked having both since it helped them choose between the problems proposed easily. One student mentioned that “While the texts were quick to read, the videos gave more in-depth information that I could access if I wanted to.”

6.4. Researchers's Observations

Toward making richer observations, the MH session in the Seekers room was recorded. In total, 75 min of video recordings were obtained. Figures 7a,b depict the point of view of the researchers observations and recordings. For the analysis, a snapshot was taken at the end of every minute of the video, generating 75 snapshots that were analyzed individually. The analysis entailed researchers annotating the number of students in each room, their relative position to each other, to identify when they formed groups, as well as their general behavior exploring the room. Next, we describe students' attendance, behavior, and communication patterns based on the video recordings and observations by one of the researchers that walked around the room during the session (MacKenzie, 2012).

6.4.1. Students' Attendance

Room 1, containing problem proposals 1 to 4, consistently received more students than room 2 (containing projects 5 to 8). On average, Room 1 welcomed 8.8 participants (SD = 3.46, Max = 14) during the 75-min session, while room 2 received on average just 2.4 participants (SD = 1.93, Max = 6). Besides receiving less participants, while room 1 was never empty during the session (Min = 3), room 2 was vacant for 15 of the 75 min, receiving its first participants 10 min after the beginning of the session. Additionally, while all the students who visited MH during the 75-min session visited room 1 (26 distinct students), room 2 was visited only by 17 students. Figure 8 shows the number of participants per room during the MH session's duration, minute by minute.

FIGURE 8

Figure 8. Seekers room: attendance over time. Over the 75 min session in the Seekers room, Room 1 received consistently more students when compared to Room 2.

6.4.2. Students' Behavior in the Room

Regarding the students' behavior when first entering the room, we observed similar patterns compared to the Pioneers of VR room. Most participants went through three stages: (1) Read the instructions, (2) Take a short walk around the room to understand the overall layout, (3) Circle back to the first problem proposal and start examining them sequentially. Since students were requested to leave texts with their comments in the environment, such as those illustrated by Figure 6b, several students left comments after reading the summary of the video. Overall, participants left thirty comments in rooms 1 and 2 combined, with most comments being left in room 1. Each project in room 1 received on average 5.25 comments (SD = 2.21), while in room 2 each project received on average just 2.5 comments (SD = 2.4). The number of comments left is directly correlated to the time students spent in each room. Most students visited and stayed for longer in room 1, and the projects there received the most comments. Several students mentioned that the texts left in the room by other students were helpful in choosing who to form teams with. One possible reason might be that students could learn about shared interests and motivations even if the authors of the messages were not there.

Similar to the behavior in the previous room, students gathered around videos. However, this time, students would form groups close to the videos and start playing them. As the video played, students would talk about them and share information about their interests staying in that formation even after the end of the videos. To analyze the impact of the Seekers room on team formation, we compared the teams formed after the MH session and the groups of students that directly interacted in the room. We found a clear correlation between the teams formed and the groups that interacted in the room. Eight teams were formed with 5 to 6 participants each. Two of those teams (teams 1 and 4) were formed by at least four students that spent time (>5 min) in the rooms talking in groups near the problem proposals they ended up choosing to work on. Three other teams (teams 3, 6, and 8) had three members that spent time in rooms together and later formed teams, while only two members of team 2 spent time together in the MH rooms. The only exception was team 7, in which none of the students that formed the team interacted in the virtual room. This strongly suggests that the rooms were successful in helping students self-form teams. The students' comments also support this affirmation. One of the students mentioned “The rooms helped meet people that shared a similar vision about the problems proposed and possibly form teams.” Another student pointed out that “It was good to talk to several people and see what group of people and problem I wanted to work with.”

One particular behavior was surprising. After exploring the rooms and interacting for some time, several students seemed to remain static in the rooms. A couple of the students that answered the surveys noted that after achieving their goals in the room, instead of disconnecting, they remained in the room while working on other matters. One student mentioned that he would glance from time to time at the room, and another noted that he kept listening, monitoring for any conversations that might interest him. As a result, this behavior caused the rooms to be populated by some avatars that would not move and interact. While this seemed odd and possibly a negative behavior, it also suggests that students were interested enough in the MH session to remain participating, even if only devoting partial attention to that task.

6.4.3. Communication Patterns

Our video analysis showed that students interacted more frequently and for longer periods than in the previous rooms. This behavior is possibly due to the nature of the task (to interact toward forming teams), but it could also be due to students getting more used to the platform. The communication pattern in this room was similar to that observed in the previous room. Students used voice for communication in small groups or on one-to-one conversations, while they resorted to the text channel to communicate to all the students in the room. This communication strategy was demonstrated in this room when several students used text to ask questions such as how to navigate from one room to the other or ask about the next steps to form teams (which would happen right after the MH room session). The instructors also found themselves using the text channel to make announcements, as it was more effective than using voice to communicate with all the students. Overall, this room served as a confirmation that students use both communication channels (voice and text) and that the use of one or the other is dependent on the target group the message is intended to reach. Text was used to communicate with all students in the room, while voice was used for small groups or one-on-one conversations.

7. Discussion

Room Design. Immersive 3D virtual social spaces in which navigation and communication are mediated through avatars, such as those developed with the Mozilla Hubs platform, offer many opportunities and challenges for the developers (or architects) of such spaces that differ from the real world. For instance, both large or small spaces can be created arguably at identical “costs,” and choosing to create rooms that resemble real-world environments or choosing to develop rooms that are radically disconnected from reality are equally possible options. The affordances of virtual worlds are also diverse from those of real life. In Mozilla Hubs, for example, flying is something as natural as walking. In our exploration of virtual spaces to support team formation in the VR4SG course, we developed several activities and created several rooms with diverse designs. The Basic Tutorial room and the UNSDG rooms were based on large open-space designs that were not made to resemble real-world scenarios. Later rooms adopted a different language. The Pioneers of VR room and the Seekers room adopted more familiar designs, resembling a real-world museums or art exhibits. Those rooms had walls, were relatively small, and were rich in details (such as lighting, furniture, and decorations) making them more similar to the real-world. The design of those rooms was mainly based on the students' feedback and researchers observations. Students reported on the size, the detail, and the preferred overall layout for the rooms. The results show that the rooms with characteristics that made them more similar to the real world were reported as being “cozy,” “comfortable,” and with sizes that were more in line with their purposes. The UNSDG room, on the other hand, was reported as being “cold,” “empty,” and “too large.” These results suggest that, even though there are endless possibilities for the design of virtual worlds and spaces, users (in our case undergraduate and graduate students from the University of Florida in the United States) preferred virtual space designs that had characteristics related to “comfort,” or “coziness” that were similar to those in the real world. Further research is needed, however, to fully understand the motivation for these preferences. It may be the case that more exposure and familiarity with avatar-based virtual environments will affect users preferences in regards to room design in the long run, while we can also hypothesize that cultural background plays an important role in these preferences.

Several researchers have explored VR as a tool to elicit users' preferences in the real world. Researchers have investigated the use of VR to elicit preferences of hotel room design (Bogicevic et al., 2018), understanding lighting preferences in office spaces (Heydarian et al., 2015), and student preferences for study spaces (Applegate, 2009). However, few have investigated users' preferences concerning virtual space design, particularly those related to social spaces geared toward team formation. This paper collects our findings and observations eliciting students' preferences in a set of design guidelines that aim to support future developers of virtual 3D social spaces. However, we believe that these guidelines are bound to evolve as researchers gain more understanding of social VR platforms' capabilities and explore their unique affordances. One area for further exploration relates to the fact that in our work most affordances unique to MH environments were not fully explored (e.g., the ability to fly or teleport). Further research is needed to explore the impact the unique affordances of virtual environments can have on room design and users' preferences.

Students' Attendance. Besides the rooms' design, we also analyzed the participants' attendance in the virtual spaces. Surprisingly, we found that naming the rooms can have a significant impact on attendance. For instance, in section 6.4.1, we describe the attendance pattern to the Seekers rooms in which most students chose to begin interacting by entering room 1 and stayed there for longer when compared to room 2. This attendance behavior was surprising since there were no substantial differences between the two rooms or their content. The rooms were numbered randomly, and the distribution of problem proposals in the rooms was also random. Having students choose to visit room 1 first may be partially explained by the fact that we chose to label the rooms “one” and “two,” which naturally gave them a sense of order, and this may have prompted students to begin visiting the rooms by going to room 1 first. The fact that most students chose to stay together in room 1 may be explained by the fact that to form effective teams and engage in collaborative tasks require that individuals identify, and preferably associate with, others who have compatible preferences and a shared background knowledge (Curry and Dunbar, 2013). For this reason, students may have been drawn to the room that offered them a higher chance to identify similarities in prospective team members, and this may have caused them to choose to stay in the most populated room for longer. This is also supported by the fact that several students that started the session in room 1 would visit room 2 for a few minutes, possibly note that the room was emptier, and then return to room 1 for the remainder of their visit. Toward better distributing students in the rooms, it seems that a better approach to label them would be to adopt names that do not imply sequence or order. This may have prevented students from being biased when choosing which room to enter when multiple options are available. Another approach would be to assign a balanced number of students to each of the rooms at the beginning of the MH session.

Avatar Choices. During the tutorial on avatar customization, the instructors emphasized to students the importance of avatars as their self-representation in the virtual environments. Prior work has suggested that avatar characteristics can augment or limit people's ability to self-present (Nowak and Fox, 2018). Therefore, it was hypothesized that students would customize their avatars and select avatars that would communicate to others their potential to become good team members. However, based on survey responses and observations, few students customized their avatars. Most students saw the opportunity to choose among existing avatars as an opportunity to have “fun,” opting to use cartoon-like and non-human-like avatars that did not necessarily represent any of their real-life characteristics. While these avatar choices may be interpreted as lost opportunities to support team formation, they can also be interpreted under a more positive light. Some students reported that the avatars complemented the experience of being in the virtual world. Besides making it more “fun,” some students mentioned that choosing and interacting with other avatars was something they looked forward to. Hence, this may have motivated students to join and participate in the MH sessions to a greater extent than if strict rules had been imposed regarding the students' avatars choices. More research, however, is necessary to evaluate further the avatar choices and their impacts on students' attendance.

Communication Channels. The fact that students used two communication channels for distinct purposes is of great relevance to further our understanding of how to promote communication in such virtual environments. One important aspect is that the dual communication channel (simultaneous voice and text) represents an affordance not present in the real world. Still, it is one that users seem to perceive and adopt in their interactions in the virtual environment quickly. The fact that the behavior of using text for group communication and voice for one-to-one or small-group communication emerged organically is also notable, meaning that no instructions or guidelines were provided that motivated this behavior. Further research is necessary to understand how best to utilize both of these communication channels in class environments. More research is also needed to understand the impact the use of HMDs have on interaction and communication since typing while wearing HMDs is usually not possible or significantly cumbersome (Boustila et al., 2019; Menzner et al., 2019). It might be that while HMDs can increase the sense of presence and immersion (Kwon, 2019), they may limit the users' ability to communicate and may also limit the amount of time students are willing to spend in the virtual rooms.

7.1. Design Considerations

In this section, we outline some considerations for the design and development of avatar-based virtual 3D social spaces and activities, factoring in lessons we learned conceptualizing, evaluating, and critiquing the three rooms described in this manuscript for the VR4SG course. The trends that emerge often reinforce the existing literature on group interaction and communication but are important to note here as frequent trends we observed on avatar-based virtual environments.

• Activity Duration. Set limits on the time students spend in the virtual rooms for an activity. Concentrating the participation increases the chances that students will meet and interact in the virtual environment.

• Room Size. Develop small, rather than large rooms for students' activities. While size is most likely dependent on the purpose of the virtual environments and more research is needed to better characterize the relationship between room size and the tasks performed, smaller rooms were preferred over larger spaces. Small rooms meant that students did not have to walk long distances, which also increased students' opportunity to be close to each other, facilitating communication and interaction.

• Room Layout. Develop virtual rooms with layouts that resemble real spaces. Also, develop closed rooms (with doors and walls) instead of open spaces. Students reported feeling more comfortable and have found it easier to locate themselves and understand the rooms' layout when they were delimited by walls.

• Level of Detail. In our observations, students preferred rooms with higher levels of detail. We suggest building rooms with furniture, lighting, and decorations (such as pictures on the walls) since rooms with these characteristics were associated with being “warmer” and more “comfortable.” Rooms with low levels of detail were associated with being “cold” and were overall less favored.

• Number of Rooms. The MH platform limits the number of concurrent users in a room to 24. For this reason, often, multiple rooms must be created to accommodate all students. We suggest creating the smallest number of rooms possible to maximize students' opportunities to meet, interact, and communicate. Additionally, we suggest creating rooms with complementing content, instead of replicas of rooms with the same content.

8. Conclusion and Future Work

In this paper, we describe our efforts to develop and apply avatar-based virtual 3D social spaces created with the Mozilla Hubs platform to complement Zoom lectures in teaching the VR4SG course online. We presented activities intended to prepare the students to access the MH platform, customize their avatars, and develop for the MH platform. We report on our iterative efforts toward developing three rooms that promoted interaction and communication. In UNSDG room, students created content for a United Nations Sustainable Development Goal of their choice. In the Pioneers of VR room, students consumed video content related to the history of virtual reality. Finally, in the Seekers room students learned about problem proposals, consumed and created content to help them form teams to work on group projects. We observed and surveyed the students regarding each of those rooms in aspects such as their perception of the rooms' size, levels of detail, and overall impressions. We also observed the way students interacted in the rooms as well as their communication patterns. We noted that students preferred high-detailed, closed spaces that resembled real-life rooms. We also noted a preference for smaller rooms over large ones. Regarding interaction and communication, smaller rooms created more opportunities for students to meet each other and interact. Similarly, sessions or events scheduled to happen for a short period (about 1 h) concentrated students' attendance and facilitated interaction. In terms of communication, we noticed that clear patterns emerged regarding the use of voice and text. Similar to the real-world, students formed circles to talk in groups and used voice in the virtual rooms. However, to communicate with all participants in the room, students often used text. Overall, our efforts in supporting students to self-form teams were successful. In the end, seven out of the eight teams formed to work in the main class project met and discussed the problem proposal they chose to work in the virtual space. Overall, this work sheds light on several aspects regarding avatar use and customization, besides users' interaction, and communication patterns in virtual 3D social spaces. We discuss aspects related to room design, students' attendance, avatar choices and the communication channels used. We contribute with a set of design considerations for 3D virtual environments that aim to support others in developing future activities and virtual spaces geared toward promoting student interaction and communication.

We noted a few limitations in this work. We recognize that the small number of students that answered the surveys limits the representativeness of their responses. Also, all the students who answered the surveys were male and used desktop or laptop machines. The limited number of participants may be related to the fact that participation incentives were a percentage of the grades. Since most students performed well in the class, and several other extra credit opportunities were available, answering the surveys may not have seemed attractive. In future work, we will seek to provide more attractive incentives so that a more representative portion of the students participate. It is also valuable to examine our finding with female and mixed populations, since there may be gender specific preferences and behaviors that our study was not able to elicit. We also recognize that having experimenters in the room observing students may have introduced bias on the results. We have however made efforts to keep a distance from students toward minimizing our influence in the environment, while also assuming avatars that would not draw attention.

In our future endeavors, we aim to evaluate cultural and gender-related aspects in the users' perceptions of virtual environments, as well as further investigate existing real-world architecture and room-building theories to develop the virtual spaces. Additionally, we will investigate the impact that other devices (such as smartphones and HMDs) may have in users' behavior and communication patterns in avatar-based virtual environments built to facilitate learning.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Institutional Review Board, University of Florida. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

AG, JS, and PF-G conceptualized the surveys. AG and JS carried out observations. AS carried out the data analysis and wrote the manuscript in consultation with PF-G, JS, and BL. All authors contributed to the conceptualization of the paper.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Alt (2020). Altspacevr | Be There, Together. Available online at: https://altvr.com/ (accessed on December 22, 2020).

Applegate, R. (2009). The library is for studying: student preferences for study space. J. Acad. Librariansh. 35, 341–346. doi: 10.1016/j.acalib.2009.04.004

CrossRef Full Text | Google Scholar

ARE (2020). United Nations res/71/313. Available online at: https://undocs.org/A/RES/71/313 (accessed on December 16, 2020).

Barrett, P., Zhang, Y., Davies, F., and Barrett, L. (2015). Clever Classrooms. Summary report of the HEAD project (holistic evidence and design), University of Salford, Salford, United Kingdom.

Google Scholar

Beck, S., Kunert, A., Kulik, A., and Froehlich, B. (2013). Immersive group-to-group telepresence. IEEE Trans. Vis. Comput. Graph. 19, 616–625. doi: 10.1109/TVCG.2013.33

PubMed Abstract | CrossRef Full Text | Google Scholar

Ble (2020). Blender.org - Home of the Blender Project - Free and Open 3d Creation Software. Available online at: https://www.blender.org/ (accessed on December 15, 2020).

Bogicevic, V., Bujisic, M., Cobanoglu, C., and Feinstein, A. H. (2018). Gender and age preferences of hotel room design. Int. J. Contemp. Hosp. Manage. 30, 874–899. doi: 10.1108/IJCHM-08-2016-0450

CrossRef Full Text | Google Scholar

Boustila, S., Guégan, T., Takashima, K., and Kitamura, Y. (2019). “Text typing in VR using smartphones touchscreen and hmd,” in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Osaka: IEEE), 860–861.

Google Scholar

Bowden, J. A., and Marton, F. (1998). The University1 of Learning. London: Psychology Press.

Bredikhina, L., Sakaguchi, T., and Shirai, A. (2020). “Web3d distance live workshop for children in mozilla hubs,” in The 25th International Conference on 3D Web Technology (Seoul), 1–2.

Google Scholar

Chamberlain, A., Crabtree, A., Rodden, T., Jones, M., and Rogers, Y. (2012). “Research in the wild: understanding'in the wild'approaches to design and development,” in Proceedings of the Designing Interactive Systems Conference (Newcastle Upon Tyne), 795–796.

Google Scholar

Curry, O., and Dunbar, R. I. (2013). Do birds of a feather flock together? Hum. Nat. 24, 336–347. doi: 10.1007/s12110-013-9174-z

CrossRef Full Text | Google Scholar

Damer, B., Judson, J., Dove, J., DiPaola, S., Ebtekar, A., McGehee, S., et al. (1997). Avatars!; Exploring and Building Virtual Worlds on the Internet. San Francisco, CA: Peachpit Press.

Google Scholar

Delaney, G., Jacob, S., Iedema, R., Winters, M., and Barton, M. (2004). Comparison of face-to-face and videoconferenced multidisciplinary clinical meetings. Aust. Radiol. 48, 487–492. doi: 10.1111/j.1440-1673.2004.01349.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Delwiche, A. (2006). Massively multiplayer online games (mmos) in the new media classroom. J. Educ. Technol. Soc. 9, 160–172.

Google Scholar

Dickey, M. D. (1999). 3D virtual worlds and learning: an analysis of the impact of design affordances and limitations in active worlds, Blaxxun interactive, and OnLive! traveler; and a study of the implementation of active worlds for formal and informal education (Ph.D. thesis). The Ohio State University, Columbus, OH, United States.

Google Scholar

Dis (2020). Discord | Your place To talk and hang out. Available online at:https://discord.com/ (accessed on December 22, 2020).

Doherty-Sneddon, G., Anderson, A., O'malley, C., Langton, S., Garrod, S., and Bruce, V. (1997). Face-to-face and video-mediated communication: a comparison of dialogue structure and task performance. J. Exp. Psychol. Appl. 3:105. doi: 10.1037/1076-898X.3.2.105

CrossRef Full Text | Google Scholar

Eiris, R., Wen, J., and Gheisari, M. (2020). “ivisit: digital interactive construction site visits using 360-degree panoramas and virtual humans,” in Construction Research Congress 2020: Computer Applications (Reston, VA: American Society of Civil Engineers), 1106–1116.

Google Scholar

Felder, R. M., and Brent, R. (2009). Active learning: an introduction. ASQ High. Educ. Brief 2, 1–5.

Google Scholar

Foley, J. D. (1987). Interfaces for advanced computing. Sci. Am. 257, 126–135.

Google Scholar

Frécon, E., and Stenius, M. (1998). Dive: a scaleable network architecture for distributed virtual environments. Distribut. Syst. Eng. 5:91. doi: 10.1088/0967-1846/5/3/002

CrossRef Full Text | Google Scholar

glT (2020). gltf - Wikipedia. Available online at: https://en.wikipedia.org/wiki/GlTF#:~:text=The%20GLB%20file%20format%20is,incorporated%20directly%20into%20glTF%202.0. (accessed on December 15, 2020).

Heilig, M. L. (1962). Sensorama Simulator. US Patent 3,050,870, Long Beach, NY.

Google Scholar

Herring, S. C. (1996). Computer-Mediated Communication: Linguistic, Social, and Cross-Cultural Perspectives, Vol. 39. Amsterdam: John Benjamins Publishing.

Google Scholar

Heydarian, A., Pantazis, E., Carneiro, J. P., Gerber, D., and Becerik-Gerber, B. (2015). “Towards understanding end-user lighting preferences in office spaces by using immersive virtual environments,” in Computing in Civil Engineering 2015 (Austin, TX), 475–482.

Google Scholar

Holt, E. A., Heim, A. B., Tessens, E., and Walker, R. (2020). Thanks for inviting me to the party: virtual poster sessions as a way to connect in a time of disconnection. Ecol. Evol. 10, 12423–12430. doi: 10.1002/ece3.6756

PubMed Abstract | CrossRef Full Text | Google Scholar

Hub (2020). Hubs - Private Social VR in Your Web Browser. Available online at: https://hubs.mozilla.com/ (accessed on December 22, 2020).

Isaacs, E. A., and Tang, J. C. (1994). What video can and cannot do for collaboration: a case study. Multimedia Syst. 2, 63–73. doi: 10.1007/BF01274181

CrossRef Full Text | Google Scholar

Isaacs, E. A., and Tang, J. C. (1997). “Studying video-based collaboration in context: from small workgroups to large organizations,” in Video-Mediated Communication, 1st Edn, eds K. E. Finn, A. J. Sellen, S. B. Wilbur, A. Sellen, and S. Wilbur (Menlo Park, CA: Routledge), 173–197.

Google Scholar

Jerald, J., Giokaris, P., Woodall, D., Hartbolt, A., Chandak, A., and Kuntz, S. (2014). “Developing virtual reality applications with unity,” in 2014 IEEE Virtual Reality (VR) (Minneapolis, MN: IEEE), 1–3.

Google Scholar

Khadka, R., Money, J. H., and Banic, A. (2018). “Evaluation of scientific workflow effectiveness for a distributed multi-user multi-platform support system for collaborative visualization,” in Proceedings of the Practice and Experience on Advanced Research Computing (New York, NY), 1–8.

Google Scholar

Knipe, D., and Lee, M. (2002). The quality of teaching and learning via videoconferencing. Br. J. Educ. Technol. 33, 301–311. doi: 10.1111/1467-8535.00265

CrossRef Full Text | Google Scholar

Kolesnichenko, A., McVeigh-Schultz, J., and Isbister, K. (2019). “Understanding emerging design practices for avatar systems in the commercial social VR ecology,” in Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA), 241–252.

Google Scholar

Kwon, C. (2019). Verification of the possibility and effectiveness of experiential learning using hmd-based immersive VR technologies. Virtual Real. 23, 101–118. doi: 10.1007/s10055-018-0364-1

CrossRef Full Text | Google Scholar

MacKenzie, I. Scott. (2012). Human-Computer Interaction: An Empirical Research Perspective. Oxford: Morgan Kaufmann.

Google Scholar

Mayrath, M., Sanchez, J., Traphagan, T., Heikes, J., and Trivedi, A. (2007). “Using second life in an english course: designing class activities to address learning objectives,” in EdMedia+ Innovate Learning [Association for the Advancement of Computing in Education (AACE)] (Vancouver, BC), 4219–4224.

Google Scholar

McVeigh-Schultz, J., Kolesnichenko, A., and Isbister, K. (2019). “Shaping pro-social interaction in VR: an emerging design framework,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow), 1–12.

Google Scholar

Meehan, B., and Thomas, I. (2006). Teamwork: education for entrants to the environment professions. Environ. Educ. Res. 12, 609–623. doi: 10.1080/13504620601053571

CrossRef Full Text | Google Scholar

Menzner, T., Otte, A., Gesslein, T., Grubert, J., Gagel, P., and Schneider, D. (2019). “A capacitive-sensing physical keyboard for VR text entry,” in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Osaka: IEEE), 1080–1081.

Google Scholar

Messenger, F. (2020). Messenger | Facebook. Available online at: https://www.facebook.com/messenger/ (accessed on December 22, 2020).

Motamedi, V. (2001). A critical look at the use of videoconferencing in united states distance education. Education (Mobile, AL), 122:386.

Google Scholar

Nardi, B. A., and Whittaker, S. (2002). The place of face-to-face communication in distributed work. Distrib. Work 83:112. doi: 10.7551/mitpress/2464.003.0008

CrossRef Full Text | Google Scholar

Nardi, B. A., Whittaker, S., and Bradner, E. (2000). “Interaction and outeraction: instant messaging in action,” in Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (Cambridge, MA), 79–88.

Google Scholar

Neale, D. C., McGee, M. K., Amento, B. S., and Brooks, P. C. (1998). Making Media Spaces Useful: Video Support and Telepresence. Blacksburg, VA: Virginia Polytechnic Institute and State University, 28.

Google Scholar

Nowak, K. L., and Fox, J. (2018). Avatars and computer-mediated communication: a review of the definitions, uses, and effects of digital representations. Rev. Commun. Res. 6, 30–53. doi: 10.12840/issn.2255-4165.2018.06.01.015

CrossRef Full Text | Google Scholar

Onl (2020). Microsoft Teams. Available online at: https://www.microsoft.com/en-us/microsoft-365/microsoft-teams/online-meetings (accessed on December 22, 2020).

Pan, Z., Cheok, A. D., Yang, H., Zhu, J., and Shi, J. (2006). Virtual reality and mixed reality for virtual learning environments. Comput. Graph. 30, 20–28. doi: 10.1016/j.cag.2005.10.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Plenert, G. J. (2011). Lean Management Principles for Information Technology. Boca Raton, FL: CRC Press.

Google Scholar

Pol (2020). Poly. Available online at: https://poly.google.com/ (accessed on December 23, 2020).

Qui (2020). Quilt. Available online at: http://tryquilt.io/ (accessed on December 15, 2020).

Rea (2020). Ready Player Me - Personal 3d Avatars for Games and VR. Available online at: https://readyplayer.me/ (accessed on December 15, 2020).

Scavarelli, A., Arya, A., and Teather, R. J. (2019). “Towards a framework on accessible and social VR in education,” in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Osaka: IEEE), 1148–1149.

Google Scholar

Serhan, D. (2020). Transitioning from face-to-face to remote learning: students' attitudes and perceptions of using zoom during covid-19 pandemic. Int. J. Technol. Educ. Sci. 4, 335–342. doi: 10.46328/ijtes.v4i4.148

CrossRef Full Text | Google Scholar

Ske (2020). Sketchfab - The Best 3d Viewer on the Web. Available online at: https://sketchfab.com/ (accessed on December 15, 2020).

Spo (2020). Spoke by Mozilla. Available online at: https://hubs.mozilla.com/spoke (accessed on December 22, 2020).

Sutherland, I. E. (1965). “The ultimate display,” in Multimedia: From Wagner to Virtual Reality, ed W. A. Kalenich (London: Macmillan and Co.), 1.

Google Scholar

Usi (2021). Using Mozilla Hubs With a Screen Reader | Equal Entry. Available online at: https://equalentry.com/mozilla-hubs-with-screen-reader/. (accessed on Feburary 24, 2021).

USR (2020). Us Remote Work Survey: Pwc. Available online at: https://www.pwc.com/us/en/library/covid-19/us-remote-work-survey.html (accessed on December 13, 2020).

Vergara, D., Rubio, M. P., Lorenzo, M., and Rodríguez, S. (2019). “On the importance of the design of virtual reality learning environments,” in International Conference in Methodologies and Intelligent Systems for Techhnology Enhanced Learning (Ávila: Springer), 146–152.

Google Scholar

Vid (2020). Video Conferencing - Zoom. Available online at: https://zoom.us/ (accessed on December 22, 2020).

Vilhjálmsson, H. H. (2003). Avatar augmented online conversation (Ph.D. thesis). Massachusetts Institute of Technology, Cambridge, MA, United States.

Google Scholar

VRC (2020). Vrchat. Available online at: https://hello.vrchat.com/ (accessed on December 22, 2020).

VRF (2020). VR for the Social Good. Available online at: http://www.vrforthesocialgood.com/ (accessed on December 22, 2020).

WEF (2020). The Rise of Online Learning During the Covid-19 Pandemic | World Economic Forum. Available online at: https://www.weforum.org/agenda/2020/04/coronavirus-education-global-covid19-online-digital-learning/. (accessed on December 13, 2020).

Wel (2020). Slack. Available online at: https://slack.com/ (accessed on December 22, 2020).

Wen, J., and Gheisari, M. (2020). Using virtual reality to facilitate communication in the aec domain: a systematic review. Constr. Innov. 20, 509–542. doi: 10.1108/CI-11-2019-0122

CrossRef Full Text | Google Scholar

Wha (2020). Whatsapp. Available online at: https://www.whatsapp.com/?lang=en (accessed on December 22, 2020).

Whittaker, S., and O'Conaill, B. (1997). “The role of vision in face-to-face and mediated communication,” in Computers, Cognition, and Work. Video-Mediated Communication, eds. K. E. Finn, A. J. Sellen, and S. B. Wilbur (Lawrence Erlbaum Associates Publishers), 23–49.

Google Scholar

Williams, L. (2012). What agile teams think of agile principles. Commun. ACM 55, 71–76. doi: 10.1145/2133806.2133823

PubMed Abstract | CrossRef Full Text | Google Scholar

Williamson, J., Li, J., Vinayagamoorthy, V., Shamma, D. A., and Cesar, P. (2021). Proxemics and social interactions in an instrumented virtual reality workshop. arXiv preprint arXiv:2101.05300.

Google Scholar