Metamorphosis of EduX: a comparative study for education in the metaverse

Introduction

The accelerated technological advances on different fronts continue to reshape the educational systems around the world. The transition was further fueled by the sudden shift to distance mode forced by the Covid-19 pandemic (McCann, 2019; Garavaglia and Petti, 2020; Krawczyk-Bryłka et al., 2020; Albakri and Albakri, 2021; Katalnikova et al., 2021; Zai and Akhunzada, 2021; Almanie, 2022). The majority of institutions struggled during the early months of the pandemic facing a host of challenges due to demands for upscaled platforms to support a large community of educators and learners online.

A plethora of literature surged during the pandemic in attempts to understand and address the ensued multifaceted challenges faced by institutions (Reis et al., 2021; Abdullah et al., 2022; Baniomar, 2022; Berruecos-Vila and Ochoa-Carrasco, 2022), students (Salih and Ibrahim, 2021; Austin Sanders et al., 2022; Di Malta et al., 2022), educators (Katalnikova et al., 2021; Akhunzada et al., 2022; Baniomar, 2022; Demir et al., 2022; Edirisingha, 2022), and parents (Sari and Maningtyas, 2020; Agaton and Cueto, 2021; Domhnaill et al., 2021; Öçal et al., 2021; Demir and Yildizli, 2022; Korukluoğlu and Bavli, 2022) all of which indicating a high magnitude of impact on teaching and learning manifesting at micro and macro levels of the educational system (Ahmed and Lataifeh, 2023).

The transition of education using ICT has already been well established before Covid-19 as evident by a number of studies (Ramirez et al., 2018) but it was not mainstream as face-to-face learning was still a preferred option. The advent of the Covid-19 pandemic had a devastating impact on the learning process and resulted in a hasty transition towards full-time remote learning (Rodrigues et al., 2020). Several tools and technologies were quickly adopted and incorporated into existing Learning Management Systems (LMS) in efforts to facilitate the remote learning process, moving from classroom to Zoom, MS teams, and Black Board Ultra (BBUltra) rooms. Fundamentally these tools were swiftly used for the dissemination of knowledge from the instructor to students, not necessarily designed to foster and encourage collaboration (Zhou et al., 2021), which resulted in lower engagement levels. Furthermore, students’ role in the process did not encourage much interactivity compared to a face-to-face classroom (Lucas et al., 2020), leading to poor interpersonal communications within the course platform (Katz and Kedem-Yemini, 2021), which substantially affects learning outcomes and overall performance (Jaggars and Xu, 2016).

Students and instructors moved from an environment where natural interactions among instructors and peers were afforded with rich means of oral, visual, and physical modalities, to the adopted distance mode. Compelled to be embraced by educational institutions around the world (Davidovitch and Wadmany, 2021; Mahmud et al., 2022), most of these tools offered limited social interaction and collaboration, which are some of the most important qualities of an effective educational environment (Garcia-Sanjuan et al., 2018). Perhaps this should not be surprising as the adopted tools were essentially made for video conferencing with a basic communications toolset (Stecuła et al., 2022). Zoom, for instance, was barely surviving as an ageing technology model facing fierce competition from similar solutions, next to an existential crisis when compared with the new technologies embedded with a wide range of communication modalities blurring the boundaries of physical and virtual embodiments of users and actions such as virtual (VR), augmented (AR), mixed (MR), and extended realities (XR). Zoom, not only did manage to survive but became a verb brand due to the pandemic (Bowles, 2021).

Active engagement and collaboration require people to be in the same space or a resemblance of one, where teaching and learning are coupled not just with two-way communications between instructors and peers, but enriched with a range of physical models, and visual materials among other tools. These tools play a pivotal role as anchors of knowledge, and educators use such anchors to mediate knowledge transfer to engaged learners. Therefore, while communication allows for a more traditional learning mechanism in the form of instructions in a synchronous collaborative environment, the visibility and modalities of feedback and collaboration between instructors and learners play an equally significant role and must be strategically planned (Attiogbe et al., 2023). Especially, as many programs and courses continue to be offered in distance mode as the mainstream despite the questionable effectiveness of this mode (Regmi and Jones, 2020). Investigating the most influential factors for distance education, agility and quickness of feedback from instructors was detrimental to the success of the transitional model (Özbey and Kayri, 2022), which for different reasons, seems to be here to stay. The goal we believe should be to develop a sustainable system that is well-planned, user-centric, and provides viable solutions to the proven defects of the traditional distance model.

While this work is not aimed to discuss an already saturated topic, it is critical to highlight that the escalated challenges of distance education range from maintaining the quality of learning outcomes (Regmi and Jones, 2020; Alkabaa, 2022; Mahmud et al., 2022), phycological and mental (Stachteas and Stachteas, 2020), motivation (Issayev et al., 2022; Snezhko et al., 2022), emotional (Abdalla et al., 2022; Huang et al., 2022; Šorgo et al., 2022), and overall performance (Amro, 2022).

There is growing evidence supporting the successful deployment of immersive realities in education across different domains of knowledge (Martirosov and Kopecek, 2017; Allcoat et al., 2021). Immersive VR and the metaverse offer a unique and engaging environment that can enhance the educational experience and provide new opportunities for learning and development. According to the most recent systematic survey (van der Meer et al., 2023), immersive, collaborative learning systems in VR can encourage further engagement and motivation, as students take an active role in their learning through the provided interactive experience (Garduño et al., 2021). Additionally, the positive impact is also coupled with improving carefully planned learning outcomes where students at different educational levels can capitalize on novel experiential learning environments (Kamińska et al., 2019; Huang et al., 2020; Allcoat et al., 2021; Garduño et al., 2021).

Furthermore, a collaborative, networked, immersive learning environment is critically increasing accessibility and inclusivity eliminating sociotechnical and geographical barriers, while allowing instructors and experience designers to maintain control over tailored experiences that cater to specific students’ needs (Dwivedi et al., 2022), which years ago has proven particularly effective with technically perceived or difficult topics (Seidman, 2009). The highly influential computer supported collaborative learning model (CSCL) by Collazos et al. (2007) that shows that collaboration with teammates is an equally important skill in comparison to having the skillset of performing a particular task. Nowadays, the metaverse is considered to be the future of the internet where the advent of 5G has the potential to transform it into one of the most ubiquitous and transformative communication mediums for the masses (Norton, 2023). The ubiquitous metaverse allows for seamless navigation, interaction, and anywhere-and-anytime immersive experience that can prove invaluable for the next generation of educational environments (Li et al., 2023). Therefore, it is very important to evaluate its effectiveness as an alternative platform to traditional educational environments.

In this work, we propose EduX as an educational platform prototyped to substantiate the value of ongoing transition toward a ubiquitous immersive metaverse. As a collaborative immersive environment leveraged with new modalities of interactions, Unity (2022) was the engine of choice to develop the platform due to the complexity of the requirements to enable instructors to deliver typical educational sessions on different topics. It is designed to be accessible by different devices, including wired and wireless VR, AR and MR devices, mobile devices, as well as typical web browsers. EduX participants are invited to join the sessions using private links. The same educational materials are also used for the parallel sessions delivered to the second control group using the university system BBUltra. Both parallel groups were asked to complete a short usability survey, followed by a post-formal assessment at the end of the sessions.

A novel structural model was devised based on NASA Task Load Index (TLX) (Hart and Staveland, 1988), followed by an extensive mixed methods usability study was performed to evaluate the usability, effectiveness, affectivity, and impact of the proposed system in comparison to the traditional BBUltra used in distance mode. The user study enrolls participants from different academic levels and four courses in computer science, graphic design, multimedia design, and a general university requirement course in information technology. Using a range of qualitative and quantitative methods, the collected data from several instruments were then presented and discussed followed by conclusions of the study.

The main contributions of our work are:

1. Developed a new structural model to statistically validate the interactions and significance of the leveraged platform on students’ engagement and outcomes fulfilment.

2. Designed a novel collaborative ubiquitous immersive metaverse environment deployed with WebGL and WebXR for wider support and compatibility along with relevant assets and educational materials related to the selected domain of knowledge.

3. Performed a comprehensive user study to evaluate the usability and effectiveness and learning impact of the proposed platform in comparison to traditional BBUltra.

In the following sections, we first present the devised research model and hypothesis. Followed by the design and development of the proposed environment with a brief view of the domain-specific experiments prepared for the participating students. Finally, the results are presented together with a detailed discussion, followed by conclusions, limitations, and future work.

Research model

Based on the literature discussed in the introduction, there has been a clear consensus on the role of engagement in educational systems regardless of the mode of delivery, in person, at a distance, or within an immersive environment. To effectively compare the leveraged abilities of the examined environments, BBUltra and EduX, we had to evaluate beyond traditional usability metrics or system acceptance because engagement as a dependent variable might appear to be influenced by several independent variables, the interplaying relationships among which had to be evaluated to explore the extent to which they are affecting the outcome (Xu and Yang, 2024).

The value of metaverse in transforming a traditional real-world experience, be it educational, or business has been an active area of study. As shown by Mancuso et al. (2023), metaverse can transform real-economy to a virtual-economy in a very successful manner. Recent studies compared the differences in retail channels when they are offered in three-dimensional virtual world compared to a traditional website (Zarifis, 2019). A similar study in the transformation of an educational environment is currently lacking, and it is important to not only deploy a traditional educational environment in metaverse but also analyze it from multiple viewpoints and relationships to correctly identify its feasibility and effectiveness (Collazos et al., 2007; Xu and Yang, 2024).

NASA Task Load Index (TLX) is a subjective workload assessment tool developed by the National Aeronautics and Space Administration (NASA) (Hart and Staveland, 1988). It has been widely used in various domains, including aviation, healthcare, and human-computer interaction, to evaluate task demands and inform design improvements. A later update on the work (Hart, 2006) identified 550 studies in which NASA-TLX was used either as devised or in modified adaptations.

The TLX consists of six measures that assess different dimensions of workload, including mental demand (including cognitive processing, decision-making, and information processing), physical demand (extent of physical exertion and effort required to perform the task), temporal demand (the perceived time pressure or time constraints associated with the task), performance (the individual’s perception of their perceived effectiveness, accuracy, and success in accomplishing the task), effort (overall level of effort, both mental and physical), and frustration (the degree of annoyance, stress, and dissatisfaction experienced). Each measure is rated by users on a scale from 0 to 100, with higher scores indicating a greater workload (NASA, 2020).

The devised model which forms the basis of our usability evaluation for this work is illustrated in Figure 1. We combine the model factors into two groups to expose internal interplaying relationships which we assume are mediated by different factors (MacKinnon, 2000). The first group is related to the perceived mental demand based on the exerted physical demand and mediated by temporal demand. The second group is devised to evaluate the perceived performance based on the efforts exerted and mediated by frustration. However, based on its established value in the literature (Garduño et al., 2021; Fredricks, 2022; Pimentel et al., 2022), and informed by the role of epistemic curiosity and situational awareness (Cheng et al., 2023), we added “engagement” as another factor that we believe will be dependent on the interactions of the two groups mentioned earlier. Users’ scores for the seven factors are collected after each session and several statistical methods are used to evaluate the interplaying relationship in effect, and we compare that to the results attained in the formal post-assessment for further validation. Additionally, we triangulate the results obtained with other collected data including qualitative visual analysis of participant behaviors, actions, and movements and the artefacts that they create in space (Gong and Xiang, 2011).

Figure 1

Figure 1. The devised research model adapted from NASA TLX.

The main hypothesis of the work is pivoted around the engagement of students during the sessions which we believe can significantly influence students’ attainment of the learning objectives measured using a post-assessment test.

System design and implementation

Our system had several challenges to overcome, most of which were related to the deployment options to ensure the maximum accessibility and compatibility of the platform. The development engine of choice was settled for Unity due to the versatile capabilities and wide plugin extensibility that support the development of the desired functionalities.

Our motivation for this work is to substantiate the value of the proposed system in comparison with the traditionally used conference-based educational tools. Therefore, the design of the space includes wider spatial and structural components to host the planned educational sessions. One of the generic environments is similar to a traditional class as shown in Figure 2.

Figure 2

Figure 2. EduX: a generic classroom environment.

To design an interactive, collaborative, and immersive system, several tools and technologies were used, most of which were tightly integrated within Unity’s echo system. Unity’s XR Interaction Toolkit (Unity, 2019) enabled a wide range of interactions, haptics, and object manipulation all within a device-independent framework for both VR and AR.

For networking and real-time multi-user voice support, Photon Unity Networking (PUN) was implemented allowing simultaneous interaction between multiple devices over the network (PUN, 2021). Developed as a cross-platform framework, contributed immensely to the fulfilment of the aspired platform. As part of the planned deployment for different educational domains, we created different scenes in the platform equipped with different objects and materials that were modelled in Blender® and 3Dshapr® as per the scenario of the intended topic, the details of which will be presented next.

As for the avatar creation process, we used the Ready Player Me plugin for Unity (RPM-Unity, 2021). To translate participants, walk, jump, and other expressive moves (dancing, cheering, etc.) with the avatars, we used the Adobe Mixamo plugin within Blender to automate the process. Several other features were also added to the platform including but not limited to importing documents, 3d objects, images, video, audio, post-it notes, emojis, drawing simple 3d mesh in space, and an embedded browser container to call different web resources if, and when needed.

The prototype was then deployed over a private web server allowing participants to join the sessions as shown in Figure 3.

Figure 3

Figure 3. Joining EduX session hosted on a private server.

Designing the educational sessions experiments

The designed materials for each of the topics included in the study were based on the provided materials used by instructors during a typical BBUltra session, such as presentations, videos, and additional reading materials.

The first experiment was designed for a graphic design course addressing the concept of design processes for visual materials. The experiment space in Figure 4 was designed to represent the design process phases from a design brief, visual research, and final concept development. Several examples of previous work for students at the same levels were exhibited and upon concluding the short introduction, students toured the space following the instructor in a virtual journey through the process anchored in materialized examples. Some of the exhibited work included additional video and visual iterations illustrating the progress available for students to view on demand, as the space was made available for them before and after the actual session time, as is usually the case with different educational materials shared with students using typical LMS systems.

Figure 4

Figure 4. Overview of the first experiment space for the graphic design course (some parts are overlayed with a grey rectangle to maintain the anonymity of the work).

The second experiment was prepared for a foundational course in computer science (problem-solving) as shown in Figure 5. The course preceded core programming courses in the program, and the topic was related to multidimensional arrays. The experiment included a 3d cube used to illustrate three-dimensional arrays, in parallel to an embedded browser with live JavaScript code and a visual compiler (p5js.org) to reflect the changes made for the array’s components, locations, and values.

Figure 5

Figure 5. Overview of the second experiment space for the problem-solving course.

The third experiment included a topic on animation design (Figure 6), one of the areas covered within an Interactive Multimedia course. The experiment included different 3D models imported into space to demonstrate functions related to rigging and kinematic animation, next to videos and other visual materials.

Figure 6

Figure 6. Overview of the third experiment space for the interactive multimedia course.

The fourth experiment was prepared for one of the general university compulsory courses on information technology presenting the structure and networking of the internet. The space in Figure 7 was prepared with several models representing different nodes and network devices involved in daily communication. The instructor and students used different annotation tools in the space to trace connections between a client request and a server response.

Figure 7

Figure 7. Overview of the fourth experiment space for the information technology course.

User study

In the user study, the system was evaluated in terms of its perceived usability factors that were devised in the proposed research model to encourage further engagement during the educational sessions. The consented participants who signed up to join the experiments were informed of the project goals where the collected data are encrypted and securely saved to be only accessed by the researchers for analysis purposes. Upon the concluded sessions, participants were asked to complete a form to mark the perceived score\value for each of the seven variables namely: mental demand, temporal demand, physical demand, efforts, frustration, performance, and engagement, all ranked from 1 to 10. Students were briefed on the connotation of each factor. An additional text field was also provided to participants to leave open-ended reflections and feedback about the user experience (UX) and other issues. Participants were then directed to complete a short post-assessment test on the educational topics presented during the session.

Direct and indirect observations were collected during the live and recorded sessions to help interpret some of the behaviors observed. Ten random students were also called for a follow-up discussion using a semi-structured interview format. Finally, students’ results in the post-assessment test were used to corroborate as a measure of the effectiveness of the system as a learning tool, and its impact on learning.

To test the system, we enlisted the help of n = 396 participants over three academic semesters, with a median age of 20 years old. Gender proportions were approximately equal with a slightly higher number of females. Students registered in these courses were randomly split between the control BBUltra and the EduX groups. Further demographic details are provided in Table 1. Henceforth, the groups are tagged with EduX-# to identify the EduX metaverse groups, and BBUltra-# for the BBUltra groups.

Table 1

Table 1. Groups formation and distribution.

As can be seen in Table 1, the number of participants over three academic terms allowed us to collect sufficient data that will be validated through different statistical methods. The higher number of participants allowed us to run the experiment in bigger collaborative groups. The first two groups were a combination of results gathered over two different terms, while the last two groups were conducted in the same academic term. For each session, the instructor explained and demonstrated the communication tools and features of each platform, including but not limited to how to interact in the session (how to add, remove or manipulate objects, how to ask questions, etc.). The instructors would then start the educational sessions during which students are encouraged to be active and use the demonstrated tools and features incentivized by offering an active engagement percentage to their grades in the course.

One passive observer joined every group session without any interference to record users’ interactions, collaborations, exploration, and similar observational data for later analysis. These interactions were noted when students engaged with each other verbally to discuss the experiment or helped their fellow students to move and explore the space. At the end of each session, students were guided to complete the usability metrics devised for this work followed by a short post-assessment test.

The devised survey along with all the quantitative and qualitative results are presented in the next section.

Results and discussion

As explained in the previous section, different quantitative and qualitative data were obtained from both controlled (BBUltra) and experimental (EduX) sessions. Participants were asked to complete a form to mark the perceived score for each of the seven variables namely: mental demand, temporal demand, physical demand, efforts, frustration, performance, and engagement, all ranked from 1 to 10. The post-assessment was performed at the end of sessions and mapped to the same scale of 1–10. The overall data was analyzed from multiple viewpoints. This allowed us to first evaluate the validity of the devised research model and hypothesis as proposed and then investigate different factors’ interactions and their significant impact on the overall attained learning outcomes. In general, all students were familiar with the BBUltra system as a distance LMS system, and we hypothesized that the EduX will have significant differences not only in terms of the attained learning outcomes but also within the perceived user experience and overall fulfilments. Additional views are also evaluated in terms of affectional influence. Once again, the null hypothesis states that the devised usability factors are perceived similarly with no statistically significant differences between the two platforms.

In this section, we will present a detailed evaluation of the devised research model, followed by a detailed discussion of the work, its limitations, and conclusions in the next section.

The devised research model evaluation

The acquired data allowed us to first evaluate the validity of the devised research model and hypothesis as proposed and then investigate different factors’ interactions and their significant impact on the overall attained learning outcomes. We evaluated both systems separately. The means for each factor for the EduX platform can be seen in Figure 8, whereas the means for each factor for BBUltra can be seen in Figure 9. Figure 10 shows the combined means of both groups for an overall comparison.

Figure 8

Figure 8. Means comparison all variable – EduX.

Figure 9

Figure 9. Means comparison all variable – BBUltra.

Figure 10

Figure 10. Combined means – EduX versus BBUltra.

To understand the differences exhibited in Figure 10, observation data and participant feedback gathered from the survey and interviews indicated that while participants within EduX groups indicated higher physical demand, the actual activity was perceived by proxy through their avatars as they followed the instructor and peers within the virtual space. Hence, we concur that participants wanted to move around and direct their focus on areas, elements, or objects, other than the one that could be highlighted by the instructor or a shared screen for lecture materials within specific fixed x-y screen coordinates.

As can be seen in the results, there is a significant difference (27%) in post-assessment results between EduX and BBUltra. A single-factor ANOVA has shown that there is a significant difference between the categorical variable Group and the variable Post Assessment F = 29.94, and a p < 0.001 (Table 2). Thus, the null hypothesis is rejected indicating a strong interaction between the platform used and post-assessment results.

Table 2

Table 2. One factor ANOVA – post assessment.

The results also show that the physical demand for BBUltra was very low as expected, but its Temporal demand was slightly higher. All the factors except temporal demand were found to be significantly different statistically using the ANOVA test. For BBUltra, the temporal demand was noted to increase as they try to catch up with the ongoing discourse of the session in case they zone-out.

Similar to the post assessment, there is also a significant difference (26%) in engagement results between EduX and BBUltra. A single factor ANOVA for the engagement over both platforms, F = 21.08, p ≤ 0.001, also clearly shows a statistically significant difference (Table 3). This is further explained by some of the feedback collected from the semi-structured interviews where BBUltra participants noted that they did not feel that the platform offered comparative means to be engaged or excited about the topic in discussion. Hence, we maintain the reported results from the combined groups to be sufficiently representative of the finding.

Table 3

Table 3. One-factor ANOVA engagement.

Finally, a Pareto diagram of standardized effect is provided in Figure 11 to highlight the relative comparisons of such influence when compared to the other variables. The results here are seen as a positive confirmation supporting the earlier justification for adding engagement and an additional variable within the devised research model since it acts as a collective aggregator for the interplaying influences between the six variables in the initial NASA TLX model.

Figure 11

Figure 11. Pareto diagram of standardized effect.

Additionally, to confirm the insight left by participants, we looked at the recorded video to make sense of movements within the virtual space which indeed were used by instructors to recharge focus and keep students moving between objects or elements of knowledge within the virtual space. Compared to the parallel sessions within BBUltra, where instructors were limited by sequential use of the same fixed space as the window/screen shared among participants. Hence, with students trying to remain actively involved, within BBUltra this was made difficult for them, and as they tried harder as marked by their perceived effort scores, they slowly felt frustrated and as indicated by one of the participants “felt like phasing out” as an indication of disengagement. We believe the sense of movement within space is critical and would be of greater value to be explored in future research as we intend to implement an automated log for gaze and heatmaps for all participants within the virtual space. Our results also align well with the CSCL theory results as presented by Collazos et al. (2007). The feedback and analysis clearly show the effectiveness of a collaborative environment that validates that students not only learn from the instructor but observing and learning from their peers is an equally important aspect of the educational process. We would like to integrate additional tools to measure the effectiveness of collaboration among students to further increase students’ collaboration within the EduX environment.

To conclude with the results, we also tested if gender as a variable had a different influence on the results discussed so far. There has been a number of studies that have analyzed differences due to gender when it comes to technology and education (Zhou and Xu, 2007; Tondeur et al., 2016; Punter et al., 2017; Reychav and McHaney, 2017). As some of the studies found gender specific differences in the educational technology adoption, we found it imperative to analyze EduX from this point of view. A two-tailed t-test for independent samples (equal variances not assumed) showed that the difference between females and males with respect to the dependent variable Engagement was not statistically significant, t(382.41) = 1.05, p = 0.296, 95% confidence interval [−0.27, 0.88]. The same was observed with respect to Post Assessment with t(375.61) = 1.17, p = 0.242, and a 95% confidence interval (Table 4). Therefore, in both tests, the null hypothesis is retained with no proven significance for gender as an independent factor. This result is in line with recent literature in XR and collaborative education (Ahmed and Lataifeh, 2023).

Table 4

Table 4. T-test for independent samples – gender, engagement and post assessment.

Having discussed various aspects of the results, the following section will present the conclusion of this work.

Conclusion

This work presents “EduX,” a networked, collaborative, ubiquitous, and immersive platform for educational content dissemination using metaverse. The proposed platform allows the hosting of educational sessions in a virtual environment without the constraints and limitations of a physical educational environment and in addition to traditional interaction mechanisms, also provides multiple types of spatial interactions within the metaverse for both instructors and students. We employed NASA Task Load Index and devised a novel research model that measures temporal demand using physical and mental parameters, along with the frustration level using effort and performance metrics to evaluate the user’s engagement level, which is then estimated based on post-assessment results. The devised research model allowed us to statistically validate the interactions and significance of the leveraged platform on students’ engagement and learning outcomes fulfilment.

We presented a very comprehensive evaluation of our devised research model from multiple viewpoints. We employed four unique scenarios with different learning goals for the user study deployed on both EduX and BBUltra platforms. A large-scale user study (n = 396) over multiple terms was conducted to collect both qualitative and quantitative data. Both EduX and BBUltra platforms were compared with respect to each scenario and showed significantly improved usability and user experience along with a higher level of engagement and lower level of frustration for the EduX. In addition, our results do not show any differences in terms of gender when it comes to using the metaverse.

Throughout the presented cycles for developing and evaluating the proposed functionalities within the metaverse, we would like to emphasize the critical need for solution providers (hardware and software) to converge toward a homogenous instead of proprietary systems that are often seen as an impairment for further exploration across different platforms and devices. Whether it is education, gaming, or social applications; to prototype and evaluate new systems in the metaverse we foresee an open, flexible, cross-compatible platform to empower educators, to create, interact, and deploy.

We would like to extend our work to strengthen some of the aspects of the system related to educational scenarios and the underlying metaverse platforms. Even though we employed four unique learning scenarios, the courses were confined to information technology and multimedia-related subjects. This is not a principal limitation of the work as the acquired knowledge and measured parameters are generic and invariant of the underlying subject. Nevertheless, for future work, we plan to add multiple learning scenarios from wider domains including but not limited to medical, business, literature, and social sciences to further strengthen our work. We would also like to further improve EduX by incorporating both augmented and mixed reality scenarios to rectify the current limitation of only using a complete virtual environment. Work is underway to also include biometric data including EEG, including an eye gazing sensor to capture different users’ parameters (gaze, fixation, and saccades within 3D environment). We are also testing the system with new XR devices that allow the integration of other sensors for heart rate, and facial camera sensors upon which the inferential confidence of cognitive states for users shall be better informed. We also plan to introduce and integrate further tools and enhancement that will encourage students’ collaboration and measure the collaborative activities to further strengthen our analysis of the collaborative environment.

Despite the limitations, our work shows that the ubiquitous immersive metaverse is a viable platform for educational content delivery as compared to traditional two-dimensional methods. Our proposed platform EduX shows that it results in higher engagement, and lower frustration, while also providing superior usability and user experience.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

This manuscript does not contain any research achievements that have been published or written by other individuals or groups. We are the only authors of this manuscript. We shall bear the legal responsibility of this statement.

Author contributions

ML: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Validation. NA: Formal analysis, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Acknowledgments

We would like to express our heartfelt gratitude to all the individuals who contributed to the success of this research project including all participating students who were committed to this project. Our senior students (name anonymized for double-blind review) who experimented with early prototypes as part of their senior project, and many others within the department and the college. Last but not least, we extend our deepest appreciation to the participating students and collaborating instructors who played a crucial role in shaping this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abdalla, E., Id, E., Raja Mohd, N., Naimie, Z., and Ahmadid, A. (2022). Undergraduate dental students’ perspective of online learning and their physical and mental health during COVID-19 pandemic. PLoS One 17:e0270091. doi: 10.1371/journal.pone.0270091

PubMed Abstract | Crossref Full Text | Google Scholar

Abdullah, M. H. T., Roslim, N., and Salleh, M. I. M. (2022). Open and distance learning during the COVID-19 pandemic: university students’ learning experiences and academic achievements. Asian J. Univ. Educ. 18:106. doi: 10.24191/ajue.v18i1.17177

Crossref Full Text | Google Scholar

Agaton, C. B., and Cueto, L. J. (2021). Learning at home: parents’ lived experiences on distance learning during COVID-19 pandemic in the Philippines. Int. J. Eval. Res. Educ. 10:901. doi: 10.11591/ijere.v10i3.21136

Crossref Full Text | Google Scholar

Ahmed, N., and Lataifeh, M. (2023). Impact and analysis of a collaborative augmented reality educational environment. J. Comput. Educ. doi: 10.1007/s40692-023-00275-x

Crossref Full Text | Google Scholar

Akhunzada, A. S., Lodhy, I. S., Munshi, P., and Jumani, S. (2022). Blended learning: past and present with special reference to COVID-19 pandemic. Int. J. Dist. Educ. E-Learn. 7, 61–70. doi: 10.36261/ijdeel.v7i1.2226

Crossref Full Text | Google Scholar

Albakri, O. M. A., and Albakri, A. (2021). “Exploring the impact of digital learning platforms on distance learning amidst the COVID-19 pandemic” in Mohammed Banu Ali and Trevor Wood-Harper Fostering Communication and Learning with Underutilized Technologies in Higher Education, IGI Global 150–164.

Google Scholar

Alkabaa, A. S. (2022). Effectiveness of using E-learning systems during COVID-19 in Saudi Arabia: experiences and perceptions analysis of engineering students. Educ Inf Technol (Dordr) 27, 10625–10645. doi: 10.1007/s10639-022-11054-z

PubMed Abstract | Crossref Full Text | Google Scholar

Allcoat, D., Hatchard, T., Azmat, F., Stansfield, K., Watson, D., and von Mühlenen, A. (2021). Education in the digital age: learning experience in virtual and mixed realities. J. Educ. Comput. Res. 59, 795–816. doi: 10.1177/0735633120985120

Crossref Full Text | Google Scholar

Almanie, A. (2022). Predictors of quality of distance education during the COVID-19 pandemic. Cypriot J. Educ. Sci. 17, 95–104. doi: 10.18844/cjes.v17i1.6685

Crossref Full Text | Google Scholar

Amro, D. K. (2022). The impact of COVID-19 lockdown on design students’ performance case study in the UAE. Int. J. Art Des. Educ. 41, 108–124. doi: 10.1111/JADE.12378

Crossref Full Text | Google Scholar

Attiogbe, E. J. K., Oheneba-Sakyi, Y., Kwapong, O. A. T. F., and Boateng, J. (2023). Assessing the relationship between feedback strategies and learning improvement from a distance learning perspective. J. Res. Innov. Teach. Learn. doi: 10.1108/JRIT-10-2022-0061

Crossref Full Text | Google Scholar

Austin Sanders, MS, Ballen, Brittany, Okoro, Judith, Afterman, Melissa, Barr, Alan, Robertson, Michelle, et al. (2022). The Impact of Distance Education on Physical and Mental Health among 4th Graders during the COVID-19 Pandemic. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66, pp. 1782–1784.

Google Scholar

Baniomar, K. A. (2022). The impact of the shift to distance learning on the seven principles of good practices in university education in light of the COVID-19 pandemic. Cypriot J. Educ. Sci. 17, 1533–1548. doi: 10.18844/cjes.v17i5.7218

Crossref Full Text | Google Scholar

Berruecos-Vila, A. M., and Ochoa-Carrasco, I. (2022). A 660 Hour Rally to Prepare Teachers for the Shift to Digital Distance Education during the COVID-19’S Health Contingency: The Importance of Formal TRAINING. INTED2022 Proceedings.

Google Scholar

Bowles, Jerry . (2021). How Zoom Defied Its Critics and Became the Go-To Video Conferencing App for Surviving the Pandemic. Available at: https://diginomica.com/how-zoom-defied-its-critics-and-became-go-video-conferencing-app-surviving-pandemic (Accessed May 2024).

Google Scholar

Cheng, K. H., Lee, S. W. Y., and Hsu, Y. T. (2023). The roles of epistemic curiosity and situational interest in students’ attitudinal learning in immersive virtual reality environments. J. Educ. Comput. Res. 61, 494–519. doi: 10.1177/07356331221121284

Crossref Full Text | Google Scholar

Collazos, C., Renzi, S., Guerrero, L., Pino, J., Ortega, M., Bravo, C., et al. (2007). Evaluating collaborative learning processes using system-based measurement. J. Educ. Technol. Soc. 10, 257–274.

Google Scholar

Davidovitch, N., and Wadmany, R. (2021). E-learning in times of crisis: an incidental or facilitative event? Int. Ser. Oper. Res. Manag. Sci. 309, 453–479. doi: 10.1007/978-3-030-74051-1_15/COVER

Crossref Full Text | Google Scholar

Demir, S., Shagiakhmetova, M. N., Bystritskaya, E. V., Stepanov, R. A., Grishnova, E. E., and Kryukova, N. I. (2022). Primary teachers difficulties related to compulsory distance education during COVID-19. Contemp. Educ. Technol. 14:ep357. doi: 10.30935/cedtech/11589

Crossref Full Text | Google Scholar

Demir, M. R., and Yildizli, H. (2022). Educational processes and learning at home during COVID-19: parents’ experiences with distance education. Int. Rev. Res. Open Distrib. Learn. 23, 1–20. doi: 10.19173/irrodl.v23i2.5870

Crossref Full Text | Google Scholar

Di Malta, G., Bond, J., Conroy, D., Smith, K., and Moller, N. (2022). Distance education students’ mental health, connectedness and academic performance during COVID-19: a mixed-methods study. Distance Educ. 43, 97–118. doi: 10.1080/01587919.2022.2029352

Crossref Full Text | Google Scholar

Domhnaill, C. M., Mohan, G., and McCoy, S. (2021). Home broadband and student engagement during COVID-19 emergency remote teaching. Distance Educ. 42, 465–493. doi: 10.1080/01587919.2021.1986372

Crossref Full Text | Google Scholar

Dwivedi, Y. K., Hughes, L., Baabdullah, A. M., Ribeiro-Navarrete, S., Giannakis, M., Al-Debei, M. M., et al. (2022). Metaverse beyond the hype: multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. Int. J. Inf. Manag. 66:102542. doi: 10.1016/J.IJINFOMGT.2022.102542

Crossref Full Text | Google Scholar

Edirisingha, P. (2022). Online and Distance Learning during Lockdown Times: COVID-19 Stories (Volume 1) : MDPI.

Google Scholar

Fredricks, J. (2022). “Measuring student engagement with observational techniques” in Handbook of Research on Student Engagement, Springer 617–627.

Google Scholar

Garavaglia, A., and Petti, L. (2020). Sudden shift to distance learning: analysis of the didactic choices made by Italian secondary school teachers in the first COVID-19 lockdown. Res. Educ. Media 12, 19–28. doi: 10.2478/rem-2020-0015

Crossref Full Text | Google Scholar

Garcia-Sanjuan, F., Jurdi, S., Jaen, J., and Nacher, V. (2018). Evaluating a tactile and a tangible multi-tablet gamified quiz system for collaborative learning in primary education. Comput. Educ. 123, 65–84. doi: 10.1016/j.compedu.2018.04.011

Crossref Full Text | Google Scholar

Garduño, H. A. S., Martínez, M. I. E., and Castro, M. P. (2021). Impact of virtual reality on student motivation in a high school science course. Appl. Sci. 11:9516. doi: 10.3390/APP11209516

Crossref Full Text | Google Scholar

Gong, S., and Xiang, T. (2011). Visual Analysis of Behaviour. London: Springer.

Google Scholar

Hart, S. G. (2006). NASA-Task Load Index (NASA-TLX); 50 Years Later, Elsivier 904–908.

Google Scholar

Hart, S. G., and Staveland, L. E. (1988). Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183. doi: 10.1016/S0166-4115(08)62386-9

Crossref Full Text | Google Scholar

Huang, C. L., Luo, Y. F., Yang, S. C., Lu, C. M., and Chen, A. S. (2020). Influence of students’ learning style, sense of presence, and cognitive load on learning outcomes in an immersive virtual reality learning environment. J. Educ. Comput. Res. 58, 596–615. doi: 10.1177/0735633119867422

Crossref Full Text | Google Scholar

Huang, W., Roscoe, R. D., Craig, S. D., and Johnson-Glenberg, M. C. (2022). Extending the cognitive-affective theory of learning with Media in Virtual Reality Learning: a structural equation modeling approach. J. Educ. Comput. Res. 60, 807–842. doi: 10.1177/07356331211053630

Crossref Full Text | Google Scholar

Issayev, G., Bagdat, B., Abay, D., Aizhan, S., Baimanova, L., and Almazhai, Y. (2022). Application of information technologies in distance learning in the field of higher education. World J. Educ. Technol. 14, 1017–1024. doi: 10.18844/WJET.V14I4.7650

Crossref Full Text | Google Scholar

Jaggars, S. S., and Xu, D. (2016). How do online course design features influence student performance? Comput. Educ. 95, 270–284. doi: 10.1016/J.COMPEDU.2016.01.014

Crossref Full Text | Google Scholar

Kamińska, D., Sapiński, T., Wiak, S., Tikk, T., Haamer, R. E., Avots, E., et al. (2019). Virtual reality and its applications in education: survey. Information 10:318. doi: 10.3390/INFO10100318

Crossref Full Text | Google Scholar

Katalnikova, S., Zagulova, D., Prokofjeva, N., Špakova, A., Ziborova, V., and Averin, A. (2021). Forced Distance Learning in Covid-19: Peculiarities of the Position of Lecturers, Students of IT and Humanitarian Specialties. 7th International Conference on Higher Education Advances (HEAd’21).

Google Scholar

Katz, A., and Kedem-Yemini, S. (2021). From classrooms to zoom rooms: preserving effective communication in distance education. J. Inf. Technol. Case Appl. Res. 23, 173–212. doi: 10.1080/15228053.2021.1922248

Crossref Full Text | Google Scholar

Korukluoğlu, P., and Bavli, B. (2022). Being a parent in the distance education process of the COVID-19 pandemic period: a phenomenological study. Afyon Kocatepe Üniversitesi Sosyal Bilimler Dergisi. 24, 445–466. doi: 10.32709/akusosbil.894043

Crossref Full Text | Google Scholar

Krawczyk-Bryłka, B., Stankiewicz, K., Ziemiański, P., and Tomczak, M. T. (2020). Effective collaboration of entrepreneurial teams—implications for entrepreneurial education. Educ. Sci. 10:364. doi: 10.3390/educsci10120364

Crossref Full Text | Google Scholar

Li, K., Lau, B. P. L., Yuan, X., Ni, W., Guizani, M., and Yuen, C. (2023). Towards ubiquitous semantic Metaverse: challenges, approaches, and opportunities. IEEE Internet Things J. 10, 21855–21872. doi: 10.1109/JIOT.2023.3302159

Crossref Full Text | Google Scholar

Lucas, M., Nelson, J., and Sims, D. (2020). Schools’ Responses to COVID-19: Pupil Engagement in Remote Learning : National Foundation for Educational Research.

Google Scholar

MacKinnon, D. P. (2000). “Contrasts in multiple mediator models” in Multivariate Applications in Substance Use Research: New Methods for New Questions. eds. J. S. Rose, L. Chassin, C. C. Presson, and S. J. Sherman (Lawrence Erlbaum Associates Publishers), 141–160.

Google Scholar

Mahmud, M., Alsamh, H. M. A., Sarfaraz, T., Arshad, M. A., Mahmood, A., and Nikku, B. R. (2022). “Impact on teaching strategies in Saudi Arabian universities owing to COVID-19 pandemic” in Teaching in the Pandemic Era in Saudi Arabia (Brill).

Google Scholar

Mancuso, I., Messeni Petruzzelli, A., and Panniello, U. (2023). Digital business model innovation in metaverse: how to approach virtual economy opportunities. Inf. Process. Manag. 60:103457. doi: 10.1016/j.ipm.2023.103457

Crossref Full Text | Google Scholar

Martirosov, S., and Kopecek, P. (2017). “Virtual reality and its influence on Training and education – literature review” in 28th DAAAM International Symposium. ed. B. Katalinic (DAAAM International), 708–717.

Google Scholar

McCann, S. (2019). “A case for leveraging Mobile learning in action learning teams” in Dominic Mentor Advancing Mobile Learning in Contemporary Educational Spaces, IRMA 1–23.

Google Scholar

NASA (2020). The Official NASA Task Load Index (TLX) : NASA.

Google Scholar

Norton, Niall . (2023). Overcoming Barriers to the Metaverse Requires Ubiquitous Connectivity. Available at: https://www.amdocs.com/insights/blog/overcoming-barriers-metaverse-requires-ubiquitous-connectivity (Accessed May 2024).

Google Scholar

Öçal, T., Halmatov, M., and Ata, S. (2021). Distance education in COVID-19 pandemic: an evaluation of parent’s, child’s and teacher’s competences. Educ. Inf. Technol. 26, 6901–6921. doi: 10.1007/s10639-021-10551-x

PubMed Abstract | Crossref Full Text | Google Scholar

Özbey, M., and Kayri, M. (2022). Investigation of factors affecting transactional distance in E-learning environment with artificial neural networks. Educ. Inf. Technol. 28, 4399–4427. doi: 10.1007/S10639-022-11346

Crossref Full Text | Google Scholar

Pimentel, D., Fauville, G., Frazier, K., Mcgivney, E., Rosas, S., and Woolsey, E. (2022). Learning in the Metaverse: A Guide for Practitioners Letter from the Authors. Available at: https://www.meridiantreehouse.com/metaverse-education-guide (Accessed May 2024).

Google Scholar

PUN . (2021). Photon Engine SDKs | Photon Unity Packages. Available at: https://www.photonengine.com/sdks#pun-unity-sdkpununity (Accessed May 2024).

Google Scholar

Punter, R. A., Meelissen, M. R., and Glas, C. A. (2017). Gender differences in computer and information literacy: an exploration of the performances of girls and boys in ICILS 2013. Eur. Educ. Res. J. 16, 762–780. doi: 10.1177/1474904116672468

Crossref Full Text | Google Scholar

Ramirez, G. M., Collazos, C. A., and Moreira, F. (2018). All-learning: the state of the art of the models and the methodologies educational with ICT. Telematics Inform. 35, 944–953. doi: 10.1016/j.tele.2017.10.004

Crossref Full Text | Google Scholar

Regmi, K., and Jones, L. (2020). A systematic review of the factors - enablers and barriers - affecting e-learning in health sciences education. BMC Med. Educ. 20, 1–18. doi: 10.1186/S12909-020-02007

Crossref Full Text | Google Scholar

Reis, F., Vicente, J., and Pimentel, H. (2021). Challenges of Online Teaching-Learning/Distance Learning Context in Higher Education due to Covid-19: An Empirical Study Applied to Lusófona University Humanities and Technologies Students (Lisbon). ICERI2021 Proceedings.

Google Scholar

Reychav, I., and McHaney, R. (2017). The relationship between gender and mobile technology use in collaborative learning settings: an empirical investigation. Comput. Educ. 113, 61–74. doi: 10.1016/j.compedu.2017.05.005

Crossref Full Text | Google Scholar

Rodrigues, M., Franco, M., and Silva, R. (2020). COVID-19 and disruption in management and education academics: bibliometric mapping and analysis. Sustain. For. MDPI 12. doi: 10.3390/su12187362

Crossref Full Text | Google Scholar

RPM-Unity . (2021). Ready Player Me for Unity. Available at: https://docs.readyplayer.me/ready-player-me/integration-guides/unity (Accessed May 2024).

Google Scholar

Salih, Y., and Ibrahim, R. A. (2021). Impact of distance learning on mental health during COVID-19: a review. Int. J. Ethno-Sci. Educ. Res. 1, 76–79. doi: 10.46336/ijeer.v1i4.238

Crossref Full Text | Google Scholar

Sari, D. K., and Maningtyas, R. T. (2020). Parents’ Involvement in Distance Learning during the Covid-19 Pandemic. Proceedings of the 2nd Early Childhood and Primary Childhood Education (ECPE 2020).

Google Scholar

Seidman, R. (2009). Alice First: 3D Interactive Game Programming. Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education, 345.

Google Scholar

Snezhko, Z., Babaskin, D., Vanina, E., Rogulin, R., and Egorova, Z. (2022). Motivation for mobile learning: teacher engagement and built-in mechanisms. Int. J. Interact. Mobile Technol. 16, 78–93. doi: 10.3991/IJIM.V16I01.26321

Crossref Full Text | Google Scholar

Šorgo, A., Crnkovič, N., Gabrovec, B., Cesar, K., and Selak, Š. (2022). Influence of forced online distance education during the COVID-19 pandemic on the perceived stress of postsecondary students: cross-sectional study. J. Med. Internet Res. 24:e30778. doi: 10.2196/30778

PubMed Abstract | Crossref Full Text | Google Scholar

Stachteas, P., and Stachteas, C. H. (2020). The psychological impact of the COVID-19 pandemic on secondary school teachers. Psychiatriki 31, 293–301. doi: 10.22365/jpsych.2020.314.293

Crossref Full Text | Google Scholar

Stecuła, K., Wolniak, R., Stecuła, K., and Wolniak, R. (2022). Influence of COVID-19 pandemic on dissemination of innovative E-learning tools in higher education in Poland. J. Open Innov. Technol. Mark. Complex. 8, doi: 10.3390/joitmc8020089

Crossref Full Text | Google Scholar

Tondeur, J., Van de Velde, S., Vermeersch, H., and Van Houtte, M. (2016). Gender differences in the ICT profile of university students: a quantitative analysis. DiGeSt. J. Divers. Gend. Stud. 3:57. doi: 10.11116/jdivegendstud.3.1.0057

Crossref Full Text | Google Scholar

Unity . (2019). XR Interaction Toolkit Preview Package Is Here | Unity Blog. Available at: https://blog.unity.com/technology/xr-interaction-toolkit-preview-package-is-here (Accessed May 2024).

Google Scholar

Unity . (2022). Unity: The Leading Platform for Creating Interactive, Real-Time Content. Available at: https://unity.com/ (Accessed May 2024).

Google Scholar

van der Meer, N., van der Werf, V., Brinkman, W.-P., and Specht, M. (2023). Virtual reality and collaborative learning: a systematic literature review. Front. Virt. Reality 4:47. doi: 10.3389/FRVIR.2023.1159905

Crossref Full Text | Google Scholar

Xu, Y., and Yang, Y. (2024). Research on the patterns of the organic growth knowledge production based on a dual perspective of content and behaviour. Br. J. Educ. Technol. 55, 1145–1166. doi: 10.1111/bjet.13418

Crossref Full Text | Google Scholar

Zai, S. A. Y., and Akhunzada, A. S. (2021). Challenges and responses of higher education institutions to align higher education with COVID-19 realities: a thematic analysis. Int. J. Distance Educ. E-Learn. 6, 109–129. doi: 10.36261/ijdeel.v6i1.1425

Crossref Full Text | Google Scholar

Zarifis, A. (2019). The Six Relative Advantages in Multichannel Retail for Three-Dimensional Virtual Worlds and Two-Dimensional Websites. Proceedings of the 10th ACM Conference on Web Science, 363–372.

Google Scholar

Zhou, Q., Suraworachet, W., Pozdniakov, S., Martinez-Maldonado, R., Bartindale, T., Chen, P., et al. (2021). Investigating Students’ Experiences with Collaboration Analytics for Remote Group Meetings. Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part 12748, 472–485.

Google Scholar

Zhou, G., and Xu, J. (2007). Adoption of educational technology: how does gender matter? Int. J. Teach. Learn. High. Educ. 19, 140–153.

Google Scholar