Re-Live History: An immersive virtual reality learning experience of prehistoric intangible cultural heritage

Introduction

The use of immersive virtual reality (IVR) for learning is not a new concept and several surveys have been published to assess the academic research surrounding this field (Freina and Ott, 2015; Stavroulia et al., 2019; Radianti et al., 2020). However, there are as yet mixed feelings about the technology’s affordances for the delivery of knowledge (Ochs and Sonderegger, 2022) together with a lack of its application in actual teaching (Radianti et al., 2020), particularly with students in the 11 to 12 year bracket (Stavroulia et al., 2019). This is a pity as a great opportunity exists to use such immersive experiences to teach about the history of our past cultures, addressing learning outcomes in school curricula that focus on periods of time about which little documentary evidence is available—such as prehistoric cultures. The project Re-Live History, which is being reported upon here, aims to contribute toward this gap by working closely with heritage experts and teachers in developing an IVR to teach prehistoric intangible cultural heritage to secondary school students of history. By intangible cultural heritage we refer to “practices, representations, expressions, knowledge, skills—as well as the instruments, objects, artifacts, and cultural spaces associated therewith—that communities, groups and, in some cases, individuals recognize as part of their cultural heritage” (UNESCO, 2003, Article 2 Definition 1) in contrast with tangible cultural heritage which refers to the physical artifacts and sites. The methods of safekeeping tangible cultural heritage cannot be simply applied to the protection and curatorship of intangible cultural heritage, and thus new techniques and methodologies need to be developed (UNESCO, 2004).

The research question of this study is how to go about designing an immersive learning experience for secondary school teachers to teach 11 to 12 year old students about prehistoric cultural heritage on which very little documented evidence is available? When there is no written record of events except for relatively recent descriptions of artifacts and sites upon their discovery, the challenge is that the interpretation of such heritage may be rife with uncertainty and thus open to speculation making the learning of authentic knowledge a non-trivial affair.

To address our research question, we explore the literature to identify issues that have challenged such attempts in the past. Based on this information, we then consider three design aspects of such an IVR learning experience to identify our project’s design objectives. An experiment is then designed to test the experience’s pedagogical value, with its evaluation serving as a point of discussion.

Background

Immersive virtual reality in education

Virtual reality technology has become more accessible to the public because of a recent emphasis on its technological development, mass production making it affordable and, through the use of smartphones for VR, foregoing the need for complex expensive devices (Martín-Gutiérrez et al., 2017). This resulted in more and more research on the use of IVR in education being carried out (Checa and Bustillo, 2020), and we now look at some results of surveys carried out on papers on the subject.

A survey carried out in 2015 identified 93 papers from three scientific digital libraries that were published between 2013 and 2014 and that report on the use of IVR in Education (Freina and Ott, 2015). The large majority of these refer to tertiary or adult education, with only two papers reporting on the use of IVR in teaching children. One of these used Second Life (Linden Research, Inc, 2003) to present a 3D virtual museum of Asian art experienced on a flat screen and so did not use any head-mounted devices (HMDs) (Huang and Han, 2014) while the other used Augmented Reality rather than VR (Eleftheria et al., 2013). A later study comments that technological limitations, financial accessibility, health and safety issues, as well as lack of experience in the use of the technology by teachers and students were factors behind the lack of studies on the use of IVR in education (Stavroulia et al., 2019).

Yet another study of VR serious games for education and training in 2019 analyzed 86 articles from two library sources. Amongst their results, they report that user satisfaction and learning rate were the two key factors measured in these articles, prompting them to suggest the consideration of user experience factors such as immersion and usability. They further concluded that most of the studies find VR serious games as a suitable tool for education regardless of the technical solution, but found the Oculus Rift and Unity as the most common platforms used (Checa and Bustillo, 2020). Another contribution of this paper is identifying areas with growth potential: in education these were target audiences of primary school students and the general public, the use of explorative interaction, and combining more than one type of evaluation (Checa and Bustillo, 2020). A literature review study encompassing 18 years (2003–2019) of research within the Child-Computer-Interaction (CCI) community explored the discourse of ethics within the field of technology and children, including education (Van Mechelen et al., 2020). Results show a lack of shared theoretical understanding of ethics, and a general lack of situational ethics (how to handle situations ethically), participation ethics (participation in the research process), and design ethics (the impact of technology on society) in academic papers. The survey identifies formal education as one of the main context for ethics while researchers and designers, the children and their parents/guardians/care givers, the educators, and the domain experts are identified as the main actors concerned with ethics (Van Mechelen et al., 2020).

A more recent study published in 2020 carried out a literature survey looking for immersive virtual reality (VR) applications in higher education. After analyzing 80 articles, gathered from four different scientific digital libraries and filtered out of an original 3219 articles, the study reports that (i) such applications did not consider learning theories, (ii) focused more on usability rather than its ability to help students achieve learning outcomes, and (iii) none of them have been applied in actual teaching (Radianti et al., 2020). This suggests the adoption of learning theories, the assessment of achievable learning outcomes and to seek their adoption in the classroom.

Immersive virtual reality in learning of cultural heritage

The use of computer simulation to represent history has a long history (Taylor, 2003) with many games such as Civilization (Meier, 1991), Age of Empires (Microsoft, 1997), and Assassin’s Creed (Ubisoft Montreal, 2007) being used for the purpose of teaching history. However, such earlier games usually have fun as the main or sole objective (Malegiannaki and Daradoumis, 2017), which may cause problems by diminishing the value of the depicted cultural heritage (Champion, 2015) and supersede any learning outcomes intended to be delivered through the use of the experience.

More recent endeavors have taken on the VR platform and shifted away from entertainment toward education. A study in 2019 on the application of VR in education and training found 171 such applications on the Oculus store, 19 of which (11.1%) dealt with the topic of History (Smutny et al., 2019). Amongst these are Nefertari: Journey to Eternity (Experius VR, and Curiosity Stream, 2018) and Chauvet: Dawn of Art (Tanant et al., 2020) which present the VR visitor with a virtual representation of the physical tomb and prehistoric cave, respectively.

However, Ch’ng had earlier called for “the experiential aspect of cultural heritage” through VR, beyond the “preservation through digital capture or detailed reconstructions” (Ch’ng et al., 2018, p. v). This has caused a shift of interest to not only use VR to portray ‘tangible cultural heritage’ (UNESCO, 1972), but also the human behavior around such sites and artifacts, collectively known as ‘intangible cultural heritage’ (UNESCO, 2003).

This reflects an earlier call for the portrayal of cultural rituals through role-play in cultural heritage-oriented games by Erik Champion, a developer of VR applications for heritage sites and computer games scholar, where he proposes that “virtual worlds and related computer games have dimensions that could afford environmental presence, social presence and cultural presence and should do so when employed for pedagogical objectives” (Champion, 2015, p. 2). He defines environmental presence as the interaction between the player and the virtual world, social presence as the interaction with its inhabitants, and cultural presence as a “distinctly situated sense of inhabitation, of social values and behaviors preserved and transmitted through ritual, artifact and inscription” (Champion, 2015, p. 7). This suggests that IVRs could well deliver the intangible aspects of cultural heritage.

Theoretical framework

An IVR learning experience for Intangible Cultural Heritage is at the intersection of three aspects: Education, Cultural Heritage, and Technology. These are now explored to see what they can contribute to the design of the experience and their intended use in this project is then described.

Educational aspects

Educational aspects of IVR learning experiences include relevant learning theories, the measurement of attained learning outcomes, assessment by stakeholders, and the willingness to adopt such a learning activity in the classroom.

Learning theories

The literature suggests that using 3D virtual worlds for learning allows students to “learn by doing” and subsequently “apply learned concepts to the real world,” thus lending themselves positively to experiential learning (Jarmon et al., 2009; Le et al., 2014). This also applies to VR, as the technological successor to 3D virtual worlds: “students’ involvement in the immersion of an authentic virtual environment helps in enhancing their experiential learning” (Asad et al., 2021, p. 9; Hsiao, 2021). Experiential learning is a four-stage learning cycle involving having the actual experience, reflecting on the experience, learning from it, and trying out what has been learned (Kolb, 1984). IVRs are very well able to deliver the first cycle of experiential learning (Aiello et al., 2012) while their use in the later cycles depends on the complexity of the interaction. Research has shown that IVRs used in experiential learning have had a positive impact on students’ attainment of learning outcomes (see below) but its magnitude was less than expected, possibly due to the removal of the teacher from the learning experience (Feng et al., 2021). This suggests designing an IVR to serve as the delivery of the experience in an experiential learning context and give space for the teacher to facilitate learning alongside the IVR experience.

Learning outcomes

Learning Outcomes are defined as “statements of what a learner is expected to know, understand and/or be able to demonstrate at the end of a period of learning” (Adam, 2006, p. 2) and are valued in curriculum development for their clarity and precision. However, clarity and precision are very subjective, sometimes interpreted by both managers and academics in terms of their own knowledge and experience (Hussey and Smith, 2002). This suggests that any analysis in terms of learning outcomes should involve both teachers as well as their managers.

In the context of the prehistory subject being taught at secondary schools in Malta, where the Re-Live History project was being developed, the learning outcomes are split across 5 subject foci: Time and Chronology in History, Historical Terms and Concepts, Working with Historical Sources, Historical Interpretation and Empathic Understanding, and Communicating History¹. The project aimed to help deliver a number of these learning outcomes, the specific selection being part of the research process described later in section 0.

Evaluation by stakeholders

In the context of IVR experiences as a cultural heritage learning tool in secondary education on prehistory, one can identify three main stakeholders: the heritage expert, the history student, and the history teacher. The heritage expert’s main concern is that the correct knowledge is being passed on to the general public through the history education, and any speculation is identified as such. Thus they seek a balance between realism and correctness based on knowledge. Realism requires details—but details that may not be known when dealing with prehistory. Another stakeholder is the history student, who will seek answers to their questions and may be frustrated when these are not available. Their inquisitive mind will seek to fill in any remaining knowledge gaps with what they know from elsewhere. In fact, an earlier experiment that sought to seek what kind of interaction students attributed to a prehistoric sites, they gamified the experience by repurposing existing game genres into the historical site, allowing them to explore the space using known mechanics (Barbara, 2020). A third stakeholder is the history teacher, who is tasked with delivering the learning outcomes expected by the level being taught. With limited time made available both by the timetable as well as by the students’ attention span, they seek delivery methods that are as efficient as possible, ideally meeting multiple learning outcomes at the same time. In view of the above biases of the heritage experts and the students, and in order to further facilitate the adoption of the experience in the classroom (see the next section), the teachers were chosen as the end-users for this project.

Adoption and educational applicability

Studies show that VR has a positive impact on students’ motivation for learning (Huang et al., 2019) by increased engagement (Kaplan-Rakowski and Wojdynski, 2018), satisfaction (Hodgson et al., 2019), attention, and relevance (Casu et al., 2015). There is also a positive outlook toward the use of the technology in the classroom in journals on education and training (Velev and Zlateva, 2017). However, as we have already mentioned, there is an issue with the lack of adoption of IVR experiences by teachers in the classroom (Radianti et al., 2020). A large-scale study on teachers’ attitudes toward the use of VR for education shows that more traditional teaching approaches were associated with lower level of VR integration in the classroom. Studies have found that expected effort and social influence are determinant factors of adoption of VR technology by the teacher in the classroom (Hussin et al., 2011).

Another factor is the availability of suitable IVR learning experiences (Velev and Zlateva, 2017). Off-the-shelf historical video games that are intent on selling to the entertainment industry will favor gameplay over instruction (Champion, 2015) while one must ensure that educational objectives are given priority (Daniela and Aierken, 2020). Thus, educational applicability is one of the suggested principles for the design of such experiences (Egea-Vivancos and Arias-Ferrer, 2021).

Educational applicability states that motivating teachers to use such a serious game requires consideration of the curriculum and social reality in schools to allow for easier integration into the classroom practice. This further suggests the identification of the immersive learning experience’s end users as being the teachers, helping them to embrace and adopt the technology in the classroom.

Cultural heritage aspects

Tangible cultural heritage

Choosing prehistoric cultural heritage as the subject of the IVR learning experience presents its own challenges. Much of the surviving primary evidence is only in the form of sites and artifacts found within them. These artifacts would survive because they are either made of stone (or clay), bronze or iron, and depending on their age of provenance: Stone Age, Bronze Age, or Iron Age, respectively—given that later ages may have artifacts made in previous ages. This means that any other artifact made with some other material at the time, such as wood, would have disintegrated with time. Secondary sources would likewise be limited: only carvings or well-preserved paintings on stone would survive from the Stone Age. These may have been intentionally made, such as red ochre paintings, or remains of workmanship, such as tool marks left in the walls and roof after excavation of a Neolithic hypogeum (Cardona and Zammit, 2018).

Intangible cultural heritage

If tangible cultural heritage from the prehistoric periods is lacking solid evidence due to disintegration over time, the situation is much worse with respect to intangible cultural heritage: the behaviors and rituals of the society who inhabited these sites at those times. We have the sites where any ritual may have been carried out, we have the artifacts with which they would have been carried out but we have no documentary evidence to tell us how and why these were done. Any speculation is based either on some function attributed to the artifacts found in situ, possibly based on traces found on site—such as the abovementioned tool marks—or based on knowledge about other spatially and chronologically proximal cultures. Heritage experts would be highly knowledgeable on the evidence available on site and the potential behaviors that such evidence supports with sufficient confidence, as well as highly aware and wary of speculative conclusions without any supporting evidence. This suggests using their expertise in order to ensure proper representation of accepted intangible cultural heritage.

Technological aspects

As VR can be defined as “the sum of the hardware and software systems that seek to perfect an all-inclusive, sensory illusion of being present in another environment” (Biocca and Delaney, 1995, p. 63), the actual delivery of the IVR experience ultimately depends on the technology used. CAVE systems offer a sited experience without encumbering the visitor with any hardware by projecting the virtual surroundings onto a room’s walls (Manjrekar et al., 2014). This has the ability to correct the perspective based on the visitor’s position in the room with the caveat that it can only handle a single individual. Head-mounted devices, such as Oculus Rift and HTC Vive, offer individual virtual experiences with stereoscopic rendering of the virtual world allowing depth perception and six degrees of freedom through sensors of the visitor’s movement.

Realism

A challenge in representing intangible cultural heritage in VR is that the movement of the character needs to match the realism depicted by the tangible cultural heritage representation. An earlier project of the authors was developed using a LIDAR scan of an underground heritage site, with millimetrical precision. Any non-player character (NPC) representation that falls short of such realism would break the visitor’s immersion and distract from any learning outcomes being addressed. On the other hand, having believable and authentic NPCs as cultural agents makes for a stronger cultural presence, as inferred by Champion (2015). Just as LIDAR scans measure the physical contours of a site, motion capture suits can be used to measure and capture the physical movement of actual humans performing the ritual which can then be applied to a realistic 3D model (Slater and Sanchez-Vives, 2014).

Ethical issues

Virtual reality also brings with it questions of ethics. For example, Madary and Metzinger (2016) argue that this sense of embodiment can have a manipulative effect on deep behavior if it is misused. They thus insist on having the users of VR informed about any lasting psychological effects of VR as one can develop a condition where they “experience the real world and their real bodies as unreal, effectively shifting their sense of reality exclusively to the virtual environment” Madary and Metzinger, 2016).

Ethical issues abound when bringing IVR into a classroom of 11 to 12 year old students (Southgate et al., 2019). In a study embedding IVR into ICT and science classes in low-income high schools in Australia, Southgate et al. (2019) describe ethical, organizational and educational challenges encountered in their project. With respect to ethics, their recommendations include participatory design, which is time-consuming but required in order to build trust between researchers and stakeholders, as well as to develop an ethical practice that responded well to both planned and emerging ethical issues. Another issue is that of empathy, understood here as “a positive outcome in the viewers, due to change in attitude or belief, with a likelihood that this change may result in pro-social behavior and even actions advancing justice” (Rouse, 2021, p. 4). Chris Milk described VR as the “ultimate empathy machine” (Milk, 2015), suggesting “a mass medium that can seamlessly reach scores of individuals, necessitate no actual interpersonal contact with the ‘others’ these individuals seek to empathize with, and smoothly and easily change minds in minutes” (Rouse, 2021, p. 5). The fallacy of this claim is evidenced by “improper distance,” when one’s truths take over the experience of others making them “indistinguishable from ourselves” (Nash, 2018, p. 120).

This suggests that, while environmental realism can be met through the use of motion capture suits, the level of expected realism should be reasonable both in terms of embodiment as well as empathy.

Application of the theoretical framework

The project aims to follow the Experiential Learning theory by providing students with an immersive virtual learning experience that situates them inside a realistic virtual representation of a Neolithic Hypogeum inside which evidence of human remains and human activity is shown. Through visual and aural stimuli, the students will experience the excavation of one of the hypogeum’s niches, allowing for reflection on the purpose, motivation, and limitations of such behavior.

What sets apart games from educational digital experiences is the authenticity of the content. As we wish to give greater importance to learning outcomes rather than gameplay (Champion, 2015), we will seek the guidance of the heritage experts in order to ensure authentic information is given to the teachers to deliver to the students.

The project is specifically driven by the learning outcomes of the History subject. As it is not the intention to cover all of the learning outcomes, the assistance of the subject’s educators in choosing which learning outcomes are potentially achievable with an IVR will be sought and then the project will focus on a further subset of these, as driven by technological and time limitations.

Heritage experts and educators are also identified as actors concerned with ethics involving technology with children (Van Mechelen et al., 2020). We seek to collaborate with curators and historians as domain experts to ensure an ethical representation of history within the IVR experience. We will also involve educators in the design process to help us plan to address situational ethical issues that may arise.

Teachers are resistant to imposed teaching resources, limiting adoption and educational applicability (Radianti et al., 2020). We will thus seek to attract teachers and other educational professionals early on in the project to assist in the design of the IVR experience. This is also inline with the ethical recommendations of Southgate et al. (2019) who recommend participatory inquiry. The choice to focus on educators in this project allows us to start a long-term relationship with teachers, also recommended by Southgate et al. (2019) in order to build trust that will allow us to follow up this project with future work that will test the IVR experience in the field with actual students.

The choice of participants is also driven by the definition of learning outcomes, specifically ‘what a learner is expected to know,’ and such expectations are better known to the educators than to the learners themselves. Thus we seek to recruit teachers, head of departments, and other educational officers to assess the feasibility of using such an IVR in the teaching of history.

The realism requirements of our project are determined by the highly realistic representation of the tangible cultural heritage site due to its data acquisition by LIDAR technology. To reach the realism required for believable and authentic NPCs (Champion, 2015), we aim to use motion capture technology to provide the Neolithic character with believable animation.

It is unethical to expect a person living in the 21st century to empathize with another person from 5000 years ago (Barbara et al., 2021). To this end, we will not seek to provide the virtual learner with an avatar through whose eyes and body they can inhabit the virtual representation of the hypogeum, but rather will be given a disembodied drone-link presence allowing for free navigation throughout the site and around the NPC.

Methods

In conclusion of the above analysis, the design objectives below were identified in terms of technological implementation as well as evaluation approach. A short description of the project’s scope is given and then the conceptual framework to be used in the design of the IVR experience and its implementation and outcomes are explained.

Design objective 1 (technology)

Use of IVRs and motion capture suits to deliver intangible aspects of cultural heritage, being guided by heritage experts in order to ensure its proper representation, while keeping expectations of both embodiment and empathy in check.

Design objective 2 (evaluation)

Use an IVR as the delivery of the experience in the first cycle of an experiential learning context, whilst allowing the teacher to facilitate the remaining three cycles of learning outside the IVR experience, including the assessment of attained learning outcomes (chosen by both teachers and their managers), thus helping teachers to embrace and adopt the technology in the classroom.

The project

The Re-Live History project was a yearlong endeavor undertaken by the Creative Team in the Computing Department of Saint Martin’s Institute of Higher Education (SMI), Ħamrun, Malta. Building upon the team’s previous project of representing the tangible cultural heritage that is the Neolithic Hypogeum of Ħal Saflieni (Barbara et al., 2020), the project’s technological aim was to introduce realistic character behavior that portrayed intangible cultural heritage while meeting the realism expectations raised by the site’s LIDAR scan. Moreover, the project also had its pedagogical aim: to test if the introduction of motion-controlled 3D characters re-enacting intangible heritage in immersive VR experiences would increase the learners’ ability to achieve learning outcomes that deal with understanding prehistoric life.

Conceptual framework

In order to address the design of a virtual heritage environment for cultural learning, we adapted and applied a conceptual framework and its design activities from the literature (see Figure 1). This starts with a content requirement study from a heritage expert’s and end user’s perspective, the output of which will be used to inform evaluation criteria and the design of a prototype to acquire feedback from the heritage experts, before presenting it for evaluation to the end users (Ibrahim and Ali, 2018).

FIGURE 1

Figure 1. Design activities for a conceptual framework adapted from Ibrahim and Ali (2018).

Content requirement study

The output of the previous self-financed project undertaken by SMI (Barbara et al., 2020) was a VR navigation experience through the three levels of the underground Hypogeum of Ħal Saflieni, a UNESCO World Heritage Site provided in three forms: an interactive experience delivered through an Oculus Rift head-mounted device connected to a powerful PC with narrated pop-up information panels giving descriptions of the main areas in the site; a lightweight version² that offered 360 screenshots from within the virtual experience accompanied by voice-over narrations and navigated via gaze-driven interaction; and a rail-driven 360° footage³ through the virtual experience with a voice-over of the personified site provided for public consumption over YouTube during the pandemic lockdown in 2020. These three experiences were presented as the starting point for the Re-Live History project, allowing heritage experts and history teachers to understand the level reached so far and have them envisage what intangible cultural heritage could be delivered to students through modeling character behavior.

Heritage experts’ input (study 1 of conceptual framework)

A meeting was held toward the beginning of the project to have the heritage experts’ opinion on what could be presented as intangible cultural heritage of the Neolithic hypogeum of Ħal Saflieni. These were two curators from the national agency in charge of the restoration and upkeep of all historical sites in Malta: The Principal Curator of Prehistoric Sites and a Senior Curator from Heritage Malta. They provided guidance on the proper dimensions and appearance of any Neolithic characters to be introduced as well as the identification of locations where digging and painting behavior could be depicted as well as where whole skeletons were found. Based on evidence in the form of artifacts and tool marks in the site’s walls, the digging process using heavy stones hitting on bone tools or horns was described and demonstrated. The lack of fire residue on the site’s roof indicated that fire was possibly carried about on hand-held torches.

End user perspective (study 2 of conceptual framework)

The content requirement study with the teachers as our end users had three target outcomes: (i) a list of learning outcomes addressable by the project, (ii) a list of character behaviors that can be developed into the VR experience to represent intangible cultural heritage from the Neolithic period, and (iii) a list of criteria to evaluate the final experience from which a research instrument, such as a questionnaire, may be developed. To acquire participants for this initial study, we reached out to the Secretariat for Catholic Education in Malta as well as the heads of department of History and Environmental studies, providing an information leaflet explaining the scope of the study, the participation required, and how data will be collected, used and disseminated to be shared with teachers in the area. Four participants responded to our call split into two groups according to their availability (see Figure 2). In line with formal procedural research ethics and participatory ethics, participants were given a consent form allowing anonymized and grouped data to be disseminated as a result of the project (see Datasheet 1 in Supplementary material).

FIGURE 2

Figure 2. Content requirement study with teachers.

Outcomes

The outcomes of study 2 were (i) a list of learning outcomes from History Level 7 for students aged 11 to 12 years that could be addressed by such an IVR experience (see Table 1), (ii) a list of three potential character behaviors to be presented, and (iii) a list of evaluation criteria that could be used to assess the final IVR learning experience (see Table 2). As for the suggested character behaviors these were the painting of the roof and wall spirals in red ochre, the digging of the niches inside the site, and the hypothetical high priest role inspired by the echoic nature of the site (Debertolis et al., 2015; Till, 2017; Wolfe et al., 2020).

TABLE 1

Table 1. Learning outcomes in history level 7.

TABLE 2

Table 2. Suggested evaluation criteria for final IVR learning experience.

Experiment design

Based on the outcomes of the first three studies, the evaluation criteria where used to design the research instrument while the input from the heritage experts and the teachers’ suggestions were used to design the IVR experience itself.

Research instrument design

The research instrument design followed the suggestions gathered in the content requirements study. This will be used to measure the evaluation of the VR experience from the teachers’ perspective (see Datasheet 2 in Supplementary material).

As the dependent variable in the hypothesis statement is an “increase the learners’ ability to achieve learning outcomes that deal with understanding prehistoric life,” the instrument should primarily evaluate whether the experience will help learners increase their ability to achieve the learning outcomes (LOs) identified above.

This evaluation is subjective and depends on the teacher’s experience in teaching the subject, familiarity with the historic site in question, and their exposure to technology. Thus an initial demographic data acquisition section is warranted.

Other evaluation aspects to consider include:

• the authenticity of the experience relative to the historic site;

• the freedom of navigation in the experience;

• the expected impact of the provided information toward the achievement of each LO;

• the expected utility of the experience relative to the history lesson;

Each section now will be dealt with in detail.

Demographics

Since this research was interested in teachers of history or environmental studies teaching 11 to 12 year old students at Level 7, we needed to establish their experience in teaching this level in terms of duration (in years but also per week) and how recent. For purpose of evaluating authenticity, we needed to record whether participants have visited the Neolithic Hypogeum of Ħal Saflieni, and how long ago. Finally, due to the technological medium being used, their experience with technology in and outside their professional role can be measured, in general but also specific to VR technology.

Authenticity

The authenticity of the experience can be measured both in relation to the historic site itself for those who visited the site (as recorded in the demographics section above) and also in terms of a ‘real place’ for those who never visited the site. Such authenticity will be assessed in terms of:

• the visual aspects of the experience (Lighting, Texture, and Animation);

• the aural aspects of the experience (Sound effects, Psychoacoustics, and Ambience).

Navigation

Due to the uneven ground of the hypogeum, the chosen navigation mode is that of a drone-like avatar in order to reduce the sensation of virtual sickness, as developed in a previous project within the department (Grech et al., 2020). This may prove to be unnatural as opposed to bipedal navigation and might have an impact on the authenticity of the experience. Thus its difficulty should be measured as a confusing variable for authenticity. It is expected that technology-savvy individuals will find the navigation controls less challenging than those less exposed to the technology (as measured in the demographics section). Given the existing free navigation mode and the planned guide-driven mode, the research instrument can help measure the impact of the navigation mode on the learner’s experience by asking for their preference of a freer control or more guided control scheme. Items measuring having problems in navigation and getting stuck were negatively scored.

Impact

This is the major criterion of this experiment: how useful is it for the 11 to 12 year old learner’s ability to achieve the identified Level 7 learning outcomes. This can be measured with a generic question on how prepared would students be to answer a relevant quiz after the experience but more specifically, it can be measured using a Likert-scale for each learning outcome.

Utility

Tied to the above evaluation, is the usefulness of this experience for the teachers: whether they would be willing to use the experience as part of their teaching. Moreover, it would be interesting to know whether they would start off with the experience and then have a follow-up lesson, or deliver the main theory during the lesson and then use the experience as a demonstration.

Immersive experience design and development

To match the realism offered by the LIDAR scan of the hypogeum carried out in 2008 during a study commissioned by Heritage Malta and provided to SMI for development into a VR experience, the project underwent substantial improvements over the previous experience in terms of lighting and landscaping of the tangible cultural heritage while a motion capture suit was used for capturing the character movements needed to represent the intangible cultural heritage.

Working upon the brief given by the heritage experts and the recommendations from the teachers’ focus group, the digging behavior imagined to have been carried out in the original excavation of the underground complex from the ground rock was chosen for development (see Figure 3). This was preferred in favor of the holy priest role, because the concept of the holy priest was speculation and not supported by any historical evidence whatsoever.

FIGURE 3

Figure 3. Character animation sequence: (A) brushing, (B) picking up tool, (C,D) digging, (E) wiping sweat off brow, (F) screwing, and (G) resting.

Expert review (study 3)

Having developed an advanced prototype, a second meeting was held with the curators and two other heritage experts who confirmed the direction taken in the experience and suggested improvements to further enhance the experience such as the hairstyles, inspired by those sculpted on figurines found at the site, and a loincloth to be worn by the character. The bone and stone tools used for digging, as well as, the oil lamp receptacles used to light up the space and make the digging process plausible and visible to the virtual visitor, were based upon artifacts found within the complex and acknowledged by the heritage experts.

Framework and prototype enhancement

Ibrahim and Ali’s original conceptual framework was composed of four elements from two sets of features: cultural information design and presentation, and virtual heritage environment navigation and setting (Ibrahim and Ali, 2018). These were followed to further enhance the prototype as follows.

As for information design, the chosen character behavior of digging a niche into the wall provided a significant and relevant piece of information regarding how the site was excavated. The contextual information offered by the experience showed the dim lighting conditions within which such excavations were carried out and the primitive tools with which such hard labor had to be performed. Whilst absent in the experience itself, the setup makes it easier for the history teacher to engage with the students by asking them why they think the niche is being dug, between the paintings room and the echoic chamber. This information was presented via a realistic 3D character animation carrying out the digging process complete with sounds for the brushing of the surface, the pummeling, and the screwing of tools into the wall.

As for navigation, the virtual visitor was placed just inside the middle layer, with a curled up skeleton within view, lit up by torchlight hidden behind a rock column. The sound of digging coming from behind the column serves as an attraction for the visitor’s navigation toward it, revealing the character—whose size was foreshadowed by the aforementioned skeleton—carrying out the digging process. The site’s uneven ground, with large steps and disheveled chambers, made bipedal movement without causing virtual sickness difficult, and thus a drone-like navigation, with direction guided by the visitor’s gaze, was implemented. As for the setting, besides the abovementioned fauna, landscape and lighting effects mentioned, a smoothening of the concrete walkway, captured by the LIDAR scan, was carried out to minimize the appearance of modern interventions.

Evaluation (study 4)

The final stage of the project, where the IVR learning experience was concerned, was the evaluation of the experience by teachers and their managers as suggested in section Learning outcomes. The participants’ enrolment, the experiment setup, and the data gathering through both the questionnaire survey, focus groups and interviews are now described.

Participants

Invitations, including information sheets, were sent out to a group of career guidance teachers, the four teachers involved in Study 2, another four history teachers, and the state’s Education Officer on History. From these, two career guidance teachers, two of the original four teachers and the Education Officer came along, the latter accompanied by the Head of Department of History in the Ministry of Education, the Head of Department of History at a state middle school and a Deputy Chair of Humanities at a church school. The other teachers were unable to attend due to restrictions imposed by COVID-19 related symptoms.

Setup

The participants were informed of the scope of the project, referring to the information sheets they had been given earlier via email and again presented physically on site, and then participated in evaluating the VR experiences, two participants at a time on two separate machines, spending roughly 10 min each.

Questionnaire survey

Right after the experience each participant was led into an adjoining room where a consent form was presented and subsequently signed, following which, each participant completed the questionnaire survey described in section Research instrument design.

Qualitative data gathering

The Education Officer and the Heads of Department and Deputy Chair participated in a focus group led by the project lead as soon as they had completed the survey, discussing clarifications and extra comments on the questionnaire survey were explored and noted. Due to time restrictions from the guidance teachers’ delegation, no qualitative data could be gathered from them, whilst the two teachers involved in the original study gave individual interviews.

Results

The outcomes of the evaluation stage are now presented in terms of the authenticity relative to the historic site, navigation in terms of freedom of movement, amount of information provided, impact in terms of achievable learning outcomes, and utility in the classroom. Whilst detailed results are presented in Data Sheet 3 in the Supplementary Material, box and whiskers charts for each category are presented and described.

Participant demographics

Of the 8 participants, 1 was an Education Officer, 3 were Heads of Department or Deputy Chairs, 2 were teachers, and 2 were guidance teachers. Five were from state education while three from church schools and none from private schools. Three of the participants were currently teaching history, while six of them had 4 or more years of experience teaching history. Half of the participants had visited the Hypogeum site over 4 years ago. Three participants had never used VR but had seen a 360° film while the rest had used VR with only one not having seen a 360° film. VR and technology were equally used both in a teaching context as well as for entertainment.

Instrument reliability

The questionnaire was found to be reliable both across the participants (α = 0.96) and across the questionnaire items (α = 0.79). A reliability analysis by questionnaire section reveals that navigation was less reliable (α = 0.62) when compared to authenticity (α = 0.89), Impact (α = 0.83), and utility (α = 0.89). There was a weak inverse correlation between the participant’s use of VR and having problems navigating (r = 0.55), getting unstuck without help (r = 0.45) and with help (r = 0.3).

Authenticity

The authenticity of the experience was measured as overall quality, in terms of visuals and audio, and then specifically in terms of lighting, textures, characters, sound effects and psychoacoustics, relative to the Hypogeum itself and also in comparison with a generic real site (see Figure 4). Participants who had not visited the site were still prompted to compare the experience in relation to whatever impression they may have of the place, maybe from the history books, photos, or videos.

FIGURE 4

Figure 4. Authenticity.

For the average authenticity across all participants, visual and aural authenticity scored high whilst the overall physical authenticity was lowest rated a result of two of the teachers noting the absence of other sensorial experiences such as smell, ambient sounds like insects and birds, and the sensation of temperature and humidity. While studies suggest that added sensory stimuli enhance the sense of immersion (Jones and Dawkins, 2018), they come with the following caveats: thermoceptive and olfactory senses are secondary to visual and aural sensations; need to be very well controlled or otherwise they break, rather than enhance, immersion; and having such control may require extra hardware that may itself limit immersion—such as a heavier headset. As we seek to deliver such an experience en masse to a class of students, we seek to diminish hardware, rather than increase its complexity, and thus do not foresee a transparent well-controlled implementation of this additional sensory layer in the near future.

Participants who had visited the Ħal Saflieni Hypogeum in the past scored the authenticity of the place higher than those who had not (see Figure 5). The deep echoic nature of the site leaves a deep impression on the visitor, which was not fully addressed in this project as its scope was to focus on the visual animations, thus lower scores here were expected. The aural aspect of the experience is the subject of a separate study.

FIGURE 5

Figure 5. Authenticity by hypogeum visit.

Navigation

Due to the site’s uneven ground, a drone-like navigation feature was provided rather than bipedal movement. There was a general agreement with the naturalness of the navigation, with one outlier who needed to be physically seated during the experience. There were no particular problems with navigation and most participants who got stuck were able to resume navigation without outside help. As for preference between free navigation and a guided tour, there was equal scores. A later discussion revealed that the guided tour was preferred for the 11 to 12 year old students while the free navigation being more attractive for the 13 to 15 year old students (see Figure 6).

FIGURE 6

Figure 6. Navigation ease and preference.

Impact

The most critical aspect of the evaluation was the ability of the experience to help students achieve a set of learning outcomes in the subject of history as identified during the content requirements study. These learning outcomes were chosen from the following sections: (A) Time and Chronology in History (B) Historical Terms and Concepts (C) Working with Historical Sources (D) Historical Interpretation and Empathic understanding and (E) Communicating History. Overall, there was agreement among the participants that the experience sufficiently prepares students for a quiz on the historical site.

For the learning outcomes concerning time and chronology (A7.1–A7.8) and maps and diagrams (C7.5) there was neutral agreement with the experience’s positive impact on their understanding. Participants strongly agreed, however, that the experience helped understand primary historical concepts such as cause and consequence and empathy, measured in outcome B7.3, as well as, describing concrete evidence (C7.2) and distinguishing between primary and secondary sources (C7.3)—bar an outlier (see Figure 7).

FIGURE 7

Figure 7. Learning outcomes set 1.

Interpretation (D7.6) and Empathy (D7.7) attracted strong to moderate agreement with very consistent responses across the participants, as did identification of values and beliefs (D7.8). For the Communication of History set of learning outcomes, there was quite a varied response. Recalling historical facts (E7.1) and events (E7.2, E7.8), communicating with reference to sources (E7.6), and in various forms (E7.9) showed moderate to strong agreement between the participants. Meanwhile, narrating activities (E7.3), use of historical terms in communication (E7.4, E7.7), and giving examples of cause and effects (E7.6) show weak to neutral agreement (see Figure 8).

FIGURE 8

Figure 8. Learning outcomes set 2.

Utility

Another key evaluation is whether teachers find utility in such an experience: how willing are they to adopt this experience as part of their teaching? For the purpose of these questions, the input of the two guidance teachers is not taken into consideration as they left some ratings unanswered due to feeling inadequate to answer them (see Figure 9).

FIGURE 9

Figure 9. Utility of experience to the teacher.

The participants showed an overall moderate to strong agreement to using the experience in the classroom, with the participant who was an outlier in evaluating the learning outcomes being consistent in his critique and being neutral about using it. He later explained that in his opinion the experience lacked signage and information panels that one usually expects to see in a museum or heritage site. There was a consistent consensus that the experience was suitable for Level 7 students of history aged 11 to 12 years. Participants also felt mostly able to manage the use of the technology in the classroom, albeit with some technological assistance. They disagreed that they would only deliver a normal lesson if they had the experience available but were then quite scattered in how they would use it in the classroom. Using the experience first and then follow it up with a lesson attracted the most diverse of answers. Delivering the main theory in a lesson, and then using the experience to reinforce it, left participants mostly undecided but moderate agreement was achieved on giving a minimal lesson as an introduction and then focusing on the experience.

Further input was acquired from the focus group activity held with the heads of department and the confirmatory interviews held with the teachers. A typical lesson plan involving the IVR would be starting off with a preamble, then having the IVR experience with the students, followed by a quiz on the experience involving the relevant learning outcomes, and then an epilogue to wrap up the lesson.

However, concerns of managing a class of 15 to 24 students were raised as well as the costs in purchasing and maintaining such a system, although splitting into groups of 15 and situated into the computer room was suggested as a solution.

Discussion

The research question of the Re-Live History research project was how to go about designing an immersive learning experience for secondary school teachers to teach students about prehistoric cultural heritage on which very little documented evidence is available. Secondary research provided us with two design objectives:

Design objective 1 (technology)

Use IVRs and motion capture suits to deliver intangible aspects of cultural heritage, being guided by heritage experts in order to ensure its proper representation, while keeping expectations of both embodiment and empathy in check.

Design objective 2 (evaluation)

Use an IVR as the delivery of the experience in the first cycle of an experiential learning context, whilst allowing the teacher to facilitate the remaining three cycles of learning outside the IVR experience, including the assessment of attained learning outcomes (chosen by both teachers and their managers), thus helping teachers to embrace and adopt the technology in the classroom.

Based on these design objectives and guided by Ibrahim and Ali’s plan of activities toward building their contextual framework (Ibrahim and Ali, 2018), an IVR experience was designed and developed as part of our primary research with heritage experts, history teachers, and their heads of department. An evaluation of the IVR experience measured via a questionnaire and a subsequent discussion has provided the following key findings:

Focusing our efforts on presenting a realistic rendering of a Neolithic character in the process of digging out a niche out of the hypogeum attracted high authenticity scores in terms of visuals and to a certain extent sounds—even though minimal efforts were carried out in contextualizing the latter as they were beyond the scope of this project. Another participant compared the project with its predecessor (Barbara et al., 2020) and suggested that the realism of this version offered an immersive experience that would be very attractive to mature students aged 14 to 16 years while the previous project, with its information panels and narrations, offered a more informative visitor mode suitable for 11 to 12 year old students. Visitors to the actual hypogeum appreciated the fidelity of the representation more than those who did not, even when comparing to a generic physical site, suggesting the utility of such an experience even in conjunction with a visit to the site itself, as it does not detract from its value.

The feedback on the navigation mechanism confirms the positive feedback received in the previous project (Barbara et al., 2020) and encourages the reuse of the same system in future projects, unless the VR visitor is assigned a human avatar. The free vs. guided navigation preference showed potential for both, with the teachers’ recommendation of using the guided tour for the 11 to 12 year old students and the freer mode for the more mature 14 to 16 year old students. In order to accommodate the teacher-led instruction of design objective 2, such a guided tour ought to be in the control of the teacher. This is supported by Rogers et al. who claim that using “full-face and body motion capture can make social interaction in VR very similar to face-to-face interaction” (Rogers et al., 2022) which is very pertinent in a post-pandemic learning experience that has suffered from lack of face-to-face teaching in view of health restrictions (Basar et al., 2021).

With respect to impact, it was pleasantly surprising to hear the deputy chair of humanities recommending the use of this IVR learning experience for subjects other than history. She suggested its use in teaching various subjects such as the Self and Community in Environmental Studies, about Digital Literacy by showing how VR can be used to teach, and for the topic of Estimates and Approximations in Mathematics. Such use of the experience beyond the history syllabus would further increase the return on investment made on the technology, training, and effort spent in implementing such an experience in a school.

As for learning outcomes, it was expected that the Time and Chronology outcomes would not score highly in the evaluation as the project only presented a specific period and not a range of eras that could show the evolution of the site as it was dug up, say. However, it scored quite high in the empathy outcome even if we didn’t follow Milk’s ‘ultimate empathy machine’ (Milk, 2015) and its interpretation as of having the VR visitor fit into the target persona’s body and suffer their toils and troubles. Rather, we placed the visitor into the same time and space as the Neolithic man as he was engaged in digging the site, pausing to wipe his brow from sweat and resting on his haunches every now and then to show the effort such work needed. No maps or diagrams were offered, as these too were seen as immersion-breakers, and so the low scores for the working with historical sources were expected. We did score highly on the description of concrete evidence, however, and this was expected since the students would be visually shown what it looked like to be living in those times—unlike the verbal descriptions accompanying photos of artifacts given by the curators in the video used in history class in recent years.

Interpretation also scored highly with the participants, as a learning outcome that could be helped by the IVR learning experience. Whilst the experience does not prompt the students to interpret what they are seeing, the teachers were well able to identify its potential for class discussion, enabling questions such as what is he doing? What is he using? Why is he doing it? Of what value are the niches in the hypogeum to him? What could they be used for? What did he believe in? The visual representation of this intangible cultural heritage also allowed students to understand such cultures not just by reading off textbooks and looking at pictures, or looking at acted filmic presentations but be, at least virtually, present inside the site and witness the actual activity in the best alternative experience they can get. They can be prompted to explain what they are seeing, in words or in writing, without regurgitating written descriptions or narrated actions.

The user-centric design and development, bringing the heritage experts early on in the project and also to assess our prototype, as well as involving the teachers into the choices made and the evaluation criteria to use were all meant to facilitate the adoption of such an IVR learning experience in class. Thus it was satisfying to receive a positive response to its potential use in class and to have teachers already planning how to use the experience as part of their lesson delivery. While one or two participants commented on the lack of guidance within the experience, others saw this as an opportunity for them to make it their own, and choose how to use it and in what context—even in subjects outside history. Having specific elements inside an experience will assist, yes, in delivering that specific learning outcome but will also diminish its ability to be used elsewhere. By minimizing the specifics inside the experience gives the teacher more control over its use and thus increasing its impact on the lesson, as suggested in the literature (Feng et al., 2021).

The final key finding is linked to utility: teachers are happy to provide an introductory explanation and then let the students explore the IVR learning experience about the Hypogeum’s both tangible and intangible cultural heritage. This would cover the first cycle of Experiential learning (Kolb, 1984). The flexibility offered by the experience’s diegetic, and thus fully immersive, nature allows the teacher to direct the students toward any specific topic they would like to address: be it architectural features, tools used, geology, habitat, etc. An assessment, be it a quiz or presentation, presents the second cycle of Experiential learning that is reflection, followed by an epilogue that wraps up and reaffirms the learning outcomes as the third cycle of learning. The fourth cycle could then be delivered as homework in which students apply what has been learned to other sites and situations.

Shortcomings and limitations

Re-Live History is a 1-year long project that carried out a teacher-driven design and development of an immersive learning experience of prehistoric intangible cultural heritage. The availability of history teachers for the end-user perspective (Study 2) was one of the shortcomings of the project, with factors including resistance to the technology and expected effort (Hussin et al., 2011) as well as skepticism about its utility (Radianti et al., 2020).

Another limitation was the portrayal of a single character behavior, that of digging. Other behaviors, such as the painting of the spirals on the wall, would have provided further context and study subjects but its implementation was hampered by a 4-month delay in the delivery of the motion capture suit due to worldwide logistics and chip supply problems⁴.

Another limitation is empirical evidence of students’ experiences. A longer longitudinal study would complement the teachers’ evaluation with putting the IVR experience to the task and test the achievement of specific learning outcomes by students using the experience as against those following traditional learning procedures. For this to be possible, a method of mass participation needs to be developed and experimented, because it is far from adequate to have each student in a class participating singularly especially if it is to be made accessible to schools without state-of-the-art technology and students from low-income families (Southgate et al., 2019). A research direction would be to involve all the class in a single teacher-led IVR experience.

A further limitation of the project’s utility is the powerful hardware needed to run the experience that may be less affordable for schools. A solution could be the provision of a 360° video shot within the virtual environment that takes the viewer along an established route throughout the site whilst allowing them to look around them along the journey. This is viewable on personal mobile phones—as long as they have a gyroscope and accelerometer and a cheap mobile-housing headset—thus providing the guided tour feature requested by teachers for Level 7 students aged 11 to 12 years⁵.

Project’s contribution

Recent studies have shown that the lack of studies on the use of IVR in younger students presents an area of growth potential particularly through the use of explorative interaction (Checa and Bustillo, 2020). This lack of studies was due to technological limitations, financial accessibility, health and safety issues, as well as lack of experience in the use of the technology by teachers and students (Stavroulia et al., 2019).

This project contributes by addressing the teaching of intangible cultural heritage to 11 to 12 year old students at Level 7 through the use of an IVR learning experience that was custom built for the purpose of learning, rather than by repurposing off-the-shelf entertainment products which favor game objectives over learning outcomes (Champion, 2015). By using the History level 7 learning outcomes as our starting point, we sought to develop an IVR experience that focused on assisting teachers in delivering content that helps 11 to 12 year old students attain these learning outcomes. Through its non-prescriptive format—by not introducing text that would bias interpretation—it provides the flexibility of usage at any point in the lesson as the teacher deems fit. Response from assessors shows its applicability to subjects even outside the topic of history, such as Environmental Studies, Digital Literacy, and Mathematics.

The project also contributes in its evaluation methodology: not only was the experience measured in terms of usability (navigation and utility) but also in terms of its potential ability to help 11 to 12 year old students achieve learning outcomes specific to the subject of History at Level 7 (Radianti et al., 2020). These were evaluated in terms of authenticity and impact and by using a wider spectrum of assessors that includes both teachers, heads of department, and education officers, to compensate for the criticism of their subjective interpretation (Hussey and Smith, 2002).

With respect to IVR in Learning of Cultural Heritage, this project contributes by shifting its emphasis away from the representation of tangible cultural heritage to that of intangible cultural heritage. Ch’ng et al. (2018) had remarked on how preservation and accessibility to tangible heritage has been the staple of 20 years relationship between cultural heritage and VR, and thus called for the provision of the ‘experiential aspect of cultural heritage.’ By realistically representing a Neolithic character performing digging of the Ħal Saflieni Hypogeum, our project aims to provide an experience of intangible cultural heritage with the added advantage that it supports Experiential learning (Kolb, 1984).

Conclusion

The teaching of history has always been limited by the primary sources’ restricted accessibility: be it geographical or due to conservation. Students’ exposure to such sites has been made possible through the written word, the drawn diagrams, the published photo or the produced video—always framed within some medium. The immersive experience offered by VR systems can bypass this remediated form and allow the student to feel present on site, surrounded by its physical and cultural presence.

With Re-Live History, the student is now also witness to the prehistoric culture as they watch the Neolithic man engaged in the digging of the Ħal Saflieni Hypogeum. Re-Live History is a learning tool, designed and developed together with heritage experts and history teachers that can be used in the classroom to help students achieve a number of learning outcomes related to prehistory. Further empirical research comparing traditional pedagogy with IVR learning among students would provide further evidence for, or against, the use of IVRs in learning. Future research may well explore other forms of intangible heritage that may be thus represented, from other periods of history, and may then test whether providing interaction with such characters can help students better achieve more learning outcomes.

Saint Martin’s Institute Creative Research Team

• Charles Theuma (Project coordinator).

• JB (Project lead).

• Jeremy Grech (Unity developer).

• Joseph Camilleri (Artistic director).

• Silvio McGurk (Hardware procurement and training).

• Kluivert Bonello (Motion capture suit actor).

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.

Author contributions

JB: the main author of the article. The members of the SMI Creative Research Team were given specific sections to review and modify according to their contribution to the project, as follows: CT: sections Educational aspects, Evaluation, and Results. JG: sections Technological aspects, Experiment design, and framework prototype enhancement. JC: sections Cultural heritage aspects, Content requirement study, and Methods. SM and KB: sections Realism.

Funding

The Re-Live History Project (REP-2021-014) was partially funded by Malta Council for Science and Technology (MCST) under the Research Excellence Programme.

Acknowledgments

The SMI Creative Research Team would like to thank the guidance and support provided by Heritage Malta, the Secretariat of Catholic Education, Malta, and the Ministry of Education in Malta.

Conflict of interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2022.1032108/full#supplementary-material

Footnotes

1. ^ https://curriculum.gov.mt/en/new_syllabi/Documents/Year_07_08/History_Learning_Outcomes_Levels_7_to_10_Sept_2018.pdf
2. ^ https://stmartins.edu/lovemycountry/assets/apps/HypogeumLiteCardboardv3.apk
3. ^ https://www.youtube.com/watch?v=TQpvMWG9fws
4. ^ https://techmonitor.ai/technology/chip-shortage-why-global
5. ^ Said 360° video is available at https://www.youtube.com/watch?v=W8LyCTMDSHE.

References

Adam, S. (2006). An Introduction to Learning Outcomes. Princeton, NJ: Citeseer.

Google Scholar

Aiello, P., D’elia, F., Di Tore, S., and Sibilio, M. (2012). A constructivist approach to virtual reality for experiential learning. E-learn. Digital Med. 9, 317–324. doi: 10.1016/j.nedt.2022.105518

PubMed Abstract | CrossRef Full Text | Google Scholar

Asad, M. M., Naz Ansari, A., Churi, P., and Mehdi Tahanzadeh, M. (2021). Virtual reality as pedagogical tool to enhance experiential learning: a systematic literature review. Educ. Res. Int. 2021:7061623. doi: 10.1155/2021/7061623

CrossRef Full Text | Google Scholar

Barbara, J. (2020). “Classification of gameplay interaction in digital cultural heritage,” in Proceedings of the Digital Games Research Association Conference

Google Scholar

Barbara, J., Grech, J., Camilleri, J., McGurk, S., and Theuma, C. (2020). “Digitizing the neolithic hypogeum,” in Proceedings of the Euro-Mediterranean Conference (Berlin: Springer), 703–710. doi: 10.1007/978-3-030-73043-7_62

CrossRef Full Text | Google Scholar

Barbara, J., Koenitz, H., and Bakk, ÁK. (2021). “The ethics of virtual reality interactive digital narratives in cultural heritage,” in Proceedings of the International Conference on Interactive Digital Storytelling (Cham: Springer). doi: 10.1007/978-3-030-92300-6_27

CrossRef Full Text | Google Scholar

Basar, Z. M., Mansor, A. N., Jamaludin, K. A., and Alias, B. S. (2021). The effectiveness and challenges of online learning for secondary school students-a case study. Asian J. University Educ. 17, 119–129. doi: 10.24191/ajue.v17i3.14514

CrossRef Full Text | Google Scholar

Biocca, F., and Delaney, B. (1995). Immersive virtual reality technology. Commun. Age Virtual Real. 15, 10–5555.

Google Scholar

Cardona, D., and Zammit, M. (2018). “Shaping spaces for the dead: an analysis of tool marks at the Ħal saflieni hypogeum, Malta,” in The lure of the antique: essays on malta and mediterranean archaeology in honour of anthony bonanno, eds N. C. Vella, A. J. Frendo, and H. C. R. Vella (Leuven: Peeters Publishers). doi: 10.2307/j.ctv1q26t57.14

CrossRef Full Text | Google Scholar

Casu, A., Spano, L. D., Sorrentino, F., and Scateni, R. (2015). “RiftArt: bringing masterpieces in the classroom through immersive virtual reality,” in Proceedings of the STAG (Italy: University of Cagliari).

Google Scholar

Champion, E. (2015). “Roleplaying and rituals for cultural heritage-orientated games,” in DiGRA’15-Proceedings of the 2015 DiGRA International Conference, Digital Games Research Association. Digital Games Research Association DiGRA (Lüneburg, Germany).

Google Scholar

Checa, D., and Bustillo, A. (2020). A review of immersive virtual reality serious games to enhance learning and training. Multimedia Tools Appl. 79, 5501–5527. doi: 10.1007/s11042-019-08348-8349

CrossRef Full Text | Google Scholar

Ch’ng, E., Cai, Y., and Thwaites, H. (2018). Special Issue on VR for culture and heritage: the experience of cultural heritage with virtual reality: guest editors introduction. PRESENCE: Teleoperators Virtual Environ. 26, iii–vi. doi: 10.1162/pres_e_00302

CrossRef Full Text | Google Scholar

Daniela, L., and Aierken, Y. (2020). “The educational perspective on virtual reality experiences of cultural heritage,” in New Perspectives on Virtual and Augmented Reality: Finding New Ways to Teach in aTransformed Learning Environment, ed. L. Daniela (London: Routledge). doi: 10.4324/9781003001874

PubMed Abstract | CrossRef Full Text | Google Scholar

Debertolis, P., Coimbra, F., and Eneix, L. (2015). Archaeoacoustic analysis of the Ħal saflieni hypogeum in Malta. J. Anthropol. Archaeol. 3, 59–79.

Google Scholar

Egea-Vivancos, A., and Arias-Ferrer, L. (2021). Principles for the design of a history and heritage game based on the evaluation of immersive virtual reality video games. E-Learn. Digital Media 18, 383–402. doi: 10.1177/2042753020980103

CrossRef Full Text | Google Scholar

Eleftheria, C. A., Charikleia, P., Gkanas Iason, C., Tsakalidis, A., and Tsolis, D. (2013). “An innovative augmented reality educational platform using gamification to enhance lifelong learning and cultural education,” in IISA 2013, (Piscataway, NJ: IEEE). doi: 10.1109/IISA.2013.6623724

CrossRef Full Text | Google Scholar

Experius VR, and Curiosity Stream (2018). Nefertari : Journey to Eternity. Available online at: https://store.steampowered.com/app/861400/Nefertari_Journey_to_Eternity/ (accessed October 10, 2022).

Google Scholar

Feng, Q., Luo, H., Li, W., Chen, Y., and Zhang, J. (2021). The moderating effect of debriefing on learning outcomes of IVR-Based instruction: an experimental research. Appl. Sci. 11:10426. doi: 10.3390/app112110426

CrossRef Full Text | Google Scholar

Freina, L., and Ott, M. (2015). “A literature review on immersive virtual reality in education: state of the art and perspectives,” in Proceedings of the International Scientific Conference Elearning and Software for Education (Bucharest). doi: 10.1016/j.nedt.2021.104868

PubMed Abstract | CrossRef Full Text | Google Scholar

Grech, J., Bugeja, M., and Seychell, D. (2020). “A case study into the user experience of an application of virtual reality at the saint paul’s catacombs, Malta,” in Rediscovering heritage through technology, eds D. Seychell and A. Dingli (Cham: Springer). doi: 10.1007/978-3-030-36107-5_8

CrossRef Full Text | Google Scholar

Hodgson, P., Lee, V. W. Y., Chan, J. C. S., Fong, A., Tang, C. S. Y., Chan, L., et al. (2019). “Immersive virtual reality (IVR) in higher education: development and implementation,” in Augmented Reality and Virtual Reality. Progress in IS, eds M. tom Dieck and T. Jung (Cham: Springer). doi: 10.1007/978-3-030-06246-0_12

CrossRef Full Text | Google Scholar

Hsiao, S. C. (2021). Effects of the application of virtual reality to experiential education on self-efficacy and learning motivation of social workers. Front. Psychol. 12:770481. doi: 10.3389/fpsyg.2021.770481

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, Y.-C., Backman, S., Backman, K. F., McGuire, F. A., and Moore, D. (2019). An investigation of motivation and experience in virtual learning environments: a self-determination theory. Educ. Inform. Technol. 24, 591–611. doi: 10.1007/s10639-018-9784-5

CrossRef Full Text | Google Scholar

Huang, Y.-C., and Han, S. R. (2014). “An immersive virtual reality museum via second life,” in Proceedings of the International Conference on Human-Computer Interaction (Berlin: Springer), 579–584. doi: 10.1007/978-3-319-07857-1_102

CrossRef Full Text | Google Scholar

Hussey, T., and Smith, P. (2002). The trouble with learning outcomes. Act. Learn. Higher Educ. 3, 220–233. doi: 10.1177/1469787402003003003

CrossRef Full Text | Google Scholar

Hussin, N. H., Jaafar, J., and Downe, A. G. (2011). “Assessing educators acceptance of Virtual Reality (VR) in the classroom using the Unified Theory of Acceptance and Use of Technology (UTAUT),” in Proceedings of the International Visual Informatics Conference (Berlin: Springer), 216–225. doi: 10.1007/978-3-642-25191-7_21

CrossRef Full Text | Google Scholar

Ibrahim, N., and Ali, N. M. (2018). A conceptual framework for designing virtual heritage environment for cultural learning. J. Comp. Cultural Heritage (JOCCH) 11, 1–27. doi: 10.1145/3117801

CrossRef Full Text | Google Scholar

Jarmon, L., Traphagan, T., Mayrath, M., and Trivedi, A. (2009). Virtual world teaching, experiential learning, and assessment: an interdisciplinary communication course in Second Life. Comp. Educ. 53, 169–182. doi: 10.1016/j.compedu.2009.01.010

CrossRef Full Text | Google Scholar

Jones, S., and Dawkins, S. (2018). “The sensorama revisited: evaluating the application of multi-sensory input on the sense of presence in 360-degree immersive film in virtual reality,” in Augmented Reality and Virtual Reality, eds M. tom Dieck and T. Jung (Berlin: Springer). doi: 10.1007/978-3-319-64027-3_13

CrossRef Full Text | Google Scholar

Kaplan-Rakowski, R., and Wojdynski, T. (2018). “Students attitudes toward high-immersion virtual reality assisted language learning,” in Future-proof CALL: Language learning as exploration and encounters–short papers from EUROCALL, eds P. Taalas, J. Jalkanen, L. Bradley, and S. y (Dublin: Research-publishing.net).

Google Scholar

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.

Google Scholar

Le, Q. T., Pedro, A., and Park, C. S. (2014). A social virtual reality based construction safety education system for experiential learning. J. Intell. Robotic Systems 79, 487–506. doi: 10.1007/s10846-014-0112-z

CrossRef Full Text | Google Scholar

Linden Research, Inc (2003). Second life. Available online at: http://secondlife.com/ (accessed October 10, 2022).

Google Scholar

Madary, M., and Metzinger, T. K. (2016). Recommendations for good scientific practice and the consumers of VR-Technology. Front. Robotics and AI 3:3. doi: 10.3389/frobt.2016.00003

CrossRef Full Text | Google Scholar

Manjrekar, S., Sandilya, S., Bhosale, D., Kanchi, S., Pitkar, A., and Gondhalekar, M. (2014). “CAVE: an emerging immersive technology-a review,” in Proceedings of the 2014 UKSim-AMSS 16th International Conference on Computer Modelling and Simulation (Piscataway, NJ: IEEE). doi: 10.1109/UKSim.2014.20

CrossRef Full Text | Google Scholar

Malegiannaki, I., and Daradoumis, T. (2017). Analyzing the educational design, use and effect of spatial games for cultural heritage: A literature review. Comput. Educ. 108, 1–10.

Google Scholar

Martín-Gutiérrez, J., Mora, C. E., Añorbe-Díaz, B., and González-Marrero, A. (2017). Virtual technologies trends in education. Eur. J. Mathematics Sci. Technol. Educ. 13, 469–486. doi: 10.12973/eurasia.2017.00626a

CrossRef Full Text | Google Scholar

Meier, S. (1991). Civilization. Hunt Valley, MD: Microprose.

Google Scholar

Microsoft (1997). Age of Empires. Redmond, DC: Microsoft.

Google Scholar

Milk, C. (2015). How Virtual Reality Can Create the Ultimate Empathy Machine. Available online at: https://www.ted.com/talks/chris_milk_how_virtual_reality_can_create_the_ultimate_ empathy_machine (accessed January 30, 2021).

Google Scholar

Nash, K. (2018). Virtual reality witness: exploring the ethics of mediated presence. Stud. Doc. Film 12, 119–131. doi: 10.1080/17503280.2017.1340796

CrossRef Full Text | Google Scholar

Ochs, C., and Sonderegger, A. (2022). The interplay between presence and learning. Front. Virtual Real. 3:742509. doi: 10.3389/frvir.2022.742509

CrossRef Full Text | Google Scholar

Radianti, J., Majchrzak, T. A., Fromm, J., and Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comp. Educ. 147:103778. doi: 10.1016/j.compedu.2019.103778

CrossRef Full Text | Google Scholar

Rogers, S. L., Broadbent, R., Brown, J., Fraser, A., Speelman, C. P., et al. (2022). Realistic motion avatars are the future for social interaction in virtual reality. Front. Virtual Real. 163:750729. doi: 10.3389/frvir.2021.750729

CrossRef Full Text | Google Scholar

Rouse, R. (2021). “Against the instrumentalization of empathy: immersive technologies and social change,” in Augmented and mixed reality for communities, ed. J. Fisher (Boca Raton, FL: CRC Press). doi: 10.1201/9781003052838-2

CrossRef Full Text | Google Scholar

Slater, M., and Sanchez-Vives, M. V. (2014). Transcending the self in immersive virtual reality. Computer 47, 24–30. doi: 10.1109/MC.2014.198

CrossRef Full Text | Google Scholar

Smutny, P., Babiuch, M., and Foltynek, P. (2019). “A review of the virtual reality applications in education and training,” in Proceedings of the 2019 20th International Carpathian Control Conference (ICCC) (Piscataway, NJ: IEEE). doi: 10.1109/CarpathianCC.2019.8765930

CrossRef Full Text | Google Scholar

Southgate, E., Smith, S. P., Cividino, C., Saxby, S., Kilham, J., Eather, G., et al. (2019). Embedding immersive virtual reality in classrooms: ethical, organisational and educational lessons in bridging research and practice. Int. J. Child-Comp. Interact. 19, 19–29. doi: 10.1016/j.ijcci.2018.10.002

CrossRef Full Text | Google Scholar

Stavroulia, K.-E., Christofi, M., Zarraonandia, T., Michael-Grigoriou, D., Lanitis, A., et al. (2019). “Virtual reality environments (VREs) for training and learning,” in Learning in a Digital World. Smart Computing and Intelligence, eds P. Díaz, A. Ioannou, K. Bhagat, and J. Spector (Springer: Singapore). doi: 10.1109/ICORR.2019.8779420

PubMed Abstract | CrossRef Full Text | Google Scholar

Tanant, J., Google Arts Culture, and Atlas V (2020). Chauvet: The Dawn of Art. Available online at: https://experiments.withgoogle.com/chauvet (accessed October 10, 2022).

Google Scholar

Taylor, T. (2003). Historical Simulations and the Future of the Historical Narrative. Ann Arbor, MI: MPublishing, University of Michigan Library.

Google Scholar

Till, R. (2017). An archaeoacoustic study of the Ħal Saflieni Hypogeum on Malta. Antiquity. 91, 74–89. doi: 10.15184/aqy.2016.258

CrossRef Full Text | Google Scholar

Ubisoft Montreal (2007). Assassin’s Creed [PC]. France: Ubisoft.

Google Scholar

UNESCO (1972). Convention Concerning the Protection of the World Cultural and Natural Heritage. Paris: UNESCO.

Google Scholar

UNESCO (2003). Text of the Convention for the Safeguarding of the Intangible Cultural Heritage. Paris: UNESCO.

Google Scholar

UNESCO (2004). Proceedings of the International Conference on the Safeguarding of Tangible and Intangible Cultural Heritage: Towards an Integrated Approach, Accessed October 20-23, 2004. Japan: UNESCO.

Google Scholar

Van Mechelen, M., Baykal, G. E., Dindler, C., Eriksson, E., and Iversen, O. S. (2020). “18 years of ethics in child-computer interaction research: a systematic literature review,” in Proceedings of the Interaction Design and Children Conference (London). doi: 10.1145/3392063.3394407

CrossRef Full Text | Google Scholar

Velev, D., and Zlateva, P. (2017). Virtual reality challenges in education and training. Int. J. Learn. Teach. 3, 33–37. doi: 10.18178/ijlt.3.1.33-37

CrossRef Full Text | Google Scholar

Wolfe, K., Swanson, D., and Till, R. (2020). The frequency spectrum and geometry of the Ħal Saflieni Hypogeum appear tuned. J. Archaeol. Sci. Rep. 34:102623. doi: 10.1016/j.jasrep.2020.102623

CrossRef Full Text | Google Scholar