A Virtual Tour of a Hardly Accessible Archaeological Site: The Effect of Immersive Virtual Reality on User Experience, Learning and Attitude Change

Introduction

People around the world visit archaeological and historical sites mainly for pleasure or to learn about the place as such and its history, or even the combination of both, in which case it is called educational tourism. Important archaeological sites around the world are not always easily accessed by everyone due to the remote location of the site itself or mobility difficulties of possible visitors. With the current advances in technology, such places can be accessed remotely through interactive multimedia applications. With ubiquitous computing evolving and devices, such as VR headsets allowing more immersive experiences compared to traditional desktop computing, virtual heritage is gaining increasing interest. According to Roussou (2002), a well-designed virtual heritage application aiming to benefit public education should visualize and provide “access” to sites and places that no longer exist, are geographically remote, or are unreachable.

The archaeological site of Choirokoitia (pronounced “Khirokitia”), a Neolithic settlement in Cyprus, is the site of interest in this work. The site was selected carefully in order to meet all the criteria we would like to address. These include the following:

(i) It is a site that is not easily accessible by everyone: the physical terrain of Choirokoitia's settlement is rough and non-uniform, with a lot of changes in elevations, and it is located on a natural steep-sloped hill, making access to people with mobility difficulties (e.g. wheelchairs) impossible, according to the official website [The Deputy Ministry of Tourism (2019)].

(ii) It is a site that attracts educational interest: the country's Ministry of Education and Culture organizes educational visits for primary school students since learning material on this specific settlement is included in the history curriculum in public schools. However, the visits are possible only for students in nearby cities. The latter adds an extra dimension to the limited access to the settlement as listed above in (i), in this case, due to geographical restrictions.

(iii) It is one of the most visited tourist attractions in Cyprus: Choirokoitia is an archaeological site with ruins dated back to 7000BC and is ranked among the top oldest ruins in the world. It was also listed by UNESCO as a United Nations Educational, Scientific and Cultural Organization World Heritage Site (1998), giving additional motivation for tourists to visit it.

The aim of this work is two-fold:

(i) Enable virtual access to the archaeological site of Choirokoitia that can be beneficial for people with disabilities and for those separated from it by geographical and political boundaries.

(ii) Suggest the most appropriate technology to use according to the user's aims, which may be related to learning or to user experience.

The first goal was addressed with the development of two versions of the application, one for VR systems and the other for personal computers. The second goal has been addressed by performing a feasibility study comparing the two different versions and assessing them on a number of different aspects, namely, participants' sense of presence, level of user experience, change in attitudes toward the archaeology of Cyprus, and learning performance.

Although several studies have been conducted on Virtual Heritage applications the topic needs to be further addressed since there have been no clear conclusions. A previous study (Christofi et al., 2018) conducted specifically for Choirokoitia was limited to assessing a VR application integrating its virtual reconstruction, and it lacked a comparison between different types of technologies. Also, some other limitations of the study, such as the small number of participants and the motion sickness experienced by a substantial percentage of them, made the results less robust.

In another study, Tost and Economou (2009) investigated the suitability of immersive VR for learning about archaeology and the past in cultural heritage settings at the Hellenic Cosmos (the exhibition center of the Foundation of the Hellenic World in Athens) but again lacked in comparison with other technologies as to its effectiveness. The results of their study demonstrated that each exhibit supported a different kind of learning, and the exhibits were considered suitable for obtaining a global idea of spatial details. In a study by Michael et al. (2010), different museum exhibits, one traditional and five interactive Information and Communication Technologies exhibits, were compared with the main emphasis being on the assessment of user experience (UX). Their results showed that the Information and Communication Technologies exhibits were rated higher than the traditional exhibit. In another comparative study, Wrzesien and Raya (2010) compared the E-Junior application, a serious virtual world (SVW) for teaching children natural science and ecology, with a traditional type of class. With regard to learning effectiveness, the results did not present statistically significant differences between the two groups. However, students from the virtual group reported enjoying the class more, being more engaged, and having greater intentions to participate than students from the traditional group. Similarly, a study by Zaharias et al. (2013) investigated the user experience (UX) and learning effectiveness of the “Walls of Nicosia,” a 3D multi-touch table, as compared to a traditional approach where a group of students took a guided tour throughout the museum and learned about the walls of Nicosia through printed maps exhibited at the museum. Results showed no statistically significant differences in the learning performance, but the virtual group reported user experience at significantly higher levels. Finally, Loizides et al. (2014), presented two museums using an immersive head-mounted display (HMD)-based technology and a stereoscopic Powerwall and evaluated the users' overall experience. The results of the user evaluation revealed that both ways of presenting the museum received an equal usability score among the users.

In the next section, we describe the two versions (VR and Desktop) that were developed of an interactive application representing the Choirokoitia settlement and the experimental study that was conducted to evaluate their feasibility. The analysis of the collected data and results are then described, while a discussion elaborating on findings, describing the present study's limitations, and outlining potential future work follows.

Materials and Methods

Materials

We used a three-dimensional (3D) digital reconstruction of the archaeological site of Choirokoitia (Figures 1, 2), which was created based on real photographs taken on the site and highly accurate modeling of the geometry. We also recorded a set of audio clips with historical information about Choirokoitia. Using the 3D reconstruction and the recorded audio, we developed a new version of an existing Interactive VR Application described in Christofi et al. (2018). Additionally, we developed an Interactive Desktop Application in such a way so as to provide exactly the same information with a VR Application in order to compare the medium. The two applications, the VR Application and the Desktop Application, that have been developed and evaluated through the feasibility study are described below.

FIGURE 1

Figure 1. Views from the 3D reconstruction of the archaeological site of Choirokoitia where information points are shown. The virtual reconstruction visualizes the archaeological site as it stands today, including the real reconstructed houses located near the settlement (left) and the ancient ruins (right) of the settlement.

FIGURE 2

Figure 2. Views from the 3D reconstruction of the archaeological site of Choirokoitia, as shown in the VR Application (top left and bottom left) and the Desktop Application (top right and bottom right).

VR Application

The VR Application used in this study is an improved version of the previously developed VR Choirokoitia application. The new version integrates a recent technique developed to minimize the dizziness that was occurring to a great extent to the participants, as is stated in the relevant publication (Fernandes and Feiner, 2016). This technique requires a dynamic reduction of the field of view (FoV) depending on the user's movement and rotation in the virtual environment. Specifically, the FoV was gradually decreased, using a vignetting effect (Figure 3) as the user's navigation (movement and rotation) speed increased, and it increased as the user's navigation speed decreased. The change in the FoV was slow and gradual, so as not to distract the user. Note that the dynamic reduction of the FoV only takes into account the navigation using the video game controller, that causes the dizziness, and not the movement and rotation applied using the HMD's head tracking. To further improve the previous version of the VR Application, we also minimized the intervention with the participants during the time that they were immersed in the VR Application. In this new version, we gave them all of the instructions through the application and achieved their familiarization with the application's functionality by having a training session at the very beginning of the experience. The training phase took place in a virtual space outside the virtual archaeological site, in a neutral scene, to avoid receiving information about Choirokoitia before the main application began.

FIGURE 3

Figure 3. Participants using the VR Application (left) and Desktop Application (right).

In the main VR Application, participants were free to navigate through the 3D reconstruction of Choirokoitia using a video game controller. The application required that the participants put on a VR HMD (Figure 4, left), which offered a first-person stereoscopic view of the environment and the ability to physically change the looking direction with head rotations. While navigating, participants listened, through the HMD headsets' headphones, to eight audio recordings, in total, containing information related to the archaeological site. The audio clips lasted ~30 s each. The recordings were triggered by collision with 8 information points, visualized by floating 3D icons (Figures 1, 2, top left; Figure 2, bottom left) that were located at various parts of the virtual archaeological site. The information presented in each of those recordings was related to the specific area the participant was exploring at that time.

FIGURE 4

Figure 4. Example of dynamic reduction of the field of view in the VR Application.

Desktop Application

In order to test the impact of VR, we designed a non-immersive Desktop Application as a control. It provides the user with the same visual and audio information but without the immersive features of the VR technology (3D stereo projection, free navigation, free head movements based on head tracking). The Desktop Application was designed in such a way that it would be easy for a person with basic computer skills to develop, in contrast to the VR Application, which requires advanced skills to develop, such as programming skills. The Desktop Application was developed using 2D rendered images based on the same 3D reconstruction of the site as was used in the VR Application instead of real photographs of the archaeological site, so as to not provide different visual content. The 2D rendered images were created in regard to specific points of view selected in such a way so as to have the rendering of areas that included all eight points of interest, where information points existed, as well as areas along the route between any two consecutive information points (Figure 2). Similar to the VR Application, in order to minimize the intervention with the user while conducting the study, we provided all the instructions through digital material at the beginning of the Desktop Application.

Interactivity was achieved with the mouse button (Figure 2, top right; Figure 2, bottom right). The participants were able to move to the next rendering of a virtual area by clicking on a button that was placed on the bottom right corner of the screen. Information points were triggered by clicking on corresponding small icons similar to those in the VR group. The participants in the Desktop group received identical audio recorded information as those in the VR group while they were looking at the rendering from the corresponding viewpoint.

Technical Setup

The 3D reconstruction of Choirokoitia was created in Autodesk Maya 2015 and was textured using as reference the original photographs taken from a visit to the archaeological site. The reconstruction was done with high precision and accuracy based on photos and maps and included not only the archaeological buildings but also the surrounding area as it stands today (Figure 1). The stereo audio recordings were edited in Adobe Audition CS6. The VR Application was developed with Unity 2017.1.2 software, and the virtual environment was displayed through an Oculus Rift CV HMD. This has two 1,080 × 1,200 pixel OLEDs per eye at a 90-Hz display refresh rate, coupled with a positional tracker and built-in headphones. A video game controller was used for navigation inside the virtual space. The interactive Desktop Application was created using Microsoft PowerPoint. We used screenshots of renders of 3D reconstructions within the Unity software in order to achieve a visual resemblance to the VR Application. The application was displayed to participants through a 1,920 × 1,080 pixel 15.6-inch computer screen. A computer mouse was used for interaction with the application.

Experimental Design

The study had a between-groups design. Participants were randomly assigned to one of two experimental groups: the VR group or the Desktop group. The VR Application was used by the participants assigned to the VR group, and the Interactive Desktop Application was used by the participants in the Desktop group.

Ethics Statement

All participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individuals for the publication of any potentially identifiable images or data included in this article.

Participants

Forty adults (N = 40), male and female participants, aged 20–54 years (mean ± SD age 26.65 ± 7.53), participated in the study. After signing an informed consent form, participants were randomly allocated to either the VR or the Desktop group.

Participants had no or little prior experience with VR technology. Table 1 presents relevant descriptive data for each condition, the total number of participants, mean ages, median and IQR values for experience in VR (1 = 0, 2 =“ <1,” 3 =“1–2, … 5 =“>5”), and frequency of previous visits to archaeological sites. Codes refer to a 1–5 Likert scale, where 1 indicates least agreement and 5 most agreement with the statement.

TABLE 1

Table 1. Experimental design and distribution of participants by condition.

Measurements

Participants were assessed on their sense of presence, level of user experience, attitudes toward the archaeology of Cyprus, and learning performance through questionnaires.

Presence

The sense of presence was measured with a three-item questionnaire (Table 2a) based on the presence questionnaire developed by Slater et al. (1994). Presence can be characterized as “the illusion of ‘being there' in the environment depicted by the VR displays” (Slater and Sanchez-Vives, 2016). Presence was rated on a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). This questionnaire was given to participants right after their experience.

TABLE 2

Table 2. The questionnaires given to participants.

User Experience

In order to assess the user experience, we designed a four-item questionnaire (Table 2b) evaluating the overall experience, the image and audio quality, and whether participants would be willing to try a similar experience in the future. Participants rated their experience on a Likert scale ranging from 1 (negative) to 5 (positive). This questionnaire was also given to the participants right after their experience.

Attitudes

A four-item questionnaire (Table 2c) on a Likert scale ranging from 1 (negative) to 5 (positive) was designed to measure participants' attitudes toward the archaeology of Cyprus. This is interpreted as the intention to acquire further knowledge about Choirokoitia or to visit this or a different archaeological site in the future. This questionnaire was given to the participants before their exposure (preAttitudes) and again right after their exposure (postAttitudes). The variable of interest was the Change in Attitude (dAttitudes = postAttitudes – preAttitudes), where positive values indicate an increase in participants' intention to learn more about the archaeology of Cyprus or visit archaeological sites and negative values a decrease.

Learning Performance

Participants' learning about the archaeological site was assessed with a 10-statement multiple-choice questionnaire that evaluated participants' knowledge of the archaeological settlement of Choirokoitia (e.g., “What was the shape of the houses in the archaeological site?,” “What was the average life expectancy of the inhabitants of the archaeological site?,” etc.). These questions were based on the information that participants had been given (audio clips) while using the applications. Each correct answer received 1 point, while wrong answers received 0 points. The knowledge test was administered two times, one prior to participants' virtual exposure (preScore) and the second immediately after their virtual exposure (postScore). The response variable of interest Learning Performance was the difference between the two (dScore = postScore – preScore), which shows the degree of improvement (positive values) or decline (negative values) in score after exposure.

Procedure

Upon arriving at the laboratory, and after completing the consent form, participants were randomly assigned to the VR or the Desktop group and were asked to complete the Knowledge and Attitudes questionnaires alongside some demographic questions. Once this process was completed, participants in the VR group were asked to put on the VR HMD (Figure 4, left), and participants in the Desktop group were asked to sit in front of the computer screen (Figure 4, right). Then, both groups were instructed to follow the respective tutorial instructions in order to become familiar with the use of the corresponding system and application. Once the tutorial was completed (with an approximate duration of 5 min), the main experimental session began, which lasted ~8 min for both groups. After completing their navigation within virtual Choirokoitia, the participants of both groups again completed the Knowledge and Attitudes questionnaires and also filled in the User Experience and Presence questionnaires.

Statistical Analysis

For the questionnaire data on Presence, User Experience, and Attitudes (Table 2), factor analysis was carried out to reduce the number of questionnaire variables. This method also has the advantage of transforming ordinal variables to continuous ones. Corresponding factor scores were used, and the interpretation of each factor was identified. For factor analysis on ordinal variables, such as in our case, Polychoric PCA analysis can be used as a test (this treats ordinal variables as if they were derived from cut-offs sampled from a normally distributed variable) (Olsson, 1979) and scores derived from those, so this approach was employed. Box plots of all raw questionnaire scores are provided in the Supplementary Material. Subsequent analysis of the derived variables was done using independent samples t-test. All results were obtained with Stata 13 software.

Results

Factor Analysis of the Questionnaires

A single factor was retained in the case of Presence (Table 2a), and the factor loadings on the scoring variable (Presence) are shown in Table 3. In the interpretation of the factor loadings, these capture the amount of overall variance in the observed variables. The scoring coefficients are the coefficients of the equations describing the factor scores in terms of a linear combination of the original variables. In Table 3, F1 is explained by There, Reality, and Place (Table 2a), which is reflected in the corresponding scoring coefficients for Presence. The latter interprets this factor as “the illusion of being at the archaeological site of Choirokoitia instead of in the physical laboratory.”

TABLE 3

Table 3. Factor analysis for presence resulting in a single factor F1, and the scoring coefficients for the factor score Presence.

The User Experience questions (Table 2b) resulted in a single factor, and the factor loadings on the scoring variable UserExperience are shown in Table 4. This factor is explained by Pleasure, Image, Audio, and Repeat, which is interpreted as “the level of pleasantness of the experience.”

TABLE 4

Table 4. Factor analysis for user experience resulting in a single factor F1, and the scoring coefficients for the factor score UserExperience.

The attitudes questionnaire was administered before and after the participants' experience. Factor analysis on the questions (Table 2c) resulted in a single factor, and the factor loadings on the scoring variables preAttitudes and postAttitudes, respectively, are shown in Tables 5, 6. The factors are explained by VisitSites, VisitChoirokoitia, LearnArchaeology, and LearnChoirokoitia (Table 2c) and are interpreted as participants' “interest in acquiring new historical knowledge in the future.”

TABLE 5

Table 5. Factor analysis for participants' attitudes before the experience resulting in a single factor F1, and the scoring coefficients for the factor score preAttitudes.

TABLE 6

Table 6. Factor analysis for participants' attitudes after the experience resulting in a single factor F1, and the scoring coefficients for the factor score postAttitudes.

Presence Analysis

Figure 5 shows the bar chart of the means and standard errors of the derived factor analysis variable Presence. Participants in the VR condition felt a stronger sense of “being” at the archaeological site of Choirokoitia (0.59 ± 0.098) compared to participants in the Desktop condition (−0.59 ± 0.148). An independent samples t-test showed that the above differences are significant [t₍₃₈₎ = −4.712, p = 0.000].

FIGURE 5

Figure 5. Bar chart showing means and standard errors for the factor score Presence by condition.

User Experience Analysis

Figure 6 shows the bar chart of the means and standard errors of the derived factor score UserExperience. Participants in the VR condition rated the overall experience of using the application more positively (0.394 ± 0.172) compared to participants in the Desktop condition (−0.394 ± 0.238). An independent samples t-test showed that the above differences are significant [t₍₃₈₎ = −2.6809, p= 0.0108].

FIGURE 6

Figure 6. Bar chart showing means and standard errors for the factor score UserExperience by condition.

Attitudes Analysis

We investigated whether participants' attitudes toward archaeology were affected by the use of the VR and Desktop Applications. The variable of interest, dAttitudes = postAttitudes – preAttitudes, reflects the degree of change in attitudes regarding acquiring new knowledge in the future (positive values). Figure 7 shows the bar chart of the means and standard errors of dAttitudes, derived as the difference from the factor scores preAttitudes and postAttitudes. Although it can be seen that there is a slight positive change for the VR condition (0.013 ± 0.102) and a negative change for the Desktop condition (−0.013 ± 0.141), the differences between the two groups are not significant [t₍₃₈₎ = −0.151, p = 0.88].

FIGURE 7

Figure 7. Bar chart showing mean and standard error of dAttitudes by condition.

Learning Performance Analysis

The response variable of interest here is dScore = postScore – preScore, showing the degree of improvement (positive values) or decline (negative values) in score. The score is defined as the total number of correct responses to the knowledge questionnaire before and after the exposure. Figure 8 shows that the mean change in dScore was greater in the Desktop than in the VR condition. The means and standard errors are 2.5 ± 0.336 and 1.35 ± 0.386, respectively, with Cohen's d = 0.71, which is a medium to large effect size. An independent samples t-test showed a significant difference between the means t₍₃₈₎ = 2.247, p = 0.031. The residual errors are compatible with normality (Shapiro-Wilk P = 0.35).

FIGURE 8

Figure 8. Bar chart showing mean and standard error of dScore by condition.

Correlations

Finally, we performed a Pearson correlation analysis in order to explore relationships between the data collected. The correlations for the Desktop and the VR condition are summarized in Supplementary Tables 1, 2, respectively.

The sense of Presence strongly correlated with User Experience in both the Desktop [r₍₁₈₎ = 0.83, p < 0.001] and VR [r₍₁₈₎ = 0.836, p < 0.001] conditions. This result means that participants who reported a higher sense of Presence in the virtual environment, also reported a higher level of UserExperience (how positively they rated the overall experience of using the application).

Presence is correlated with preScore in the Desktop condition [r₍₁₈₎ = 0.61, p < 0.004]. Also, in the Desktop condition, Presence correlated with preAttitudes [r₍₁₈₎ = 0.561, p = 0.01] and postAttitudes [r₍₁₈₎ = 0.619, p = 0.004]. UserExperience and preScore were correlated in the Desktop condition [r₍₁₈₎ = 0.606, p = 0.005]. Finally, UserExperience correlated with preAttitudes [r₍₁₈₎ = 0.491, p = 0.028] and postAttitudes [r(18) = 0.569, p = 0.009] in the VR condition.

Discussion

The aims of this study were (i) to enable virtual access to the archaeological site of Choirokoitia and (ii) to examine the feasibility of the use of the immersive interactive VR application developed in comparison to an application that can be easily developed by a person with basic computer skills. For this reason, we virtually reconstructed the archaeological site of Choirokoitia and developed and compared two applications, an immersive VR Application and a Desktop Application. This comparison was focused on four main aspects, namely, participants' sense of presence, user experience, attitudes toward archaeology, and learning performance regarding historical facts of Choirokoitia, in order for the results to suggest the most appropriate technology to utilize according to a user's aims, which may be related to learning or to user experience.

First, we found that the VR Application has a critical advantage in terms of the sense of presence delivered to participants, which refers to the sense of “being there” in the virtual world (here, the virtual Choirokoitia) (Slater and Sanchez-Vives, 2016). The VR Application succeeded in giving participants the feeling of virtually visiting the archaeological site of Choirokoitia, more so than the Desktop Application, and it was also found that the overall user experience was significantly better in terms of image and audio quality, overall pleasantness, and willingness to try a similar application in the future. This is in line with previous research and proposed models for complex conceptual learning (Slater and Wilbur, 1997; Salzman et al., 1999; Lee et al., 2010; McMahan, 2013). The latter suggests that immersive environments create a strong sense of presence due to the very powerful emotional impact they can have, which then leads to higher engagement in the experience, motivation, and cognitive processing of the material.

However, our results demonstrated no advantage of the VR Application regarding participants' attitudes toward archaeology. The use of the VR Application did not seem to impact participants' motivation to acquire new knowledge in the future more than the Desktop Application. Furthermore, the VR Application was found to be less effective in acquiring and memorizing new information about the archaeological site. Interestingly, participants in the Desktop group gained more knowledge than the participants in the VR group. Previous studies have shown that low-immersion simulations, such as computer games and other desktop applications, can result in better cognitive performance and attitudes toward learning (Bonde et al., 2014; Clark et al., 2016; Makransky et al., 2016; Thisgaard and Makransky, 2017). However, research evidence as to whether high-immersive virtual reality applications lead to increased motivational outcomes and learning is still controversial. In the study described in Passig et al. (2016), the authors showed that teaching in immersive VR environments contributed to students' cognitive modifiability more than traditional learning experiences. Similar results were reported by Alhalabi (2016) when using a VR system to enhance students' education in engineering, and by Webster (2016) when aiming to improve learning on basic corrosion prevention and control in military personnel. In contrast, other studies have yielded negative results when comparing learning in immersive VR environments and desktop applications. In one example, Moreno and Mayer (2002) investigated how desktop VR and immersive VR, the second using an HMD and navigation techniques, compared with multimedia learning material. It was found that the two media did not affect students' performance differently, with equal improvement results for both. Similarly, Richards and Taylor (2015) found that biology students' knowledge did not improve after exposing them to virtual simulations with 3D models more than with simulations using two-dimensional (2D) models. The authors concluded that this could have occurred due to the additional cognitive load imposed by the 3D models. Likewise, in previous research comparing an immersive VR human anatomy application with traditional slide presentations, it was found that although both methods increased participants' performance, this was higher for participants in the slide-presentation group (Michael-Grigoriou et al., 2017). Our present results are in line with the aforementioned studies, showcasing that indeed, immersive VR technologies can have an overwhelming impact on participants, leading to diminished attention and a lower increase in their learning performance.

According to the cognitive load theory (CLT), if one engages in excessive amounts of extraneous processing (i.e., cognitive processing that does not support the goal, caused for instance by distractions), then there is not an adequate capacity for critical processing and thus meaningful learning outcomes (Sweller, 1994). Additionally, Thisgaard and Makransky (2017) suggests that highly immersive environments might not necessarily result in higher learning and transfer outcomes due to their highly hedonic or utilitarian nature. According to this theory, fun or pleasurable experiences within immersive VR environments can lead users to disregard their instrumental value, and instead, concentrate on the entertainment value such systems offer.

Based on the above and in line with previous studies, we speculate that something similar happened in the present study, with participants in the VR group spending more time exploring and navigating through the new (to them) immersive environment, thus leading to poor attention to other stimuli and consequently, worse performance in the knowledge test. This has been previously supported in the context of immersive virtual gaming apps, where the authors explain that navigation through virtual rooms can draw attention away from the main task and negatively influence players' learning (Freina and Canessa, 2015). Further, and according to Van der Heijden (2004), we could argue that participants focused more on enjoying the environment rather than on learning the material presented to them. We do not know, however, how performance would have been affected if the participants had had more experience in using VR environments. In our study, all participants had little or no prior knowledge of VR systems, and therefore, this point cannot be addressed here, but we leave it as an open question for further research. Also, regarding motivation, it should be noted that participants in both experimental groups reported positive attitudes toward archaeology even before their virtual exposure (see Supplementary Figure 3), which could possibly explain why there were no differences in attitudes after the exposure. Further, attitudes and motivation were based on self-reported questionnaires, which are oftentimes subject to social desirability bias (Yu et al., 2018). We propose that future studies should look at alternative and more indirect ways of measuring motivation, such as, for example, follow-up forum access and participation related to activities on the given subject (either on the same day or in the long-term).

A limitation of our study is that there are differences not only in the technology used in the two groups (VR vs. Desktop) but also in the features of the applications as such. For example, the VR Application offers the user the possibility to freely navigate in a 3D environment, while the Desktop version provides the visual information in 2D renders. While this simplification on the Desktop Application was intensional, as it allows its development by people with basic computer skills, it does not allow us to safely attribute the differences we detected between the VR group and Desktop group either to the type of the technology used (immersive or not) or to the application as such. A future study where exactly the same application is used, visualized through a VR system or a Desktop computer, will allow a conclusion on this aspect.

This VR Application of Choirokoitia attempted to provide the best possible representation of the archaeological site, giving the user an experience that simulates a physical visit. The reconstruction used in this paper represents the current state of the archaeological site. Achieving this goal can make the site (at least virtually) accessible, overcoming mobility or geographical constraints. But it only exploits a part of the possibilities of VR technologies. VR can overcome the boundaries of reality, time, or space. For instance, users could experience “being” in the Choirokoitia of the Neolithic period, when the settlement was inhabited. Such an application would give the user visual information on how the settlement looked and the way of life and occupations of the inhabitants. This paper has examined whether VR can enhance and promote archaeological content in comparison to a non-immersive medium. Although VR seems to have an advantage in terms of Presence and User Experience, regarding Learning Performance, new ways to convey information to the users should be explored. According to Tost and Economou (2009) and Roussou (2002), in order to be suitable for learning, cultural heritage virtual worlds should not just be visually represented in a photorealistic manner; they need to be complete, interactive and present the virtual world in a meaningful and engaging way. In our application, the information that the knowledge test was based on was conveyed using audio. It is possible, that the participants in the VR group, overwhelmed by the technology, paid less attention to the audio information, and this might explain their lower increase in learning performance. Therefore, in future studies, a Neolithic-period Choirokoitia application in which the information would be rendered mainly visually could overcome the limitation of knowledge transfer.

Data Availability Statement

All datasets generated for this study are included in the article/Supplementary Material.

Ethics Statement

The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

CK and MC made a substantial contribution to the design of the study, the development of the VR Application, the data acquisition, the analysis and interpretation of data, and the drafting of the article. DM-G contributed substantially to the conception and design of the study, the interpretation of the data, and the drafting and critical revision of the manuscript and supervised and coordinated all of the steps of the study. DB contributed substantially to the analysis and interpretation of the data for the work and to the drafting and critical revision of the manuscript. AI contributed to the interpretation of the data for the work and revising the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This project has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No. 739578 and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to acknowledge Zoe Anastasiadou, Maria Michaelidou, Ioanna Papamichael, and Katerina Pieri, students of the course MGA444: Virtual Reality of the Department of Multimedia and Graphic Arts of the Cyprus University of Technology for sharing with us the original version of the Choirokoitia VR Application, which was developed under the supervision of the course's instructor, DM-G. We would also like to thank Christos Hadjipanayi and Maria Matsangidou, research associates with the Research Centre on Interactive Media Smart Systems and Emerging Technologies for proofreading and partially reviewing some sections of the article, respectively.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomp.2020.00023/full#supplementary-material

References

Alhalabi, W. (2016). Virtual reality systems enhance students' achievements in engineering education. Behav. Inform. Technol. 35, 919–925. doi: 10.1080/0144929X.2016.1212931

CrossRef Full Text | Google Scholar

Bonde, M. T., Makransky, G., Wandall, J., Larsen, M. V., Morsing, M., Jarmer, H., et al. (2014). Improving biotech education through gamified laboratory simulations. Nat. Biotechnol. 32, 694–697. doi: 10.1038/nbt.2955

PubMed Abstract | CrossRef Full Text | Google Scholar

Christofi, M., Kyrlitsias, C., Michael-Grigoriou, D., Anastasiadou, Z., Michaelidou, M., Papamichael, I., et al. (2018). “A tour in the archaeological site of Choirokoitia using virtual reality: a learning performance and interest generation assessment,” in Advances in Digital Cultural Heritage (Cham: Springer), 208–217.

Google Scholar

Clark, D. B., Tanner-Smith, E. E., and Killingsworth, S. S. (2016). Digital games, design, and learning: a systematic review and meta-analysis. Rev. Educ. Res. 86, 79–122. doi: 10.3102/0034654315582065

PubMed Abstract | CrossRef Full Text | Google Scholar

Fernandes, A. S., and Feiner, S. K. (2016). “Combating VR sickness through subtle dynamic field-of-view modification,” in 2016 IEEE Symposium on 3D User Interfaces (3DUI) (Greenville, SC: IEEE), 201–210. doi: 10.1109/3DUI.2016.7460053

CrossRef Full Text | Google Scholar

Freina, L., and Canessa, A. (2015). “Immersive vs desktop virtual reality in game based learning,” in European Conference on Games Based Learning, Steinkjer, Norway (Reading: Academic Conferences International Limited), 195.

Google Scholar

Lee, E. A. L., Wong, K. W., and Fung, C. C. (2010). How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Comput. Educ. 55, 1424–1442. doi: 10.1016/j.compedu.2010.06.006

CrossRef Full Text | Google Scholar

Loizides, F., El Kater, A., Terlikas, C., Lanitis, A., and Michael, D. (2014). “Presenting cypriot cultural heritage in virtual reality: a user evaluation,” in Euro-Mediterranean Conference (Cham: Springer), 572–579.

Google Scholar

Makransky, G., Thisgaard, M. W., and Gadegaard, H. (2016). Virtual simulations as preparation for lab exercises: assessing learning of key laboratory skills in microbiology and improvement of essential non-cognitive skills. PLoS ONE 11:e0155895. doi: 10.1371/journal.pone.0155895

PubMed Abstract | CrossRef Full Text | Google Scholar

McMahan, A. (2013). “Immersion, engagement, and presence: a method for analyzing 3-D video games.” in The Video Game Theory Reader (New York, NY: Routledge), 89–108.

Google Scholar

Michael, D., Pelekanos, N., Chrysanthou, I., Zaharias, P., Hadjigavriel, L. L., and Chrysanthou, Y. (2010). “Comparative study of interactive systems in a museum,” in Euro-Mediterranean Conference (Berlin; Heidelberg: Springer), 250–261.

Google Scholar

Michael-Grigoriou, D., Yiannakou, P., and Christofi, M. (2017). “Intuitive interaction for exploring human anatomy in a VR setup,” in 2017 23rd International Conference on Virtual System and Multimedia (VSMM) (Dublin: IEEE), 1–4.

Google Scholar

Moreno, R., and Mayer, R. E. (2002). Learning science in virtual reality multimedia environments: role of methods and media. J. Educ. Psychol. 94:598. doi: 10.1037/0022-0663.94.3.598

CrossRef Full Text | Google Scholar

Olsson, U. (1979). Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika 44, 443–460. doi: 10.1007/BF02296207

CrossRef Full Text | Google Scholar

Passig, D., Tzuriel, D., and Eshel-Kedmi, G. (2016). Improving children's cognitive modifiability by dynamic assessment in 3D Immersive Virtual Reality environments. Comput. Educ. 95, 296–308. doi: 10.1016/j.compedu.2016.01.009

CrossRef Full Text | Google Scholar

Richards, D., and Taylor, M. (2015). A Comparison of learning gains when using a 2D simulation tool versus a 3D virtual world: an experiment to find the right representation involving the Marginal Value Theorem. Comput. Educ. 86, 157–171. doi: 10.1016/j.compedu.2015.03.009

CrossRef Full Text | Google Scholar

Roussou, M. (2002). Virtual heritage: from the research lab to the broad public. Bar Int. Ser. 1075, 93–100.

Google Scholar

Salzman, M. C., Dede, C., Loftin, R. B., and Chen, J. (1999). A model for understanding how virtual reality aids complex conceptual learning. Pres. Teleoperat. Virtual Environ. 8, 293–316. doi: 10.1162/105474699566242

CrossRef Full Text | Google Scholar

Slater, M., and Sanchez-Vives, M. V. (2016). Enhancing our lives with immersive virtual reality. Front. Robot. AI. 3:74. doi: 10.3389/frobt.2016.00074

CrossRef Full Text | Google Scholar

Slater, M., Usoh, M., and Steed, A. (1994). Depth of presence in virtual environments. Pres. Teleoperat. Virtual Environ. 3, 130–144. doi: 10.1162/pres.1994.3.2.130

CrossRef Full Text | Google Scholar

Slater, M., and Wilbur, S. (1997). A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Pres. Teleoperat. Virtual Environ. 6, 603–616. doi: 10.1162/pres.1997.6.6.603

CrossRef Full Text | Google Scholar

Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learn. Instruct. 4, 295–312. doi: 10.1016/0959-4752(94)90003-5

CrossRef Full Text | Google Scholar

The Deputy Ministry of Tourism. (2019). Choirokoitia Archaeological Site (Neolithic Settlement). Available online at: www.visitcyprus.com/index.php/en/discovercyprus/rural/sites-monuments/item/757-choirokoitia-archaeological-site-neolithic-settlement

Google Scholar

Thisgaard, M., and Makransky, G. (2017). Virtual learning simulations in high school: effects on cognitive and non-cognitive outcomes and implications on the development of STEM academic and career choice. Front. Psychol. 8:805. doi: 10.3389/fpsyg.2017.00805

PubMed Abstract | CrossRef Full Text | Google Scholar

Tost, L. P., and Economou, M. (2009). Worth a thousand words? The usefulness of immersive virtual reality for learning in cultural heritage settings. Int. J. Architect. Comput. 7, 157–176. doi: 10.1260/147807709788549367

CrossRef Full Text | Google Scholar

United Nations Educational Scientific and Cultural Organization World Heritage Site. (1998). Choirokoitia. Available online at: whc.unesco.org/en/list/848

Google Scholar

Van der Heijden, H. (2004). User acceptance of hedonic information systems. MIS Quarterly 28, 695–704. doi: 10.2307/25148660

CrossRef Full Text | Google Scholar

Webster, R. (2016). Declarative knowledge acquisition in immersive virtual learning environments. Interact. Learn. Environ. 24, 1319–1333. doi: 10.1080/10494820.2014.994533

CrossRef Full Text | Google Scholar

Wrzesien, M., and Raya, M. A. (2010). Learning in serious virtual worlds: evaluation of learning effectiveness and appeal to students in the E-Junior project. Comput. Educ. 55, 178–187. doi: 10.1016/j.compedu.2010.01.003

CrossRef Full Text | Google Scholar

Yu, P. L., Shen, C., and Smith, G. S. (2018). Relevance and Application of Heritage in Contemporary Society (New York, NY: Routledge). doi: 10.4324/9780203702277

CrossRef Full Text | Google Scholar

Zaharias, P., Michael, D., and Chrysanthou, Y. (2013). Learning through multi-touch interfaces in museum exhibits: an empirical investigation. J. Educ. Technol. Soc. 16, 374–384. Available online at: www.jstor.org/stable/jeductechsoci.16.3.374

Google Scholar