Job relevance or perceived usefulness? What features of immersive virtual reality software predict intention to use in a future project-based-learning scenario: a mixed method approach

1 Introduction

1.1 Decentral work and learning

Home office, international collaboration, and COVID-19–decentralized working and learning methods are becoming increasingly important for various reasons. Despite many advantages, collaborative work can be challenging due to temporal and spatial separation. Different software companies have recognized these challenges and are trying to remedy them with collaboration tools such as Skype, Microsoft Teams, and Cisco Webex in decentralized teams. A newer type of technical support used in this context is immersive Virtual Reality or iVR.

1.2 iVR in operational use

The worldwide turnover of iVR is constantly increasing. The market research company Super Data (2020) expects a turnover of 5.1 billion US dollars in 2023, almost double that of 2018, and a turnover of 2.6 billion US dollars at that time (Super Data Staff, 2020). This increase will be seen not only in the entertainment industry, which the broad masses see as the main field of application (Kunst, 2019). A study by Cohen et al. (2018) confirms the work and training context trend. Out of 700 automotive, manufacturing, and utilities leaders, 50% will experiment with iVR by 2023, and 82% of the companies assume that iVR will have a permanent place in their organization by 2023. However, in this working environment, the focus will be on repair, maintenance, and design tasks. The increased sense of presence and immersion that iVR enables could make collaborative work on projects more accessible and efficient (Marston and McClenaghan, 2013). For this purpose, it makes sense to determine the exact user needs. Grimm et al. (2013) point out that the most important selection criterion is not implementing what is technically possible but how well the functions fit the user’s needs. When it comes to the needs of students at universities of applied sciences, tools are needed that support student project work (Arbenz, 2016; Chu et al., 2017; Wróblewska and Okraszewska, 2020). This learning paradigm is explained in the next section.

1.3 Project-based learning

Project-based learning (PjBL) is a systematic teaching method involving students at different universities in complex and realistic practical tasks. This usually leads to a product or a presentation in front of an audience. In the process, students can acquire knowledge and life-enhancing skills (Thomas, Mergendoller, and Michaelson, 1999; Barron and Darling-Hammond, 2008). In the literature, the mean weighted effect size of project-based learning on student academic achievement is 0.71 (Chen and Yang, 2019). This indicates that project-based learning has a medium to significant positive effect compared to traditional teaching. Supporting this effective form of learning using innovative and user-oriented iVR learning technology seems promising since previous learning successes in an iVR setting provide a mixed picture. However, no studies on needs are known yet regarding features of iVR software that support project work in the student’s everyday life.

1.4 Learning in iVR

When it comes to learning success, the results of the effect of iVR in learning scenarios are mixed (Checa and Bustillo, 2020; Di Natale et al., 2020; Radianti et al., 2020; Pellas et al., 2020; Wu et al., 2020; Hamilton et al., 2021; Huang et al., 2021; Coban et al., 2022; Rojas-Sánchez et al., 2023). For example, in their meta-analysis, Wu et al. (2020) analyzed 35 experimental or quasi-experimental studies that addressed the effects of iVR on learning. Despite a positive, albeit small, effect size of ES = 0.24 of iVR on learning outcomes, 34% of the studies showed a negative or no effect. Thus, the advantage of iVR via a head-mounted display (HMD, “VR glasses”) should not be overestimated based on the results, and a look should be taken at possible influencing factors. In the review by Zhang and Song (2022), the impact of sensory cues on immersive experiences to promote technology-enhanced sustainable behavior was investigated. The authors mention the following technological design factors: Dimension, style (e.g., realism), perspective, color, haptics, the sound of cues, music, or dialogue. Regarding important factors for persuasion (scene, story), the authors see authority, sympathy, commitment, reciprocity, and scarcity as essential implementation factors. Sugand et al. (2015) observed significant learning effects in medical training (including faster task-solving and reduction of error rate) in the training of surgeons. Similarly, Webster (2016) found tremendous learning success with user-oriented design and didactically derived success criteria in iVR compared to a lecture (with the support of PowerPoint) condition. Likewise, children between 6 and 8 years of age benefited most from iVR in spatial allocation tasks (Passig et al., 2016), and engineering students benefited most from iVR compared to control and other iVR conditions (Alhalabi, 2016). Older persons also seem to have advantages in memory performance through iVR (Lokka and Çöltekin, 2019). In Phé et al. (2017), no effects were observed after iVR training in novices or experienced physicians. Only non-specialists showed improvement over time. Våpenstad et al. (2017) and Makransky et al. (2019) showed the opposite effects of iVR. In summary, studies on training in IVR show that: 1. positive outcome of learning in iVR is mixed. 2. The user’s/student’s needs (regarding the iVR environment) were not evaluated separately. As previously mentioned by Grimm et al. (2013), it is essential to determine how well the functions fit the user’s needs. This fit can be described in perceived usefulness and job relevance. Both concepts are part of the Technical Acceptance Model (TAM-2) described in the next section.

1.5 Technical Acceptance model (TAM-2)

The TAM model has been applied to many technologies for higher education to predict their intention to use, e.g., smartphones (Ahmed et al., 2017) and social media (Al-Rahmi et al., 2014). The origin of the TAM-2 model is based on the model of the theory of reasoned action (Ajzen and Fishbein, 1980). The Theory of Deliberate Action by Fishbein and Ajzen (1980) is one of the most frequently used theories of behavioral prediction. The TAM model consists of several components. These include Perceived Usefulness (PU), Perceived Ease of Use, Attitude toward Using and Actual System Use, and Design Features. According to the TAM model, a potential user’s overall attitude toward using a particular system has a decisive influence on whether or not he uses the system. The attitude towards use is influenced by perceived usefulness and perceived usability. Design features influence perceived usefulness and perceived usability (Davis, 1986). The TAM was extended by Venkatesh and Davis (2000) to include Voluntariness, Experience, Subjective Norm, Image, Job Relevance (JR), Output Quality, and Result Demonstrability. By adding the variable Job Relevance, the TAM-2 model provides a suitable basis for the required work relevance, which is essential for project-based learning (Chen and Yang, 2019). At the same time, general (cross-technical) relationships between PU, JR, and Intention to Use (ITU) are given. Whether these relationships also apply to software features in VR applications in the area of project-based learning will be examined in this study. For this, a heuristic derivation from the original TAM- 2 model was made and is shown in Figure 1.

FIGURE 1

FIGURE 1. TAM 2 model.

1.6 Research questions

Summarizing the above literature, the following prerequisites for the acceptance and later use of iVR applications in the area of iVR-supported, decentralized project-based learning are necessary: The iVR application must be developed in a user-oriented way (Grimm et al., 2013), which corresponds to the TAM 2 variable PU. The iVR application must also be relevant to the work or project. Otherwise, no learning takes place (Chen and Yang, 2019). Likewise considered in TAM 2 with the variable Job Relevance. Since no study has examined the relationship between iVR software features for project-based learning and the later intention of use, we postulate the following research questions: 1) Which (planned) software features do today’s iVR applications have for professional applications? 2) What are the most favored software features in the student sample, and how are they related to the TAM -2 derived structure (JP, PU, ItU)? The operationalization of these research questions and the research design are presented in the next chapter.

2 Methods

We developed an exploratory sequential mixed methods design to investigate these research questions above. This design combines qualitative and quantitative data collection and analysis in phases (Creswell and Plano Clark, 2018). We chose the following qual->quant->QUANT Sequence (see Figure 2): 1. Market analysis and Categorization of iVR Software Features 2. Expert ranking of importance regarding project management characteristics and development of questionnaire 3. Survey Study.

FIGURE 2

FIGURE 2. Mixed methods design.

2.1 Market analysis and categorization

The qualitative sequence aimed to get an overview of the existing iVR applications’ features. The conducted market analysis should reveal possible gaps and, most importantly, already existing tools and features. We analyzed 34 iVR applications, which (2020) summarized regarding working with distance, education, training, and design. For this purpose, the applications were divided into two groups of 17 applications each. Three raters concentrated on the first half, the other three on the second half. In the first step, the features were analyzed individually, followed by a second step, where all individual ratings were consolidated into an inventory of all features. These were categorized according to similarity and then grouped into superordinate categories. The resulting 88 features are presented in Table 1 and serve as a basis for further analysis.

TABLE 1

TABLE 1. List of the 88 features.

2.2 Expert ranking and design of questionnaire

With an expert ranking, the TAM 2 questionnaire was limited to five features to minimize aborts due to a too-long questionnaire. Within the expert ranking, two experts independently selected five of the 88 features they classified as necessary (overview in Table 2). Subsequently, the consistency of the prioritization was evaluated using Cohen’s Kappa. We obtained a Kappa value of 0.128 [κ ≤ 0.1 no agreement; 0.1 < κ ≤ 0.4 weak agreement; 0.4 < κ ≤ 0.6 explicit agreement; 0.6 < κ ≤ 0.8 strong agreement; 0.8 < κ ≤ 1 complete agreement (Landis and Koch 1977)]. Since the Kappa value indicated a weak match, further discussion with the experts took place to determine the desired number of features jointly. The five final features using the TAM-2 questions were mind maps, customizability of digital space, sharing of computer applications, speech-to-text, and recording function. To be able to make additional statements about desired features by potential users, a list of 18 other features was included in the questionnaire. For this purpose, all research group members prioritized 18 from the 88 features to compile an extended selection, as seen in Table 3. The questions on JR, PU, and ItU from TAM-2 in the questionnaire first had to be translated into German. To ensure the professionalism and correctness of the translation, it was checked by an interpreter. The items were answered on a seven-level Likert scale (1 = do not agree at all, up to agree = 7 fully). The TAM-2 items on ItU, PU, and JR are shown in Figure 3 and used in five questionnaire chapters. The word “system” was substituted by one of the selected features (mind maps, customizability of digital space, sharing of computer applications, voice-to-text annotations, and recording function). Each feature was described verbally and illustrated with a symbol picture.

TABLE 2

TABLE 2. Final selection of the rating process.

TABLE 3

TABLE 3. Selection of an additional 18 Items.

FIGURE 3

FIGURE 3. Used items.

2.3 Survey study

2.3.1 General approach

At the beginning of the survey, a short introduction video explained the topic and the basic features of VR. In addition to the feature-related questions, the following demographic data were collected: Age, gender, field of study, and duration of studies. Also, previous experience with VR and project management was collected to enable further evaluations. The respondents could complete the questionnaire on all Internet-capable end devices with web browsers (PC, smartphone, tablet).

2.3.2 Pre-test and sample acquisition

When the questionnaire design was finished, a pre-test was conducted to show possible errors and ambiguities. Twelve people completed the pre-test. Due to the pre-test, some spelling mistakes were corrected, and the wording was adjusted. Subsequently, a comparison with the research team took place once again. The final survey was sent via the mail distribution list of the University of Applied Sciences Northwestern Switzerland and the University of Bern. The aim was to obtain at least a sufficiently high N for the regression calculations (see Jenkins and Quintana-Ascencio, 2020) and, above all, to reach students at the universities contacted.

2.3.3 Detail procedure

After opening the link to the online questionnaire, the participants first reached a welcome page. Afterward, the declaration of consent had to be accepted to start the questionnaire. The instruction video was shown to get started; then, the demographic data was recorded. Afterward, the participants were asked about their experience in project management and VR. This was followed by the TAM-2 questions on PU, ItU, and JR with the five developed features. Then, the participants had to choose five from a list of 18 additional features they considered indispensable for a VR project management tool.

2.3.4 Data cleansing

The statistical software SPSS (versions 25 & 26) and Excel 2016 were used to evaluate the quantitative data. The participants’ data was cumulated and processed after the survey was completed. The data of 107 participants were imported from Unipark into SPSS. A total of four participants were excluded.

2.3.5 Statistics

First, the data were checked over a large area for their normal distribution (Kolmogorov-Smirnov test and Shapiro-Wilk test) and variance homogeneity (Levene test) to determine which tests may be used for evaluation. A reliability analysis was carried out for the answers to the TAM questions to check whether the different questions still measure the same construct. Multiple linear regressions were used to study the influence of the JR and PU on the ItU. Furthermore, the PROCESS macro for SPSS was used to check whether, as indicated by the TAM-2 model, PU is considered a mediator for the influence of the JR on the ItU. A Sobel test was performed on the quantpsy.org site to check the data. The Sobel test determines the significance of the indirect effect of the mediator by testing the hypothesis that there is no difference between the overall effect (path c) and the direct effect (path c’) (Sobel, 1982). Scatter plots were used to check whether there was a suspicion of heteroskedasticity, which was reviewed and confirmed by the Breusch-Pagan test. For this reason, bootstrapping was performed for the (multiple) linear regression analyses according to the bias-corrected and accelerated method with a sample of 20′000 and a confidence interval of 95%. In addition, the data were tested for multicollinearity using the variance inflation factor, and a Durbin-Watson test was used to check autocorrelation. Histograms were used to investigate the normal distribution of the error value. PROCESS (version 3.5) was used to perform an analysis according to model 4 (Hayes, 2013) to check a moderation effect. For the bootstrapping used for this purpose, a heteroskedasticity-compliant conclusion, according to Davidson MacKinnon, with a sample of 20,000 and a confidence interval of 95%, was used, as this is considered particularly robust for such analyses. Differences in demographics were calculated with non-parametric tests.

3 Results

3.1 Market analysis and categorization (qual)

To find out which features currently available VR applications have, 34 VR applications were examined within the scope of market analysis. The process identified 141 features and functions in 13 superordinate categories. Table 4 shows and explains the 20 features that occurred most frequently.

TABLE 4

TABLE 4. 20 features that occurred most frequently.

3.2 Sample and prior experiences

The online questionnaire was opened by 365 people and completed by 107, which corresponds to a completion rate of 31.56%. The age of the respondents varies between 18 and 46 years. The median is 25 years, meaning half of the respondents were 25 or younger. An outlier check was carried out. For this purpose, data was first removed from people who had not completed the survey correctly (for example, people who had selected the same number or pattern everywhere in the TAM questionnaire or had written obscene statements in the comment fields). Boxplot diagrams were also created for the TAM questions. After cleansing the data, 103 people remained to be analyzed. This sample is divided into 67% women and 32% men, plus one person who does not fit either of these two genders. Students from nine fields of study took part: Social Sciences (45 students/43.7%), Law and Economics (40 students/38.8%), Medicine-, Health-, Sport- Sciences (4 students, 3.9%), Mathematics/Informatics (3 students, 2.9%), Technical Sciences (3 students/2.9%), Natural and Environmental Sciences (3 students/2.9%), Art/Music/Design (1 students/1%), Language Literature Communication (1 student, 1%) and something different (3 Students/2.9%). Of the respondents, 71.8% are studying for a Bachelor’s degree and 22.3% for a Master’s degree. The clear minority (5.8%) were in another education or training program. Just over half of the students study part-time (52.4%), and 80.6% have a part-time job while learning. Most people have recently worked in the administrative and service sector (24.3%), followed by production and manufacturing (11.7%), insurance and services (9.7%), and health and social services (9.7%). As Table 5 shows, 59.2% of the participants had less than 1 hour of VR experience. Of the remaining 40.8%, just under a third (32%) already have some experience (between 1 h and 10 h), and only 8.8% have more experience than 10 h. Project management experience is also low at 58.3%, with little experience (> 1 year). In addition, the respondents stated that 6 to 7 people (57.4%) participate most frequently per session.

TABLE 5

TABLE 5. Distribution of VR experience.

3.3 Descriptive TAM data

The lowest rating of 1 (= do not agree at all) and the highest rating of 7 (=agree) can be found for all items. The highest agreement was given to the question, “Assuming I have access to sharing computer applications in VR, do I intend to use them?” (mean = 5.82). “The use of mind maps in VR is relevant in my work” only achieved a mean value of 3.77.

3.4 TAM dimensions per function

Table 6 summarizes all surveyed items per TAM dimension and function, resulting in a mean value of 5.34 and a median of 5.5 for ItU. For PU, the mean value is 4.84 and a median of 4.8. For JR, the mean value is 4.36 and a median of 4.2. All mean values are between 3.83 (mind map JR) and 5.81 (sharing PC APPs ItU). No median is below the scale mean 4 (neither). The Mind maps function received the lowest score (mean 4.41, median 5). The highest overall score was for application sharing (mean 5.56).

TABLE 6

TABLE 6. Descriptive values TAM.

3.5 Inference statistics TAM data

First, the TAM dimensions were statistically checked across all five features. Both the scatterplot and the Breusch- Pagan test for heteroskedasticity [Chi-square (1) = 14,333, p < .001, n = 103] showed that there was no homoskedasticity. The data show neither multicollinearity (VIF = 3.679) nor autocorrelation (Durbin-Watson statistics = 1.581). Multiple linear regression showed that only the PU significantly influences the ItU, F (2.100) = 112.038, p < 0.001, n = 103. If the PU increases by one point, the ItU increases by 0.889 points. 68.5% of the dispersion in the ItU is explained by the two independent variables, which correspond to an intense effect, according to Cohen 1992. Further results of the multiple linear regression are shown in Table 7. According to TAM-2, the PU serves as a mediator for the influence of the JR on the ItU. The calculation carried out for this purpose showed that JR has a significant impact on PU [b = 0.7350, t (101) = 13.3744, p < .001] and also, PU is significantly related to ItU [b = 0.8886, t (100) = 7.7512, p < 0.001]. However, JR has no direct influence on ItU [b = −0.0604, t (100) = −0.6296, p = .5304]. With the combined model, a significant effect can be determined [b = 0.5928, t (101) = 7.2549, p < 0.001], which, however, is mainly due to indirect effects (Indirect = 0.65, SE = 0.0992, 95% CI [0.4672,0.8549]). In contrast, direct effects have hardly any influence. The result of the Sobel test was significant (z-score: 6.706, p < .001). The model is also shown in Figure 4A. The same findings, as in the overview of all five functions, were also found with minor deviations in the investigations of individual functions (see Figures 4B–F). About the differences between the genders, it was noticeable that ItU was divided into the functions Mind map (median male = 5.0, median female = 5.5, Mann-Whitney-U-Test: z = −2.035, p = .042, Cohens d = .41), Individual configurability of the working environment (median male = 5.0, median female = 6.0, Mann-Whitney-U-Test: z = −2. 025, p = .043, Cohens d = .41), the recording function (median male = 5.0, median female = 4.0, Mann-Whitney-U-Test: z = −1.959, p = .050, Cohens d = . 40) and the overall value across all functions (median male = 5.3, median female = 5.6, Mann-Whitney-U-Test: z = −2,278, p = .023, Cohens d = .46) differ significantly. There was also a significant difference in the ItU characteristics of the mind maps between law and economics and the social sciences (median R&W = 5.0, median SW = 6.0, Mann-Whitney- U-Test: z = −2,487, p = .013, Cohens d = .51). According to (Cohen 1988) these are small to medium effects.

TABLE 7

TABLE 7. Regression values.

FIGURE 4

FIGURE 4. (A–F) moderation effects.

4 Discussion

In the presented study, many iVR software development companies have overlapping features implemented in 34 products intended for the professional market. Regarding our first research question, we could aggregate 13 categories with different feature counts (see Table 8). The market analysis has shown that there is already a more extensive selection of VR applications that can have very different functions. Grimm et al. (2013) provide a possible explanation for this with their finding that the exact fit of the functions of a VR application to the user needs the decisive selection factor. It is, therefore, in the interest of the application developers to have a unique range of functions that covers as many needs as possible. The results of the descriptive calculations show very high values for all properties and TAM dimensions. For most respondents, the probability that they would use the presented features if they could is very high, which underlines the expected high relevance of VR in the working context. The results of the analytical calculations could confirm some of the relationships postulated in the derived TAM-2 model and thus validate its transferability.

TABLE 8

TABLE 8. (a–d) Aggregation of 13 categories with different features.

4.1 Derived TAM model

As shown in Figures 4A–F, although there is a significant correlation between the various dimensions (PU, JR & ItU), these vary between the functions examined. Across all five functions, JR appears to have less influence on the ItU than PU. Therefore, we recommend that iVR software developers do not solely aim at high JR but also focus on functionality and usefulness for the user. For example, the quick import and display of images may not be relevant for a project manager’s job; however, if the project manager thinks it is easier and quicker this way and, therefore, more helpful to him. It will influence the intention to use much quicker/stronger than if the function is essential for the job, but the project manager would prefer something else. Regarding project-based learning scenarios (Barron and Darling-Hammond, 2008; Thomas, Mergendoller, and Michaelson, 1999; Chen and Yang, 2019), the same example could give students better possibilities for life-enhancing skills. Suppose a software function will be perceived as applicable (e.g., import of images) during virtual space in the first semester. In that case, the intention to use iVR software with the same feature outside the iVR learning environment (after graduation) becomes more likely. The higher the PU of the functions in a distance learning setting, the more it should be used in software built for later life-enhancing skills (Barron and Darling-Hammond, 2008).

4.2 Gender differences

Although we had no specific gender-related research question differences in gender regarding ItU in Mind map (higher values in females), Customizability of digital space (higher values in females), the recording function (higher values in males), and the overall value across all functions (higher values in female) demonstrate that although there are general high values of intention to use, gender effects might play a role. We do not have enough data to postulate that developing specific software features in the iVR application should be gender orientated. In literature, the personnel presumptions in iVR training are focused on variables like spatial abilities, age (Passig et al., 2016; Lokka and Çöltekin, 2019), or being a specialist (Phé et al., 2017). Further studies will show if there is a specific gender effect that could predict the intention to use certain iVR application features.

4.3 Feature vs. field of study

There was also a significant difference in the ItU characteristics of the mind maps between law and economics and the social sciences (median R&W = 5.0, median SW = 6.0, Mann-Whitney-U-Test: z = −2,487, p = .013, Cohens d = .51). This principle applies to the various functions studied. For the Mind map function, for example, it was possible to establish an almost one-to-one correspondence between the perceived usefulness and the intention to use. Whereas Job Relevance has practically no direct influence on the intention to use. So, suppose a programmer wants to integrate mind maps into his program because he thinks they are essential for the job. In that case, it is much more effective to convince the people using the software that there is a personal benefit from using the function than focusing on its relevance. In return, this could lead to more tutorials or tips and tricks integrated into such programs to show users what they might miss or how these tools could make their lives easier, which is also a part of acquiring knowledge and life-enhancing skills (Thomas, Mergendoller, and Michaelson, 1999). Once again, it is much easier to provoke a change in people’s behavior if a personal benefit is shown than a work-related benefit. For example, an advertising strategy could demonstrate a person who has saved time and enjoyed a coffee somewhere. At the same time, his colleague still struggles to create a mind map in the office and wishes something would make his work easier.

4.4 Customizability of digital space

Another intriguing effect appeared along with the customizability of digital space. Although a customizable environment has certain benefits, job relevance influences the perceived usefulness less than the perceived usefulness influences the intention to use. This could be a particular interest of people who like to individualize their surroundings. In offices with a clean desk policy, for example, a tense atmosphere can arise when employees no longer have the opportunity to individualize and decorate their environment according to their wishes and have to put away their belongings every evening. Therefore, if virtual reality offers the possibility to participate in shaping certain aspects of the environment and allowing people to be creative and free, then people are more likely to use it than a program that does not have such freedom. Also, it is proposed that changing the environment (which is very easy in iVR) could have a positive influence on retention and transfer (see contextual interference effect, e.g., Ziegler et al., 2020).

4.5 Combined effect of JR

While the influence of the JR was usually lower on the Intention to use than the perceived Usefulness, this seems to change for some functions, such as a speech-to-text function (Figure 4D). The importance of the job has a more substantial combined effect on the intention to use, almost as strong as the PU. This leads to the conclusion that this function is equally important in everyday work and general use. Due to the more difficult input in the virtual world, such a function could significantly facilitate work and increase productivity. This finding is even more significant in features 4e and 4f, where in both cases, the JB has a more powerful direct influence on the PU and a more significant combined influence on the intention to use than the perceived benefit has on the intention to use. This could indicate that functions such as sharing PC applications in such a tool are considered important enough for the job that no matter how great the self-interest of the program is, they are considered important enough to be used. If a function is regarded as essential for the respective field of work, the perceived benefit of the function and the intention to ultimately use the product increases. Thus, a distinction should be made between tools that are almost obligatory for processing tasks and functions that are useful but not mandatory. The former can be advertised well because they are present in the programs, while the latter should instead show the personal use for users to be used. Accordingly, the former tools should be preferred as they can lead to more tool sales, while the latter should be used instead as extensions. This is important because including information technical support is one of the successors of project-based learning (Chen and Yang, 2019).

4.6 Limitations and outlook

Like every study, the present research work has limitations. 1. Although a mixed methods design is helpful when exploring a new field, we cannot assume that the objectivity in some steps of result integration may be reduced because of discussions of how to get further. 2. We hoped to get a bigger sample size and were interested in expanding the TAM 2 derivation (with the addition of, e.g., ease of use) and could use structural equation modeling. Still, since we all got through the pandemic, we have to be thankful for getting the data we have. According to Tangmanee and Niruttinanon (2019), the response rate is influenced by the forced responses, question display styles, and attitude towards the questionnaire and varies between 18.8% and 85.8%. We believe that additional digital fatigue influenced the completion rate due to the forced digital switch during the pandemic (a massive increase in, e.g., Zoom and MS Teams use in class). A repeat study would now be necessary under normal conditions to describe the factors that led to our completion rate. Regarding the concepts of work and project-based learning, it can be discussed whether the relation of work as a student is comparable to the working human concept. The most significant conceptual difference is that during a project work as a student, you are not doing paid work. The extent to which our conceptual equation of both constructs affects the relationships found in our study can only be determined by further studies. These should only include working and non-student persons (although in our sample, 80.6% were employed). Further experimental studies with changes in the operationalization of PU and JR should be conducted to check for possible changeability via the regression coefficients.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical approval was unnecessary since the study involved no human manipulation. It was based on a questionnaire where participants rated their acceptance of software features. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

AT: Conceptualization, Data curation, Formal Analysis, Investigation, Visualization, Writing–original draft, Writing–review and editing. EB: Conceptualization, Formal Analysis, Visualization, Writing–original draft, Writing–review and editing. PM: Validation, Writing–original draft, Writing–review and editing. OC: Conceptualization, Investigation, Methodology, Supervision, Writing–original draft, Writing–review and editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

AT was employed by the company Post Ltd and EB was employed by HR Campus.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ahmed, I. Y. S., El-Kasim, M., and Mustapha, L. K. (2017). University students’ intention of smartphone adoption for academic activities: testing an extended TAM model. Media Watch J. Commun. 8 (2), 208–221. doi:10.15655/mw/2017/v8i2/49010

CrossRef Full Text | Google Scholar

Ajzen, I., and Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall.

Google Scholar

Alhalabi, W. (2016). Virtual reality systems enhance students’ achievements in engineering education. Behav. Inf. Technol. 35 (11), 919–925. doi:10.1080/0144929x.2016.1212931

CrossRef Full Text | Google Scholar

Al-Rahmi, W., Othman, M., and Musa, M. (2014). The improvement of student’s academic performance by using social media through collaborative learning in Malaysian higher education. Asian Soc. Sci. 10 (8), 210–221. doi:10.5539/ass.v10n8p210

CrossRef Full Text | Google Scholar

Arbenz, D. (2016). Project placement “enabling occupation” in Switzerland. World Fed. Occup. Ther. Bull. 72 (1), 43–44. doi:10.1080/14473828.2016.1151680

CrossRef Full Text | Google Scholar

Barron, B., and Darling-Hammond, L. (2008). “How can we teach for meaningful learning?,” in Powerful learning: what we know about teaching for understanding. Editor L. Darling-Hammond (San Francisco, CA: Jossey- Bass), 11–70.

Google Scholar

Checa, D., and Bustillo, A. (2020). A review of immersive virtual reality serious games to enhance learning and training. Multimedia Tools Appl. 79 (9), 5501–5527. doi:10.1007/s11042-019-08348-9

CrossRef Full Text | Google Scholar

Chen, C.-H., and Yang, Y.-C. (2019). Revisiting the effects of project-based learning on students’ academic achievement: a meta-analysis investigating moderators. Educ. Res. Rev. 26, 71–81. doi:10.1016/j.edurev.2018.11.001

CrossRef Full Text | Google Scholar

Chu, S. K. W., Reynolds, R. B., Tavares, N. J., Notari, M., and Lee, C. W. Y. (2017). Twenty-first century skills education in Switzerland: an example of project-based learning using wiki in science education, 21st century skills development through inquiry-based learning. Singapore: Springer.

CrossRef Full Text | Google Scholar

Coban, M., Bolat, Y. I., and Goksu, I. (2022). The potential of immersive virtual reality to enhance learning: a meta-analysis. Educ. Res. Rev. 36, 100452. doi:10.1016/j.edurev.2022.100452

CrossRef Full Text | Google Scholar

Cohen, L., Duboe, P., Buvat, J., Melton, D., Khadikar, A., and Shah, H. (2018). Augmented and virtual reality in operations. Capgemini Research Institute report, 1–32.

Google Scholar

Creswell, J. W., and Plano Clark, V. L. (2018). Designing and conducting mixed methods research. 3rd ed. Thousand Oaks, CA: Sage.

Google Scholar

Di Natale, A. F., Repetto, C., Riva, G., and Villani, D. (2020). Immersive virtual reality in K-12 and higher education: a 10-year systematic review of empirical research. Br. J. Educ. Technol. 51 (6), 2006–2033. doi:10.1111/bjet.13030

CrossRef Full Text | Google Scholar

Grimm, P., Herold, R., Hummel, J., and Broll, W. (2013). “VR-Eingabegeräte,” in Virtual and augmented reality (VR/AR). Editors R. Dörner, W. Broll, P. Grimm, and B. Jung (Berlin Heidelberg: Springer Vieweg)), 97–125.

CrossRef Full Text | Google Scholar

Hamilton, D., McKechnie, J., Edgerton, E., and Wilson, C. (2021). Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design. J. Comput. Educ. 8 (1), 1–32. doi:10.1007/s40692-020-00169-2

CrossRef Full Text | Google Scholar

Hayes, A. (2013). Integrating mediation and moderation analysis: fundamentals using PROCESS. Introduction to mediation, moderation, and conditional process analysis.

Google Scholar

Huang, X., Zou, D., Cheng, G., and Xie, H. (2021). A systematic review of AR and VR enhanced language learning. Sustainability 13 (9), 4639. Article 9. doi:10.3390/su13094639

CrossRef Full Text | Google Scholar

Jenkins, D. G., and Quintana-Ascencio, P. F. (2020). A solution to minimum sample size for regressions. PLOS ONE 15 (2), e0229345. doi:10.1371/journal.pone.0229345

PubMed Abstract | CrossRef Full Text | Google Scholar

Kunst, A. (2019). In welchen Bereichen werden Virtual-Reality-Brillen Ihrer Meinung nach hauptsächlich Anwendung finden? Available from: https://de.statista.com/prognosen/793226/umfrage-in-deutschland-zu-anwendungsbereichen-fuer-virtual-reality-brillen#statisticContainer (Accessed July 10, 2020).

Google Scholar

Lokka, I.-E., and Çöltekin, A. (2019). Toward optimizing the design of virtual environments for route learning: empirically assessing the effects of changing levels of realism on memory. Earth 12, 137–155. doi:10.1080/17538947.2017.1349842

CrossRef Full Text | Google Scholar

Makransky, G., Terkildsen, T. S., and Mayer, R. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn. Instr. 60, 225–236. doi:10.1016/j.learninstruc.2017.12.007

CrossRef Full Text | Google Scholar

Marston, H. R., and McClenaghan, P. A. (2013). “Play yourself fit: exercise + videogames = exergames,” in Serious games and virtual worlds in education. Editors K. Bredl, and W. Bösche, 241–257. Professional Development, and Healthcare (IGI Global).

CrossRef Full Text | Google Scholar

Paas, F., Tuovinen, J. E., Tabbers, H., and Van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educ. Psychol. 38 (1), 63–71. doi:10.1207/s15326985ep3801_8

CrossRef Full Text | Google Scholar

Passig, D., Tzuriel, D., and Eshel-Kedmi, G. (2016). Improving children's cognitive modifiability by dynamic assessment in 3D Immersive Virtual Reality environments. Comput. Educ. 95, 296–308. doi:10.1016/j.compedu.2016.01.009

CrossRef Full Text | Google Scholar

Pellas, N., Dengel, A., and Christopoulos, A. (2020). A scoping review of immersive virtual reality in STEM education. IEEE Trans. Learn. Technol. 13 (4), 748–761. doi:10.1109/TLT.2020.3019405

CrossRef Full Text | Google Scholar

Phé, V., Cattarino, S., Parra, J., Bitker, M. O., Ambrogi, V., Vaessen, C., et al. (2017). Outcomes of a virtual-reality simulator-training programme on basic surgical skills in robot-assisted laparoscopic surgery. Int. J. Med. Robotics + Computer Assisted Surgery MRCAS 13 (2). doi:10.1002/rcs.1740

CrossRef Full Text | Google Scholar

Radianti Radianti, J., Majchrzak, T. A., Fromm, J., and Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147 103778. https://doi.org/10.1016/j.compedu.2019.103778

CrossRef Full Text | Google Scholar

Rojas-Sánchez, M. A., Palos-Sánchez, P. R., and Folgado-Fernández, J. A. (2023). Systematic literature review and bibliometric analysis on virtual reality and education. Education and Information Technologies 28 (1), 155–192. doi:10.1007/s10639-022-11167-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Santos, A., Zarraonandia, T., Díaz, P., and Aedo, I. (2017). “A comparative study of menus in virtual reality environments,” in Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces. Editor S Subramanian. (New York, NY, USA: ACM), 294–299.

CrossRef Full Text | Google Scholar

Schinauer, T., and Lachmann, T. (2009). Order preferences as a function of the size and orientation of the display. Proceedings of Fechner days (25), 299–304.

Google Scholar

Sobel, M. E. (1982). “Asymptotic confidence intervals for indirect effects in structural equation models,” in Sociological methodology. Editor S. Leinhart (San Francisco: American Sociological Association), 13, 290–312.

CrossRef Full Text | Google Scholar

Sugand, K., Akhtar, K., Khatri, C., Cobb, J., and Gupt, C. (2015). Training effect of a virtual reality haptics-enabled dynamic hip screw simulator. Acta Orthop 86 (6), 695–701. doi:10.3109/17453674.2015.1071111

PubMed Abstract | CrossRef Full Text | Google Scholar

Super Data Staff (2020). Super data XR Q1 2020 update. Super data perspectives. Available from: https://www.superdataresearch.com/blog/superdata-xr-update (Accessed July 10, 2020).

Google Scholar

Tangmanee, C., and Niruttinanon, P. (2019). Web survey’s completion rates: effects of forced responses, question display styles, and subjects’ attitude. International Journal of Research in Business and Social Science 8, 20–29. doi:10.20525/ijrbs.v8i1.183

CrossRef Full Text | Google Scholar

Thomas, J. W., Mergendoller, J. R., and Michaelson, A. (1999). Project-based learning: a handbook for middle and high school teachers. 1st ed. Novato, CA: Buck Institute for Education.

Google Scholar

Våpenstad, C., Hofstad, E. F., Bø, L. E., Kuhry, E., Johnsen, G., Mårvik, R., et al. (2017). Lack of transfer of skills after virtual reality simulator training with haptic feedback. Minim. Invasive Ther. Allied Technol. 9, 346–354. doi:10.1080/13645706.2017.1319866

CrossRef Full Text | Google Scholar

Webster, R. (2016). Declarative knowledge acquisition in immersive virtual learning environments. Learn. Environ. 24 (6), 1319–1333. doi:10.1080/10494820.2014.994533

CrossRef Full Text | Google Scholar

Wróblewska, D., and Okraszewska, R. (2020). Project-based learning as a method for interdisciplinary adaptation to climate change—reda valley case study. Sustainability 12, 4360. doi:10.3390/su12114360

CrossRef Full Text | Google Scholar

Wu, B., Yu, X., and Gu, X. (2020). Effectiveness of immersive virtual reality using head-mounted displays on learning performance: a meta-analysis. British Journal of Education Technology 51 (6), 1991–2005. doi:10.1111/bjet.13023

CrossRef Full Text | Google Scholar

Zhang, Y., and Song, Y. (2022). The effects of sensory cues on immersive experiences for fostering technology-assisted sustainable behavior: a systematic review. Behavioral Sciences 12 (10), 361. doi:10.3390/bs12100361

PubMed Abstract | CrossRef Full Text | Google Scholar

Ziegler, C., Papageorgiou, A., Hirschi, M., Genovese, R., and Christ, O. (2020). “Training in immersive virtual reality: a short review of presumptions and the contextual interference effect,” in Human interaction, emerging technologies and future applications II. Editors T. Ahram, R. Taiar, V. Gremeaux-Bader, and K. Aminian (Cham: Springer International Publishing)), 328–333.

CrossRef Full Text | Google Scholar

Davis, F. D. (1986). A technology acceptance model for empirically testing new end-user information systems: Theory and results. Sloan School of Management. Massachussetts Institute of Technology.

Google Scholar

Venkatesh, V., and Davis, F. D. (2000). A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science 46, 186–204.

CrossRef Full Text | Google Scholar

Lang, B. (2020). “Training, Design Review, and More,” in Road to VR, Available at: https://www.roadtovr.com/vr-apps-work-from-home-remote-office-design-review-training-education-cad-telepresence-wfh/

Google Scholar

Landis, J. R., and Koch, G. (1977). The Measurement of Observer Agreement for Categorical Data. Biometrics 33, 159–174.

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, J. (1992). A Power Primer. Psychological Bulletin 112, 155–159.

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2. Aufl.) (NJ: Erlbaum: Hillsdale).

Google Scholar