Glove versus controller: the effect of VR gloves and controllers on presence, embodiment, and cognitive absorption

1 Introduction

We learn from an early age that using our own hands allows us to interact with the real world. Virtual reality (VR) gloves allow a similar interaction by transferring hand movements in VR. Recently, the technology has reached a point where it can be applied in commercial VR headsets. The controls profit from natural movement capabilities that match the user’s real-world mental models (Tomborini and Skalski, 2006). The ability of a system to map its controls to changes in the mediated environment in a natural and predictable manner is called natural mapping (Steuer, 1992). VR gloves have a strong potential to address natural mapping in VR, as they adopt natural hand interaction in the real world (Kim et al., 2017). According to Tamborini et al. (2004), natural mapping is a possible determining factor for video-game-induced presence. This suggests that VR gloves can also induce presence in VR scenarios due to natural hand interaction controls. However, to this point, natural mapping has received little empirical attention in the literature on VR gloves and presence. In a search of abstracts in the Web of Science and Elsevier Scopus Search using the search string “mapping AND virtual AND reality AND glove”, 16 and 24 documents, respectively, were identified. A supplementation with presence led to two results. A qualitative user perspective on natural mapping could not be found (search string + “qualitative”). As qualitative research is still widely missing in this field, we hypothesize that a more human-centered explorative perspective on presence, embodiment, and cognitive absorption would generate new insights and suggestions for the design of VR glove systems. Consequently, the research question of this study is: “How do naturally mapped controls of VR gloves enhance perceived presence, embodiment, and cognitive absorption in VR in comparison to VR controllers?”

This study attempts to a) specify a theoretical background on natural mapping, presence, and cognitive absorption with VR input devices, and b) empirically test the notion of naturally mapped controls on presence and cognitive absorption in a qualitative experimental setting. We want to deliver explorative data that could influence new design principles for upcoming VR glove and VR input device systems. The devices were selected due to different natural mapping levels, realistic natural mapping controls and actions with the Noitom Hi5 VR glove, incomplete tangible mapping with the Valve index controller, and directional natural mapping with the HTC Vive controller (Skalski et al., 2011).

2 Materials and methods

2.1 State of VR input devices

At the time of this investigation, controllers are standard input devices for VR. Controllers are held in the hand of the user and allow interaction through buttons that abstract the real movement of the hand (Tomborini and Skalski, 2006; Skalski et al., 2011; Gusai et al., 2017). A standard operation that is triggered with the devices is, for instance, the grabbing motion, where the virtual hand grabs a virtual object in VR and releases it when the user releases a button in reality at the same time (Hufnal et al., 2019; Allgaier et al., 2022). VR gloves offer more advanced movement capabilities than VR controllers. However, a study on VR gloves by Fahmi et al. (2020) found that a VR glove does not necessarily achieve a better user experience than standard VR controllers and that the interplay of controls in VR is not 1:1 comparable with a real-world interaction.

2.1.1 VR gloves and optical hand tracking

The method of optical hand tracking is a type of VR input device that scans full hand motion, including natural mapping, in the full range of motion and motor control. Optical hand-tracking devices capture the movement of hands with cameras, either applied to HMD or as local sensors connected to a PC. As no device is connected to the user’s hand, no haptic feedback is included (Fahmi et al., 2020). Lower-cost optical hand-tracking devices are available as VR gloves, but these devices come with some limitations that must be considered. Optical tracking can only be conducted when the camera can see the hands. Tracking outside the camera field of view is not included. Additionally, precision, tolerance (e.g., towards light inference), limited data capture, and operating system compatibility are issues with optical hand-tracking devices that impede the technology (Gamboa et al., 2019). These limitations make the devices problematic in fast changing environments, as, for instance, when needed for the replication of manufacturing tasks (see implications for practice). This gave a further note of the importance of choosing a VR glove over an optical system in an operational use case (see Section 2.2.1). The use case resulted from the initial research gap of lacking qualitative studies on natural mapping with VR gloves.

2.2 Theoretical background

2.2.1 Natural mapping and controller naturalness

To deliver an extensive picture between naturally mapped devices, differences between input devices must be outlined. The opposite of an abstracted movement with a controller is a naturally mapped movement (McEwan et al., 2014). Steuer (1992) defines natural mapping as the “ability of a system to map its controls to changes in the mediated environment in a natural and predictable manner.” The perception of naturalness is based on the ability to complete the mental models people use for real-world activities (Tomborini and Skalski, 2006). Mental models are internal representations of the external world that allow individuals to anticipate and predict the outcomes of their actions. McEwan et al. (2014) give an insightful note on natural mapping: “Naturally mapped control interfaces (NMCIs) for video games involve interactions with less abstraction between the task to be virtually achieved and the action required to achieve it” (McEwan et al., 2014). Hence, a VR glove can be considered closer to a realistic haptic interaction due to higher natural mapping capabilities than other VR input devices. Caeiro-Rodriguez et al. (2021) noted three ways in which commercial VR gloves map human actions. I) Hand and finger pose estimation motion tracking (degree of freedom), II) tactile feedback, and III) kinesthetic feedback. Caeiro-Rodriguez et al. (2021) depicted that a wide variety of active movements and natural mapping influences the sense of agency. Agency refers to the experience of control in one’s own actions (motor control), which contributes to perceived naturalness (McEwan et al., 2014).

Skalski et al. (2011) differentiated in a typology between four different mapping types: Directional natural mapping refers to devices that produce a correspondence between a direction used with a control and the actual movement of a body part. Devices that inherit directional natural mapping are, for instance, video game controllers. Kinesic natural mapping involves body movements that correspond to real-life actions. No tangible stimulation is included in this type of mapping. Optical hand-tracking devices fall into this category. Incomplete natural mapping is included in devices that partly map the “feel” of an object. However, the tangibility is not fully able to reproduce real-life sensations. Realistic natural mapping is the highest level of natural mapping, according to Skalski et al. (2011). It provides a realistic, tangible element to achieve the highest level of natural mapping. Considering a hand movement, including grabbing and touching, as natural tangible elements, a VR glove can be considered a realistic, naturally mapped device for hand operations. The motion capabilities of a human hand are displayed in Figure 1.

Figure 1

Figure 1. DoF in human hand (belongs here).

At this point, the gap between kinesic and realistic natural mapping for hand movements is not constantly given. According to Biocca (1997), close mapping of body movements and tangible stimulation is needed to address mental models for real-life behavior. Backed on this definition, Skalski et al. (2011) investigated a car steering wheel as a control device. This brings direct features of a clear mental model in relationship to driving into the simulation. This allows classifying the steering wheel as a realistic natural controller. The definition of a VR glove as a realistic, naturally mapped device is, therefore, also related to the mental model that it fulfills.

2.2.2 Presence

Perceived naturalness with controllers can lead to perceived presence (Skalski et al., 2011). Presence is a feeling of being virtually immersed in the media. It can be classified as the response of the human brain to the system (Berkman and Akan, 2019). In the investigation of Skalski et al. (2011) on natural mapping, perceived presence is directly influenced by perceived naturalness with natural mapped controllers. According to Tamborini and Bowman (2010), increased accessibility of mental models is expected to enhance spatial presence with naturally mapped devices (Tamborini and Bowman, 2010). Spatial presence is a type of presence that concerns the ability of a device to create a mediated experience to be immersed in a virtual environment or game space. (Tomborini and Skalski, 2006). Consequently, spatial presence is primarily of concern for this investigation of VR gloves.

2.2.3 Embodiment

If the input device creates a strong perceived presence, the user may also experience a situation in which they accept the virtual hand model as their own. This is a state of embodiment, as the virtual hand embodies the hand of the user (Gonzalez-Franco and Peck, 2018). Input devices in VR are mainly displayed as hand models (Caeiro-Rodríguez et al., 2021). For instance, the HTC Vive controller represents a model of a hand that is shown at the location of the VR input device. When the user pushes a button, they can observe how the virtual hand is grabbing an object in VR. Embodiment is also highlighted as a strong level of self-presence in the scientific literature and is strongly interrelated to cognitive absorption in VR (Schultze, 2010; Gonzalez-Franco and Peck, 2018). As the VR glove is a device that aims to mimic hand movements in VR, a state of embodiment should be reached where a user feels fully embodied with the hand model. We assume that the users have perceived embodiment with the hand model due to mapped controls.

2.2.4 Mapping and haptic feedback

Haptic feedback can affect presence with VR input devices and lead to tangible natural mapping (Skalski et al., 2011). The objective of haptic feedback in VR is to increase the user’s sense of reality (Gibbs et al., 2022). Haptic feedback in VR allows users to perceive the physical properties of virtual objects (e.g., weight and motion patterns) (Wang et al., 2022). Monteiro et al. (2020) highlighted a distinct dependency between new input device types and perceived presence in new interactions (Monteiro et al., 2020). On the other hand, a lack of haptic sensations can disorientate the users’ experience of presence (Wang et al., 2022). In current research on VR gloves, distinctions are made between certain aspects of a system. Gibbs et al. (2022) provided an extensive overview of the effect on presence between visuals and haptics of a VR system. In this research, we deliver a primary investigation on motor control and navigation aspects related to natural mapping (McEwan et al., 2014; Moon et al., 2023). Haptic feedback is included with VR controllers but is not of primary concern for the study of natural mapping movement, as we orient on the natural mapping definition from Steuer (1992), which defines natural mapping primarily as a feature of the system to map its controls to changes in the mediated environment (see Section 2.2.1). Haptic feedback is a complex field of study and implies a set of different technologies (Caeiro-Rodríguez et al., 2021) that must be examined in specialized investigations. Furthermore, the haptic feedback of current VR gloves does not realistically map real-life haptic processes. The human hand creates haptics in a complex interplay of multimodal stimulations (Kim et al., 2017). To this point, no commercial solutions can realistically map haptic feedback as in reality (Kim et al., 2017). Haptic tangible feedback is multimodal and involves a variety of senses. The interplay between haptics, changing features of the grabbed or touched object, sounds, etc., is highly complex (Gibbs et al., 2022) and would exceed the scope of this study with a focus on control mechanisms on naturally mapped VR controllers and VR input devices after Steuer (1992).

2.2.5 Cognitive absorption

To this point, the connection of naturally mapped VR gloves on presence is outlined as a direct effect of the system. However, presence does not describe the perception of the task that is operated with a VR glove. To fully understand naturally mapped interactivity, a perspective on the operated task is necessary. Deep involvement in a task is described as cognitive absorption (CA) (Ryan et al., 2015). This is related to the trait absorption on its own, the engagement of the user in VR, and the state of flow that is experienced with the system (Agarwal and Karahanna, 2000). Cognitive absorption represents the extent to which a user is absorbed when using a system (Agarwal and Karahanna, 2000). Flow can be understood as a self-motivated state of immersion that results from an optimal balance between the challenge and skill with a system (Csikszentmihalyi, 1993). Therefore, this research gives an extensive overview of the presence response to the immersive capabilities of the VR glove. CA is additionally a precondition for learning, as it is described in the field of learning as a state in which an individual is completely involved in a task (Csikszentmihalyi, 1990). Based on this literature, we assume that natural mapping with VR gloves can lead to a high level of cognitive absorption into a task.

The scope of interest for this study is outlined toward this point. The goal of this research is to learn if a VR glove, due to natural mapping, automatically has stronger effects on presence and how this affects cognitive absorption and embodiment in the VR environment. The research question is accordingly:

“How do naturally mapped controls of VR gloves enhance perceived presence, embodiment, and cognitive absorption in VR in comparison to VR controllers?”

The study aims

1) To analyze participant perceptions while using a VR glove compared to two other input devices on lower mapping levels.

2) To identify patterns corresponding to the effect of natural mapping in VR gloves and perceived spatial presence, cognitive absorption, and embodiment.

2.3 Participants and recruitment

The present study recruited N = 20 participants aged 22–36 (9 women, 10 men, and 1 nonbinary). The average age in the investigation was 27 years, and six participants were from an international background. More than half (60%) had previous experience in VR, and 55% were actively playing video games.

2.4 Materials

2.4.1 VR experience

The entire VR scenario was programmed with the Unreal Engine 4 Game Engine on a PC with 32 GB RAM, an i7 12th Generation-12700K processor, and a GeForce RTX 3080 Ti Graphics card for use with an HTC Vive Pro Eye VR headset. The HMD included an FOV of 110° and a frame rate of 90 Hz. The headset was tracked in the experiment through two HTC Vive 2.0 base stations that were attached in the corners of the laboratory. The HTC Vive was additionally equipped with a wireless adapter toolkit so that no direct cable connection to the computer was needed. For the experiment, three input devices on different natural mapping levels were chosen and are displayed in Figure 2.

Figure 2

Figure 2. VR input devices in the experiment (belongs here).

The HTC Vive controller, the first device, is an input device with the lowest level of natural mapping. The HTC Vive controller is displayed in the VR as a virtual hand model that represents the hand of the user. The controller inherits directional natural mapping through applied buttons for grabbing and abstracts the grabbing motion through the directional pushing of a button with the index finger, which triggers a grabbing animation of the hand model. The positioning is accurately tracked in the VR through the two external sensors in the room that capture the position in the real world and set the controller model at the same position in VR. This creates partly realistic movement natural mapping by representing the basic hand movements. The HTC Vive controller delivers a full range of motion in the arm movement (three DoF) but no more complex finger movements. Haptic feedback is given in the HTC Vive controller through tactile vibration (Fahmi et al., 2020).

The second device, the index controller, partly abstracts grabbing physically with buttons for the index finger and thumb but also captures the movements of the middle, ring finger, and pinkie finger through sensors. The grabbing can be triggered by bending the fingers through flexion in contact with a virtual object. In contrast to the HTC Vive controller, the index controller mimics the finger movements through motion tracking but cannot completely map the finger movements of the user. For that reason, the index controller is considered here as an incomplete mapping input device. The index controller reaches six DoF in the hand movements and three DoF in the arm movements. Haptic feedback was included in the experiment with the VR controllers. The standard controller functions for Unreal Engine 4 were used for the HTC Vive and the index controller. The last of the chosen devices is the Noitom Hi5 VR glove, a fabric VR glove that allows almost full hand movement and real-time position tracking in VR. Each finger movement of the hand except for the side movements (abduction) was directly captured 1:1 from the real world into the VR while grabbing. In total, the Notom Hi5 glove supports 19 DoF in the hand; the real human hand has 23 DoF, as displayed in Table 1. However, as side movements are not necessarily needed for everyday operations (Skalski et al., 2011), the Hi5 glove comes close to the finger movement and mapping capabilities of the human hand. No tactile vibration feedback was provided for the grabbing due to limitations of the basic version of the Noitom Hi5-glove in Unreal Engine 4.

Table 1

Table 1. DoF capabilities of VR input devices (belongs here).

2.5 Procedure

2.5.1 The experimental setting

VR gloves are still highly complex and costly devices. However, in training fields, including surgery training, education, and industrial security training, VR gloves bring new possibilities to simulate critical situations (Caeiro-Rodriguez et al., 2021). Considering the current application of VR gloves and naturally mapped devices in the field, a construction scenario in an industrial context with a perspective on the transferability of practice was chosen (see the implication section for the practice). The Noitom Hi5 glove was selected for this investigation due to its strong motion-capturing capabilities and its light fiber design. A lightweight device was selected so as to not affect the movement and motion range capabilities of the system and allow optimal movement-mapping conditions. Secondary for the experiment design was good affordability, low latency, and a solid integration into Unreal Engine 4. The selection of the VR glove was made on the basis of a systematic analysis of VR gloves by Caeiro-Rodriguez et al. (2021). A scenario was selected for the experimental case where the participants practiced assembling an industrial medium-sized crane in VR. The VR scenario for investigating immersion effects was designed with a focus on the implication for practice (see Section 4.2). At the time of this study, VR gloves are in an experimental state and not an affordable option for the private consumer market (Caeiro-Rodríguez et al., 2021). Current customers for VR gloves are mainly companies that use VR gloves to display their processes in VR (Kim et al., 2017).

2.5.2 Virtual reality environment

The VR level was designed as a plain white room to reduce distractions and allow full concentration on the operation with the input devices. The participant must assemble a crane from five crane parts. When the construction process with one device was finished, the construction procedure started again with the subsequent device in order. The order of the input devices switched randomly to avoid bias on the devices as the user already knew more about the construction process with each assembly round. The qualitative research process was divided into an experimental part and an interview part. The investigation took 1 hour.

Each participant received a short introduction to the procedure and the context of the investigation. To that, a short definition of immersion was given to explain the field of interest. After the introduction with the narrative, the participant was given a sheet that showed the finished constructed crane in the real world and the crane model with text labels to explain each construction part. The participant needed to remember the arrangement of the parts and construct the crane in VR from the crane parts displayed in the picture. The preparation of the experiment took an average of 15 min. After the interview, the participants were thanked for their attendance and asked for possible improvements. An interview took, on average, for 25–30 min. The five crane parts were placed in the VR level of the white room. A ghost model of the crane was presented next to the parts in the VR environment to represent where the crane should be assembled in the scenario (see Figure 3). The ghost model consisted of five blue transparent boxes, one box for one crane part. The user was instructed to take the parts and find their corresponding box in the ghost model. Here, the user should remember the position from the construction manual that was presented beforehand. When the user found the right location box and overlapped the part, the box changed color to green and showed a ghost model of the crane part. The VR controllers provided haptic vibration feedback during the grabbing activities. When the user released the crane part, it snapped to the equivalent position in the ghost model. The standard input mappings for the HTC Vive controller and the index controller were integrated into Unreal Engine 4. For the Noitom Hi5 glove, the grabbing mechanism was developed manually, as no input mapping pattern for the movements was provided by the manufacturer. The grabbing trigger with the VR glove was programmed with the Noitom Hi5 software development kit (SDK), which inherited special functions for data retrieval. The grabbing was equivalent to the movement of the fingertips to the hand base. An average between the distances of the fingers was counted. When the average distance between the fingers and the hand base was less than a predetermined threshold, the crane parts snapped to the hand model when the user operated a grabbing pose. The crane parts were released when the fingers opened, and the distance exceeded the threshold again.

Figure 3

Figure 3. VR scenario for the experiment (belongs here).

2.5.3 Data collection

The experiment took 20 min on average. After the experiment, the attendants were again asked for their permission on the audio recording. Then, they were interviewed according to the interview guidelines. In the interview, the participants replied to a general entry question to share their thoughts on immersion so they could express their impressions without being influenced by the interviewer. Then, more detailed categorical questions were asked. The interviewer could dynamically ask open follow-up questions on a category if needed to get more details on an insight. After sampling, a final question asked the users to give their overall impression of the devices and explain what they would need to be immersed with an input device.

2.6 Transcription

The participants of the interview were informed that their data were treated anonymously. The recordings in the field were made to have objectivity in the data sampling process. The interviews were recorded on a PC with the AI recording and transcription program Otter.AI. The program records the interview and creates a first transcription. The transcription program was connected to a Thomann T-bone SC 420 microphone for good sound quality.

2.7 Data analysis

The researchers used a qualitative content analysis (QCA) to analyze data from the focus group. The QCA after Mayring (2022) was chosen as research on VR gloves and naturally mapped input devices is, to the best of our knowledge, a relatively uninspected topic in information systems and human–computer interaction (HCI) research. QCA is an inductive approach to identifying patterns from data through selective coding. To additionally identify a comprehensive picture of the capabilities, a quantitative operation was conducted for essential categories and presented in the following section. The first preparation step in the QCA is the development of a category catalog (Mayring, 2022). For the investigation, categories were defined for the expected observable phenomena, such as findings and expressions regarding immersion effects in VR. The development of a category catalog was based on findings of the secondary literature (deductive category application or structurization), as the structure of the investigation was refined through the application of the natural mapping model by Skalski et al. (2011). The questionnaire for the survey was derived from the category catalog. For each category, a set of questions was derived to initiate findings on the categories. In the initial phase, a pre-study with 25% of the focus group was conducted, and a first coding sprint was facilitated. Minor changes to the questions were made and implemented in the subsequent main studies. During the evaluation, a supplementary quantitative evaluation matrix of coherence scores was set up to create an initial picture of the satisfaction of users in certain categories (Yu et al., 2018). The evaluation sheets are included in the attachment to give the reader a more comprehensive picture of the device’s performance. However, only the qualitative findings are outlined here in this research article to limit the scope of analysis.

3 Results

The study identifies the perspective of a young focus group (average age 27 years) on naturally mapped VR devices. The results represent the perspective of the focus group on naturally mapped interactions with VR input devices and their influence on perceived special presence, cognitive absorption, and embodiment.

3.1 Controller naturalness

The VR glove was claimed as natural in half of the interviewed cases and in a further six cases as natural with limitations. The missing vibrotactile feedback of the VR glove was not seen as a considerable limitation in the experiment. However, the participants who would not categorize the glove as natural saw haptic feedback as the main reason for that (FG20): “The glove is the most natural from all controllers, but I would love also to feel something when I grab. With the controllers you feel but only the controller; that’s unnatural”. In contrast, the HTC Vive controller was observed in only three cases as natural and in four cases as partly natural; it was declared to be not natural and more artificial device in four cases. The index controller had a negative ratio with four participants observing it as an artificial device and one observing it as natural. The main reason for the natural perception was the fluent movement capabilities with the VR glove.

“I think the movements were the most fluent with the glove because when you know your brain sends out the signal to move your hand. And it was directly like the movement in the VR was the same. So that was natural” (FG4).

The main reason for the artificial feeling, according to the interviewees, was the different movement mechanisms of the index controller that resulted in increased thinking effort: “With the glove yes. With the remote control close with the index control not really because I pushed several buttons to figure out how to grab something that was okay” (FG2).

3.1.1 Natural mapping

Regarding natural mapping, the perceived freedom in the scenario was indicated. “I think the glove simulates grabbing in real life the best because you move your hand freely” (FG4). Most people perceived a gap between holding a device in their hand for interaction and moving their own hand in VR that limited the mapped activity: “I would say it does not affect my task with holding the controller in the hand, but it did not feel like really free. You had always to grab something. And that was a bit, I would say, a little bit limiting my task” (FG1). Regarding the index controller, the controls were criticized for not matching actual hand movements, which was seen as artificial.

“I needed more attention to, I don't know, somehow my brain was like: Okay, first thumb, then the index finger, and I didn't manage to pull up both at the same time. So, I think the grabbing motion would be easier but somehow, I did two motions. That was not so natural” (FG4).

3.1.2 Realism

In terms of realism, 13 of 20 participants claimed the glove was, in general, a realistic device. The normal controller, in comparison, was seen in three cases as realistic but also criticized in four cases as unrealistic. As an indication of the perception of the realism of the HTC Vive controller, the capability of young people to abstract experiences, and the decreasing importance of realism in VR and video game experiences, one participant noted:

“The glove was more realistic; I felt I used my hand” (FG5)

3.2 Spatial presence

In the VR scenario, the majority felt more present while using the VR glove. One participant said, “I was present. I was there. I felt that I was there, and I was building something. It really felt like it” (FG5). “The glove was so seamlessly, and the fact that I didn't need to hold anything” (FG9). “I still feel my fingers in real life, but I see it in the VR as it is in real life. So that was really convincing” (FG1). With the HTC Vive controller, three participants noticed a state of presence. Furthermore, three participants agreed that they had experienced presence in the VR with the index controller. Although some participants lost their feeling of time while using devices, some stated that the duration was too short to lose the sense of time during the experiment: “I wasn't thinking about time because the tests were not too long” (FG1).

3.3 Cognitive absorption

In the experiment, 14 people stated that they felt strongly absorbed into the task with the VR glove. One participant commented,

“If I had the controllers in my hand, I was more like, I had more to think of the controller, and how I should grab it and which button I had to use to grab it, and with the glove, I could really dive into the process of building” (FG3).

“It felt like a natural thing of my body, and I fully dived into it” (FG4). Three participants perceived strong absorption with the index controller.

3.3.1 Engagement

When using the glove, 60% of the participants perceived to be more engaged in the scenario. In comparison, four participants claimed the HTC Vive controller was engaging. Two people rated the index controller as an engaging device in the scenario. The capability of the glove to be seamlessly in the scenario was valued by most participants: “With the glove, I was really focused on the crane, on the scenario. With the other devices, I had to focus more on the devices instead of the scenario” (FG2). Most participants noticed a thinking effort with the other devices that kept them from diving into the experience:

“I had more to think of the controller, and how I should grab it and which button I had to use to grab it, and with the glove, I could really dive into the process” (FG3)

3.4 Embodiment

Half of the participants noticed they unconsciously accepted the VR glove in the VR as their hand.

“There was no distance between this virtual hand and my hand because it was my hand. It's kind of hard to describe. What I mean does no difference for me that the virtual hand is my hand because it does every movement that I do” (FG2)

A phenomenon that appeared as a consequence of embodiment with the VR glove was the operation of poses: “With the glove, it was I can move my hands, and then I remembered I can do poses” (FG4). A collection of operated poses is displayed in Figure 4. With the HTC Vive controller, in contrast, one participant experienced embodiment, and two participants experienced embodiment with the index controller.

Figure 4

Figure 4. Poses in VR (belongs here).

4 Discussion

4.1 Key findings

The study aimed to investigate how natural mapping in VR contributes to spatial presence, cognitive absorption, and embodiment compared to VR controllers. The research question of this work is: How do naturally mapped controls of VR gloves enhance perceived presence, embodiment, and cognitive absorption in VR compared to VR controllers?

4.1.1 Perceived naturalness of naturally mapped input devices

The analysis indicated that the VR glove created strong associations to natural hand movements compared to other state-of-the-art VR input devices. The prediction that the natural mapping motion capture of the VR glove would be perceived as more natural than VR controllers was highly supported (Skalski et al., 2011). However, flexibility does not directly imply a natural perception. The index controller allowed a similar range of movements as the VR glove in the experiment but could not reach a natural perception in most cases. According to the participants, the mapping must be in line with the expectations of the users to create a perception of naturalness and realism (Skalski et al., 2011). In contrast, a device with higher movement capabilities but weak mapping (the index controller) was perceived as less natural than a normal controller. In addition, participants highlighted a correlation between a stronger thinking effort of “translating” movements. This presents that a simpler device, such as the HTC Vive controller, can be perceived as natural and more realistic than a device with plenty of movement capabilities if the thinking effort with the complex device is too high. The VR glove, in contrast, shows that activation of existing movement patterns from mental models can work to create a naturally perceived interaction (McGloin & Krcmar, 2011).

4.1.2 Cognitive absorption and learning

The results with the devices indicate that the VR glove allowed the participants to be involved and engaged in a task, which could, according to Agarwal and Karahanna (2000), be related to increased CA. Jahn et al. (2018) highlight that CA can influence attitudes toward an intention to use a system that can positively influence learning outcomes. Kampling (2018) extensively outlines the importance of CA in VR learning environments due to CA, presence, and embodiment (Kampling, 2018). Nevertheless, further research would be needed to isolate a relationship between CA and learning for VR gloves.

4.1.3 Embodiment with input devices

The results indicate that the participants experienced different levels of spatial presence with the input devices. The perceived embodiment is a reliable indicator that perceived spatial presence due to a naturally mapped device can change the perception of the hand as a body part in VR (Gonzalez-Franco and Peck, 2018). The fact that the VR glove in this early development stage, without haptic feedback, can induce a strong perception of embodiment indicates the potential that can be unlocked in this field with further technological advancements.

4.2 Implication for practice

The findings of this study are relevant for readers who need to select the optimal input device for their use case and lack an understanding of the full potential that VR gloves and naturally mapped input devices provide. The investigation suggests that VR gloves are a suitable device in scenarios where a high degree of spatial presence is needed. Furthermore, the findings of this investigation show the potential of VR gloves for application in training scenarios, where a strong cognitive absorption into a task should be reached. To this point, VR gloves can be a suitable device, especially in training scenarios where haptic tasks are taught, as displayed in the crane construction scenario for this research. Considering design possibilities and principles, our findings could be valuable, for instance, for the development of VR glove systems that use the haptic sensation of human hands as natural feedback, as stated by participant FG1. Moreover, the use of poses as elements for attention and presence in VR could change the existing design of VR input devices, as indicated in the results.

4.3 Limitations

This study comes with some limitations that must be considered. Due to the interpretative nature of the QCA, the findings were generated for a smaller sample of 20 probands. It aimed to validate the identified patterns in ongoing quantitative research on mental models and natural mapping with VR input devices. Furthermore, the experimental setting had limitations that could not be addressed in the experimental design. The missing vibrotactile feedback in the experiment with the VR glove is one factor that limited the results; tactile feedback could have generated new findings on haptic interactions in VR. In this regard, no direct comparison between optical hand-tracking devices and VR gloves could be provided. For that, a more distinct evaluation of mental models due to the multiple movements of these devices (DoF) would be necessary to deliver a comprehensive perspective on directly compared control capabilities (Skalski et al., 2011). This could not be done in the scope of this explorative analysis (see implications for theory). Further minor limitations of this study were that the interviewees were of similar ages, between 22 and 36. It would be valuable to learn if an older focus group reacts differently to the natural interaction with their hand in VR. Individual traits (e.g., individual behaviors, past experiences with video games, decision making in VR) were not objectively considered during the study. The present study focused on changing spatial presence and cognitive absorption perception with VR gloves and VR controllers.

4.4 Implications for theory

An investigation on learning and mental model information processing would be a reasonable extension of the research topic, as these mechanisms are, to the best of our knowledge, not successfully outlined for naturally mapped input devices in VR. Additionally, an investigation of related concepts on presence, including social presence and telepresence, could allow us to differentiate presence mechanisms with VR gloves and naturally mapped devices. We create a comprehensive picture of the differences between mapped input device controls (Steuer, 1992). Related to Skalski et al. (2011), haptic feedback, as a specific type of tangible natural mapping, would be an insightful layer on our findings. This type of investigation could be helpful in delivering a clear definition of VR gloves as a realistic natural mapping device. Regarding related concepts to haptics and mapping, we recommend qualitative observations on inertial load that could give new insights into how natural mapping perception changes with the introduction of inertial load to VR glove design (Tang et al., 2023). We anticipate that new results and control insights could be gained that could complement the picture of natural mapping and haptic control mechanisms. We also suggest a more precise investigation of several mental models to understand in which situations realistic natural mapping can be achieved and what the primary mental models are that should be addressed in the design of VR gloves. At this point, we propose an analysis on a kinesic naturally mapped device, for instance, an optical hand-tracking device compared to a VR glove with vibrotactile feedback. Such a study could provide more information on the question of whether mental models strongly differ between optical representation of movements with an optical hand-tracking device and physical tracking with a VR glove. Thus, for that case, a distinct evaluation between situational mental models and movements for these types would be needed. A situational picture of mapping capabilities between tangible and realistic natural mapping devices could be delivered in this way (see Section 2.2.1). It would be valuable for further enhanced mapping and input device design. Studies on collaboration would be a further valuable extension of this study, as hand movements and gestures often depend on the interaction with other actors (Mütterlein et al., 2018). How these affect social presence and collaboration could complement the findings of this study.

4.5 Conclusion

In this research, three VR input devices with different natural mapping levels were investigated in a qualitative study on their effect on spatial presence, embodiment, and cognitive absorption. Twenty participants tried a learning scenario from an industrial construction context. According to the participants in the study, the VR glove offered the strongest perceived effects on the investigated constructs as a naturally mapped device. Therefore, a connection between constructs and natural mapping could be highlighted. We propose that VR experiences like this could allow users to be more strongly immersed in VR with existing controllers. Moreover, the VR glove induced a feeling of perceived naturalness and realism that let the user experience a high degree of absorption and embodiment. Even if the findings are still to be validated in an extensive quantitative study, the insights of this research can deliver a promising avenue for further examination. An investigation on cognitive absorption with VR gloves would be needed to unlock the full potential of this technology for virtual task operation and training conditions, which are currently the main applications of this technology in the field (see implications for practice) (Caeiro-Rodríguez et al., 2021). Haptic feedback, as a system operation factor, could set a further layer for this research to highlight the distinct influence of haptic feedback on natural mapping and motor control. We suggest a general distinction between haptic systems, as given by Caeiro-Rodriguez et al. (2021), and investigate actual feedback mechanisms on their natural mapping capabilities in system operation and changing outcomes on presence. Furthermore, research on new information processing and learning possibilities could be a valuable extension.

This work provides a clear picture of how natural mapping and presence are interrelated to the concept of cognitive absorption. The responses in the interviews clearly indicate a relationship between these variables. From this point of view, natural mapping can be taken into action as a possible mechanism for presence facilitation with VR input devices and VR gloves. Future research can profit from our study by receiving a user-centric perspective on natural mapping. The qualitative findings highlight the relationship between naturally mapped input devices and users and present possible fields for new research. With this article at hand, science on naturally mapped input devices can set the users’ experience in the focus of the development of new input devices and avoid a too-specific focus on single psychological mechanisms that solely inspect these phenomena on a technologically driven level.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

RP: writing–original draft. SW: writing–review and editing. MW: writing–review and editing. BN: writing–review and editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research was funded by the German Federal Ministry of Education and Research in the context of the VR-Chain project at the University of Bremen. In the scope of funding are projects on digitization in small- and medium-sized companies.

Acknowledgments

The authors thankfully acknowledge our project partners from Vetter Krantechnik GmbH, ISF München, and AERO-Lift GmbH for giving valuable insights about the research design phase. They are tremendously grateful to each participant who participated in the experiment. Furthermore, they would like to appreciate the work of Andreas Weigel, who made this project and the research conditions possible and gave further valuable input to the research design. The authors hope this research helps design training scenarios and highly immersive VR levels that implement VR gloves and naturally mapped input devices for their purpose.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2024.1337959/full#supplementary-material

References

Agarwal, R., and Karahanna, E. (2000). Time flies when you're having fun: cognitive absorption and beliefs about information technology usage. MIS Q. 24 (4), 665. doi:10.2307/3250951

CrossRef Full Text | Google Scholar

Allgaier, M., Chheang, V., Saalfeld, P., Apilla, V., Huber, T., Huettl, F., et al. (2022). A comparison of input devices for precise interaction tasks in VR-based surgical planning and training. Comput. Biol. Med. 145 (145), 105429–9. doi:10.1016/j.compbiomed.2022.105429

PubMed Abstract | CrossRef Full Text | Google Scholar

Berkman, M. I., and Akan, E. (2019). “Presence and immersion in virtual reality,” in Encyclopedia of computer Graphics and games (Cham: Springer), 1–10. doi:10.1007/978-3-319-08234-9_162-1

CrossRef Full Text | Google Scholar

Caeiro-Rodríguez, M., Otero-González, I., Mikic-Fonte, F. A., and Llamas-Nistal, M. (2021). A systematic review of commercial smart gloves: current status and applications. Sensors Basel, Switz. 21 (8), 2667. doi:10.3390/s21082667

CrossRef Full Text | Google Scholar

Csikszentmihalyi, M. (1990). Flow: the psychology of optimal experience. 1st ed. HarperPerennial.

Google Scholar

Csikszentmihalyi, M. (1993). The evolving self: a psychology for the third millennium. Soc. Hum. Ecol. (1). Article 1, 190–192. doi:10.24251/HICSS.2018.173

CrossRef Full Text | Google Scholar

Fahmi, F., Tanjung, K., Nainggolan, F., Siregar, B., Mubarakah, N., and Zarlis, M. (2020). Comparison study of user experience between virtual reality controllers, leap motion controllers, and senso glove for anatomy learning systems in a virtual reality environment. IOP Conf. Ser. Mater. Sci. Eng. 851 (1), 012024. doi:10.1088/1757-899X/851/1/012024

CrossRef Full Text | Google Scholar

Gamboa, E., Serrato, A., Toro, D., and Trujillo, M. (2019). “Advantages and limitations of leap motion for developing physical rehabilitation exergames (PREGs): problem,” in Proceedings of the 5th workshop on ICTs for improving patients rehabilitation research techniques. Editors H. M. Fardoun, S. A. Pérez, A. A. M. Hassan, and E. de La Guia (ACM Digital Library), 43–46. doi:10.1145/3364138.3364149

CrossRef Full Text | Google Scholar

Gibbs, J. K., Gillies, M., and Pan, X. (2022). A comparison of the effects of haptic and visual feedback on presence in virtual reality. Int. J. Human-Computer Stud. 157, 102717. doi:10.1016/j.ijhcs.2021.102717

CrossRef Full Text | Google Scholar

Gonzalez-Franco, M., and Peck, T. C. (2018). Avatar embodiment. Towards a standardized questionnaire. Front. Robotics AI 5, 74. doi:10.3389/frobt.2018.00074

CrossRef Full Text | Google Scholar

Gusai, E., Bassano, C., Solari, F., and Chessa, M. (2017). “Interaction in an immersive collaborative virtual reality environment: a comparison between leap motion and HTC controllers,” in Lecture notes in computer science. New trends in image analysis and processing – iciap 2017. Editors S. Battiato, G. M. Farinella, M. Leo, and G. Gallo (Cham: Springer International Publishing), 290–300. doi:10.1007/978-3-319-70742-6_27

CrossRef Full Text | Google Scholar

Hufnal, D., Osborne, E., Johnson, T., and Yildirim, C. (2019). “The impact of controller type on video game user experience in virtual reality,” in 2019 IEEE Games, Entertainment, Media Conference (GEM), New Haven, CT, USA, 18-21 June 2019. doi:10.1109/GEM.2019.8811543

CrossRef Full Text | Google Scholar

Jahn, K., Kampling, H., Klein, H. C., Kuru, Y., and Niehaves, B. (2018). Towards an explanatory design theory for context-dependent learning in immersive virtual reality. In PACIS, p. 235.

Google Scholar

Kampling, H. (2018). The role of immersive virtual reality in individual learning. HICCS 2018 (51). doi:10.24251/HICSS.2018.173

CrossRef Full Text | Google Scholar

Kim, M., Jeon, C., and Kim, J. (2017). A study on immersion and presence of a portable hand haptic system for immersive virtual reality. Sensors Basel, Switz. 17 (5), 1141. doi:10.3390/s17051141

CrossRef Full Text | Google Scholar

Mayring, P. (2022). Qualitative Inhaltsanalyse: grundlagen und Techniken [Qualitative Content Analysis] (Neuausgabe). Beltz Verlagsgruppe. Available at: http://nbn-resolving.org/urn:nbn:de:bsz:31-epflicht-2019387.

Google Scholar

McEwan, M. W., Blackler, A. L., Johnson, D. M., and Wyeth, P. A. (2014). “Natural mapping and intuitive interaction in videogames,” in Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play. Editors L. E. Nacke, and T. N. Graham (ACM Digital Library), 191–200. doi:10.1145/2658537.2658541

CrossRef Full Text | Google Scholar

McGloin, R., and Krcmar, M. (2011). The impact of controller naturalness on spatial presence, gamer enjoyment, and perceived realism in a tennis simulation video game. Presence Teleoperators Virtual Environ. 20 (4), 309–324. doi:10.1162/PRES_a_00053

CrossRef Full Text | Google Scholar

Monteiro, D., Liang, H.-N., Wang, J., Chen, H., and Baghaie, N. (2020). “An in-depth exploration of the effect of 2D/3D views and controller,” in 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Porto de Galinhas, Brazil, 09-13 November 2020.

Google Scholar

Moon, H. S., Orr, G., and Jeon, M. (2023). Hand tracking with vibrotactile feedback enhanced presence, engagement, usability, and performance in a virtual reality rhythm game. Int. J. Human–Computer Interact. 39 (14), 2840–2851. doi:10.1080/10447318.2022.2087000

CrossRef Full Text | Google Scholar

Mütterlein, J., Jelsch, S., and Hess, T. (2018). Specifics of collaboration in virtual reality: how immersion drives the intention to collaborate. PACIS Proc.

Google Scholar

Ryan, R., Bowman, N., and Oliver, M. B. (2015). It’s not the model that doesn’t fit, it’s the controller! The role of cognitive skills in understanding the links between natural mapping, performance, and enjoyment of console video games. Comput. Hum. Behav. 49, 588–596. doi:10.1016/j.chb.2015.03.027

CrossRef Full Text | Google Scholar

Schultze, U. (2010). Embodiment and presence in virtual worlds: a review. J. Inf. Technol. 25 (4), 434–449. doi:10.1057/jit.2010.25

CrossRef Full Text | Google Scholar

Skalski, P., Tamborini, R., Shelton, A., Buncher, M., and Lindmark, P. (2011). Mapping the road to fun: natural video game controllers, presence, and game enjoyment. New Media & Soc. 13 (2), 224–242. doi:10.1177/1461444810370949

CrossRef Full Text | Google Scholar

Steuer, J. (1992). Defining virtual reality: dimensions determining telepresence. J. Commun. 42 (4), 73–93. doi:10.1111/j.1460-2466.1992.tb00812.x

CrossRef Full Text | Google Scholar

Tamborini, R., and Bowman, N. (2010). Presence in video games.

Google Scholar

Tang, M., Liu, X., Dong, Y., Tang, Z., Huo, H., Fan, L., et al. (2023). Absence of inertial load on hand decreases task performance in virtual reality interaction. Int. J. Human–Computer Interact., 2023. 1–15. doi:10.1080/10447318.2023.2184960

CrossRef Full Text | Google Scholar

Tomborini, R., and Skalski, P. (2006). The role of presence in the experience of electronic games. Lawrence Erlbaum Associates Publishers, 225–240.

Google Scholar

Wang, X., Monteiro, D., Lee, L.-H., Hui, P., and Liang, H.-N. (2022). VibroWeight: simulating weight and center of gravity changes of objects in virtual reality for enhanced realism. Available at: http://arxiv.org/pdf/2201.07078v1.

CrossRef Full Text | Google Scholar

Yu, C.-M., Chang, H.-T., and Chen, K.-S. (2018). Developing a performance evaluation matrix to enhance the learner satisfaction of an e-learning system. Total Qual. Manag. Bus. Excell. 29 (7-8), 727–745. doi:10.1080/14783363.2016.1233809

CrossRef Full Text | Google Scholar