Triggermuscle: Exploring Weight Perception for Virtual Reality Through Adaptive Trigger Resistance in a Haptic VR Controller
Carolin Stellmacher
Michael Bonfert
Ernst Kruijff
Johannes Schöning- 1Faculty of Mathematics and Computer Science, University of Bremen, Bremen, Germany
- 2Institute of Visual Computing, Bonn-Rhein-Sieg University of Applied Sciences, Sankt Augustin, Germany
- 3School of Computer Science , University of St. Gallen, St. Gallen, Switzerland
It is challenging to provide users with a haptic weight sensation of virtual objects in VR since current consumer VR controllers and software-based approaches such as pseudo-haptics cannot render appropriate haptic stimuli. To overcome these limitations, we developed a haptic VR controller named Triggermuscle that adjusts its trigger resistance according to the weight of a virtual object. Therefore, users need to adapt their index finger force to grab objects of different virtual weights. Dynamic and continuous adjustment is enabled by a spring mechanism inside the casing of an HTC Vive controller. In two user studies, we explored the effect on weight perception and found large differences between participants for sensing change in trigger resistance and thus for discriminating virtual weights. The variations were easily distinguished and associated with weight by some participants while others did not notice them at all. We discuss possible limitations, confounding factors, how to overcome them in future research and the pros and cons of this novel technology.
1 Introduction
Grabbing objects in reality provides humans with a haptic sensation of weight (Loomis and Lederman, 1986). Muscles, tendons and skin receptors sense the gravitational pull through proprioceptive and cutaneous stimuli (McCloskey, 1974; Brodie and Ross, 1984) and enable the natural perception of weights. Users of consumer virtual reality (VR) systems, however, cannot experience the same haptic weight sensation when grabbing virtual objects. Current consumer VR controllers are unable to render appropriate haptic feedback associated with weight, leaving users with identical haptic weight perception of virtual objects. In many cases, only visual cues are ambiguous and insufficient in conveying weight information. This limits a natural and realistic experience of weight in VR and results in a discrepancy between what users are familiar with from the real world and what they experience in virtual environments (VE).
Handheld controllers are typically used in consumer VR systems. With increasing consumer attention, these lightweight and mobile VR controllers become more relevant for designing interactions in VEs. They offer various components to register user input such as buttons, a trigger, or a trackpad, which provide users with different techniques to interact with the VE. However, to provide a haptic response during the interaction (e.g., the weight of a virtual object), current state-of-the-art VR controllers such as the HTC Vive or Oculus Touch controllers only offer vibrotactile rendering. This prevents users from having an appropriate haptic experience. It also rules out haptic feedback, which is specific to user input. To nevertheless enhance weight perception with consumer VR controllers and to overcome their current hardware limitations, pseudo-haptics were explored. This software-based approach manipulates the control/display (C/D) ratio between a user’s hand movements and the rendered position in the VE during a lifting motion (Rietzler et al., 2018). This sensation of weight is caused by the user’s arm movements being amplified when lifting heavier virtual objects with the controller. This effect was also observed when users lifted physical props in VR (Samad et al., 2019). While this method enables conveying a sense of weight in current consumer VR, it cannot resemble an actual haptic sensation of weight since users only receive visual feedback, but no haptic weight information.
Other researchers have addressed the hardware limitations of haptic weight rendering by proposing various lightweight and mobile VR devices. For such ungrounded haptic devices, it is particularly challenging to provide proprioceptive information as they cannot generate externally grounded forces that act on users. As an alternative, cutaneous stimuli are often used as a substitute to offset the lack of force feedback. For instance, a simulation of muscular grip forces is imitated through skin deformation at the finger pads of the index finger and thumb (Minamizawa et al., 2007a; Schorr and Okamura, 2017; Suchoski et al., 2018) or through skin stretch with asymmetric vibrations (Choi et al., 2017). Other wearable technology utilises electrical muscle stimulation to resemble a proprioceptive sensation by artificially pulling the user’s arm downwards (Lopes et al., 2017). Other research has explored handheld devices with liquid-based haptic feedback to simulate the weight of fluid objects (Cheng et al., 2018), shape-changing abilities to provide haptic feedback through air resistance (Zenner and Krüger, 2019) or weight-shifting abilities (Zenner and Krüger, 2017) to generate haptic sensations for weight.
While these proposed haptic interfaces have succeeded in enhancing weight perception in VR, they are either designed for specific cases, rely on complex and expensive systems or need to be manufactured for different hand sizes. This makes them currently unsuitable for mass production and the consumer market. So far, no approach has considered established input components of current consumer VR controllers to render haptic stimuli for weight perception in VR.
In this paper, we propose an approach that—in contrast to related studies—utilises a standard button available in any consumer VR controller: the trigger. By varying the resistance of the trigger, which is normally constant by default, we extend the input component through output rendering to provide users with haptic feedback during the interaction. Hence, when users pull the trigger to grab a virtual object in VR, they need to scale their index finger force accordingly to the configured resistance displaying the virtual weight: The heavier a held virtual object is, the more finger force needs to be exerted onto the trigger.
We present Triggermuscle, a novel haptic controller that simulates the weight of virtual objects in VR through adjustable trigger resistance. As proof of concept, our system is built into the casing of an HTC Vive controller (see Figure 1). The novel spring mechanism is connected internally to the original trigger and dynamically modifies the trigger resistance according to the weight of the grabbed virtual object. A demonstration video of Triggermuscle is submitted as a Supplementary Material. The mechanism is built with inexpensive hardware components and can be easily tailored to other form factors due to the principle of force redirection, which demonstrates the potential for haptic weight rendering in different VR controllers. Additionally, as a handheld device, Triggermuscle fits a large range of users with various hand sizes. Enriching buttons with additional haptic feedback is increasingly evident in input device development. For instance, Sony recently released the DualSense controller for PlayStation 5 with actuated triggers (Sony, 2020), while Microsoft announced a locking feature for the triggers of the Xbox Elite controller (Microsoft, 2021).
FIGURE 1. (left) Our haptic controller Triggermuscle simulates the weight of virtual objects in VR through adapting the trigger resistance. The spring mechanism for a dynamic adjustment is built into the casing of an HTC Vive controller. An HTC Vive tracker is mounted to the top to enable spatial tracking. (right) Triggermuscle allows a continuous regulation of the trigger resistance according to the weight of the grabbed virtual object.
What follows explores the capacity of variable trigger resistance to display virtual weight in VR. To do so, we present the design and implementation of a preliminary prototype and of Triggermuscle as well as their different spring mechanisms, which we developed in an iterative and human-centred design process. In two user studies, we evaluate both controllers’ hardware designs and investigate users’ ability to discriminate different levels of resistance as well as the effect on weight perception in VR. Our research addresses two research questions:
RQ1: Do different trigger resistances influence the haptic perception of different virtual weights in VR?
RQ2: How can the intensity of the trigger resistance be quantified and mapped to convey distinguishable virtual weights?
We found large differences between participants for sensing change in trigger resistance and thus for discriminating different virtual weights based on the resistance. The variations were easily distinguished by some participants while others did not notice them at all. This points towards an association between trigger resistance and a sense of virtual weight in VR, but also towards confounding factors interfering with the perception of the trigger resistance. We discuss these issues to provide better insight into the problem space, to illustrate potential perceptual mechanisms that may have affected the experiment outcomes and to propose further studies and hardware designs based on our findings. We thus provide a novel hardware solution and, better understanding of the underlying (perceptual) mechanisms.
2 Related Work
Extensive research in VR haptics has explored various strategies to enable experiencing physical properties of virtual objects during the interaction. This section presents an overview of grounded and handheld haptic technologies and emphasises approaches that convey a sense of weight in VR.
2.1 World- and Body-Grounded Haptic Devices
Humans feel an object’s mass through the gravitational force pulling down the hand. To simulate this external pull in a force feedback display, the interface can be anchored in the environment or to the user’s body. A widespread example for world-grounded interfaces is the Phantom Premium by 3D Systems (Massie and Salisbury, 1994; Systems, D., 2020), a 6 degrees-of-freedom (DoF) interface with a stylus as an effector mounted to the desk. A different approach anchors the force to the users’ fingers via strings as done by the various SPIDAR interfaces (Sato, 2002) for simulating virtual weight in an early implementation (Ishii and Sato, 1994). Such wire-based force display systems were also adapted into body-grounded technologies to increase portability such as the HapticGEAR (Hirose et al., 2001) or Wireality for rendering complex shapes (Fang et al., 2020). Another wire-based variant anchors the user’s hand to the respective upper arm (Tsai et al., 2019). By stretching the arm, the wire gets tightened and the user experiences multilevel resistive force and impact. Thanks to their mobility, these body-grounded interfaces allow users to move freely within the tracking area and are suitable if a larger space is required. At the same time, they typically need cumbersome and complex equipment that is time-consuming to set up.
Haptic interfaces worn on users’ hands such as exoskeletons provide force feedback (Burdea et al., 1992; Bouzit et al., 2002; Ben-Tzvi and Ma, 2015; Gu et al., 2016; HaptX, 2020) that is grounded to different parts of the hand or arm (Nisar et al., 2019) and that can actively restrict the finger movement with motors and complex mechanics. This can make such devices bulky, tethered, expensive or limit hand flexibility. In contrast, gloves (Giannopoulos et al., 2012; Martínez et al., 2016; Marquardt et al., 2018; Manus, 2020) typically do not use motors and can, therefore, overcome shortcomings related to the actuation. Gloves can also track users’ fingers and provide vibrotactile feedback or other cutaneous stimuli. Researchers rendered various physical properties of virtual objects including shape (Solazzi et al., 2007), contact forces (Leonardis et al., 2015), texture (Gabardi et al., 2016), or inertia (Girard et al., 2016). Nonetheless, putting on gloves or hand-mounted equipment like exoskeletons and thimbles can be time-consuming and requires hygienic considerations. Further, they need to be adjustable or manufactured in different sizes to fit a diverse range of users.
2.2 Handheld Haptic Devices
In contrast to such wearable haptic interfaces, handheld haptic devices are ready for use when picked up. They do not physically restrict the user’s movements and flexibly fit a large range of hand sizes. Most current consumer VR systems include such handheld controllers by default. In recent years, the development of handheld haptic devices that generate physical forces for haptic feedback as well as controller-based interaction techniques has received considerable attention. For instance, the CapstanCrunch allows to feel rigid and compliant objects (Sinclair et al., 2019), the TORC creates a haptic sensation for texture and compliance (Lee et al., 2019) and the controllers NormalTouch and TextureTouch render shape and texture through tilting a platform at the user’s index finger pad (Benko et al., 2016). PaCaPa is a haptic display for an object’s size, shape and stiffness by tilting movable wings, but cannot render resistance (Sun et al., 2019). Haptic Links generates resistance by mechanically constraining the relative movement between two controllers (Strasnick et al., 2018), while Thor’s Hammer creates force feedback through airflow with propellers (Heo et al., 2018). The CLAW is a handheld device that integrates multiple haptic technologies to simulate a range of haptic sensations (Choi et al., 2018). It renders kinaesthetic forces at the index finger during grasping and touching which allows feeling the shape and stiffness of virtual objects. Additionally, a voice coil actuator produces vibrations for different surface textures. Another interface for displaying surface properties is the Haptic Revolver (Whitmire et al., 2018). Shapes and shear forces that occur when gliding along a surface are rendered at the fingertip by rotating a wheel with a direct current (DC) motor. The haptic wheels are customisable and can provide various textures and shapes, but can also comprise active electronic components such as buttons, switches and joysticks. Transcalibur enables shape perception through inertia (Shigeyama et al., 2019). Variable weight distributions along the controller are realised with shifting weights. Moving the controller through space makes the inertia noticeable for users and creates a haptic shape illusion. A similar concept was proposed with ShapeSense. Movable surface elements increase or decrease the surface area of the controller (Liu et al., 2019). While these mechanisms can render the distribution of an object’s mass, they cannot render the absolute mass of objects. Furthermore, even though the proposed devices allow haptic rendering for various object properties, they do not address the weight of virtual objects.
2.3 Haptic Devices for Weight Simulation
To provide users with a weight sensation during the interaction with virtual objects in VR, previous work explored diverse approaches. For example, electrical muscle stimulation was used to induce contractions of the user’s muscles while lifting virtual objects. The system actuates the user’s triceps to simulate the weight of a held virtual object by inducing a downward movement of the arms (Lopes et al., 2017). Another concept of inducing a sense of weight for VR applications is skin deformation on the finger pad. This imitates the stretch of the skin from the downward pull of the object’s surface. One way to do so is using small actuated belts that are strapped around users’ finger pads which has been shown to generate a reliable weight sensation (Minamizawa et al., 2007b). The refined implementation of this approach induced the impression of grip force, gravity and inertia by stretching the skin on the finger pad with the attached belt, without the need for proprioceptive sensations (Minamizawa et al., 2007a). This type of approach was also explored in the context of augmented reality (AR) rendering weight and shear forces (Scheggi et al., 2010). To account for the combination of physical and virtual objects in AR, another implementation placed the finger-worn device as a ring around users’ fingers leaving the finger pads free for the interaction with physical objects (Pacchierotti et al., 2016; Maisto et al., 2017). Other haptic devices use actuated plates to achieve skin stretch such as a finger-worn device that slides the contact area at the user’s index finger pad to mimic weight and friction (Kurita et al., 2011). Further, scaling inertial forces rendered with a 3DoF wearable device on the finger pad showed an increase of the perceived weight of an object moved by the user (Suchoski et al., 2018). A different implementation demonstrates a handheld controller with movable plates in its handle to resemble the friction between an object and the hand during grasping (Provancher, 2014). Another finger-mounted device is Grabity, which simulates grip forces and a sensation of weight. The device is mounted on the thumb, index and middle finger and applies kinaesthetic forces for rendering shape. To render weight, asymmetric vibrations of voice coil actuators stretch the skin at the finger pad resembling the pull of gravity. The participants in the evaluation successfully distinguished the objects of different weight but felt the vibration cues even stronger than the weight cues (Choi et al., 2017).
The handheld VR controller Drag:on adjusts its surface area to generate varying haptic sensations for experiencing drag and weight. As the concept depends on air resistance, the different object properties are only noticeable when the device is moved through space. Due to the flat controller design built with fans, the effect is dependent on the orientation of the controller (Zenner and Krüger, 2019). The same authors also created Shifty, which enhances the perception of the dimensions of virtual objects by changing the controller’s weight distribution. An internal weight is moved along the longitudinal axis shifting the centre of mass away from the hand. This increases the leverage and therefore feels like holding a heavier object (Zenner and Krüger, 2017). An increase in the possible rendered shift was achieved through combining the haptic device with haptic retargeting (Zenner et al., 2021). Rendering shifting weights on a 2D plane was also achieved in a handheld controller using jet propellers. Aero-plane generates force feedback with up to 14 N that can be interpreted by the user as weight while holding the device level. This was found to increase the perceived immersion and realism (Je et al., 2019). Finally, in GravityCup, the actual weight of the device changes. It is filled with water or emptied again to render inertia and weight of liquids. The user holds the interface by a handle like a cup. The interface requires a separate wearable bag with water to fill the haptic display as needed (Cheng et al., 2018). So far, these proposed devices often rely on complex hardware, might feel cumbersome to users or target specific use cases which limits their use for haptic weight rendering in commercial VR.
2.4 Software-Based Approaches for Weight Simulation
As discussed, haptic devices are limited in their application by a number of factors. Moreover, one additional constraint is the availability of the hardware. Beyond technical feasibility, a haptic display needs to be universal, flexible and affordable enough to be established as a standard interface in VR interaction. To overcome hardware limitations and deliver haptic experiences readily available to users, researchers proposed various software solutions for pseudo-haptics. This term describes haptic illusions through visual, auditory or multimodal stimuli without actual touch.
In terms of weight perception, the manipulation of the C/D ratio between users’ hand movements and the rendered position in VR has proven effective. It was demonstrated in a non-VR setup that this mismatch strongly influences the perception of mass (Dominjon et al., 2005). The effect has been replicated successfully in VR. In an experiment, participants lifted two physical boxes with their hands. An increase in the offset for heavier virtual boxes resulted in an amplification of users’ hand movements and a heavier perceived weight (Samad et al., 2019). This method was also applied to the interaction with a consumer VR controller and produced corresponding results (Rietzler et al., 2018).
Analogously to the modification of the translational C/D ratio, another approach changed the rotational C/D ratio depending on an object’s weight, thus, the rotational motion is scaled relative to the mass of the object. A user study confirmed that this method effectively, realistically and robustly conveys different weights. At the same time, it does not compromise the perceived controllability. Furthermore, the authors proposed the manipulation of the pivot point during rotation and the scaling of rotational motion to convey the distribution of mass within an object (Yu and Bowman, 2020).
Such software-based approaches have been shown to provide users with an experience of virtual weight, but they cannot render actual force stimuli. This limits their ability to haptically convey weight in VR. Unlike previous devices or software-based approaches, Triggermuscle offers a novel hardware solution built into a commercial handheld VR controller to enhance weight perception during the interaction with a virtual object in VR. With our technology, we extend the trigger’s capabilities towards generating haptic feedback and explore the effect of adaptive trigger resistance on the perception of virtual weight in VR.
3 Adaptive Trigger
This section describes the background in haptic weight perception during grasping and how we addressed this in the concept and the first implementation of the adaptive trigger inside a prototype. The section concludes with the evaluation of the prototype in a pilot study and the findings that influenced the development of Triggermuscle.
3.1 Background in Weight Perception
Our adaptive trigger is informed by humans’ perception of weight cues through the haptic sense (Loomis and Lederman, 1986). In addition, to grasp and lift an object the human brain initially incorporates visual cues (Gordon et al., 1991) and previous lifting experiences (Van Polanen and Davare, 2015) to predict an object’s weight and scale finger forces accordingly. Touching and lifting the object then supplies simultaneous haptic cues obtained from cutaneous stimuli registered by receptors in the skin and proprioceptive stimuli obtained from muscles and tendons (McCloskey, 1974; Brodie and Ross, 1984). Depending on the updated weight perception, enough grip force is applied to overcome the gravitational pull, but at the same time causing no damage to the object (Westling and Johansson, 1984). The result is a direct relationship between the physical weight and the applied grip forces: The heavier the object, the more manual force needs to be applied. This principle forms the main instigator for our hardware design. Increasing the grip force to a sufficient amount is enabled by isometric contractions in the muscles of the hand and arm (Johansson and Westling, 1988), meaning the muscle tension is adjusted accordingly to the weight, but no muscle movement takes place. When lifting the object away from the supporting surface, the contractions switch to isotonic which keeps the muscle tension static while the length of the muscle changes, e.g., to flex the elbow.
Grip forces are not only scaled according to the gravitational pull, but are also influenced by various tactile cues derived from material properties such as surface texture (surface-weight illusion) (Johansson and Westling, 1984; Flanagan et al., 1995) or material (material-weight illusion) (Ellis and Lederman, 1999) and spatial properties such as shape (shape-weight illusion) (Jenmalm and Johansson, 1997) or size (size-weight illusion) (Ellis and Lederman, 1993). These studies have shown that the illusions provoke modulated grip forces and influence the perceived weight. For instance, an increased grip force due to smoother surface texture leads to higher perceived weight (Johansson and Westling, 1984; Flanagan et al., 1995). In such cases, cutaneous receptors detect less frictional force between the skin and the object’s surface leading to higher grip forces to prevent the object from slipping. Based on their findings, the researchers argued that “grip force may be a useful cue for discriminating weight” (Flanagan et al., 1995).
Such haptic illusions caused by stimuli unrelated to gravity could contribute to a successful substitution of haptic weight cues occurring during the grip. The sensory substitution implies that haptic stimuli are registered through another sense as they normally are or at a different location (Kaczmarek et al., 1991). This is necessary for most haptic interfaces to compensate for the lack of corresponding physical stimulation when virtually interacting with an object. Especially handheld VR devices displaying virtual weight rely on substitutional stimuli to compensate for the lack of gravitational force pulling down the user’s hand. Simulating this force has been previously done, e.g., by deforming the user’s skin at the fingertips through stretching the skin or through asymmetric vibrations (Choi et al., 2017). Moreover, as haptic stimuli unrelated to gravity such as surface texture or material have been shown to create weight illusions and induce a modulation of grip forces, we assume that providing variable resistances as a haptic stimulus at the users’ index fingers and thereby provoking a modulation of grip forces might enable a haptic weight perception in VR.
3.2 Concept
The concept of our adaptive trigger incorporates the previously described relation between grip forces and the perceived weight and transfers it to the established interaction technique of any consumer VR controller. With such devices, grabbing a virtual object typically involves pulling the trigger, which requires muscle force of users’ index fingers to overcome the constant resistance. By adjusting this resisting force, substitutional stimuli (Kaczmarek et al., 1991) are displayed as weight cues and users need to adjust their index finger forces according to the weight of the grabbed virtual object. For example, the heavier the virtual object, the stronger the trigger must be pulled. An illustration of the intended effect is shown in Figure 2. Early evidence has demonstrated that the pull of the trigger can be interpreted as varying the grip force that the user exerts onto a virtual object enabling loose and firm grasping for controlling the object’s rotation (Bonfert et al., 2019). As the grip force required for holding an object correlates with its weight, higher trigger resistance provoking higher index finger forces might consequentially be interpreted as increased weight.
FIGURE 2. Modulated finger forces from reality occurring during the grasping of objects with different weights are translated into the resistance of the trigger.
While our adaptive trigger transfers the haptic recognition of weight onto a one-finger interaction, other haptic interfaces have demonstrated that rendering haptic stimuli only at users’ index fingers can be sufficient to enhance haptic object perception in VR (Benko et al., 2016; Choi et al., 2018; Whitmire et al., 2018). In particular, applying resistive forces to restrict the index finger’s movement during grasping has been shown to enhance the perception of rigid and compliant objects (Sinclair et al., 2019). With Sony and Microsoft incorporating haptic feedback into their triggers for game experience, the future availability thereof is another strong argument for exploring the potential of triggers with adaptive resistance for the perception of various haptic events, in our case weight perception.
We implemented our adaptive trigger in an iterative process following a human-centred design approach. First, we built a prototype of a spring mechanism to dynamically adjust the trigger resistance which we present in section 3.3. We evaluated the effectiveness of the resistance range and the technical implementation in a pilot study. Based on our findings, we revised the spring mechanism and built the improved haptic controller Triggermuscle, shown in section 4.
3.3 Prototype
The first implementation of our adaptive trigger is based on the typical construction of standard triggers: Pulling the trigger compresses one leg of a torsion spring whereby its tension exerts a force in the opposite direction, i.e., resisting the finger’s pull. To establish a change in tension force, we constructed a mechanism that rotates the spring’s second leg before a pull motion, increasing or decreasing its angle. The resulting adjustment of the trigger resistance is dynamically performed by a high-voltage (7.4 V) digital micro servo (BMS-115HV) which rotates the usually fixed leg via a connected tilting platform. The entire mechanism is built into the casing of an HTC Vive controller and connected to the original trigger, as illustrated in Figure 3. In contrast to the later Triggermuscle controller, the prototype fully accommodates the servo inside the casing.
FIGURE 3. (top left) Schematic illustration of the spring mechanism of our prototype utilising a torsion spring. (bottom left) Spring mechanism embedded into the casing of an HTC Vive controller. (right) Our prototype for adaptive trigger resistance. Electrical components are connected via cable to the prototype’s bottom and carried in a small bag. An HTC Vive tracker attached on top ensures the spatial tracking of the device.
With our used spring model, the prototype renders a continuous range of resistance between 19.27 Nmm and 47.61 Nmm with a fully pulled trigger. Beyond that, users could increase the finger force further, but without moving the trigger or changing the input, identical to a conventional trigger. The chosen spring model offered the greatest possible resistance range when installed inside the mechanism and its level of resistance was closely located to the one of an original Vive controller. Informal testing in the lab with three users suggested comparability between the middle value of the prototype’s range and the trigger resistance of the original HTC Vive controller. The resistance values are calculated based on the path-force ratio of the spring model (29.44 Nmm/103.89◦), the respective compression angle computationally set by the servo, the additional 18◦ compression when the trigger is pulled and the spring’s preloaded angle of 50◦ when installed inside the mechanism. The latter two angles were carefully measured manually using visual scales. The prototype’s total resistance range achieves a maximum increase of 147%. Humans are known to perceive a difference in spring stiffness between 15 and 22%, also known as the Weber Fraction (WF) (Jones and Tan, 2013).
Apart from the modified resistance, the haptic sensation of pulling the trigger is maintained, including the final click. This occurs when users fully pull the trigger which then mechanically pushes the original mini button that registers the digital signal at the maximum limit. The signal is send via cable to an ESP32 microcontroller unit (MCU) which also drives the servo and communicates with Unity 2018.3 via Bluetooth. Along with a 11.1 V lithium polymer battery and a battery eliminator circuit (BEC) component, the MCU is carried in a small bag on the user’s back and connected to the controller’s bottom via cable. Since the original tracking components were removed, an HTC Vive tracker 1.0 is mounted to the top of the controller.
3.4 Pilot Study
Our pilot study evaluated users’ ability to perceive and discriminate different resistances in VR while using the prototype. Similar to previous haptic device research (Dominjon et al., 2005; Maereg et al., 2017; Suchoski et al., 2018; Ryu et al., 2020), we conducted a psychophysical experiment to measure the just noticeable difference (JND) of adaptive trigger resistance in VR. In addition, we also carried out semi-structured interviews to qualitatively assess subjective perception. In accordance with our research goal, to explore weight perception through adaptive trigger resistance, our initial objective also included the prototype’s effect on the perception of virtual weight. However, preliminary testing, in which eight participants lifted and compared the weight of two boxes in VR (see Figure 4 for the VE), suggested that weight perception was not influenced by the varying intensity of the trigger resistance. Based on participants’ reports, we assumed that the visual modality of the box-lifting task dominated the perception. To address this issue, we simplified the visual input and focused, as a first step in the pilot study, only on users’ ability to discriminate different trigger resistances, without additional weight perception. The simplified pilot study is described below.
FIGURE 4. Setup in the VE used for the main user study. Both boxes had to be lifted and placed onto the platform right next to it. Virtual “HEAVIER”-buttons on both target platforms allowed participants to log in their response. The same setup was used for the preliminary testing of the pilot study.
3.4.1 Experiment
We recruited nine participants (two females, seven males) aged 21–50 (M = 28, SD = 9.22), of which the majority (7) reported previous VR experience. One participant did not produce valid data and was excluded from further analysis. At the beginning of the experiment, participants gave their consent to take part in the study and were not made aware of the altered trigger resistance to ensure unbiased experience. Throughout the experiment, participants used the prototype and wore the head-mounted display (HMD) of the HTC Vive system. Noise-cancelling headphones played neutral music to block the motion noise of the servo and to avoid possible bias.
To implement an interaction task that involved pulling the trigger, participants were asked to change the colour of a grey virtual wall in VR by pressing the button. In each trial, participants consecutively activated two colours, magenta or green. For each activation, a different trigger resistance was rendered. Each colour was then deactivated, returning the wall to grey, as soon as the trigger was released. At the end of each trial, participants chose that colour that felt heavier to activate and logged their response by touching the virtual interface button in the respective colour. To assess participants’ ability to discriminate different resistances, we followed the method of constant stimuli with a two-alternative forced choice (2AFC) paradigm (Jones and Tan, 2013), as this is said to produce more accurate results than alternative methods (Simpson, 1988; Guilford, 1955). Each activation was haptically rendered with different trigger resistances using the prototype described in section 3.3. In each trial, participants were presented with the same standard resistance (19.27 Nmm) and with one of four preselected comparison resistances (26.35, 33.44, 40.52, 47.61 Nmm). These were equally distributed along the prototype’s resistance range. The comparison resistances were computed based on the compression angles set by the servo and by the path-force ratio of our used torsion spring model. Typically, the comparison values are spaced on either side of the standard value (Jones and Tan, 2013). However, we were concerned that presenting only half of the range as the maximum resistance change might be too subtle to be noticed by participants. We, therefore, chose the minimum resistance as the standard value, similar to (Maiero et al., 2019). Each comparison stimulus was tested 10 times, resulting in a total of 40 trials. The order of all trials and the appearance of the standard and comparison resistance as well as the appearance of green and magenta within one trial were randomised. Three pre-task trials allowed participants to familiarise themselves with the procedure. Upon task completion, we carried out semi-structured interviews to assess participants’ self-reported experience.
3.4.2 Results
We measured the proportions of “heavier”-responses for each tested comparison resistance, plotted psychometric functions (PFs) and assessed the goodness-of-fit (Schütt et al., 2016) using the MATLAB toolbox psignifit 4 (Schütt, 2019). Based on the results, only three participants qualified for further JND computation, the point of subjective equality (PSE) and the WF. Three of the excluded data sets performed around the guess rate. The other two performed almost perfectly, not allowing an assessment of discrimination sensitivity, since the method of constant stimuli of psychophysical testing requires a decreasing range of correct responses between 100 and 50% in the case of our 2AFC task. For the remaining three participants, the average JND was 3.90 Nmm (SD = 0.79), resulting in an average WF of 14.70% (SD = 3.60). This level of sensory precision is slightly below the previously mentioned 15–22% WF in the literature of spring stiffness discrimination. However, due to the small number of considered data sets, the results should be treated with caution.
3.4.3 Discussion and Implications
At this point, the results remain inconclusive if the adaptive trigger resistance can be discriminated sufficiently in VR. Nonetheless, our findings offer the incentive to continue exploring the adaptive trigger resistance since the data sets of three participants suggest that their identification of the heavier activation was influenced by the intensity of the resistance. Further, two participants achieved an almost always perfect identification which further indicates an influence of the adaptive trigger resistance on their perception. Our findings also highlight two key limitations with the prototype’s haptic feedback which we will discuss in the following.
Simplifying the visual input after the preliminary testing showed an improvement in participants’ perception of the adaptive trigger resistance. We assume it enabled a shift in their attention and allowed participants to detect the provided haptic feedback by themselves, as they were kept unaware of it. This assumption is in line with other haptic VR research also observing a domination of the haptic sense by the visual sense (Ban et al., 2012; Azmandian et al., 2016; Choi et al., 2018; Degraen et al., 2019). Providing stronger haptic stimuli in future investigation might, therefore, achieve a more balanced perception of both senses. Consecutively, this might allow more users to sense the resistance as well as sense the resistance more strongly. To continue the exploration and investigate whether higher resisting forces improve the sensing of the adaptive trigger resistance and if they can influence the perception of weight in VR, we plan to establish a wider resistance range.
Furthermore, statements obtained from the interview showed that a subtle vibration occurring as a side effect during the servo’s adjustment was noticed by all participants. This suggests another possible diversion of participants’ haptic attention, additionally preventing them from recognising the resistance change. This could have again been amplified by the fact that participants did not know about the adaptive trigger resistance. We plan to address the vibration as a possible side effect through modifications in the hardware as well as software design. Modifying the hardware design by increasing the distance between the servo and participants’ hands, adding additional damping or by using a different type of actuation might help to reduce the exposure to the servo’s subtle adjustment vibrations. Implications for the software design are derived from our observations during the preliminary testing with the box-lifting task. The implemented software for that task adjusted the servo angle when participants reached for the virtual box and intersected its collider with the one from the virtual controller. To decouple the controller’s adjustment from the moment of participants preparing to pull the trigger, we plan to set the servo’s angle independent from the lifting motion.
4 Triggermuscle
To overcome the drawbacks of the prototype, we built Triggermuscle, which is shown in Figure 1. The key change implemented in Triggermuscle is the revised spring mechanism that utilises an extension spring. This allows a larger manipulation of the exerted force in contrast to the torsion spring used for the prototype. Therefore, pulling the trigger of Triggermuscle stretches the attached extension spring and makes the exerted force noticeable to users’ index fingers as the trigger resistance. Thus, changing the length of the extension spring enables the adjustment of the trigger resistance.
4.1 Implementation
The revised spring mechanism is again built into the casing of an HTC Vive controller and attached to the original trigger, as illustrated in Figure 5. The dynamic adjustment of the extension spring is established with a high-voltage (6.0 V) digital micro servo (BMS-210DMH). However, in contrast to the prototype, the servo was moved upwards and installed into the original trackpad component of the controller. The revised servo position increases the distance to the user’s hand and was intended to reduce the sensed vibrations reported in the pilot study. To provide further damping, a protective silicon cover for the HTC Vive controller was wrapped around the handle. With an attached pulley on top, the servo is connected to the extension spring via a thin 1 mm wire rope. Despite its small diameter, the wire rope is strong enough to handle the forces and inelastic to ensure accurate translation. Changing the servo’s angle rotates the attached pulley and winds the connected wire rope, thus pulling or releasing the spring. A second, smaller redirect pulley in the controller’s bottom secures a stable guiding of the wire rope. Due to this principle of force redirect, the mechanism can be tailored to other form factors for different interaction devices.
FIGURE 5. Schematic illustration of Triggermuscle’s spring mechanism utilising an extension spring.
The dimensions of the controller casing allow a spring stretch of up to 20.3 mm. For the selection of the suitable spring model, the same selection criteria were applied for Triggermuscle as for the prototype. In combination with the used extension spring model (spring rate of R = 0.592 N/mm, minimum force 1.33 N), a continuous regulation of the resistance is achieved from approximately 4.29–16.36 N with a fully pulled trigger (5 mm stretch adding 2.96 N). Increasing the finger force further beyond the set maximum limit does not move the trigger or change the input, identical to a conventional trigger. Note that we use the force unit [N] for the extension spring which differs from the unit [Nmm] for the torsion spring torque of the prototype to emphasise the correct units of the spring types and for the readers to easily identify the prototype and Triggermuscle. The total resistance range illustrates an increase of over 281%. In contrast, the prototype allowed a smaller increase of 147%. Triggermuscle, therefore, exceeds the previously tested range of trigger resistance of the prototype by 134%. For the purpose of comparison, both ranges are illustrated in Figure 6 in which the resistance values of the prototype were converted into [N].
FIGURE 6. Both ranges of trigger resistance of the prototype and Triggermuscle. The torsion spring’s torque values [Nmm] of the prototype were converted into [N] for the purpose of comparability.
Similar to the prototype, the haptic sensation of pulling the trigger is maintained, apart from the modified resistance. The MCU, the lithium polymer battery and the BEC are again carried in the bag previously used for the prototype. The final controller is shown in Figure 7. An HTC Vive tracker 1.0 attached to the controller’s top provides spatial tracking since the original tracking components were removed from the casing. The total weight of Triggermuscle including the HTC Vive tracker 1.0 (90 g) and its mounting (10 g) is 300 g. In comparison, the original HTC Vive controller weighs 200 g. The bag carrying the battery (250 g), MCU (10 g) and BEC (22 g) weighs 350 g. Further details are described in the workshop paper (Stellmacher, 2021).
FIGURE 7. (left) Our haptic VR controller Triggermuscle. (top right) Attachment of the HTC Vive tracker to enable spatial tracking. (right bottom) Electronic components (MCU, the battery and the BEC) are carried in a small bag.
5 Study
Our main user study evaluated the revised technical implementation of the adaptive trigger that is built into Triggermuscle. Our main objective was to explore whether the increased resistance range makes more users notice the change in intensity. In addition, we investigated whether differently intense trigger resistances resemble a perception of virtual weight in VR. To do so, this study repeated the psychophysical experiment of our pilot study by using the method of constant stimuli. However, we adapted the interaction task to the main goal of the user study and repeated the box-lifting task used in the preliminary testing of the pilot study.
5.1 Participants
We recruited 21 participants (five females, 16 males) aged 19–29 (M = 22.67, SD = 2.78). Most of them (19) stated previous VR experience, more than half (16) were familiar with non-VR game controllers. No participant was previously involved in the pilot study.
5.2 Task and Stimuli
Participants lifted and compared two visually identical boxes and identified the heavier one. The setup in the VE is shown in Figure 4. The virtual weight of both boxes was haptically rendered with different trigger resistances. In each trial, participants were presented with the same standard resistance and with one of five preselected comparison resistances. The value of the standard resistance was 4.29 N (0% of the range). The five comparison resistances were 4.46 N (2%), 4.79 N (5%), 6.09 N (19%), 8.67 N (46%) and 13.82 N (100%), with each being repeated ten times. Therefore, a total of 50 trials were conducted. The order of all trials and the appearance of the standard and comparison resistance within one trial was randomised. The possible maximum value of 16.36 N of Triggermuscle was restricted to 13.82 N for this user study to avoid wearing out the components. The lower half of the tested resistance range was covered by four comparison values since we considered the highest resistance to be easily recognised. We also assumed that participants would easily become aware of the change in the haptic feedback and consequently be more attentive to smaller changes. We also expected our sample to include participants who almost always successfully identify the heavier resistance as observed in two participants in the pilot study, and thus motivating the small 2% value. The crooked percentage values are due to the resolution of the servo angles.
5.3 Procedure
At the beginning of the task, participants received instructions but were kept unaware of the adaptive trigger resistance and possible vibrations as a side effect. They performed three pre-task trials in the VE to familiarise themselves with the procedure of lifting the boxes and selecting the heavier one. Three comparison resistances were rendered during those trials: 7.29 N (25% of the range), 7.72 N (29%) and 8.15 N (32%). Since we considered higher resistances of Triggermuscle to be easily recognised, the values were distributed around the lower third of the range to ensure an unbiased starting position for the psychophysical testing during the experiment task. To avoid possible influences of the servo’s motion noise, participants wore noise-cancelling headphones and listened to neutral music. After completing the experiment task, we carried out semi-structured interviews to assess the self-reported experience of participants.
6 Results
We assessed the influence of adaptive trigger resistance on the perception of virtual weight based on quantitative data recorded during the task and qualitative data from the interview.
6.1 Trigger Resistance Discrimination
The average number of “heavier”-responses for each tested resistance intensity are shown in Figure 8. We expected the number of “heavier”-responses to clearly decrease with decreasing resistance intensity. This behaviour, however, could not be observed in the average responses. For a more differentiated assessment, PFs for all 21 participants were fitted, again using the MATLAB toolbox psignifit 4 (Schütt, 2019). We also reassessed goodness-of-fit based on the calculated deviance to assess the proximity between the fitted dataset and the underlying model. It asymptotically converges to 1.0. However, the lower the mean proportion of correct responses, the higher the expected deviance is. A “typical cut off [is] around the value 2 for what is often regarded as a still” well behaved” data set” (Schütt et al., 2016).
FIGURE 8. Average number of “heavier”-responses for each tested resistance intensity from our main user study with error bars representing the standard errors. The maximum possible value is ten due to the total amount of ten trials per intensity level. A decrease was expected for the number of “heavier”-responses over the decreasing range of the resistance intensity, which, however, cannot be observed in the average responses.
The results demonstrate highly diverse subjective perception of trigger resistance. The observed perception can be categorised into three behavioural patterns based on their deviance values and average percentages of “heavier”-responses. Each category is represented by one of the three exemplary PFs (see Figure 9). Seven PFs are located at the lower end of the spectrum (represented by Figure 9 (left)). The average percentage was of 50.3% (SD = 5.50) for the “heavier”-responses with a deviance often well above 2.00. Since these responses are located closely around the guess rate of 50% (in our 2AFC case), not enough evidence exists to support our assumption that they were influenced by the intensity of the trigger resistance. The second category includes four data sets that exhibit the expected decrease in discrimination ability with a decrease in resistance level (represented by Figure 9 (middle)). Another seven PFs are located at the higher end (represented by Figure 9 (right)). They show an average success rate of 94.30% (SD = 1.80). Five exhibited a deviance well above 2.00, two a value of 0.00 and 0.02, respectively. These participants almost always perfectly identified the heavier box, even with smaller resistances, and did not produce data within a resistance range where their perception changes according to the level of resistance. Three PFs showed a deviance above 2.0, indicating a high discrepancy from the underlying model. Since the method of constant stimuli of psychophysical testing requires a decreasing range of correct responses between 100 and 50% in the case of our 2AFC task, only the four data sets represented by Figure 9 (middle) qualify for JND calculation. The remaining 17 data sets did not produce data that would allow understanding the discrimination sensitivity of the sensory system around the threshold and the resulting perception. Most of these participants are associated with both ends of the spectrum depicted in Figure 9. The results of the four qualified participants for JND, WF and PSE are listed in Table 1. Their sensory precision was determined with an average JND of 0.28 N (SD = 0.35). This resulted in an average WF of 5.47% (SD = 7.21), which is below the reported WF of 15–22% in the literature of spring stiffness discrimination. However, due to the small number of included data sets these values can only be interpreted as a first indicator of future studies. Nonetheless, the low average is caused by three out of four participants exhibiting a WF of equal or below 2.23%. Only one participant produced a WF of 16.27%, which is in line with the literature.
FIGURE 9. Psychometric functions of three participants representing three commonly observed behaviours in our user study for the perception of Triggermuscle’s adaptive trigger resistance. (left) “Heavier”-responses are located closely around and below guess rate not allowing a direct conclusion for the perception of different resistances. (middle) Proportion of “heavier”-responses decreases with the decrease of resistance intensity. This perception shows the full range of discrimination sensitivity of the sensory system. (right) High proportion of “heavier”-responses even with smaller resistances.
TABLE 1. Results of four participants of our psychophysical user study.
6.2 Interview Feedback
Statements obtained in the interview revealed that one-third of participants self-reported the adaptive trigger resistance as the cue for identifying the heavier box. Two of them additionally incorporated subtle vibrations which occur as a side effect during the servo’s adjustment. The perception of virtual weight was described through “the trigger was harder to press for heavier boxes” or “depending on how much I had to press the index finger”. Lifting the boxes was described as “heavier or less heavy”, lifting heavier boxes was characterised as “more demanding” and one participant reported that grabbing a heavier box took “much longer” to grab than lighter ones. One participant stated he was initially unable to identify the cause of his sense of weight before noticing that the trigger “was harder to press”. However, he stated that he stopped being consciously aware of the change after some time since he felt so immersed in the virtual world.
The remaining two-thirds of participants reported vibrations only, visual input or sound as the basis for their decision process, or they were unable to tell. Vibrations only were mentioned by six participants who further described different intensities, moments of appearance and different ways of how vibrations ended. One participant stated he focused on the vibration and perceived a sound when he experienced more vibrations, despite the noise-cancelling headphones with neutral music. One participant who was unable to identify the reason for his selection process still described the boxes as “much heavier” and “much lighter”.
During the interview, participants were also asked if they had spontaneous associations for the content of the boxes during the task. Nine did not, but twelve did. For them, light boxes felt empty and were associated with feathers while heavy boxes felt solid and as if they were filled with sand, stones, gravel, brick or a book. One participant stated he imagined the boxes empty, but made from different materials.
7 Discussion and Future Work
The evaluation of Triggermuscle showed that the revised spring mechanism with a larger resistance range improved the perception of adaptive trigger resistance compared to the pilot study using the prototype system. However, it also showed some limitations of the current spring mechanism to simulate weight over variable resistance. In this section, we examine our results by identifying the advantages and limitations of our design from a perceptual and a hardware perspective and laying out potential directions for improvement in hardware design and follow-up studies.
7.1 RQ1—Do Different Trigger Resistances Influence the Haptic Perception of Different Virtual Weights in VR?
The interview responses of our main study revealed seven participants—one third of the sample—self-reporting the change in trigger resistance and discriminating different virtual weights according to the intensity of the resistance. These reports about the weight experience demonstrate that higher resistances were associated with heavier virtual weights, smaller resistances with lighter virtual weights. While this applies only to a limited number of our participants, it presents early indications that different trigger resistances can provide substitutional haptic weight cues at the index finger and, therefore, induce a sense of lighter and heavier virtual objects. However, at the current time, our results remain inconclusive and we cannot fully confirm our assumptions as a considerable number of users were also unable to effectively sense, interpret and associate the cues. This raises the question of why some participants experienced virtual weight using Triggermuscle while others did not.
The quantitative data of our main study highlights the users’ ability to discriminate resistances for 13 data sets. This suggests that 3/5 of the participants were able to successfully differentiate between the resistance intensities rendered by Triggermuscle. The data sets consist of four participants with moderate success rates qualifying for the JND calculation, seven participants almost always identifying the heavier box and two additional participants showing an average success rate of 80 and 84%. Surprisingly, this number of data sets exceeds the number of self-reports about the change in trigger resistance by six. This discrepancy may indicate that some participants registered different resistances, but were not consciously aware of it.
7.1.1 Visual Dominance
We believe that the often reported visual dominance in human perception (Posner et al., 1976; Hecht and Reiner, 2009) might have caused the differences in the individual haptic sensing of the trigger resistance and shifted the focus for some participants away from the haptic sensation at their index finger towards the visually identical-looking boxes. Our assumption is supported by the results of our pilot study in which more participants noticed the change in trigger resistance after we simplified the visual input in response to the observations during the preliminary testing. The modification removed the visual sensation of identical-looking boxes which might have previously dominated the perception of the varying haptic feedback. In addition, the visual dominance has been previously reported in VR in the context of pseudo-haptics to overwrite haptic cues, e.g., in haptic retargeting (Azmandian et al., 2016; Zenner et al., 2021), shape rendering (Ban et al., 2012) or the perception of different surface textures in VR (Choi et al., 2018; Degraen et al., 2019). While these approaches purposely made use of this effect, it might have disrupted the haptic perception of the trigger resistance in case of Triggermuscle. We intend to follow this up in future work by exploring the ability to discriminate the trigger resistance in a non-VR setting where participants are blindfolded. This might focus participants’ attention on the haptic sense and, further, remove the aspect of weight association. Since this setup would be similar to typical studies investigating the discrimination of stiffness, we would expect the results to be in line with the reported weber fractions from such studies. Nonetheless, we would also like to explore the pseudo-haptics approach through meaningfully combining the trigger resistance with virtual objects that visually indicate different weights. Comparing the weight perception based on only the adaptive resistance, only visual input and a combination of both could identify if visual input could also improve the ability to discriminate between resistances and facilitates an association with virtual weight in VR.
7.1.2 Subtle Vibrations
To better understand participants’ discrimination mechanisms and to detect the causes of individual differences, we cross-referenced our qualitative and quantitative results for each participant by comparing the reported decision factors with the psychometric functions. This helped to identify the environmental vibration of the controller’s handle, which occurred as an unintended side effect of the servo’s adjustment, as a potential haptic confounding factor. Psychometric functions indicating an influence of the resistance on the perception system belonged to interview statements reporting either only the altered trigger resistance, or only vibration, or a combination of both. More precisely, participants who stated that their decision relied entirely on trigger resistance exhibited the expected decrease in their discrimination ability for smaller resistances. While these cases emphasise a relationship between the level of trigger resistance and the perceived weight, those participants who performed best stated both resistance change and vibration, but also vibration on its own as decisive factors. This reveals that vibrations, as an additionally perceived haptic cue, might have interfered with the ability to sense different resistances and their association with weight difference. This assumption, however, is challenged by our data. Five of our participants also stated that they based their decisions on the sensed vibrations while producing poor success rates closely around the guess rate. Due to these two extremes, we cannot make a clear assumption about the influence of vibrations, positive or negative, on the ability to discriminate variable resistances rendered by Triggermuscle and on the association with virtual weight. While previous work has intentionally utilised vibrotactile feedback to render contact forces in addition to asymmetric skin deformation for weight sensation, it has also identified vibration amplitude as a possible confounding factor with an unknown effect on the perception of skin deformation (Choi et al., 2017).
One possible explanation for the differences in differentiating resistances could be the weber fraction of vibrotactile frequency, which ranges widely from 3 to 30% (Jones and Tan, 2013). Some participants might have been, therefore, more receptive to vibrations than others, causing different degrees of distraction away from the change in trigger resistance. This assumption is supported by participants describing vibrations to different extents: One participant reported that he did not notice any vibrations, while others described them as a side effect and yet others focused on them as the main indicator of virtual weight. Since participants were not informed about the adaptive trigger, this might have consolidated a focus on vibration. Further, this possible shift in attention to vibration could also have been promoted by servo adjustment (i.e., the occurrence of vibration). To prevent adjustment during grasping in the main study, the servo changed its angle at the beginning of each trial and before grasping the second box. However, observations during the task showed that some participants released the trigger very slowly and carefully when placing the box. In the case of the first box, the servo’s adjustment for the second box was then provoked when participants were still focused on the previous box. In addition, the very first servo adjustment in the initial trial appeared even before participants pulled the trigger for the first time, hence before they experienced any resistance. Nonetheless, the occurrence of vibration does not follow a clear pattern. Vibrations occur not only as soon as the servo registers a pulling force inside the mechanism (e.g., when resistance exceeds the standard value), but also when the trigger is pulled during standard value configuration. Additionally, in this state, a reverberation sometimes occurs when the trigger is released, meaning the servo is active, i.e., causing subtle vibrations. Vibrations as a distraction in haptic devices were previously described as “one of the most noticeable disturbances in a force reflecting device” (Tan et al., 1994). To quantify participants’ exposure to the reported subtle vibrations, we took measurements using the digital vibration meter no. 480 600 from VOGEL GERMANY (VOGEL GERMANY, 2021). While all servo adjustments from the main study were tested, only the switch between the minimum and maximum resistances created measurable vibrations. However, these vibrations are below the perceivable range. Importantly, this effect occurs mainly only during the brief period of servo adjustment and does not mature into a continuous vibration.
In future studies, we would like to clarify these possible limitations concerning vibrations. To approach a consistent sensing of the varying resistance, we would like to investigate the role of attention by informing users about the adaptive trigger and by additionally instructing them to ignore the current mechanism’s side effect. In addition, shifting the focus away from vibration and to the trigger’s intended feedback could be further facilitated by implementing a short adjustment phase. This would ensure that the servo’s angle is not configured while participants are close to a virtual object.
7.1.3 Hardware
To address this matter in future hardware design and decrease users’ exposure to unintended vibrations, other types of actuation technologies could be tested, such as a micro linear actuator for controlling the spring’s length or exploring magnetic repulsion forces which are used in magnetic force-feedback joysticks. This could, additionally, increase the stimulus range and emphasise the trigger resistance as the weight cue. Apart from that, vibrations could also be intentionally used to enhance the experience of virtual weight. A perspective on this matter is demonstrated by previous work which explored vibratory stimulation and patterns for weight perception during the interaction with a vision-tactile-force display (Mizuno et al., 2013). As a movable weight shifts along the display’s back towards one of the display’s handles, the perceived weight of that handle seems heavier when strong vibrations are rendered. Vibrations were also observed to enhance other virtual object properties such as virtual stiffness in combination with visual information (Maereg et al., 2017). To better understand the possibility to enhance Triggermuscle’s weight perception meaningfully with vibrations, further research is necessary.
To achieve a simple weight rendering technique that could potentially be integrated into consumer VR controllers, our approach took only into account the trigger and its level of resistance. While participants were, therefore, presented with different resistive forces at their fingertip, Triggermuscle did not render skin stretch as an additional weight cue. This stimulus is often provided in other haptic devices also focusing on rendering haptic feedback at users’ fingers, such as (Minamizawa et al., 2007a; Scheggi et al., 2010; Kurita et al., 2011; Suchoski et al., 2018). While it is common for approaches of sensory substitution to not address the receptors that are addressed in reality, this limitation could have contributed to the different reactions towards the same haptic feedback of Triggermuscle. As the previous finger-worn haptic interfaces for skin stretch use belts or plates stretching and pressing the finger pad, users passively experience the skin modulation in mid-air without physical counter forces to the hand. Since, in contrast, users of the handheld Triggermuscle actively apply pressure on their skin by pulling the trigger and experiencing the counter force of the held device, an additional integration of skin stretch rendering at the trigger might enhance the weight sensation of Triggermuscle. While previous research has shown to successfully render haptic feedback only to users’ index fingers for object properties (Benko et al., 2016; Choi et al., 2018; Whitmire et al., 2018; Sinclair et al., 2019), not receiving a sensation to the full hand during grasping might have further impacted the weight sensation of Triggermuscle.
7.2 RQ2—How can the Intensity of the Trigger be Quantified and Mapped to Convey Distinguishable Virtual Weights?
Our findings show that some participants easily detected smaller changes in the resistance, thus enabling them to haptically and precisely render lighter objects, while other participants did not notice any influence of the intensity level even with large differences. Sensitivity differences towards resistance might have been affected by the previously discussed limitations. Nonetheless, self-reports on absolute weight and on the respective box content associations revealed comparable statements for lower and higher resistances. While these reported impressions indicated a possible influence of the visual appearance of the virtual boxes used in the experimental task, they suggest the possibility of using visual input to map the level of resistance and the perceived absolute virtual weight. Future investigating adaptive trigger resistance as a weight metaphor could determine whether haptic feedback could be used to convey a large range of different relative virtual weights. This could be achieved by creating a visual weight context through visually rendering objects of a similar weight class in the VE and through testing the same resistances in different visual weight classes. To advance the experience of virtual weight during interaction, auditory cues could also be presented to convey weight information. For example, a hollow sound could convey an empty, light object whereas a dull sound could convey a filled, heavy object. This arrangement could also be applied to objects of different materials due to different weights.
In summary, designing triggers with adaptive resistance could be one way of equipping VR controllers with enriched haptic feedback in the future. Since the trigger is a commonly used button that can be found in other VR controllers such as Oculus Touch or game controllers, actuated triggers could be integrated into a various interaction devices, also beyond the VR domain. To achieve the intended effect, Triggermuscle’s spring mechanism could be tailored to other controller shapes. Alternatively, actuation could be modified with various technical approaches including motors, springs, magnets, and gears, to flexibly fit a wide array of form factors. Particular attention should be paid to the discussed limitations in order to achieve a consistent sensing of the haptic feedback for all users. These limitations should be addressed through hardware design by minimising unintended vibrations. Our findings also indicate possible cross-modal effects, which need to be considered when designing adaptive trigger resistance for VR. One possible limitation of our study is that we asked participants about the object’s weight, as this may have biased their association with the provided feedback. Although, participants were not informed of the type of haptic feedback provided, one third linked the stimulus to weight perception. Based on user reports, we further assume that some participants had a real sense of weight as they spontaneously imagined light boxes being filled with feathers, and the heavy ones with sand, stones or gravel.
In the future, we are also interested in expanding the scope of applications for adaptive trigger resistance in VR beyond weight perception by exploring visual cues for other physical properties. As studies (Johansson and Westling, 1984; Flanagan et al., 1995; Ellis and Lederman, 1999) have documented that different material properties impact the level of applied grip forces when lifting an object and its perceived weight, this effect could be potentially be used to further enhance weight rendering in VR and additionally convey different materials of the lifted object. Adapting the trigger resistance then not only to the object’s weight but also to its surface roughness or smoothness could account for the naturally performed grip force modulation as reported in the studies and contribute to a more realistic weight experience. Further, we are interested in whether the trigger resistance can substitute haptic cues for virtual stiffness and if different grasp animations can further convey various surface tensions of different materials. We also imagine an integration of resistive forces beyond the trigger in other haptic devices, such as the haptic VR controller Haptic Revolver (Whitmire et al., 2018) to simulate surface stiffness. Users of this device are presented with textured wheels at their index finger to haptically experience shear forces and textures. By adding resistive forces to the textured wheel, varying surface stiffness could be rendered when the textured wheel is pressed down. Additionally, introducing an active modulation of the resistance during the pull of the trigger could render a modulation of surface tension, e.g., depending on how much a deformable material is squeezed. The resistance modulation could also convey weight shifts, e.g., as the liquid inside a cup moves around when balancing it or when an object is accelerated in order to be thrown. The brief increase or decrease of trigger resistance could furthermore be applied for haptic feedback of operating virtual UI elements.
8 Conclusion
We explored adaptive trigger resistance as a novel approach for weight perception in VR. The adaptive trigger inside our haptic VR controller Triggermuscle modifies the level of resistance according to the weight of a grabbed virtual object. Users, therefore, need to scale their index finger force to grab objects with different virtual weights. We presented the design and implementation of the prototype’s initial and Triggermuscle’s revised spring mechanism, which adjusts the trigger resistance dynamically and continuously. As proof of concept, our systems are built into the casing of an HTC Vive controller. The mechanisms, however, can easily be adapted to various of form factors of different interaction devices. In two user studies, we evaluated both technical implementations and explored the positive as well as the negative effects of actuated adaptive trigger resistance on users’ discrimination ability and associations with virtual object weight in VR. Our findings show large differences between participants. While Triggermuscle’s adaptive trigger enabled 3 out of 5 participants to sense and discriminate different levels of resistance, only 1 out of 3 participants self-reported an association with weight. In the successful weight illusion, lower resistances were associated with lighter objects and higher resistances with heavier objects. While these findings reveal early indications for using adaptive trigger resistance to simulate virtual weight in VR, they reveal limitations regarding the sensing of variable resistance, perceptual mechanisms and hardware design. We have discussed and wish to address these limitations in future studies. Our findings provide a first important step towards using adaptive trigger resistance in VR. We hope that this work motivates further research on transforming established input elements into input-output components, so as to enhance the haptic experience of handheld VR controllers.
Data Availability Statement
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.
Ethics Statement
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.
Author Contributions
CS was the primary author and developed both systems, conducted user studies, data analysis, literature review, manuscript writing and manuscript review. MB contributed to the discussion of the research idea as well as the literature review, manuscript writing and manuscript review. EK contributed to the discussion of the research idea, provided feedback on the prototypes, and took part in manuscript writing and manuscript review. JS contributed to the discussion of the research idea, provided feedback on the prototypes, took part in the manuscript writing and manuscript review and supervised the project. All authors contributed to the article and approved the submitted version.
Funding
This work was partially funded by a Lichtenberg Professorship of the Volkswagen Foundation as well as the BMBF project InviDas (16SV853). In addition, the Klaus Tschira Foundation supported a co-author with a PhD scholarship.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2021.754511/full#supplementary-material
References
Azmandian, M., Hancock, M., Benko, H., Ofek, E., and Wilson, A. D. (2016). “Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences,” in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1968–1979. CHI ’16. doi:10.1145/2858036.2858226
Ban, Y., Kajinami, T., Narumi, T., Tanikawa, T., and Hirose, M. (2012). “Modifying an Identified Curved Surface Shape Using Pseudo-haptic Effect,” in Proceeding of the 2012 IEEE Haptics Symposium (HAPTICS), Vancouver, BC, Canada, 4-7 March 2012 (Vancouver, BC: IEEE), 211–216. doi:10.1109/HAPTIC.2012.6183793
Ben-Tzvi, P., and Ma, Z. (2015). Sensing and Force-Feedback Exoskeleton (SAFE) Robotic Glove. IEEE Trans. Neural Syst. Rehabil. Eng. 23, 992–1002. doi:10.1109/TNSRE.2014.2378171
Benko, H., Holz, C., Sinclair, M., and Ofek, E. (2016). “NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers,” in Proceedings of the 29th Annual Symposium on User Interface Software and Technology (New York, NY, USAUIST ’16: Association for Computing Machinery), 717–728. doi:10.1145/2984511.2984526
Bonfert, M., Porzel, R., and Malaka, R. (2019). “Get a Grip! Introducing Variable Grip for Controller-Based VR Systems,” in Proceeding of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan, 23-27 March 2019 (IEEE), 604–612. doi:10.1109/VR.2019.8797824
Bouzit, M., Burdea, G., Popescu, G., and Boian, R. (2002). The Rutgers Master II-New Design Force-Feedback Glove. Ieee/asme Trans. Mechatron. 7, 256–263. doi:10.1109/TMECH.2002.1011262
Brodie, E. E., and Ross, H. E. (1984). Sensorimotor Mechanisms in Weight Discrimination. Perception & Psychophysics 36, 477–481. doi:10.3758/BF03207502
Burdea, G., Zhuang, J., Roskos, E., Silver, D., and Langrana, N. (1992). A Portable Dextrous Master with Force Feedback. Presence: Teleoperators & Virtual Environments 1, 18–28. doi:10.1162/pres.1992.1.1.18
Cheng, C.-H., Chang, C.-C., Chen, Y.-H., Lin, Y.-L., Huang, J.-Y., Han, P.-H., et al. (2018). “GravityCup: A Liquid-Based Haptics for Simulating Dynamic Weight in Virtual Reality,” in Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (New York, NY, USA: Association for Computing Machinery), 1–2. VRST ’18. doi:10.1145/3281505.3281569
Choi, I., Culbertson, H., Miller, M. R., Olwal, A., and Follmer, S. (2017). “Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality,” in Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (New York, NY, USA: Association for Computing Machinery), 119–130. UIST ’17. doi:10.1145/3126594.3126599
Choi, I., Ofek, E., Benko, H., Sinclair, M., and Holz, C. (2018). “CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery). CHI ’18, 1–13. doi:10.1145/3173574.3174228
Degraen, D., Zenner, A., and Krüger, A. (2019). “Enhancing Texture Perception in Virtual Reality Using 3D-Printed Hair Structures,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–12. CHI ’19. doi:10.1145/3290605.3300479
Dominjon, L., Lecuyer, A., Burkhardt, J., Richard, P., and Richir, S. (2005). “Influence of Control/Display Ratio on the Perception of Mass of Manipulated Objects in Virtual Environments,” in Proceedings - IEEE Virtual Reality, Bonn, Germany, 12-16 March 2005 (IEEE). doi:10.1109/vr.2005.1492749
Ellis, R. R., and Lederman, S. J. (1999). The Material-Weight Illusion Revisited. Perception & Psychophysics 61, 1564–1576. doi:10.3758/BF03213118
Ellis, R. R., and Lederman, S. J. (1993). The Role of Haptic versus Visual Volume Cues in the Size-Weight Illusion. Perception & Psychophysics 53, 315–324. doi:10.3758/BF03205186
Fang, C., Zhang, Y., Dworman, M., and Harrison, C. (2020). “Wireality: Enabling Complex Tangible Geometries in Virtual Reality with Worn Multi-String Haptics,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery). CHI ’20, 1–10. doi:10.1145/3313831.3376470
Flanagan, J. R., Wing, A. M., Allison, S., and Spenceley, A. (1995). Effects of Surface Texture on Weight Perception when Lifting Objects with a Precision Grip. Perception & Psychophysics 57, 282–290. doi:10.3758/BF03213054
Gabardi, M., Solazzi, M., Leonardis, D., and Frisoli, A. (2016). “A New Wearable Fingertip Haptic Interface for the Rendering of Virtual Shapes and Surface Features,” in Proceeding of the 2016 IEEE Haptics Symposium, HAPTICS, Philadelphia, PA, USA, 8-11 April 2016 (IEEE), 140–146. doi:10.1109/haptics.2016.7463168
Giannopoulos, E., Pomes, A., and Slater, M. (2012). Touching the Void: Exploring Virtual Objects through a Vibrotactile Glove. Ijvr 11, 19–24. doi:10.20870/IJVR.2012.11.3.2847
Girard, A., Marchal, M., Gosselin, F., Chabrier, A., Louveau, F., and Lécuyer, A. (2016). Haptip: Displaying Haptic Shear Forces at the Fingertips for Multi-finger Interaction in Virtual Environments. Front. ICT 3, 718. doi:10.3389/fict.2016.00006
Gordon, A. M., Forssberg, H., Johansson, R. S., and Westling, G. (1991). Visual Size Cues in the Programming of Manipulative Forces during Precision Grip. Exp. Brain Res. 83, 477–482. doi:10.1007/BF00229824
Gu, X., Zhang, Y., Sun, W., Bian, Y., Zhou, D., and Kristensson, P. O. (2016). “Dexmo: An Inexpensive and Lightweight Mechanical Exoskeleton for Motion Capture and Force Feedback in VR,” in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1991–1995. CHI ’16. doi:10.1145/2858036.2858487
HaptX (2020). Haptx Gloves. Available at: https://haptx.com/ (Accessed June 15, 2020).
Hecht, D., and Reiner, M. (2009). Sensory Dominance in Combinations of Audio, Visual and Haptic Stimuli. Exp. Brain Res. 193, 307–314. doi:10.1007/s00221-008-1626-z
Heo, S., Chung, C., Lee, G., and Wigdor, D. (2018). “Thor’s Hammer: An Ungrounded Force Feedback Device Utilizing Propeller-Induced Propulsive Force,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–11.
Hirose, M., Hirota, K., Ogi, T., Yano, H., Kakehi, N., Saito, M., et al. (2001). “Hapticgear: the Development of a Wearable Force Display System for Immersive Projection Displays,” in Proceedings IEEE Virtual Reality 2001, Yokohama, Japan, 13-17 March 2001 (IEEE), 123–129. doi:10.1109/VR.2001.913778
Ishii, M., and Sato, M. (1994). A 3D Spatial Interface Device Using Tensed Strings. Presence: Teleoperators & Virtual Environments 3, 81–86. doi:10.1162/pres.1994.3.1.81
Je, S., Kim, M. J., Lee, W., Lee, B., Yang, X.-D., Lopes, P., et al. (2019). “Aero-plane: A Handheld Force-Feedback Device that Renders Weight Motion Illusion on a Virtual 2d Plane,” in Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New York, NY, USA: Association for Computing Machinery), 763–775. UIST ’19. doi:10.1145/3332165.3347926
Jenmalm, P., and Johansson, R. S. (1997). Visual and Somatosensory Information about Object Shape Control Manipulative Fingertip Forces. J. Neurosci. 17, 4486–4499. doi:10.1523/jneurosci.17-11-04486.1997
Johansson, R. S., and Westling, G. (1988). Coordinated Isometric Muscle Commands Adequately and Erroneously Programmed for the Weight during Lifting Task with Precision Grip. Exp. Brain Res. 71, 59–71. doi:10.1007/BF00247522
Johansson, R. S., and Westling, G. (1984). Roles of Glabrous Skin Receptors and Sensorimotor Memory in Automatic Control of Precision Grip when Lifting Rougher or More Slippery Objects. Exp. Brain Res. 56, 550–564. doi:10.1007/BF00237997
Jones, L. A., and Tan, H. Z. (2013). Application of Psychophysical Techniques to Haptic Research. IEEE Trans. Haptics 6, 268–284. doi:10.1109/TOH.2012.74
Kaczmarek, K. A., Webster, J. G., Bach-y-Rita, P., and Tompkins, W. J. (1991). Electrotactile and Vibrotactile Displays for Sensory Substitution Systems. IEEE Trans. Biomed. Eng. 38, 1–16. doi:10.1109/10.682010.1109/10.68204
Kurita, Y., Yonezawa, S., Ikeda, A., and Ogasawara, T. (2011). “Weight and Friction Display Device by Controlling the Slip Condition of a Fingertip,” in Proceeding of the 2011 IEEE International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, 25-30 Sept. 2011 (IEEE), 2127–2132. doi:10.1109/iros.2011.6094613
Lee, J., Sinclair, M., Gonzalez-Franco, M., Ofek, E., and Holz, C. (2019). “TORC: A Virtual Reality Controller for In-Hand High-Dexterity Finger Interaction,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–13. CHI ’19. doi:10.1145/3290605.3300301
Leonardis, D., Solazzi, M., Bortone, I., and Frisoli, A. (2015). “A Wearable Fingertip Haptic Device with 3 Dof Asymmetric 3-rsr Kinematics,” in Proceeding of the 2015 IEEE World Haptics Conference (WHC), Evanston, IL, USA, 22-26 June 2015 (IEEE), 388–393. doi:10.1109/WHC.2015.7177743
Liu, Y., Hashimoto, T., Yoshida, S., Narumi, T., Tanikawa, T., and Hirose, M. (2019). “ShapeSense: A 2D Shape Rendering VR Device with Moving Surfaces that Controls Mass Properties and Air Resistance,” in ACM SIGGRAPH 2019 Emerging Technologies (New York, NY, USA: Association for Computing Machinery), 1–2. SIGGRAPH ’19. doi:10.1145/3305367.3327991
Loomis, J. M., and Lederman, S. J. (1986). “Tactual Perception,” in Handbook of Perception and Human Performances (New York, NY: John Wiley & Sons), 2, 1–41.
Lopes, P., You, S., Cheng, L.-P., Marwecki, S., and Baudisch, P. (2017). “Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation,” in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1471–1482. CHI ’17. doi:10.1145/3025453.3025600
Maereg, A. T., Nagar, A., Reid, D., and Secco, E. L. (2017). Wearable Vibrotactile Haptic Device for Stiffness Discrimination during Virtual Interactions. Front. Robot. AI 4, 42. doi:10.3389/frobt.2017.00042
Maiero, J., Eibich, D., Kruijff, E., Hinkenjann, A., Stuerzlinger, W., Benko, H., et al. (2019). Back-of-Device Force Feedback Improves Touchscreen Interaction for Mobile Devices. IEEE Trans. Haptics 12, 483–496. doi:10.1109/toh.2019.2911519
Maisto, M., Pacchierotti, C., Chinello, F., Salvietti, G., De Luca, A., and Prattichizzo, D. (2017). Evaluation of Wearable Haptic Systems for the Fingers in Augmented Reality Applications. IEEE Trans. Haptics 10, 511–522. doi:10.1109/TOH.2017.2691328
Manus (2020). Prime II Mocap Gloves. Available at: https://www.manus-vr.com/mocapgloves (Accessed June 17, 2020).
Marquardt, A., Kruijff, E., Trepkowski, C., Maiero, J., Schwandt, A., Hinkenjann, A., et al. (2018). “Audio-Tactile Proximity Feedback for Enhancing 3D Manipulation,” in Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (New York, NY, USA: Association for Computing Machinery), 1–10. VRST ’18. doi:10.1145/3281505.3281525
Martínez, J., García, A., Oliver, M., Molina, J. P., and González, P. (2016). Identifying Virtual 3d Geometric Shapes with a Vibrotactile Glove. IEEE Computer Graphics Appl. 36, 42–51. doi:10.1109/MCG.2014.81
Massie, T. H., and Salisbury, J. K. (1994). “The Phantom Haptic Interface: A Device for Probing Virtual Objects,” in Proceedings of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (Chicago), 295–300.
McCloskey, D. I. (1974). Muscular and Cutaneous Mechanisms in the Estimation of the Weights of Grasped Objects. Neuropsychologia 12, 513–520. doi:10.1016/0028-3932(74)90081-5
Microsoft (2021). Xbox Elite Wireless Controller. Available at: https://www.xbox.com/en-US/accessories/controllers/elite-wireless-controller (Accessed June 3, 2021).
Minamizawa, K., Fukamachi, S., Kajimoto, H., Kawakami, N., and Tachi, S. (2007a). “Gravity Grabber: Wearable Haptic Display to Present Virtual Mass Sensation,” in ACM SIGGRAPH 2007 Emerging Technologies (New York, NY, USA: Association for Computing Machinery), 3–6. SIGGRAPH ’07. doi:10.1145/1278280.1278289
Minamizawa, K., Kajimoto, H., Kawakami, N., and Tachi, S. (2007b). “A Wearable Haptic Display to Present the Gravity Sensation - Preliminary Observations and Device Design,” in Proceeding of the 2013 Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC’07), Tsukuba, Japan, 22-24 March 2007 (IEEE), 133–138. doi:10.1109/WHC.2007.15
Mizuno, T., Maeda, J., and Kume, Y. (2013). “Weight Sensation Affected by Vibrotactile Stimulation with a Handheld Vision-Tactile-Force Display Device,” in Proceeding of the 2013 10th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, Krabi, Thailand, 15-17 May 2013 (IEEE), 1–6. doi:10.1109/ECTICon.2013.6559590
Nisar, S., Martinez, M. O., Endo, T., Matsuno, F., and Okamura, A. M. (2019). Effects of Different Hand-Grounding Locations on Haptic Performance with a Wearable Kinesthetic Haptic Device. IEEE Robot. Autom. Lett. 4, 351–358. doi:10.1109/lra.2018.2890198
Pacchierotti, C., Salvietti, G., Hussain, I., Meli, L., and Prattichizzo, D. (2016). “The hRing: A Wearable Haptic Device to Avoid Occlusions in Hand Tracking,” in Proceeding of the 2016 IEEE Haptics Symposium (HAPTICS), Philadelphia, PA, USA, 8-11 April 2016 (IEEE), 134–139. doi:10.1109/haptics.2016.7463167
Posner, M. I., Nissen, M. J., and Klein, R. M. (1976). Visual Dominance: An Information-Processing Account of its Origins and Significance. Psychol. Rev. 83, 157–171. doi:10.1037/0033-295X.83.2.157
Provancher, W. (2014). Creating Greater VR Immersion by Emulating Force Feedback with Ungrounded Tactile Feedback. IQT Quarterly 6, 18–21.
Rietzler, M., Geiselhart, F., Gugenheimer, J., and Rukzio, E. (2018). “Breaking the Tracking: Enabling Weight Perception using Perceivable Tracking Offsets,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–12. CHI ’18. doi:10.1145/3173574.3173702
Ryu, N., Lee, W., Kim, M. J., and Bianchi, A. (2020). “ElaStick: A Handheld Variable Stiffness Display for Rendering Dynamic Haptic Response of Flexible Object,” in Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (New York, NY: Association for Computing Machinery). UIST ’20. doi:10.1145/3379337.3415862
Samad, M., Gatti, E., Hermes, A., Benko, H., and Parise, C. (2019). “Pseudo-Haptic Weight: Changing the Perceived Weight of Virtual Objects By Manipulating Control-Display Ratio,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery). CHI ’19, 1–13. doi:10.1145/3290605.3300550
Sato, M. (2002). Development of String-Based Force Display: Spidar. 8th International Conference on Virtual Systems and Multimedia. 1034–1039. Citeseer.
Scheggi, S., Salvietti, G., and Prattichizzo, D. (2010). “Shape and Weight Rendering for Haptic Augmented Reality,” in Proceedings - IEEE International Workshop on Robot and Human Interactive Communication, Viareggio, Italy, 13-15 Sept. 2010 (IEEE), 44–49. doi:10.1109/roman.2010.5598632
Schorr, S. B., and Okamura, A. M. (2017). “Fingertip Tactile Devices for Virtual Object Manipulation and Exploration,” in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (New York, NY: Association for Computing Machinery), 3115–3119. CHI ’17. doi:10.1145/3025453.3025744
Schütt, H. H., Harmeling, S., Macke, J. H., and Wichmann, F. A. (2016). Painfree and Accurate Bayesian Estimation of Psychometric Functions for (Potentially) Overdispersed Data. Vis. Res. 122, 105–123. doi:10.1016/j.visres.2016.02.002
Schütt, H. H. (2019). Psignifit. Available at: https://github.com/wichmann-lab/psignifit/wiki (Accessed June 16, 2020).
Shigeyama, J., Hashimoto, T., Yoshida, S., Narumi, T., Tanikawa, T., and Hirose, M. (2019). “Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering based on Computational Perception Model,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–11. CHI ’19. doi:10.1145/3290605.3300241
Simpson, W. A. (1988). The Method of Constant Stimuli Is Efficient. Perception & Psychophysics 44, 433–436. doi:10.3758/bf03210427
Sinclair, M., Ofek, E., Gonzalez-Franco, M., and Holz, C. (2019). “CapstanCrunch: A Haptic VR Controller with User-supplied Force Feedback,” in Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New York, NY, USA: Association for Computing Machinery), 815–829. UIST ’19. doi:10.1145/3332165.3347891
Solazzi, M., Frisoli, A., Salsedo, F., and Bergamasco, M. (2007). “A Fingertip Haptic Display for Improving Local Perception of Shape Cues,” in Proceeding of the Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC’07), Tsukuba, Japan, 22-24 March 2007 (IEEE), 409–414. doi:10.1109/WHC.2007.3
Sony (2020). Dualsense Wireless Controller. Available at: https://www.playstation.com/en-gb/accessories/dualsense-wireless-controller/ (Accessed September 4, 2020).
Stellmacher, C. (2021). “Haptic-Enabled Buttons through Adaptive Trigger Resistance,” in Proceeding of the 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Lisbon, Portugal, 27 March-1 April 2021 (IEEE), 201–204. doi:10.1109/VRW52623.2021.00044
Strasnick, E., Holz, C., Ofek, E., Sinclair, M., and Benko, H. (2018). “Haptic Links: Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–12.
Suchoski, J. M., Martinez, S., and Okamura, A. M. (2018). “Scaling Inertial Forces to Alter Weight Perception in Virtual Reality,” in Proceeding of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 21-25 May 2018 (IEEE), 484–489. doi:10.1109/icra.2018.8462874
Sun, Y., Yoshida, S., Narumi, T., and Hirose, M. (2019). “PaCaPa: A Handheld VR Device for Rendering Size, Shape, and Stiffness of Virtual Objects in Tool-based Interactions,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–12. CHI ’19. doi:10.1145/3290605.3300682
3D Systems (2021). Phantom Premium. Available at: https://www.3dsystems.com/haptics-devices/3d-systems-phantom-premium (Accessed June 15, 2020).
Tan, H. Z., Srinivasan, M. A., Eberman, B., and Cheng, B. (1994). “Human Factors for the Design of Force-Reflecting Haptic,” in Dynamics Systems and Control (American Society of Mechanical Engineers (ASME)), 55, 353–359.
Tsai, H.-R., Rekimoto, J., and Chen, B.-Y. (2019). “ElasticVR: Providing Multilevel Continuously-Changing Resistive Force and Instant Impact Using Elasticity for VR,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY: Association for Computing Machinery), 1–10. CHI’19. doi:10.1145/3290605.3300450
Van Polanen, V., and Davare, M. (2015). Sensorimotor Memory Biases Weight Perception during Object Lifting. Front. Hum. Neurosci. 9, 1–11. doi:10.3389/fnhum.2015.00700
VOGEL GERMANY (2021). Electr. Digital Vibration Meter. Available at: https://www.vogel-germany.de/en/produkte-kataloge-prospekte-edition.html.
Westling, G., and Johansson, R. S. (1984). Factors Influencing the Force Control during Precision Grip. Exp. Brain Res. 53, 277–284. doi:10.1007/BF00238156
Whitmire, E., Benko, H., Holz, C., Ofek, E., and Sinclair, M. (2018). “Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–12. CHI ’18. doi:10.1145/3173574.3173660
Yu, R., and Bowman, D. A. (2020). Pseudo-haptic Display of Mass and Mass Distribution during Object Rotation in Virtual Reality. IEEE Trans. Vis. Comput. Graphics 26, 2094–2103. doi:10.1109/TVCG.2020.2973056
Zenner, A., and Krüger, A. (2019). “Drag:on: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), 1–12. CHI ’19. doi:10.1145/3290605.3300441
Zenner, A., and Krüger, A. (2017). Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality. IEEE Trans. Vis. Comput. Graphics 23, 1285–1294. doi:10.1109/TVCG.2017.2656978
Keywords: haptics, virtual reality, weight perception, adaptive trigger, controller design, psychophysics
Citation: Stellmacher C, Bonfert M, Kruijff E and Schöning J (2022) Triggermuscle: Exploring Weight Perception for Virtual Reality Through Adaptive Trigger Resistance in a Haptic VR Controller. Front. Virtual Real. 2:754511. doi: 10.3389/frvir.2021.754511
Received: 06 August 2021; Accepted: 20 December 2021;
Published: 14 January 2022.
Edited by:
Manivannan Muniyandi, Indian Institute of Technology Madras, IndiaReviewed by:
Claudio Pacchierotti, Italian Institute of Technology (IIT), ItalyGionata Salvietti, University of Siena, Italy
Ravali Gourishetti, Max Planck Institute for Intelligent Systems, Germany
Copyright © 2022 Stellmacher, Bonfert, Kruijff and Schöning. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Carolin Stellmacher, cstellma@uni-bremen.de