Traditional art exhibitions are designed for visitors to wander around and enjoy a visual engagement. When interaction is part of the art exhibition, it engages the user with the contents and creates a shared experience. Interaction can take multiple forms such as gesture, virtual drawing or engagement, body motion, or gaze tracking (Chisholm, 2018). The movement toward “Immersive art” was catalyzed by Yayoi Kusama’s 1st Infinity Room in 1965 Applin, (2012), which created the foundation for groups like the art collective founded in 2001 known as Teamlab to offer a variety of immersive art installations (Lee, 2022). The ideal of providing an immersive experience is now a requisite quality of art entertainment as is exemplified in the highly popular “The Immersive Van Gogh Experience”. This exhibition has been held in North America, Europe, Asia, and the Middle East. It most often involves visitors moving in large spaces with projected paintings on the floor, walls, and ceilings. Some include virtual reality headsets to experience the artist’s viewpoint and life. Most of these experiences are often limited to the visual and auditory senses (Richardson et al., 2013; Gao and Xie, 2018), with few attempts incorporating the remaining senses (Carbon and Jakesch, 2012). In this study, we assess the usage of passive haptics in a non-interactive virtual walking experience to explore visual art. More specifically, we combined vibrations around the ankles and ultrasound sensations under the sole of the feet when the participant is walking on a 3D landscape of an oil canvas.

Passive haptics refers to tactile sensations received on the skin without an active exploration from the user end (Ziat, 2022). Passive haptics has been shown to augment visual virtual environments and improve sense of presence, spatial knowledge, navigation, and object manipulation (Insko, 2001; Ziat et al., 2014; Azmandian et al., 2016). In both interactive (i.e., user walking) and non-interactive (i.e., user standing or sitting) conditions, passive haptic feedback when provided to the feet, enhanced the sense of vection, which is an illusion of self-motion that occurs when the perceiver feel bodily motion despite no movement taking place (Turchet et al., 2012; Riecke and Schulte-Pelkum, 2013; Kruijff et al., 2016). Vibrations at the foot can also help reduce visual attention and stress during navigation (Meier et al., 2015).

The human foot is highly sensitive to touch stimulation ((Dim and Ren, 2017)) and the sole of the foot contains similar mechanoreceptors that are found in the human palm (Strzalkowski et al., 2017). Because the feet serve different functions (i.e., gait control, maintaining posture, body orientation, and walking) than the hands, the distribution of afferent receptors and their frequency responses rely on population firing rather than individual neuron firing as it seems to be the case for the hand (Strzalkowski et al., 2017; Reed and Ziat, 2018; de Grosbois et al., 2020). Haptic feedback on the feet have been used for multiple purposes such as robotic telepresence (Jones et al., 2020), illusory self-motion (Riecke and Schulte-Pelkum, 2013), gait control in elderly (Galica et al., 2009; Lipsitz et al., 2015), and improved situational awareness in blind people (Velázquez et al., 2012). Location sites around the foot area and the technology used differ from one study to another. Vibration patterns have proved to elicit illusory motion (Terziman et al., 2013) by providing directional cues to the feet to help with navigation in virtual environments. In terms of locations, vibrations have been provided to sides of the foot to convey distal information from an object and collision avoidance (Jones et al., 2020). Vibrations around the ankle have also been shown to help with gait control (Aimonetti et al., 2007; Mildren and Bent, 2016). The sole of the foot is the most common location by either having participants standing on a vibrating platform (Lovreglio et al., 2018; Zwoliński et al., 2022) or wearing shoes that have small vibrating motors while standing or sitting on a chair (Nilsson et al., 2012; Turchet et al., 2012; Kruijff et al., 2016). Other researchers opted for fluid actuators on shoes to offer more realistic VR walking experiences by feeling different ground structures (Son et al., 2018a; Son et al., 2019; Yang et al., 2020). This solution seems to be more viable than vibrations as the fluid viscosity changes based on the pressure applied by the user during walking creating a more dynamic interaction. Vibrations on shoes remains a cheap solution, but their propagation highly depends on the complexity of the device and the materials used that can attenuate their effect; specifically if they are placed under the sole of the foot. Some potential solutions is the use of force or pressure sensors to modulate the vibrations or supplement the haptic feedback with auditory feedback to create a more realistic feel of the ground texture (Turchet and Serafin, 2014).

Paintings are typically enjoyed visually. The overall aesthetic experience, however, includes multiple factors including the haptic sense. Our aesthetic experience when perceiving, exploring, or interacting with artistic objects is governed by multiple contingencies related to personal and further associative experiences (Ortlieb et al., 2020). They are also determined by the sensory modalities that are in play during the moment. We are interested in how touch can affect the multimodal experience of art and how it can enhance the immersive experience of the viewers in a virtual environment. Some solutions already suggested haptic exploration using bodysuits (Giordano et al., 2015), haptic brushes (Son et al., 2018b), textured reliefs (Reichinger et al., 2011), exoskeletons (Frisoli et al., 2005), force feedback (Dima et al., 2014), surface haptics (Ziat et al., 2021), vibrotactile (Marquardt et al., 2009), thermal (Hribar and Pawluk, 2011), or mid-air haptics (Vi et al., 2017) to interact with an art installation on a screen, in a virtual reality setting, or enhanced tactile walls in museums. In these types of exhibitions and systems where touch is at the center of the artistic piece or movement, the interaction is highly encouraged. Although the main motivation is to enhance the interaction in museums, an additional objective is to provide blind people a medium to explore the artistic pieces using the sense of touch (Lim et al., 2019; Cho, 2021). Passive haptics is of specific interest in the present work for its ease of implementation and cost effectiveness.

Motion sickness scores were significantly higher after the exposure to the virtual world with higher scores for the Disorientation subscale. This result is not surprising as virtual motion methods that instantly teleport users to new locations are usually correlated with increased users’ disorientation (Bowman et al., 1997). In our experiment, participants jumped into the painting from the art gallery and again jumped off the painting to return to the gallery, causing an increase in disorientation. Additionally, walking techniques impacted a user’s sense of presence in a virtual environment since they require varying amounts of physical motion (Ruddle and Lessels, 2006), which in turn affects the amount of simulator sickness caused by the apparatus; a well-recognized side effect of exposure to virtual environment. Real walking is well-known to reduce motion sickness and shows a great sense of presence (Usoh et al., 1999), but requires a bigger space, special treadmills, or redirected walking methods (Razzaque et al., 2002; De Luca et al., 2009; Matsumoto et al., 2016). In our experiment, the fact that participants were receiving passive haptic sensations on their feet while they were physically motionless and visually moving in the world could have increased the conflict between their proprioceptive and visual systems. In the future, this effect could be mitigated by shorter exposure times or 3D axis motion additions to the seat.

Participants’ ability to adapt and immerse themselves into the VR world was affected by their immersive tendencies, specifically their tendency to concentrate or pay attention (focus sub-scale of ITQ). The higher their immersive tendencies were, the stronger their cognitive, physical, and emotional involvement into the scenarios was. This goes hand in hand with their perception of the haptic sensations and how it affected their engagement during the walking illusion. We observed the same trend for participants’ VAS scores. The participants who reported low immersion tendencies had a VAS average score around 50 (midpoint of the scale), which can be considered as neutral. The participants with high immersion tendencies felt that their body was moving forward, they were walking forward, and their feet were striking the ground (mean score >81), regardless of the haptic conditions. The fact that there was no difference in preferences between the haptic conditions can be explained by the more even distribution of receptors on the sole of the feet (Kennedy and Inglis, 2002) as opposed to the proximal to distal increase in FAI and SAI receptors on the hand (Johansson and Vallbo, 1979), suggesting that any haptic combination enhanced the experience for those who felt more immersed. Additionally, the afferents on the sole of the foot have higher thresholds compared to those of the hand that response to specific frequency ranges (Johansson et al., 1982; Strzalkowski et al., 2015) which would require an extensive evaluation between the different locations at the foot. Although neurophysiology evidences provide support for the choice of these locations (sole and ankles), additional factors, such as the choice of the technology, the type of haptic feedback, and the practicality and the comfort for the user, could have affected the immersive experience. Evaluating which part of the foot (ankle, side, sole), their combination, and their timing provide better vection and enhanced experience would prove to be beneficial for future studies that target the foot as a site of stimulation.

In the future, the system can be improved by adding a self-view avatar that has been shown to enhance the sensation of walking, presence, and leg action, specifically when combined with passive haptics (Matsuda et al., 2021). Moreover, although sound was included, it was simply replicating the haptic sensations. Previous research has shown that audio feedback can provide indication of the ground textures in real-time virtual walking (Turchet et al., 2012). Based on the PQ answers, participants did not appear to be affected by the audio feedback. It would be interesting to explore in depth the multimodal interaction and how this information is integrated by the brain in congruent and incongruent situations (Ziat et al., 2015).