Sec. Human-Media Interaction
Volume 3 - 2021 | https://doi.org/10.3389/fcomp.2021.716846
Impossible (Food) Experiences in Extended Reality
- 1Centre for Multisensory Marketing, Department of Marketing, BI Norwegian Business School, Oslo, Norway
- 2Department of Food Science, Faculty of Technical Sciences, Aarhus University, Aarhus, Denmark
- 3Department of Marketing, Kedge Business School, Marseille, France
We introduce a model to think about impossible experiences in mixed and virtual reality, while emphasizing the role of said experiences in the context of food. This reality-impossibility model includes two continua, namely, the reality-fantasy character of objects and environments, and the extent to which they follow the laws of physics-other laws. We present a series of examples in each of the quadrants of the model and discuss both the research possibilities and implications of impossible experiences.
“Truth is a mobile army of metaphors, metonyms, anthropomorphisms, in short, a sum of human relations which were poetically and rhetorically heightened, transferred, and adorned, and after long use seem solid, canonical, and binding to a nation. Truths are illusions about which it has been forgotten that they are illusions.”
– Fredrich Nietzsche
Concrete concepts provide a connection between what people think and how the world is (Johnson, 2008). Humans often resort to metaphors, comparisons, and idioms to give substance to abstract concepts and thus ground them in sensory experiences. For example, the concept “unicorn” is used as a metaphor to describe start-ups whose dazzling success seems impossible, and even a little magical. In the food sector, the metaphor “hair-silk is ice cream” leads people to generate taste inferences in the hair silk domain (Petit et al., 2016). Thus, metaphors often help to simulate things that do not exist, or that are physically impossible to do through concrete language (Marks, 1996; Gibbs and Matlock, 2008).
Today, new technologies are bringing these metaphors to life by making it possible to create seemingly “impossible experiences.” Impossible experiences, that is, experiences that cannot occur in the physical world (e.g., talking with a virtual reality representation of oneself) and tend to involve fantasy (that is, unrealistic or improbable elements; e.g., in the world of Disney’s fantasia), are increasingly allowed and facilitated by immersive, extended reality (XR), technologies such as augmented reality (AR), augmented virtuality (AV), and virtual reality (VR). However, although there are certain models and concepts that allow researchers and experience designers to think about fantastic experiences through immersive technologies (e.g., the immersion/fantasy typology developed by (Cowan and Ketron, 2019), there is no conceptual model to think, and guide the design of, impossible experiences. In this perspective article, we introduce the concept of impossible experiences and what we call the reality-impossibility model, which addresses the aforesaid gap.
Impossible Experiences and the Reality-Virtuality Continuum
How to think about impossible experiences in the context of immersive technologies? The reality-virtuality continuum, which was introduced in the context of visual displays to characterize environments ranging from real to virtual (Milgram and Kishino, 1994), serves as a starting point. In this continuum, real physical environments are located on its left end. Mixed reality environments, which merge real and virtual environments (as in augmented reality and augmented virtuality) are located in its center, and fully virtual environments on its right end. Recently, the reality—virtuality continuum has been used to classify our experiences as a function of the level of digitalization that they involve (Velasco and Obrist, 2020). Said continuum ranges from real experiences, through those involving both real and digital elements (mixed reality), to those which are fully virtual. Mixed reality and virtual experiences, such as those involving XR technology, offer a number of opportunities to create novel experiences.
Mixed reality experiences can help to enrich our physical environments by adding digital elements (e.g., Javornik, 2016). They also allow us to recreate existing experiences that are difficult to access otherwise (e.g., being in a different location, such as the international space station or Earth lower orbit, Stepanova et al., 2019). Perhaps more interestingly, mixed and virtual reality make possible the creation of impossible experiences. In our view, these experiences do not follow the laws of physics (follow imaginary rules, e.g., objects falling up on Earth) and/or are characterized by the introduction of fantasy elements (e.g., visualizing a bear in the Ursa Major constellation).
While impossible experiences in XR are common in the realm of films and games (e.g., Zuo et al., 2019), these are relatively unexplored when it comes to eating. However, impossible experiences per se are not uncommon in eating (see Spence et al., 2020, for a review of magical food experiences). In the next sections, we specifically present our perspective on: 1) a model to classify and guide the design of impossible food experiences in mixed reality; 2) some examples of, and opportunities associated with, said experiences; and 3) the implications associated with them. This article may be of interest to researchers and practitioners interested in designing experiences that do not fully resemble reality and thus aim, through imagination, to create novel experiences for users.
A Design Model to Classify Impossible Experiences
Previous research has suggested a typology of VR research that involves the dimensions of realism–fantasy and immersion, which are instrumental to flow (e.g., Cowan and Ketron, 2019). However, said research has not covered impossible experiences, as we understand them. Indeed, the term “impossible,” as defined in the Merriam-Webster dictionary, refers to “incapable of being or of occurring”1. In that sense, the first part refers to the visible elements, in other words, real-fantasy objects and environments; the second part of the definition refers to the invisible laws governing such objects and environments. Therefore, a more encompassing “impossible experiences” concept, must involve both the realism-fantasy continuum and also one that captures the extent to which the objects in the experience follow the laws of physics as we know them, that is, the laws of physics—other laws continuum (Figure 1). Whereas the first focuses on whether the elements that are part of an experience correspond to real objects and environments (e.g., say, presenting a horse that exists in real life vs. a unicorn that does not), the second focuses on whether interactions between objects are governed by the laws of physics (for example, both the horse and unicorn run on a grass field vs. they fly).
FIGURE 1. The reality-impossibility model. The (1) Physical reality quadrant is both rooted in real scenarios and experiences follow the laws of physics (a horse that gallops), the (2) Other reality quadrant is rooted in real scenarios and experiences do not follow the laws of physics (a horse that flies), the (3) Physical fantasy quadrant is rooted in fantasy scenarios and follows the laws of physics (a unicorn that gallops), and the (4) Other fantasy is rooted in fantasy and do not follow the laws of physics (a unicorn that flies).
Altogether, these two continua compose what we call the reality-impossibility model (see Figure 1). XR technology becomes the digital, and immersive, means by which quadrants other than the real—laws of physics quadrant become possible. Importantly, immersion (Agrawal et al., 2020) and sense of presence (Schuemie et al., 2001), which are key variables in immersive technologies, are not part of the experience taxonomy but instead, in our view, may determine the compellingness of the experience (cf. Slater, 2018). Moreover, by focusing on the objects and their interactions, our model is agnostic to the way the experience is implemented (e.g., whether the experience involves augmented reality, augmented virtuality, or virtual reality, cf. Milgram et al., 1995).
Let us look at some examples of experiences that fall within each of the quadrants of the reality-impossibility model. The physical reality quadrant (quadrant 1) involves objects and environments which exist in reality and obey the laws of physics. Examples in this quadrant include those AR visualizations of foods that several companies are now utilizing to enhance the expectations of consumers before they order food online (Petit et al., 2021). Consumers can use a smartphone to overlap, say, a dish of food in their own physical environment through their cell phones to experience it as if it was in front of them, and thus, better imagine the consumption process. Another example here is National Geographic Explore VR2, which allows the user to explore, through virtual reality, some of the most iconic natural locations on the planet.
In the other reality quadrant (quadrant 2), scenarios and experiences are real but do not follow the laws of physics. An example is the VR video tour of six real exoplanets created in collaboration with researchers at the University of Exeter (Exoplanet 360°, 2018)3. The video takes the viewer on a first-person perspective, guided tour of six exoplanets (reality) viewed from space, as well as from the surface of the planets in a short time span (other laws). The video shows short information highlights in text form, and different astrophysicists verbally present more in-depth information about each exoplanet. Whereas the immersive video presents a scientifically accurate representation of the planets, the transportation between the planets happens within seconds and some of the environments are likely hostile to sustain a visitor doing the guided tour as presented (other laws). Another example of this quadrant are the AR filters that can be found on social media such as Instagram and Snapchat, that allow people to see themselves (reality) as significantly older and/or younger in real time (other laws). In both cases, the forces that govern the experience and act onto the real objects defy the laws of physics. For instance, the concept of time as part of the narrative of the experience does not follow its normal speed forwards, or goes backwards.
The physical fantasy quadrant (quadrant 3) is characterized by the inclusion of fantastical elements in a world which still follow the laws of physics. An example is the Tree VR experience developed by members of MIT Media Lab’s Fluid Interfaces group, in which the user embodies (fantasy) a forest tree (laws of physics) from seedling to its fully-grown form and lives through different significant events (Liu and Qian, 2017). During the experience, the user’s body becomes the trunk, and their arms become the branches; the full experience incorporates bodily haptics, vibration, heat generators, and fans to enhance its immersiveness. In this case, while the scenario and objects obey the ordinary laws of physics (though the representation of the growing process is speeded up, as in a timelapse), the concept of the experience is fantastical (a human becomes a tree). An AR example is where TeamLab (an art collective), together with Sagaya4 (a restaurant), designed a dining experience, in which they projected, via projection mapping, a fantastic representation of the different seasons on the dining table and room, while the diners ate.
Finally, the other fantasy scenario (quadrant 4) includes both fantastical elements and interactions which go beyond ordinary laws of physics. An example is the Dreams of Dalí VR experience5, in which the user enters Dalí’s painting (fantasy), Archaeological Reminiscence of Millet’s “Angelus,” as if it were another dimension, and explores a surreal world based on it (other laws, Dreams of Dalí, 2020) Another example here is Coca Cola’s virtual reality for Christmas6, a campaign developed by the brand in which users are immersed in Coca Cola’s animated Christmas world (fantasy), whilst flying in Santa’s sleigh (other laws), or Pokémon GO7, the augmented reality game in which both fictional characters (fantasy), with their own, other, laws (e.g., self-levitating), are augmented into the physical reality.
It is important to mention that, although the quadrants may provide a guide to classify experiences in four discrete categories, the two axes of the model are continua. In that sense, experiences may vary in extent, along them. In addition, whereas the reality vs. fantasy continuum can possibly be measured by the extent to which the objects or context exist in real life, the continuum of laws of physics vs. other laws can be slightly more difficult to measure, considering that technologies may allow interactions (e.g., flying) otherwise not possible without technology, whilst at the same time complying with the laws of physics. On a similar note, it is important to distinguish whether the laws of physics are being broken as an integral part of the plot of the experience. For instance, in the interest of time, the experience may be speeded up at will, while this manipulation is not part of the narrative per se. As such, our experience design model, at present, provides a more qualitative representation/inspiration of the variations along the two dimensions, though future research may aim at developing measures for these variations.
Impossible XR Eating Experiences
What do impossible experiences mean in the context of eating and drinking? As immersive technologies become a part of eating and drinking, the potential for novel experience design with said experiences grows. Indeed, research has already suggested that a number of immersive technologies are being used for food experience design both in research and practice (Velasco et al., 2018).
The majority of XR eating research has focused, so far, on real scenarios obeying the laws of physics (quadrant 1). Using VR, for example, studies have explored drinking coffee while immersed in a coffee farm (Barbosa Escobar et al., 2021). Using stereoscopic AR interfaces, other studies have investigated altering the luminance distribution of a slice of cake (Ueda et al., 2020) and changing the apparent size of a cookie (Narumi et al., 2012). Furthermore, projective AR systems have been used to modify the food color of sponge cake and potato crisps in real time (Nishizawa et al., 2016) and modify the appearance of the cooking state of food in a Chinese hotpot with remote commensals (Foley-Fisher et al., 2010).
However, there are a few studies which have explored quadrant 2, with scenarios impossible to test in the real world either due to ethical issues or physical impossibility. For example, Ammann et al. (2020) used VR to set up a disgust sensitivity study where participants were asked to taste chocolate that either appeared on the table or came from the bottom end of a dog. While this would have been impossible to achieve in the real world, the researchers used VR to manipulate participants’ disgust response while keeping the food identical. In another example, Wang et al. (2020) gave participants black coffee that was colored either dark brown or light brown in VR, creating coffee with “virtual milk” added. In this case, VR allowed the researchers to study how the brain integrates digitally presented visual information with physically presented chemosensory information.
Since much food research has traditionally focused on real products, quadrants 3 and 4, which involve imaginary scenarios, have, to the best of our knowledge, only been explored in the realm of HCI research. An example involving imaginary scenarios but obeying physical laws (quadrant 3) was shown in Harley et al. (2018), where participants had a picnic in Little Red Riding Hood’s forest while the wolf approached from a distance. While we are not aware of any existing XR eating studies in quadrant 4, such experiences—for example eating novel foods in space—have been conceptualized, and systems associated with them devised (e.g., Obrist et al., 2019).
Discussion and Conclusion
What other impossible experiences can be designed? Going forwards, we hope to see many more XR eating experiences in quadrants 2, 3, and 4. Experiences in quadrant 2 can play more explicitly with breaking laws of physics, for instance with self-levitating or flying foods (though they may not break the laws of physics, see Vi et al., 2020). Moreover, food appreciation can be enhanced by building in further interaction points with the food, for instance with living serving ware or even the food itself. Another interesting avenue may lie in VR experiences in which the users embody an animal and experience different stages in its value chain, which could potentially have impactful effects in individuals’ food-related behaviors, including diet and food waste generation. Experiences in quadrant 3, by exploring fantasy objects and environments, can potentially add value to food research by uncovering consumers’ state of mind and emotions towards novel foods, or by helping people build a deeper experience with the past by tasting historical and/or extinct foods.
Quadrant 4 opens up the possibility of fantasy dining scenarios where the questions of where, when, who, and what to eat are all open to experimentation. For example, how would it feel like to eat in the World of Warcraft, whereby some of the laws of physics are also broken? Or how would it feel like to dine in one of Asimov’s novels? We can also imagine a situation in which different quadrants of the model would be tested and see which quadrant is most likely to help individuals simulate impossible experiences. For example, what situation could make gruyere cheese best look like the Moon? - representing someone eating gruyere on the Moon (quadrant 2), - eating a piece of the Moon with wine and bread at the table (quadrant 3), or even eating a piece of the Moon with bread and wine, while gazing at the Earth from the surface of the Moon (quadrant 4). It may also be interesting to analyze whether XR can facilitate the embodiment of metaphors. Metaphors generally help people to represent abstract concepts in terms of more concrete ones (i.e., more grounded in bodily states), through the simulation of impossible experiences (Lakoff and Johnson, 1980; Gibbs and Matlock, 2008). Many metaphors relate to the food register (e.g., ‘walking on eggshells’, ‘the apple of my eye’, ‘pork barreling’, ‘I’m in a pickle’). If instead of imagining them, it would be possible to live these impossible experiences through XR, what level of fantasy/reality best favored the understanding of these metaphors? In public health communication, metaphors such as “Are you pouring on the pounds?” are often used to create disgust and reduce food cravings (Puhl et al., 2013; Petit et al., 2016). What XR scenarios in quadrants 2, 3, and 4 may help people experience sodas in terms of pounds?
While the reality-impossibility model enables multiple scenarios for experimentation with foods, this, as other research in XR, carries certain ethical reflections that are worth examining. Quadrant 4, for instance, facilitates the creation of experiences that do not conform to any known experiences, and as such, their implications need to be carefully thought through, given that the realism of XR is achieving unprecedented levels. Slater et al. (2020) considered some questions worth reflecting on, which may well apply to impossible food experiences: Does realism in XR lead to confusion between the real and virtual? Does it lead to emotional and behavioral impact? What are the long-term effects of virtual impossible experiences?
The answers to these questions are not black and white and need further reflection. However, following Velasco and Obrist’s (2020) three laws of multisensory experiences, one may argue that any impossible experience should 1) be used for good and must not harm others, 2) the participants should be treated fairly, and 3) the experience designer, the rationale, and the means through which the experience is created must be known.
In conclusion, impossible experiences in XR have already demonstrated their value in a wide spectrum of applications: to probe multisensory integration (e.g., Cornelio et al., 2021) and decision making (e.g., Ammann et al., 2020), optimize product/experience development (e.g., Obrist et al., 2019), and create “fun” consumer experiences (e.g., Barbosa Escobar et al., 2021; https://lepetitchef.com/; see also Wang et al., 2021, for a review).
Our perspective on impossible experiences and our suggested model present a first approach toward the conceptualization of impossible experiences. As such, it is critical for future research to further operationalize the continua and empirically support or challenge them, through qualitative and quantitative studies. XR provides a new space for creativity. While metaphors were limited to the ability of individuals to simulate experiences, by giving more space to the impossible, XR might improve our abstraction skills.
Going forward, we invite other researchers to join us to explore the vast space of opportunities in the realm of impossible experiences.
Data Availability Statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Agrawal, S., Simon, A., Bech, S., Bærentsen, K., and Forchhammer, S. (2020). Defining Immersion: Literature Review and Implications for Research on Audiovisual Experiences. J. Audio Eng. Soc. 68 (6), 404–417. doi:10.17743/jaes.2020.0039
Ammann, J., Hartmann, C., Peterhans, V., Ropelato, S., and Siegrist, M. (2020). The Relationship between Disgust Sensitivity and Behaviour: A Virtual Reality Study on Food Disgust. Food Qual. Pref. 80, 103833. doi:10.1016/j.foodqual.2019.103833
Dreams of Dalí (2020). The Dalí Museum. Available from: https://thedali.org/dreams-of-dali-2/ (Retrieved May 5, 2021).
Exoplanet 360° (2018). Exoplanet 360°. University of Exeter Research and Innovation Blog. Available from: https://researchandinnovation.co.uk/exoplanet-360/ (Retrieved May 5, 2021).
Foley-Fisher, Z., Tsao, V., Wang, J., and Fels, S. (2010). “NetPot: Easy Meal Enjoyment for Distant Diners,” in Entertainment Computing - ICEC 2010. Editors H. S. Yang, R. Malaka, J. Hoshino, and J. H. Han (Seoul, South Korea: Springer Berlin Heidelberg), 446–448. doi:10.1007/978-3-642-15399-0_56
Harley, D., Verni, A., Willis, M., Ng, A., Bozzo, L., and Mazalek, A. (2018). “Sensory VR,” in Presented at the TEI ′18: Twelfth International Conference on Tangible, Embedded, and Embodied Interaction (New York, NY, USA: ACM), 386–397. doi:10.1145/3173225.3173241
Javornik, A. (2016). Augmented Reality: Research Agenda for Studying the Impact of its media Characteristics on Consumer Behaviour. J. Retail. Consum. Serv. 30, 252–261. doi:10.1016/j.jretconser.2016.02.004
Liu, X., and Qian, Y. (2017). Project Tree. MIT Media Lab. Available from: https://www.media.mit.edu/projects/tree/overview/ (Retrieved May 5, 2021).
Milgram, P., Takemura, H., Utsumi, A., and Kishino, F. (1995). “Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum,” in Telemanipulator and Telepresence Technologies (Boston, United States: International Society for Optics and Photonics), 2351, 282–292. doi:10.1117/12.197321
Narumi, T., Ban, Y., Kajinami, T., Tanikawa, T., and Hirose, M. (2012). “Augmented Perception of Satiety: Controlling Food Consumption by Changing Apparent Size of Food with Augmented Reality,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, May 5–10, 2012, 109–118.
Nishizawa, M., Jiang, W., and Okajima, K. (2016). “Projective-AR System for Customizing the Appearance and Taste of Food,” in Proceedings of the 2016 Workshop on Multimodal Virtual and Augmented Reality, Tokyo Japan, November 16, 2016. doi:10.1145/3001959.3001966
Obrist, M., Tu, Y., Yao, L., and Velasco, C. (2019). Space Food Experiences: Designing Passenger's Eating Experiences for Future Space Travel Scenarios. Front. Comput. Sci. 1, 7–17. doi:10.3389/fcomp.2019.00003
Petit, O., Basso, F., Merunka, D., Spence, C., Cheok, A. D., and Oullier, O. (2016). Pleasure and the Control of Food Intake: An Embodied Cognition Approach to Consumer Self-Regulation. Psychol. Mark. 33 (8), 608–619. doi:10.1002/mar.20903
Petit, O., Javornik, A., and Velasco, C. (2021). We Eat First with Our (Digital) Eyes: Enhancing Simulation of Eating through Visual-Enabling Technologies. J. Retail. (in press). doi:10.1016/j.jretai.2021.04.003
Schuemie, M. J., van der Straaten, P., Krijn, M., and van der Mast, C. A. P. G. (2001). Research on Presence in Virtual Reality: A Survey. CyberPsychol. Behav. 4 (2), 183–201. doi:10.1089/109493101300117884
Slater, M., Gonzalez-Liencres, C., Haggard, P., Vinkers, C., Gregory-Clarke, R., Jelley, S., et al. (2020). The Ethics of Realism in Virtual and Augmented Reality. Front. Virtual Reality 1, 1. doi:10.3389/frvir.2020.00001
Stepanova, E. R., Quesnel, D., and Riecke, B. E. (2019). Space—a Virtual Frontier: How to Design and Evaluate a Virtual Reality Experience of the Overview Effect. Front. Digital Humanit. 6, 7. doi:10.3389/fdigh.2019.00007
Ueda, J., Spence, C., and Okajima, K. (2020). Effects of Varying the Standard Deviation of the Luminance on the Appearance of Food, Flavour Expectations, and Taste/flavour Perception. Sci. Rep. 10, 16175. doi:10.1038/s41598-020-73189-8
Vi, C. T., Marzo, A., Memoli, G., Maggioni, E., Ablart, D., Yeomans, M., et al. (2020). LeviSense: A Platform for the Multisensory Integration in Levitating Food and Insights into its Effect on Flavour Perception. Int. J. Human-Comput. Stud. 139, 102428. doi:10.1016/j.ijhcs.2020.102428
Wang, Q. J., Meyer, R., Waters, S., and Zendle, D. (2020). A dash of virtual milk: altering product color in virtual reality influences flavor perception of cold-brew coffee. Front. Psychol. 11, 3491.
Wang, Q. J., Barbosa Escobar, F., Alves Da Mota, P., and Velasco, C. (2021). Getting Started with Virtual Reality for Sensory and Consumer Science: Current Practices and Future Perspectives. Food Res. Int. 145, 110410. doi:10.1016/j.foodres.2021.110410
Zuo, T., Feijs, L., Van der Spek, E. D., and Hu, J. (2019). “A Classification of Fantasy in Serious Games,” in Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, Barcelona Spain, October 22–25, 2019, 821–828.
Keywords: mixed reality, virtual reality, impossible, experiences, reality, extended
Citation: Velasco C, Barbosa Escobar F, Petit O and Wang QJ (2021) Impossible (Food) Experiences in Extended Reality. Front. Comput. Sci. 3:716846. doi: 10.3389/fcomp.2021.716846
Received: 29 May 2021; Accepted: 27 July 2021;
Published: 05 August 2021.
Edited by:Nicola Bruno, University of Parma, Italy
Reviewed by:Silvia Ceccacci, Marche Polytechnic University, Italy
Farhan Mohamed, University of Technology Malaysia, Malaysia
Copyright © 2021 Velasco, Barbosa Escobar, Petit and Wang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Carlos Velasco, email@example.com