Sec. Organizational Psychology
Volume 13 - 2022 | https://doi.org/10.3389/fpsyg.2022.965499
From virtual reality to augmented reality: A neuromarketing perspective
- 1Department of Business, Law, Economics and Consumer Behaviour “Carlo A. Ricciardi”, Università IULM, Milan, Italy
- 2Behavior and Brain Lab IULM, Neuromarketing Research Center, Università IULM, Milan, Italy
Neuromarketing, virtual reality, and augmented reality in consumer behavior
Recent studies on consumer behaviors showed that the role of neuroscientific tools in increasing the knowledge of the neural mechanism is involved in the decision and emotional processes (Russo et al., 2022). Neuromarketing refers to the use of such tools in business practices, especially in advertising and marketing research (Ramsøy, 2019). It aims to overcome the limitations of the traditional methodologies by directly investigating emotional and cognitive reactions through electrophysiological and biometric measures (Karmarkar and Plassmann, 2019). In fact, traditional research uses self-report measures to investigate emotions, but this measurement is not sufficient to capture the complexity of the emotional experience since it is based on cognitive processes allowing to only investigate the conscious side of emotions (Micu and Plummer, 2010), and it is subjected to cognitive bias or social desirability (Missaglia et al., 2017). Neuroscience applied to consumer psychology can be crucial in understanding the role of advertising or other stimuli in the consumers’ processing phases, wherein emotions can build meaning (Passyn and Sujan, 2006).
Nowadays, consumers’ needs are not only related to the product/service as such but also to the associated buying experience that must be enchanting, captivating, and fascinating (Kazmi et al., 2021). Virtual Reality (VR) and Augmented Reality (AR) technologies represent suitable candidates, given their abilities to produce intense and enhanced experiences (Slater et al., 2020). In marketing, VR has been investigated in different areas, such as the product and offer perception (Grudzewski et al., 2018), the impact on B2B buyer perceptions (Boyd and Koles, 2019), the evaluation of brands, and how they can be perceived depending on the vividness effect of VR (Van Kerrebroeck et al., 2017), and the tourism sector with highlights for the tourism promotion (Adachi et al., 2020). In marketing, AR is considered “a strategic concept that integrates digital information or objects into the subject’s perception of the physical world, often in combination with other media, to expose, articulate, or demonstrate consumer benefits to achieve organizational goals” (Rauschnabel et al., 2019, p.44). Thanks to 3D virtual objects or environments -most of which are interactive (Javornik, 2016a)–placed into the real scene, it enhances the user experience (Rauschnabel et al., 2022) and creates new ways to deliver information and show products (Huang and Liao, 2015).
In defining AR, scholars argue that it is somehow related to VR because of its immersivity: in both cases, users feel telepresence, namely, the sensation to be present in an environment that is real, with the possibility to respond, and control and act on the experience they are living (Steffen et al., 2019). VR scenarios are separated from the person (Preece et al., 2015), computer-generated, and depleted of elements belonging to the physical environment (Javornik, 2016a), while AR combines real objects with virtual elements (Rauschnabel et al., 2018, 2022) that match the physical environment in terms of real-time perception and interactivity (Javornik, 2016b; Rauschnabel et al., 2022). Beyond interactivity, which produces higher engagement (Esteban-Millat et al., 2014) and influences consumer responses (Gao et al., 2009), augmentation represents a key element in the AR since it redraws the physical reality (Preece et al., 2015). In the marketing field, augmentation applies to the person, the space, and the product. The augmentation of the person is applied in virtual fitting rooms, belonging to the field of the so-called virtual try-on. Also known as a virtual or magic mirror (Javornik et al., 2016), this type of augmentation is particularly used by cosmetic brands (Javornik, 2016a). The augmentation of the space can be found in the product placement or interior design areas, where virtual products can be placed in a real environment, giving consumers the idea of that placement (Jessen et al., 2020). Finally, the augmentation of products, based on interactive elements, provides additional information about real objects whose logo or QR code is scanned using smartphone apps (Do et al., 2020).
Even though VR and AR are similarly evolving in both the theoretical models and applicative examples (Loureiro et al., 2020), they still differ in popularity among consumer behavior researchers: only a few studies focused on AR (Javornik, 2016b; Kazmi et al., 2021), in contrast with the VR. Thus, focusing on AR emerged as crucial. This technology can be used to verify behavioral models related to the decision-making and the consumers’ approach/avoidance experience (Loureiro et al., 2020), as well as to test predictive models for the consumer responses (Sundar et al., 2015). As technology augments the users’ experience, it better supports their needs, influencing the whole experience and enhancing emotions linked to pleasure and arousal (Kourouthanassis et al., 2015). This would be functional to better capture the users’ experience: understanding emotions, particularly in the precise moment in which they are felt, allowing them to understand the experimentation of a stimulus and the possibility that a message can influence the individuals’ behavior (Donovan and Henley, 2003). Moreover, the technology usage has psychological implications since its interactivity can give a sense of control, or the personalization can flow into a sense of agency (Sundar et al., 2015), also boosting the brand image and the willingness to repeat the experience (Obada, 2013; Esteban-Millat et al., 2014).
Considering these elements, we argue the importance of deepening the role of AR in the emotional response of consumers to a product and to an experience that is closer to reality. Together with the mentioned engagement the combination of real objects with virtual elements matching the physical environment (Javornik, 2016b; Rauschnabel et al., 2022) and redrawing the physical reality (Preece et al., 2015), AR can be a precious element to investigate real-time consumers’ experiences with neuromarketing technique. With regards to neuromarketing, although the context of VR has already been explored, AR neuromarketing is still underrepresented. Supplying this scarcity in the literature is important, as AR technology is applied more and more in different marketing fields. One example is in retail, where AR gives the augmentation of a product or a person (Rejeb et al., 2021), and beauty and cosmetics, where the use of magic mirrors provides recommendations for the makeup (Almeida et al., 2015), as well as the visual merchandising area, where augmented objects can be positioned in a real store (Jessen et al., 2020). In the e-commerce field, friendly shopping cart software can be used to boost the buying experience (Yaoyuneyong et al., 2014), similar to tourism, with tools for improving the immersion experience based on the augmentation of objects and environments, such as hotels rooms, restaurants, and museal contents (Siamionava et al., 2018). Finally, also healthcare marketing is adopting AR during the product evaluation (Renu, 2021).
Considering the characteristics of the associated media (Javornik, 2016a; Do et al., 2020), AR can be studied since they play a key role in measuring the impact of technology to the consumers, as well as in evaluating the interactions between human and technology (Li and Meshkova, 2013; Sundar et al., 2015). As argued by these studies, if at the beginning, AR was expensive and not fully accessible, now, technological development has cut the costs and made the use of AR within different devices possible. It can be embedded into fixed devices such as screens or mirrors, as well into mobile devices such as smartphones or wearable devices such as smart glasses (Yaoyuneyong et al., 2016; Rauschnabel et al., 2018, 2019). By creating an experience more immersive, enjoyable, and useful, AR technology can rise positive attitudes toward the used media, as shown, for instance, in the e-commerce field (Esteban-Millat et al., 2014; Yim et al., 2017).
Neuromarketing tools for virtual reality and augmented reality experience
As mentioned, the VR experience has already been investigated through neuromarketing techniques. A study by Uhm et al. (2019) suggested that subjects experimenting with a high sense of presence in a virtual environment show high arousal; a study by Ayata et al. (2017) tried to understand the consumers’ behavior in a virtual supermarket; whereas a study by Ostrovskaya and Vives (2020) highlighted the importance of neuromarketing tools to capture users’ preference while booking hotels rooms via VR experience. Considering the possibility of highly simulating the real environment through AR technology, it is important to also focus on neuromarketing tools to capture emotional responses in AR situations. This is particularly important as little has been done in AR with neuromarketing techniques, also with the understanding of the AR features in consumer behavior (Gill and Singh, 2022). As for the specific tools that were suggested to deepen research, the electromagnetic activity of the brain can be measured by electroencephalography (EEG) to assess, on a moment-to-moment basis, the brain changes related to memory activation, interest, or engagement (Zito et al., 2021a). Some tools record non-brain activity, such as eye tracking, which is mainly used to assess visual salience and exploration patterns, giving indications of attention (Russo et al., 2021). In addition, by measuring the pupil diameter, the eye-tracker provides information on emotional arousal and cognitive workload (Bilucaglia et al., 2019). Skin conductance is used to detect physiological activation: when arousal occurs, an increase in sweat secretion dealing with an increase in skin conductance is observed (Zito et al., 2021b). Heart rate, which reflects both the sympathetic and parasympathetic branches of the autonomous nervous system, can predict various cognitive and emotional states, such as engagement, attention, and emotional valence (Barraza et al., 2015). Finally, facial coding is used to measure, through human expressions, the experimented feelings. Based on Eckman’s studies, this technique can detect the emotional impact of stimuli by the evaluation of unobservable micro-muscle changes (Stasi et al., 2018).
It is important to consider the nature of AR and the exposure to specific visual stimuli that could engage but also ask for an effort of adaptation that can change the human’s sensitivity and sensorial perception (Baldassi et al., 2018). Therefore, neuromarketing emerges as an important element in the consumer responses to AR investigation. Neuromarketing studies could benefit from the AR technology as a new way to deliver marketing stimuli to investigate the emotional and cognitive impacts of augmented products, such as augmented food packages or augmented brand logos. In addition to different buying behaviors (e.g., perceived value, likelihood to purchase), novel eye-tracking patterns, as well as memorization, attention, and engagement values, are expected.
Neuromarketing tools have been already used within the immersive VR technology through devices applied to the head and the hands to record consumers’ responses and changes in muscle control over a long time. This was possible since VR involves the sensorimotor system more than other stimuli, with the potential of increased realistic psychological and behavioral responses (Bohil et al., 2011). Recent studies in the Brain-Computer Interfaces tested the EEG technology in the VR field, with the aim to produce a more immersive experience for the subjects, through highly interactive and imaginative situations. This procedure is also to be applied to AR technology through applications that are able to give biofeedback from brain and body signals (Cardin et al., 2016; Loureiro et al., 2020).
This article aimed to focus on the importance of deepening the role of AR in the emotional response of consumers to a product and an experience through specific tools, such as neuromarketing techniques, considering that AR could be closer to reality than other simulated situations. If VR has already been investigated, AR should be deepened by scholars, researchers, and companies for them to offer reliable answers on consumer behavior in augmented situations.
These considerations can be useful in tracing a roadmap for future research in the neuromarketing and AR fields. Neuromarketing studies could include AR as an alternative experimental environment, while AR could consider the benefits of the neuromarketing tools (Alimardani and Kaba, 2021).
Augmented Reality technologies will directly mediate the interaction between the brain and the real world, as well as the effort in adaptation and perception of risks, including cognition, memory, and decision-making (Baldassi et al., 2018). In this sense, neuromarketing tools could prevent these risks by monitoring brain and physiological signals, which can undermine the consumers’ health and decision-making process. AR has the potential to create a world similar to reality, in which consumers can experiment with objects, self-augmentation, and different spaces, allowing brands to show how to use, enjoy, and engage with products. This is also functional to fully understand consumers’ responses, through the measure of their emotions. To do this, understanding what influences the interaction between consumers and technology is crucial. This should consider both the possibility to control these responses through biofeedback detection and the usage and users’ emotional response to a specific media applied to AR (o VR) technology. This is functional and essential to plan and combine AR technology with the right product and media to offer a real and enjoyable experience that consumers will repeat, dealing with consumers’ needs to associate an experience with the product.
VR, MB, and MZ wrote the first draft and each section of the manuscript, contributed to the manuscript’s final writing and revision, read, and approved the submitted version. All authors contributed to the article and approved the submitted version.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Alimardani, M., and Kaba, M. (2021). “Deep Learning for Neuromarketing; Classification of User Preference using EEG Signals,” in 12th Augmented Human International Conference (AH2021), May 27, 28, 2021, (Geneva), doi: 10.1145/3460881
Almeida, D., Guedes, P., Silva, M., Vieira e Silva, A. L., Silva do Monte Lima, J. P., and Teichrieb, V. (2015). “Interactive Makeup Tutorial Using Face Tracking and Augmented Reality on Mobile Devices,” in XVII Symposium on Virtual and Augmented Reality (SVR) (Piscataway, NJ), doi: 10.1109/SVR.2015.39
Ayata, D., Yaslan, Y., and Kamasak, M. (2017). Emotion recognition via galvanic skin response: Comparison of machine learning algorithms and feature extraction methods. Istanbul Univ. J. Electr. Electron. Eng. 17, 3147–3156.
Barraza, J. A., Alexander, V., Beavin, L. E., Terris, E. T., and Zak, P. J. (2015). The heart of the story: Peripheral physiology during narrative exposure predicts charitable giving. Biol. Psychol. 105, 138–143. doi: 10.1016/j.biopsycho.2015.01.008
Bilucaglia, M., Laureanti, R., Zito, M., Circi, R., Fici, A., Rivetti, F., et al. (2019). “Looking through blue glasses: Bioelectrical measures to assess the awakening after a calm situation,” in 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (Piscataway, NJ), 526–529. doi: 10.1109/EMBC.2019.8856486
Cardin, S., Ogden, H., Pérez-Marcos, D., Williams, J., Ohno, T., and Tadi, T. (2016). “Neurogoggles for Multimodal Augmented Reality,” in AH ‘16: Proceedings of the 7th Augmented Human International Conference, (New York, NY) Vol. 48, 1–2. doi: 10.1145/2875194.2875242
Esteban-Millat, I., Martínez-López, F. J., Luna, D., and Rodríguez-Ardura, I. (2014). “The Concept of Flow in Online Consumer Behavior,” in Handbook of Strategic e-Business Management, ed. F. J. Martínez-López (New York, NY: Springer), 371–402. doi: 10.1007/978-3-642-39747-9_17
Gao, Q., Rau, P. L. P., and Salvendy, G. (2009). Perception of interactivity: Affects of four key variables in mobile advertising. Int. J. Hum. Comput. Interact. 25, 479–505. doi: 10.1080/10447310902963936
Gill, R., and Singh, J. (2022). A study of neuromarketing techniques for proposing cost effective information driven framework for decision making. Mater. Today 49, 2969–2981. doi: 10.1016/j.matpr.2020.08.730
Grudzewski, F., Awdziej, M., Mazurek, G., and Piotrowska, K. (2018). Virtual reality in marketing communication – the impact on the message, technology and offer perception – empirical study. Econ. Bus. Rev. 4, 36–50. doi: 10.18559/ebr.2018.3.4
Huang, T. L., and Liao, S. (2015). A model of acceptance of augmented-reality interactive technology: The moderating role of cognitive innovativeness. Electron. Commer. Res. 15, 269–295. doi: 10.1007/s10660-014-9163-2
Javornik, A. (2016a). Augmented reality: Research agenda for studying the impact of its media characteristics on consumer behavior. J. Retail Consum. Serv. 30, 252–261. doi: 10.1016/j.jretconser.2016.02.004
Javornik, A. (2016b). ‘It’s an illusion, but it looks real!’ Consumer affective, cognitive and behavioural responses to augmented reality applications. J. Mark. Manag. 32, 987–1011. doi: 10.1080/0267257X.2016.1174726
Javornik, A., Rogers, Y., Mooutinho, A. M., and Freeman, R. (2016). “Revealing the Shopper Experience of Using a “Magic Mirror” Augmented Reality Make-Up Application,” in DIS ’16: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, 871–882. doi: 10.1145/2901790.2901881
Jessen, A., Hilken, T., Chylinski, M., Mahr, D., Heller, J., Keeling, D. I., et al. (2020). The playground effect: How augmented reality drives creative customer engagement. J. Bus. Res. 116, 85–98. doi: 10.1016/j.jbusres.2020.05.002
Kazmi, S. H. A., Ahmed, R. R., Soomro, K. A., Hashem, E., Akhtar, H., and Parmar, V. (2021). Role of augmented reality in changing consumer behavior and decision making: Case of pakistan. Sustainability 13:14064. doi: 10.3390/su132414064
Kourouthanassis, P., Boletsis, C., Bardaki, C., and Chasanidou, D. (2015). Tourists responses to mobile augmented reality travel guides: The role of emotions on adoption behavior. Pervasive Mob. Comput. 18, 71–87. doi: 10.1016/j.pmcj.2014.08.009
Loureiro, S. M. C., Guerreiro, J., and Ali, F. (2020). 20 years of research on virtual reality and augmented reality in tourism context: A text-mining approach. Tour. Manag. 77:104028. doi: 10.1016/j.tourman.2019.104028
Micu, A. C., and Plummer, J. (2010). Measurable emotions: How television ads really work patterns of reactions to commercials can demonstrate advertising effectiveness. J. Advert. Res. 137–153. 91300 doi: 10.2501/S0021849910091300
Missaglia, A. L., Oppo, A., Mauri, M., Ghiringhelli, B., Ciceri, A., and Russo, V. (2017). The impact of emotions on recall: An empirical study on social ads. J. Consum. Behav. 16, 1–10. doi: 10.1002/cb.1642
Ramsøy, T. Z. (2019). Building a foundation for neuromarketing and consumer neuroscience research: How researchers can apply academic rigor to the neuroscientific study of advertising effects. J. Adv. Res. 59, 281–294. doi: 10.2501/JAR-2019-034
Rauschnabel, P. A., Babin, B. J., Tom Dieck, M. C., Krey, N., and Jung, T. (2022). What is augmented reality marketing? Its definition, complexity, and future. J. Bus. Res. 142, 1140–1150. doi: 10.1016/j.jbusres.2021.12.084
Rauschnabel, P. A., Felix, R., and Hinsch, C. (2019). Augmented reality marketing: How mobile AR-apps can improve brands through inspiration. J. Retail. Consum. Serv. 49, 43–53. doi: 10.1016/j.jretconser.2019.03.004
Rauschnabel, P. A., He, J., and Ro, Y. K. (2018). Antecedents to the adoption of augmented reality smart glasses: A closer look at privacy risks. J. Bus. Res. 92, 374–384. doi: 10.1016/j.jbusres.2018.08.008
Rejeb, A., Rejeb, K., and Treiblmaier, H. (2021). How augmented reality impacts retail marketing: A state-of-the-art review from a consumer perspective. J. Strateg. Mark. 1–31. doi: 10.1080/0965254X.2021.1972439
Russo, V., Milani Marin, L. E., Fici, A., Bilucaglia, M., Circi, R., Rivetti, F., et al. (2021). Strategic communication and neuromarketing in the fisheries sector: Generating ideas from the territory. Front. Commun. 6:49. doi: 10.3389/fcomm.2021.659484
Slater, M., Gonzalez-Liencres, C., Haggard, P., Vinkers, C., Gregory-Clarke, R., Jelley, S., et al. (2020). The ethics of realism in virtual and augmented reality. Front. Virtual Real. 1:1. doi: 10.3389/frvir.2020.00001
Stasi, A., Songa, G., Mauri, M., Ciceri, A., Diotallevi, F., Nardone, G., et al. (2018). Neuromarketing empirical approaches and food choice: A systematic review. Food Res. Int. 108, 650–664. doi: 10.1016/j.foodres.2017.11.049
Steffen, J. H., Gaskin, J. E., Meservy, T. O., Jenkins, J. L., and Wolman, I. (2019). Framework of affordances for virtual reality and augmented reality. J. Manag. Inform. Syst. 36, 683–729. doi: 10.1080/07421222.2019.1628877
Sundar, S. S., Jia, H., Waddell, T. F., and Huang, Y. (2015). “Toward a theory of interactive media effects (TIME),” in The handbook of the psychology of communication technology, ed. S. S. Sundar (Chichester: John Wiley & Sons, Inc), 47–86. doi: 10.1002/9781118426456.ch3
Uhm, J.-P., Lee, H.-W., and Han, J.-W. (2019). Creating sense of presence in a virtual reality experience: Impact on neurophysiological arousal and attitude towards a winter sport. Sport Manag. Rev. 4, 588–600. smr.2019.10.003 doi: 10.1016/j.smr.2019.10.003
Van Kerrebroeck, H., Brengman, M., and Willems, K. (2017). When brands come to life: Experimental research on the vividness effect of Virtual Reality in transformational marketing communications. Virtual Real. 21, 177–119. doi: 10.1007/s10055-017-0306-3
Yaoyuneyong, G., Foster, J., Johnson, E., and Johnson, D. (2016). Augmented reality marketing: Consumer preferences and attitudes toward hypermedia print ads. J. Interact. Advert. 16, 16–30. doi: 10.1080/15252019.2015.1125316
Yaoyuneyong, G., Foster, J. K., and Flynn, L. R. (2014). Factors impacting the efficacy of augmented reality virtual dressing room technology as a tool for online visual merchandising. J. Glob. Fash Mark. 5, 283–296. doi: 10.1080/20932685.2014.926129
Yim, M. Y. C., Chu, S. C., and Sauer, P. L. (2017). Is Augmented Reality Technology an Effective Tool for E-commerce? An Interactivity and Vividness Perspective. J. Interact. Mark. 39, 89–103. doi: 10.1016/j.intmar.2017.04.001
Zito, M., Fici, A., Bilucaglia, M., Ambrogetti, F. S., and Russo, V. (2021a). Assessing the emotional response in social communication: The role of neuromarketing. Front. Psychol. 12:625570. doi: 10.3389/fpsyg.2021.625570
Keywords: augmented reality, virtual reality, neuromarketing, consumer neuroscience, consumer behavior
Citation: Russo V, Bilucaglia M and Zito M (2022) From virtual reality to augmented reality: A neuromarketing perspective. Front. Psychol. 13:965499. doi: 10.3389/fpsyg.2022.965499
Received: 09 June 2022; Accepted: 19 August 2022;
Published: 09 September 2022.
Edited by:Debora Bettiga, Politecnico di Milano, Italy
Reviewed by:Marion Garaus, Modul University Vienna, Austria
Copyright © 2022 Russo, Bilucaglia and Zito. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Marco Bilucaglia, email@example.com