# MULTISENSORY HUMAN-FOOD INTERACTION

EDITED BY : Carlos Velasco, Anton Nijholt and Kasun Karunanayaka PUBLISHED IN : Frontiers in Psychology and Frontiers in Digital Humanities

#### Frontiers Copyright Statement

© Copyright 2007-2018 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use. ISSN 1664-8714 ISBN 978-2-88945-518-8 DOI 10.3389/978-2-88945-518-8

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# MULTISENSORY HUMAN-FOOD INTERACTION

Topic Editors:

Carlos Velasco, BI Norwegian Business School, Norway Anton Nijholt, University of Twente, Netherlands Kasun Karunanayaka, Imagineering Institute, Malaysia

Image: Melle Nieling

Our food experiences can be significantly influenced by both intrinsic and extrinsic multisensory information. Therefore, it is crucial to understand and apply the principles that govern the systematic connections that exist between the senses in the context of Human-Food Interaction (HFI). In our Research Topic, namely Multisensory Human-Food Interaction (MHFI), several studies that consider such connections in the context of HFI are presented. We also have contributions that focus on multisensory technologies that can be used to share and reproduce specific HFIs. This eBook, which resulted from the Research Topic, presents some of the most recent developments in the field of MHFI. The eBook begins with the Editorial, which provides an overview of MHFI. Then, it includes six articles that relate to principles in MHFI (Section 1) and three on technologies in MHFI (Section 2). We hope that the different contributions featured here will support future developments in MHFI research.

Citation: Velasco, C., Nijholt, A., Karunanayaka, K., eds. (2018). Multisensory Human-Food Interaction. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-518-8

# Table of Contents

*04 Editorial: Multisensory Human-Food Interaction* Carlos Velasco, Kasun Karunanayaka and Anton Nijholt

# SECTION 1 PRINCIPLES IN MHFI

*07 Music to Make Your Mouth Water? Assessing the Potential Influence of Sour Music on Salivation*

Qian J. Wang, Klemens Knoeferle and Charles Spence


Jérémy Roque, Malika Auvray and Jérémie Lafraire

*63 Visual Search for Wines With a Triangle on the Label in a Virtual Store* Hui Zhao, Fuxing Huang, Charles Spence and Xiaoang Wan

# SECTION 2

## DEVELOPING AND UTILIZING MULTISENSORY TECHNOLOGIES IN HFI

*74 Galvanic Tongue Stimulation Inhibits Five Basic Tastes Induced by Aqueous Electrolyte Solutions*

Kazuma Aoyama, Kenta Sakurai, Satoru Sakurai, Makoto Mizukami, Taro Maeda and Hideyuki Ando


# Editorial: Multisensory Human-Food Interaction

#### Carlos Velasco<sup>1</sup> \*, Kasun Karunanayaka<sup>2</sup> and Anton Nijholt <sup>3</sup>

*<sup>1</sup> Department of Marketing, Center for Multisensory Marketing, BI Norwegian Business School, Oslo, Norway, <sup>2</sup> Imagineering Institute, Iskandar Puteri, Malaysia, <sup>3</sup> Faculty EEMCS, University of Twente, Enschede, Netherlands*

Keywords: food, multisensory, technology, human-food interaction, human-computer interaction, experience

#### **Editorial on the Research Topic**

#### **Multisensory Human-Food Interaction**

In recent years, there has been a growing interest in the context of Human-Food Interaction (HFI) to capitalize on multisensory interactions in order to create, modify, and enhance our food-related experiences (Nijholt et al., 2016; Velasco et al., 2017). There are two ideas that may explain this interest. First, research has provided a strong case for the fact that eating and drinking are among the most multisensory events of our everyday lives (see Prescott, 2015; Spence, 2015, 2017, for reviews). This has paved the way to target both intrinsic and extrinsic sensory cues to design specific food experiences (e.g., Koizumi et al., 2011; Narumi et al., 2011; Spence et al., 2017). Second, given the ubiquitous nature of technology and the increasing availability of multisensory-oriented devices (Obrist et al., 2016, 2017), both researchers and practitioners have become interested in the roles that these technologies can play in food contexts (Choi et al., 2014; Petit et al., 2015).

The aforementioned ideas are the basis of the present emerging research topic, that is, Multisensory Human-Food Interaction (MHFI). Broadly speaking, research in MHFI aims to further our understanding of the principles that govern the systematic connections that exist between the senses in the context of HFI. Moreover, it also aims to build on such understanding in order to utilize and develop different technologies to "hack" the senses, that is, to modify existing, and create novel experiences, in the context of HFI (e.g., in support of healthy eating, entertainment, sensory marketing). In the present research topic, we called for investigations on these aims.

Edited and reviewed by: *Francesco Ferrise, Politecnico di Milano, Italy*

> \*Correspondence: *Carlos Velasco carlos.velasco@bi.no*

#### Specialty section:

*This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology*

Received: *23 April 2018* Accepted: *03 May 2018* Published: *23 May 2018*

#### Citation:

*Velasco C, Karunanayaka K and Nijholt A (2018) Editorial: Multisensory Human-Food Interaction. Front. Psychol. 9:796. doi: 10.3389/fpsyg.2018.00796* THEME 1: MULTISENSORY PRINCIPLES IN MHFI

Based on the idea that people associate basic tastes with music in specific ways, Wang et al. conducted a study designed to assess whether "sour music" would elicit a distinctive physiological response. In particular, their results provided evidence for the idea that salivation was greater when participants were shown an image representing a sour taste (a lemon) than when they were presented with a "sour" soundtrack or just a silent, comparable, interval of time. Note that no difference in salivation levels was found between the sour soundtrack condition compared to the silent condition. It appears that, although there might be strong links between music and tastes, music by itself is not sufficient to evoke physiological responses associated with tastes.

Reinoso Carvalho et al. investigated the effect of a beer's color on people's sensory and hedonic expectations and experience of the beer. In particular, they assessed whether dark or pale beer coloring, while keeping the taste/flavor equal, would influence people's evaluation of the beer. No differences were observed in the evaluation of the beer after tasting it, however, their participants expected to like the pale beer more than the dark one, and the latter to be more bitter, to taste stronger, and to have more "body" than the first. Here, color appears to make a difference only at the expectation level.

Taking advantage of two popular drinks, namely coffee and tea, Pramudya and Seo studied the role of temperature on people's sensory and emotional responses to such drinks. In terms of the first, their results indicated that, warmer drinks (65◦C), independently of arousal, resulted in more frequent positive emotional responses. Colder drinks (5◦C), on the other hand, resulted in higher activation/arousal and negative emotional responses and medium temperature drinks (25◦C) resulted in more frequent low activation and negative emotional responses. As for sensory attributes, specific flavors were differentially associated with the drinks at different temperatures. For example, "roasted flavor" attribute of brewed coffee was more often associated with the high temperature condition (65◦C), while characteristics such as "pungent aroma," "metallic flavor," and "skunky flavor" were more often reported in relation to the lower temperature (5◦C). Therefore, temperature can influence both sensory and emotional responses to drinks.

Fenko et al. also used coffee as a means to assess the effect of extrinsic cues on flavor and more broadly on product evaluation. In particular, they assessed the influence of visual metaphors (a lion located on top or bottom of a coffee packaging) and textual claims of strength, on the evaluation of coffee products. Their results demonstrated that both elements can influence coffee expectations and experience. For example, both the textual claim ("extra strong") and visual metaphor (image located on the bottom) of strength enhanced the perception of coffee strength and also the participants' intention to purchase the product. Therefore, visual characteristics of a drink's packaging but also any textual information used to describe it, may influence its expectations and experiences.

Through a metanalytical approach, Roque et al. presented a comprehensive review on the concept of freshness in beverages. Using multisensory flavor perception as a frame of reference, the authors suggested that freshness perception is characterized by both perceptual and semantic elements. They differentiated it from flavor by indicating that freshness is more specific in terms of the particular functions that it serves, such as to alleviate oropharyngeal symptoms. Moreover, they also suggested that the weighting of different sensory inputs in freshness also differs from that of flavor.

Broadening the understanding of multisensory principles and the use of new technologies in MHFI, Zhao et al. conducted a study to assess people's visual search behavior for wine bottles with characteristic triangles in their label. Both of the experiments in their study were conducted in virtual reality. Consistent with previous literature, their results revealed that participants identified faster bottles with downward vs. upward pointing triangles, and that this effect was modulated by the bottle's location on the shelf. Their results provided relevant information as to how extrinsic attributes (e.g., label elements) influence wine search. Moreover, Zhao et al. forwarded virtual reality means to assess product performance.

# THEME 2: DEVELOPING AND UTILIZING MULTISENSORY TECHNOLOGIES IN HFI

Aoyama et al. presented a study designed to further understand the way in which galvanic tongue stimulation influences tastes sensations. First, they tested the effect of taste inhibition for cathodal GTS in non-electrolytes aqueous and electrolytes aqueous solutions. Ayoma et al. found that cathodal GTS (1.0 mA and 2,000 ms square current) only weakened the sweet and bitter taste sensations in electrolytes aqueous solutions (which are glycine and MgCl2). In the second experiment, they investigated whether the cathodal-GTS would inhibit all five basic tastes. Their results showed that all tastes sensations were weakened by cathodal-GTS. Further, this effect was strongly correlated with the strength of the current. The findings of this study support the ion migration hypothesis for taste inhibition as cathodal-GTS only weakened the sensations produced by electrolytes aqueous solutions. This paper informs research on digital flavor technologies.

Risso et al. developed and tested a practical small-sized computer-controlled olfactometer, or Multi-Fragrance Olfactory Display (MFOD), that can be used in food and beverage research. This olfactometer used a solid fragrance release method to produce smell sensations and it had eight different odor channels. The intensity and flow rate of the fragrances were adjusted by changing the speed of the centrifugal fan and using a small anemometer. The results of their study confirmed that this olfactometer can significantly modulate the participants' evaluation of foods and beverages. This miniaturized portable olfactometer provide a low cost and efficient odor delivery solution for food and beverage experiments.

Finally, Velasco et al. contributed with a mini-review on multisensory technologies that have been forwarded both for flavor, but also more general, food and drink augmentation. Whilst their suggestion was that there are a number of interesting of these technologies that promise to transform HFI design (e.g., experience design, healthy eating, and sensory marketing), they also indicated that there are several challenges that will need to be addressed carefully before they become part of our everyday life food and drink experiences.

Overall, the research presented in this topic contributes to both multisensory principles and technologies in MHFI. We hope that the papers featured in this special issue will support and inspire further research in MHFI.

## AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

# REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Velasco, Karunanayaka and Nijholt. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Music to Make Your Mouth Water? Assessing the Potential Influence of Sour Music on Salivation

#### Qian J. Wang<sup>1</sup> \*, Klemens Knoeferle<sup>2</sup> and Charles Spence<sup>1</sup>

<sup>1</sup> Crossmodal Research Laboratory, Department of Experimental Psychology, Oxford University, Oxford, UK, <sup>2</sup> Department of Marketing, BI Norwegian Business School, Oslo, Norway

People robustly associate various sound attributes with specific smells/tastes, and soundtracks that are associated with specific tastes can influence people's evaluation of the taste of food and drink. However, it is currently unknown whether such soundtracks directly impact the eating experience via physiological changes (an embodiment account), or whether they act at a higher cognitive level, or both. The present research assessed a version of the embodiment account, where a soundtrack associated with sourness is hypothesized to induce a physiological response in the listener by increasing salivary flow. Salivation was measured while participants were exposed to three different experimental conditions – a sour soundtrack, a muted lemon video showing a man eating a lemon, and a silent baseline condition. The results revealed that salivation during the lemon video condition was significantly greater than in the sour soundtrack and baseline conditions. However, contrary to our hypothesis, there was no significant difference between salivation levels in the sour soundtrack compared to the baseline condition. These results are discussed in terms of potential mechanisms underlying the auditory modulation of taste perception/evaluation.

#### Edited by:

Anton Nijholt, University of Twente, Netherlands

#### Reviewed by:

Aleksandra Mroczko-W ˛asowicz, National Yang-Ming University, Taiwan Merijn Bruijnes, University of Twente, Netherlands

> \*Correspondence: Qian J. Wang qian.wang@psy.ox.ac.uk

#### Specialty section:

This article was submitted to Eating Behavior, a section of the journal Frontiers in Psychology

Received: 13 December 2016 Accepted: 10 April 2017 Published: 26 April 2017

#### Citation:

Wang QJ, Knoeferle K and Spence C (2017) Music to Make Your Mouth Water? Assessing the Potential Influence of Sour Music on Salivation. Front. Psychol. 8:638. doi: 10.3389/fpsyg.2017.00638 Keywords: salivation, crossmodal correspondences, taste perception, audiovisual stimuli, physiological response

# INTRODUCTION

Recently, it has been demonstrated that people tend to robustly associate attributes of sound with specific olfactory (i.e., smell) and gustatory (i.e., taste) stimuli. For instance, consonant harmonies and legato musical articulation tends to be associated with sweetness, while dissonant harmonies and staccato articulation tends to be associated with sourness instead (e.g., Mesz et al., 2011; Wang and Spence, 2016). In addition, both sweet and sour tastes are mapped to high pitch whereas bitter tastes are mapped to low pitch (Crisinel and Spence, 2010; Mesz et al., 2011; Knoeferle et al., 2015; Wang et al., 2016). Furthermore, these sound-taste correspondences can affect people's evaluation of the taste/flavor of foods. For example, ratings of juice samples on a sweet–sour scale varied significantly depending on the consonance/dissonance levels of the background musical composition that people heard in one recent study (Wang and Spence, 2016). However, what is currently still unclear is whether these changes in taste evaluation occur at a low level (i.e., by directly influencing sensory experience), and/or at a higher level, such as by priming people's expectations or by biasing their self-reported taste ratings.

A possible low level hypothesis, investigated in the present study, is an embodied account, whereby people might associate certain soundtracks with certain tastes because the soundtracks

induce a similar physiological response in the listener as ingesting foods having that taste property. More specifically, we hypothezise that people might associate a soundtrack with sourness because the soundtrack, much like sour foods, can increase the listeners' salivary flow. Salivation is a non-conscious physiological process controlled by the autonomic nervous system, which aids in the digestion process and can influence the perception of tastants in the mouth (see Spence, 2011, for a review). Salivation can also be induced by conditioned reflexes, such as seeing or smelling appetizing foods (Wooley and Wooley, 1973; Krishna et al., 2014), or even by a goal-driven material reward (Gal, 2012). Previous research has shown that while looking at a lemon does not increase salivation (Kerr, 1961; Shannon et al., 1974), sniffing or slicing lemons does (e.g., Pangborn, 1968; Pangborn et al., 1979). Looking at a video of someone else eating a lemon has also been shown to induce salivation (Hagenmuller et al., 2014). Therefore, to check the validity of our methodology, we used a video of a man eating a lemon (henceforth referred to as "lemon video condition," which should increase salivation (thus demonstrating the sensitivity of our measurement technique).

Furthermore, participants from our previous studies have occasionally commented on the "mouth-watering" effect of high-pitched and dissonant soundtracks which were composed to correspond to sourness. Previously, it has been shown that music can influence the composition of salivation (Suda et al., 2008), with major mode music reducing salivary cortisol levels as compared to minor mode music. While there are no studies relating salivary cortisol levels with effects on taste perception, those who exhibit higher cortisol level increases due to stress also tend to consume more foods, including more sweet foods, as compared to those who experience lesser cortisol level changes (Epel et al., 2001). Spoken food words have also been shown to increase salivation compared to non-food words (Staats and Hammond, 1972). Based on these results, we would expect that a putatively sour soundtrack might enhance the level of salivation in the listener. This might especially be the case if the listener explicitly associates the soundtrack with the idea of sourness (i.e., if they were to match the soundtrack to sourness in a forced-choice task with multiple taste words as options, say).

To test this embodiment account hypothesis, a study was designed to measure levels of salivation under different audio/video conditions. Several methods of saliva collection have been used over the years, including the absorption of saliva by dental cotton rolls, measuring the frequency of swallows, or the electrophysiological measurement of parotid gland activity (Nederkoorn et al., 2001). The method of cotton-roll collection is used in the current study, as it has been shown to provide a reliable, sensitive, and straightforward means of measuring salivary flow (White, 1977).

## MATERIALS AND METHODS

#### Participants

Thirty six participants (22 women, 14 men) aged between 18 and 49 years (M = 23.1, SD = 6.6) took part in the study. The participants reported no hearing impairments. The participants were recruited from the Oxford Psychology Research Participant Database and the Experimental Psychology Research Participation Scheme. The study was carried out in accordance with the recommendations of the Central University Research Ethics Committee of Oxford University, with the written informed consent of all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Central University Research Ethics Committee of Oxford University (R47262\_RE001).

# Audio/Video Stimuli

Three audio/video stimuli were used. As a sour soundtrack, we used a high-pitched and dissonant soundtrack composed by Bruno Mesz that has been shown to reliably correspond to sourness based on previous studies (Kontukoski et al., 2015). In fact, Wang et al. (2015) compared seven soundtracks that have been designed to correspond to the experience of tasting sourness. In the study, the soundtrack by Mesz was labeled as sour, as opposed to any other basic taste, by the largest number of participants<sup>1</sup> (58/100). A silent video of a man eating a lemon was used as an additional condition (the "lemon video condition") to verify the validity of the saliva measurement methodology used here, since it has previously been shown to elicit salivation (Hagenmuller et al., 2014). The specific 60-s segment of the video can be viewed at https://www.youtube.com/ watch?v=5FfHSUVBIdw#t=63s. Finally, a silent condition (via a soundtrack with the commands "start" and "stop" separated over a 60 s interval) was included as a baseline saliva measure. All three conditions were 60 s long. The sour soundtrack and baseline conditions were accompanied by a visual target (+) for participants to focus on while listening to the soundtracks.

#### Procedure

The experiment was conducted at the Crossmodal Research Laboratory at the University of Oxford. Participants were seated at a table in front of a computer monitor with a keyboard, mouse, and headphones in an experimental booth. On the side table were six small plates each with three 8 mm dental cotton rolls, a cup of water, and a napkin.

On each trial, the participants were instructed to place three cotton dental rolls in their mouth, two buccally and one under the tongue, then immediately start playing the soundtrack or video. Once the soundtrack or video had finished, the participants were asked to remove the cotton rolls immediately, place them back on the plate, and hand them to the experimenter. Each trial lasted for 60 s, and the participants were given a 5 min recovery period between trials. Each condition was repeated twice (not necessarily successively), thus giving rise to a total of six trials. The order in which the trials were presented was determined using a Williams Design Latin Square in order to minimize first order carryover effects between trials. The cotton rolls were disposed of immediately after weighing.

<sup>1</sup>Note that, if participants chose tastes at random, the soundtrack would be labeled as sour by 25/100 people.

After the saliva collection trials, the participants rated which basic taste (sweet, sour, bitter, salty) the soundtrack best matched with, and more specifically, how well the sour soundtrack matched with sweet, sour, and bitter tastes on three 1–7 scales (1 = does not match at all, 7 = matches very well). They also reported their age and gender.

The study lasted for approximately 35–40 min. The participants were paid £6 or awarded with course credit for taking part.

#### Data Analysis

To determine the level of induced salivation, the cotton rolls were weighed before and immediately after each trial, on a balance with 0.0001 g precision. The difference between the two weights was used as the amount of induced saliva. The mean weight of induced salivation was then calculated for each condition and each participant.

A repeated measures analysis of variance (RM-ANOVA) was conducted with the factor 'experimental condition' (sour soundtrack, lemon video, silence). In addition, the model included participants' rating of whether they matched the soundtrack with sourness as a between-participants variable, and the interaction term of the experimental condition and the between-participants variable.

Furthermore, we calculated the % increase of salivation for each participant while listening to the sour soundtrack as compared to the silence condition. We then calculated Pearson's correlation coefficient between this % increase and how much the participant matched the soundtrack to sourness, in order to determine if sensitivity to the soundtrack's intended taste representation influenced their level of salivation.

#### RESULTS

The average salivation level for each experimental condition is shown in **Figure 1**. RM-ANOVA with Huynh-Feldt corrections revealed a significant main effect of experimental condition on salivation [F(1.82,61.77) = 6.85, p = 0.003, η <sup>2</sup> = 0.17],

FIGURE 2 | Correlation plot between the % increase in salivation between the sour soundtrack condition as compared to the silent baseline condition, and the soundtrack-taste match rating between the sour soundtrack and sourness (1 = does not match at all, 7 = matches very well). The black line indicates line of best fit. There is no significant correlation (r<sup>36</sup> = –0.11, p = 0.51).

but no significant main effect of soundtrack-sourness match [F(1,34) < 0.000, p = 0.99] and no interaction effect between the two [F(2,68) = 0.17, p = 0.84]. More specifically, pairwise comparisons with Bonferroni corrections revealed that more salivation was measured during the lemon video condition (Mvideo = 0.93 g, SD = 0.68) as compared to the silent condition (Msilence = 0.74 g, SD = 0.53, p = 0.006) or the sour soundtrack condition (Msoundtrack = 0.74 g, SD = 0.58, p = 0.029). The sour soundtrack condition, however, did not significantly differ from the silent baseline condition (p = 1.00). This result does not support our hypothesis, which stated that listening to the sour soundtrack would induce increased salivation compared to the baseline condition.

Moreover, there was no significant correlation between % increase of salivation while listening to the sour soundtrack as compared to silence, and the rating of how much the sour soundtrack was matched to sourness (r<sup>36</sup> = −0.11, p = 0.51, see **Figure 2** for a correlation plot). In other words, the extent to which someone matched the sour soundtrack to sourness is not related to any increase in the amount of salivation while listening to the sour soundtrack.

#### DISCUSSION

The results of the present study reveal that, as reported previously, watching a video of someone eating a lemon induces increased salivation as compared to the baseline condition, i.e., silently looking at a fixation cross. This replication of previous results (Hagenmuller et al., 2014; see Spence, 2011, for a review) validates our methodology of using dental rolls to measure salivation. On average, the lemon video condition increased salivation by 0.18 g as compared to the baseline condition. This is similar to the findings reported by Hagenmuller et al. (2014) where a different lemon video increased salivation by approximately 0.25 g over a 60-s interval.

However, we found no evidence that listening to the sour soundtrack increased salivation in our participants compared to the silent baseline condition. This result is especially telling given that we used the most effective – in terms of being associated with sourness – soundtrack that has been tested to date (Wang et al., 2015). Perhaps auditory stimulation is simply not sufficient to evoke a physiological response; this may be in-line with previous research which showed that while looking at lemons does not induce increased salivation (Kerr, 1961; Shannon et al., 1974), smelling or slicing lemons does (Pangborn, 1968; Pangborn et al., 1979). Therefore, like a visual representation of lemons, the soundtrack alone might not evoke a strong enough representation of sourness to stimulate a physiological response. In line with this suggestion, it has been theorized that while sound may influence the overall eating experience, it has a relatively weak contribution (Mroczko-W ˛asowicz, 2016). The resulting percept is consciously decomposable into its component unisensory parts (e.g., hearing the sound of crunching makes potato crisps more crunchy, but it is easy to separate the sound of mastication from the flavor of the potato crisp). In contrast, olfaction has a strong (what Mroczko-W ˛asowicz termed "constitutive") contribution which binds with information from the tongue to form a unified flavor perception (Rozin, 1982; Spence et al., 2015).

In this study, we hypothesized that people might associate a soundtrack with sourness because the soundtrack, much like sour foods, could potentially increase the listeners' salivary flow. The fact that no increase in salivation was found between the soundtrack condition and the baseline condition allows us to conclude that, contrary to our initial hypothesis, we failed to observe any enhancement of salivation due to music, even when participants associated the soundtrack with sourness. Therefore, there is no evidence for this particular version of the embodiment hypothesis in the present study. Interestingly, as the lemon video did, in fact, evoke increased salivation, it implies that top–down effects are at work, where an understanding of the visual scene could trigger a mental imagery of eating a lemon, which would then produce the physiological response observed (Jenkins and Dawes, 1966).

Going back to the different high and low level mechanisms proposed in the "Introduction" section, it is worth enumerating here which other pathways could be underlying the taste modulation effects by putatively sour soundtracks, such as reported by Wang and Spence (2016). Besides physiological influences, another bottom–up mechanism could involve attentional capture. According to this view, auditory features might automatically focus our attention on taste elements in the food that crossmodally correspond to those features. This focused attention could then enhance the salience of the attended feature in a mixture (in this case, sour tastes in a food/drink), relatively to when the same feature is unattended (Driver, 2001; Spence, 2014). In terms of top–down influences, associating soundtracks with sourness might prime people's sensory expectations of sourness in the food that they are about to consume, which could then go on to influence the perceptual experience (see Deliza and MacFie, 1996; Piqueras-Fiszman and Spence, 2015, for reviews). Finally, it is worth considering the possibility that the sour soundtracks might act only to alter participants' self-reported ratings without having a genuine perceptual effect.

Another intriguing question from the present study is just where exactly such crossmodal associations between sourness and sounds might come from. One hypothesis of soundtaste correspondences – specifically that between auditory pitch and taste<sup>2</sup> – is that the correspondence originates in innate stereotypical orofacial gestures that people make in response to ingesting different tastes (Knöferle and Spence, 2012; Spence, 2012; Bredie et al., 2014). Babies protrude their tongue out and up in response to pleasant tastes such as sweetness (Rosenstein and Oster, 1988; Steiner et al., 2001). This in turn produces a high vowel sound when air is exhaled (Ladefoged and Johnson, 2011). In contrast, the tongue goes out and down in response to unpleasant tastes (e.g., bitterness), which then produces a low vowel sound upon exhalation. Unlike in the case of sweetness and bitterness, however, this does not account for the fact that sourness, which is traditionally characterized as aversive, corresponds to high pitch.

An alternative hypothesis is based on emotion mediation, which seeks to explain the association between sourness and auditory attributes such as fast tempo, high pitch, and high levels of harmonic dissonance. The experience of ingesting a sour taste is associated with higher levels of arousal as compared to the other tastes (Wang et al., 2016). Similarly, the experience of listening to these auditory parameters is also associated with high arousal levels (e.g., Blumstein et al., 2010; Van der Zwaag et al., 2011; Wang et al., 2016). Therefore, the correspondence between sourness and sound might be linked to their similar associations with high arousal states.

While the present study does not support a physiological link between sound and sourness, it is still possible that music, in conjunction with other sensory stimuli, might act to enhance physiological responses to food/drinks. An interesting future study, for instance, could compare the lemon video condition with a combined lemon video plus sour soundtrack condition, to assess if music might act to further enhance salivation.

# AUTHOR CONTRIBUTIONS

QW and CS designed the study. QW collected the data and designed the experimental stimuli. QW and KK performed data analysis. All authors participated in manuscript preparation and all authors read and approved the final manuscript.

# ACKNOWLEDGMENT

CS would like to thank the AHRC grant entitled 'Rethinking the senses' (AH/L007053/1) for supporting this research.

<sup>2</sup> It has been demonstrated that sweetness and sourness corresponds to relatively higher pitch whereas bitterness corresponds to relatively lower pitch (Crisinel and Spence, 2010; Wang et al., 2016).

## REFERENCES

fpsyg-08-00638 April 24, 2017 Time: 12:56 # 5


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer MB and handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

Copyright © 2017 Wang, Knoeferle and Spence. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Influence of Color on the Consumer's Experience of Beer

Felipe Reinoso Carvalho1,2 \*, Pieter Moors<sup>2</sup> , Johan Wagemans<sup>2</sup> and Charles Spence<sup>3</sup>

<sup>1</sup> Department of Marketing, School of Management, Universidad de Los Andes, Bogotá, Colombia, <sup>2</sup> Brain and Cognition, Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium, <sup>3</sup> Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom

Visual appearance (e.g., color) cues set expectations regarding the likely taste and flavor properties of food and drink. These expectations may, in turn, anchor the subsequent tasting experience. In the present study, we examined the influence of the color of a beer on the consumer's experience. Dark and pale beers were evaluated both before and after tasting. Importantly, these beers were indistinguishable in terms of their taste/flavor when tasted without any visual cues. The results indicate that the differing visual appearance of the beers led to clear differences in expected taste/flavor. However, after tasting, no differences in flavor ratings were observed, indicating that the expectations based on visual cues did not influence the actual tasting experience. The participants also expected the dark beer to be more expensive than the pale one. These outcomes suggest that changes in the visual appearance of a beer lead to significant changes in the way in which consumers expect the beer to taste. At the same time, however, our findings also suggest the need for more evidence to be collected in order to determine the boundary conditions on when such crossmodal expectations may vs. may not affect the tasting experience.

#### Highlights:

The expected flavor of a beer is affected by its visual appearance. No differences in flavor ratings were observed on tasting. Consumers expect dark beers to be more expensive than pale/amber beers.

#### Keywords: beer, flavor, color, crossmodal correspondences, sensory marketing, multi-sensorial

# INTRODUCTION

Over the last 80 years or so, several hundred studies have assessed the influences that visual cues (such as color) have on the experience of different food and drink items (e.g., Moir, 1936; Zellner et al., 1991; Parr et al., 2003; Zellner and Durlach, 2003; Zampini et al., 2007; Spence, 2015b; see Spence and Piqueras-Fiszman, 2014, for a review). For instance, the color of a food/drink (or the color of its packaging; Barnett and Spence, 2016; Lick et al., 2017) can influence which products consumers notice and, consequently, which they choose to buy (i.e., color influences shopping behavior), as well as influence their tasting experience.

The consumer's interaction with food or drink usually starts prior to the tasting experience itself, creating a rich context for the development of sensory/hedonic expectations (see Piqueras-Fiszman and Spence, 2015, for a review). Actually, the interaction between colors and taste/flavors,

#### Edited by:

Anton Nijholt, University of Twente, Netherlands

#### Reviewed by:

Pam Blundell-Birtill, University of Leeds, United Kingdom Nicholas T. Bello, Rutgers University, The State University of New Jersey, United States

#### \*Correspondence:

Felipe Reinoso Carvalho f.reinosoc@uniandes.edu.co; f.sound@gmail.com

#### Specialty section:

This article was submitted to Eating Behavior, a section of the journal Frontiers in Psychology

Received: 31 August 2017 Accepted: 04 December 2017 Published: 19 December 2017

#### Citation:

Reinoso Carvalho F, Moors P, Wagemans J and Spence C (2017) The Influence of Color on the Consumer's Experience of Beer. Front. Psychol. 8:2205. doi: 10.3389/fpsyg.2017.02205

in the case of taste and flavor perception, should not be understood only at the time that tasting occurs. The role of higher-level cognitive factors (e.g., consumer expectations) should also be considered (Shankar M. et al., 2010). It has been argued that under the appropriate conditions, both sensory congruency and sensory incongruity between what is expected and what is experienced while tasting can bring positive or negative reactions to the overall tasting experience (Stevenson, 2009; Spence et al., 2010; Piqueras-Fiszman and Spence, 2012; Spence, 2016; Velasco et al., 2016a; Yanagisawa, 2016). Visual cues (particularly color) set sensory and hedonic expectations regarding the likely taste and flavor properties of food and drink items. The suggestion is that these expectations may, in turn, anchor the subsequent tasting experience (e.g., Garber et al., 2000; Demattè et al., 2006; see Spence et al., 2010; Piqueras-Fiszman and Spence, 2015, for reviews).

Color-taste/flavor associations have been studied in a number of different ways. For instance, by comparing the presence versus absence of color, or by changing the characteristics of the color that is present in a food or drink (e.g., its intensity, hue, etc.). Other studies, meanwhile, have studied how the expectations triggered by food coloring influence the judgment of a food or a drink's flavor (see Spence et al., 2010, for a review). Researchers have demonstrated that the four or five well-known basic tastes (bitter, sweet, sour, salty, and umami) are individually associated with particular colors as well (e.g., Koch and Koch, 2003; Velasco et al., 2016a; see Spence et al., 2015, for a review), and that such crossmodal associations can be exploited in the context of consumer behavior (Garber et al., 2001, 2003a,b). For example, by adding yellow coloring to a sweet solution, it is possible to significantly decrease people's sensitivity to sweetness (Maga, 1974). There is also a growing awareness that cultural differences can influence the way in which people establish color–flavor associations (Shankar M.U. et al., 2010; Velasco et al., 2014; Wan et al., 2014a,b; Jacquot et al., 2016; Wan et al., 2016).

Guided by the aforementioned literature review, we wondered whether such ideas could be studied using a particular type of food, or drink, available in a wide range of colors that happens to be consumed frequently in everyday life. Therefore, we decided to work with beer. There is great diversity associated with the flavor of beer, and some of these flavor attributes are more commonly associated with a blond, pale, or perhaps a dark beer. However, such flavor diversity is not necessarily constrained within a certain color category. Nevertheless, we asked ourselves how people would judge a beer's flavor when comparing, let us say, two beers of very different colors, but very similar flavor. Could particular sensations triggered by the differences in color be transferred to the tasting experience, in a way that would lead to significantly different flavor judgements (see Cheskin, 1972, on the notion of sensation transference)?

Previous research has already approached the role of visual cues in a beer tasting experience as when, for instance, comparing the same beers under blind versus sighted tasting conditions. When given access to both visual aspects of beer (appearance and brand identity), consumers tend to report different preferences (Allison and Uhl, 1964; Guinard et al., 2000). Hedonic ratings also seem to change from blind to similar informed tasting conditions (Guinard et al., 2001; Lee et al., 2006), with significant variations across different age ranges (Guinard et al., 2001). Meanwhile, elsewhere, it has been shown that differently colored beer labels can also exert a significant influence over consumers' hedonic and flavor ratings, as well as on their purchase intent, even when drinking the beer from the glass, i.e., away from the packaging (Barnett and Spence, 2016). A recent report also measured the impact of affect, and the senses, in the experience of drinking beer in real context situations (see Gómez-Corona et al., 2017). These results revealed no significant differences in expected liking and purchase intent between the eight beers that were evaluated. However, semantic differences were reported between those phrases that resembled the more cognitive aspects of a beer experience (that were more frequently associated with craft beer types), versus phrases that indexed the more sensory and affective aspects of beer (and which were more frequently associated with industrial beers).

In the experiment reported here, participants tasted two beers sighted. Both beers were produced in order to be indistinguishable in terms of their flavor (when tasted without visual cues), but at the same time to have a very different visual appearance. One beer, which fell within a pale/amber color range, was intended to represent a kind of ale/lager beer type (pale beer). The other, darker, was intended to visually simulate a Belgian style double (abbey Trappist), or perhaps a porter type color range (i.e., a very dark type of beer). Note that the baseline beer used to create both drinks was produced following the standards for blond-ale beer, which, speaking flavor-wise, represents a filtered, light (in terms of alcohol and body), and hoppy beer.

In this study, we wanted to know whether by creating a dark version of a light beer, it would be possible to significantly influence the tasting experience, when comparing the expected and actual tasting sensory/hedonic evaluations of participants. Our main objective was to try and gain some understanding of the perceptual implications of experiencing a dark beer, which would be most likely judged as incongruent in terms of flavor, and especially when compared to a pale one. On the one hand, we were interested in the potential perceptual influence that the visual appearance of the beer might have on people's flavor judgments (e.g., Shankar M.U. et al., 2010). On the other hand, we were particularly interested in understanding how the contrasting formula of the dark beer would affect the consumer's tasting experience, and the expected price of such a product (i.e., by triggering confirmation and/or disconfirmation of expectation responses; see Oliver, 1977; see also Hovland et al., 1957; Piqueras-Fiszman and Spence, 2015, for an overview on assimilation and contrast effects). This experiential exercise was set as a comparison between such dark beer and its pale counterpart.

When thinking about the potential applicability of these ideas, we detected the dramatic increase of the microbrewery movement<sup>1</sup> , particularly in markets where the production of craft beers is, by no means, historically common - e.g., located outside Europe. The exponential rise of the demand for craft

<sup>1</sup> See https://www.ft.com/content/c9f77348-8ccc-11e6-8cb7-e7ada1d123b1 (retrieved August, 2017).

beers is providing great access – and triggering great interest – in the consumption of more unconventional types of beer, such as dark ones (Carroll and Swaminathan, 2000; Murray and O'Neill, 2012).

# MATERIALS AND METHODS

#### Participants

Between the 16th and 18th of December, 2016, visitors at the Musical Instruments Museum Brussels (MIM), in Belgium, were invited to take part in a short experiment. They were informed that they would be given complimentary beer to taste while answering a short survey. A total of 136 participants agreed to take part (45% females, 55% males, mean – M – age of 32.3 years, standard deviation – SD – of 12.4, with around 60% of participants being between 20 and 30 years old, and a total of 80% of participants being less than 40 years old). All of the participants were at least 16 years of age (the minimum legal age to drink beer in Belgium), and gave their informed consent prior to taking part in the study. None of the participants reported having a cold or any other impairment of their senses of smell, taste, or hearing at the time of the study. In general, the participants were mostly European residents, with the majority from Belgium (28.7%), France (19.0%), and the United Kingdom (18.4%).

#### Stimuli

The beers that were used in this experiment were produced under the strict supervision of the Laboratory of Enzyme, Fermentation and Brewing Technology, at KU Leuven, Belgium. Here, an inhouse-produced blond-ale beer was used as baseline (with 7 EBC; EBC<sup>2</sup> is the color based standard reference method of color grade in beers; a higher EBC means a darker colored beer, and viceversa<sup>3</sup> ). The color grade of this baseline beer was artifically altered in two ways, in order to obtain two beers with different colors, but similar flavor. In summary, one beer was fermented and filtered (baseline). Afterward, two batches were colored separately and then individually carbonated/bottled.

The color agents used for this process were provided by PureMalt<sup>4</sup> (color agents labeled as RB7, RB1500). These coloring agents were chosen to have a minimum impact on the resulting flavor. The two resultant beers were referred to as pale (17.5 EBC), and dark (50 EBC). The dark beer resulted from mixing 2.9 g of RB1500 plus 2.4 g of RB7 per liter of the original 7 EBC. The pale beer formula was the result of mixing 45% of 20 EBC (the latter was obtained by adding 1.4 g of RB7 per liter on the original 7 EBC), 45% of original 7 EBC, plus 10% of the dark – 50 EBC. The resultant formulas outputted two beers with similar-low body, smell, hopiness, and astringency. The final alcohol content of the two beers was 5.5 (% v/v), with

a bitterness of around 22 IBU<sup>5</sup> , and a carbonation level of 5.6 g/l. The suggested temperature for pouring these beers was between 5 and 10◦ Celsius (this temperature range was maintained during the entire experimental procedure). All of the beers used in the present study were consumed during a period of no more than 6 months after bottling. **Figure 1** shows the two resultant beer colors.

Following industry standards, a triangle test was implemented for defining participants' ability to differentiate between the resultant flavors of the two beers. The triangle test is a discriminative method commonly used in the sensory evaluation of foods and drinks (see British Standard ISO, 2004, for details). In this test, the participants had to choose which beer was different from three choices, without knowing anything about which beer they were tasting each time (since they could not actually see the differences in what they were drinking<sup>6</sup> ). Twenty five participants took part in this test (the

<sup>2</sup> See Beer 10-A Spectrophotometric Color Method, ASBC (American Society of Beer Chemists) methods of analysis, for details on the EBC methodology.

<sup>3</sup> See https://tinyurl.com/beer-ebc-scale for an overview of the EBC standardized scale (retrieved April, 2017).

<sup>4</sup> See http://www.puremalt.com for details of the company and their products.

<sup>5</sup> IBU is the International bitterness units in Wort, obtained by segmented flow analysis. See Method Wort-23, of the ASBC (American Society of Beer Chemists) methods of analysis.

<sup>6</sup>During the triangle test, the beers were served in darkened-black cups. The foam of the resultant dark beer was somewhat darker than that of the pale one. Therefore, in order to prevent any color bias, the participants were blindfolded while tasting. The lights in the experimental room were set at the minimum.

British Standard ISO, 2004 standard recommends a minimum of 18 assessors for triangle testing differences). Here, three darkened-and-numbered-cups were filled with two different beers (two cups filled with the same beer, and the third cup with a different beer; in our case, a distribution between our pale and dark beers). This pouring process was counter-balanced across participants. The resultant ratings were compared by means of a Pearson Chi-square distribution. The results confirmed that the participants were not able to demarcate the differences in flavor between the pale and dark beers (X 2 <sup>10</sup>,<sup>05</sup> = 2.163, p = 0.141). These results therefore suggest that naïve drinkers would not be able to differentiate the flavor of the beers that were produced here (when tasted without visual cues), leaving their differences in color as the most important factor that could lead to changes in their consuming behavior.

#### Experimental Design and Procedure

The Social and Societal Ethics Committee at KU Leuven (SMEC) approved this protocol (registered as G-2015 09 339).

#### Experimental Design

The objective of this particular study was to compare the expectations and the actual taste/flavor judgment of each participant (using a within-participants experimental design, with two conditions). The experiment took place on the 9th floor of the MIM. During the three experimental days, it was possible to have a well-controlled and stable environment, although more naturalistic as compared to a laboratory environment. The experiments were performed using computers placed on tables. The participants sat together, although fairly well separated, to prevent them from interacting with one another. Each participant joined the experiment for approximately 10 min.

#### Experimental Procedure

Each participant was seated in front of a computer screen with a computer mouse, and a keyboard to complete the survey. Each participant had two transparent-plastic cups filled with the dark and pale beers, respectively, with each cup containing no more than 5 cl. Before and after drinking the beers, the participants were advised to drink tap water, for palate cleansing (a cup filled with tap water was available for each participant).

The survey consisted of an electronic form containing three main stages. In the first step, the participants were instructed to read and accept the conditions of the informed consent before entering their demographic details. In a second step, the participants responded to a pre-questionnaire in which they rated their expectations concerning the taste (bitterness, sweetness), flavor (body, alcohol strength), and expected liking of the pale and dark beers. In the third step of the procedure, the participants tasted each beer, while again answering the same questions as in the second step (the beers were experienced separately, meaning that the participants answered the corresponding batch of questions after tasting each of the two beers). Steps two and three consisted of seven-point scales presented in a randomized order (with the number 1 of the scale representing 'not at all,' the number 4 representing 'balanced'/'moderated'/'neutral' - depending on the question; and '7' representing 'very much'; for example, 1 for 'not at all bitter,' 4 for 'moderate,' and 7 for 'very bitter' ratings). Finally, the participants indicated which of the two beers they preferred, and which beer they thought was the most expensive (here, orderrandomized multiple-choice questions were used). Note that only the color ratings were based on a bipolar scale (with 1 being 'very dark,' 4 being 'moderate,' and 7 'very pale').

The order of presentation of the beers was counterbalanced across participants. Hence, the participants were advised to follow the survey instructions carefully, in order to drink the appropriate beer at the appropriate time. Note that there were always at least two supervisors present during the entire experimental process for extra guidance, coordination, and support, in addition to the self-guiding written experimentalinstructions. Upon finishing the study, the participants were instructed to leave the room without discussing any details with the next group of participants.

#### Analysis

As the data for each rating scale were based on a 2 × 2 (pretaste vs. post-taste by dark vs. pale) within-subjects experimental design, we subjected the data to a 2 × 2 repeated measures ANOVA for each scale, separately. As the analysis of many scales might not guarantee proper Type I error control for α = 0.05, we applied a conservative Bonferroni correction for multiple comparisons. In addition to considering the six scales, we also consider the three statistical tests conducted in each ANOVA as potentially inflating Type I error (see Cramer et al., 2016). Therefore, we set α = 0.05/18 = 0.00277 for our main analyses reported below. As a measure of effect size, we report generalized η 2 as proposed by Olejnik and Algina (2003). All analyses were conducted in R, using the RStudio IDE, and mainly relying on the tidyverse and afex packages.

# RESULTS

#### Expected versus After-tasting Ratings

**Figure 2** summarizes the mean ratings of the participants prior to (triangles), and after (circles) tasting the dark and pale beers.

For liking, we observed a main effect of beer type [F(1,135) = 25.42, p < 0.0001, η 2 G = 0.05], no main effect of time [F(1,135) = 0.89, p = 0.35, η 2 G = 0.0009], and an interaction effect between beer type and time [F(1,135) = 15.73, p = 0.0001, η 2 G = 0.02]. As can be seen from **Figure 2**, the pale beer was liked more, on average. However, this main effect was qualified by an interaction which indicated that the liking ratings actually converged. Indeed, when considering the simple effects, participants expected to like the pale beer more than the dark beer [t(253.66) = 6.415, p < 0.0001] prior to tasting, yet after tasting there was no difference [t(253.66) = 1.57, p = 0.12].

A similar pattern of results emerged for bitterness ratings. Again, a main effect of beer type was observed [F(1,135) = 20.63, p < 0.0001, η 2 G = 0.04]. There was no main effect of time [F(1,135) = 0.34, p = 0.56, η 2 G = 0.0005], and there was an interaction effect between beer type and time [F(1,135) = 16.4, p < 0.0001, η 2 G = 0.03]. On average, the dark beer was expected to ratings for the pale beer.

fpsyg-08-02205 December 17, 2017 Time: 16:35 # 5

be more bitter than the pale one, yet this main effect was again qualified by an interaction indicating that the corresponding ratings converged. Prior to tasting, the dark beer was expected to taste more bitter than the pale beer [t(266.88) = 6.085, p < 0.0001]. Yet, after tasting, the aforementioned difference disappeared [t(266.88) = 0.676, p = 0.5).

For sweetness ratings, no main effect of beer type was observed [F(1,135) = 1.44, p = 0.23, η 2 G = 0.003]. A main effect of time was observed, but did not survive the correction for multiple comparisons [F(1,135) = 8.56, p = 0.004, η 2 G = 0.01]. No interaction between beer type and time emerged [F(1,135) = 2.21, p = 0.14, η 2 G = 0.004].

Similarly for sourness ratings, no differences emerged in the analysis either for the main effects of beer type [F(1,135) = 0.005, p = 0.94, η 2 <sup>G</sup> < 0.0001] or time [F(1,135) = 4.51, p = 0.04, η 2 G = 0.007] nor for the interaction [F(1,135) = 2.13, p = 0.15, η 2 G = 0.003].

For the ratings of strength, we observed a main effect of beer type [F(1,135) = 14.16, p = 0.0003, η 2 G = 0.03], a main effect of time [F(1,135) = 18.18, p < 0.0001, η 2 G = 0.02], and an interaction between beer type and time [F(1,135) = 36.85, p < 0.0001, η 2 G = 0.06]. The main effects indicated that the dark beer was rated as stronger on average compared to the pale beer, and that the expected strength was higher prior to tasting compared to after tasting. This interpretation should again be qualified by the results of the interaction, which again converges to the results of the previous scales. Prior to tasting, the dark beer was expected to taste stronger compared to the pale one [t(268.95) = 6.898, p < 0.0001], yet this difference (prior to tasting) was no longer observed when comparing the after-tasting ratings of both beers [t(268.95) = 1.414, p = 0.16].

The same pattern emerged for the ratings of body. Main effects of beer type [F(1,135) = 73.14, p < 0.0001, η 2 G = 0.13] and time [F(1,135) = 20.65, p < 0.0001, η 2 G = 0.03] as well

FIGURE 3 | Scatterplot depicting average ratings for each combination of beer type, time and rating scale, split up by country. Error bars indicate means ±2 SEM. The dotted line is the identity line. Points closer to the dotted line indicate average ratings for both beer types are similar. Points below the dotted line indicate higher ratings for the dark beer, while points above the line indicate higher ratings for the pale beer.

as an interaction between beer type and time were observed [F(1,135) = 64.81, p < 0.0001, η 2 G = 0.09]. The dark beer was rated to have more body, on average, as compared to the pale beer, and ratings of body were higher prior to tasting compared to after tasting. The interaction depicts a similar pattern, where the dark beer was expected to have more body compared to the pale one [t(264.22) = 11.734, p < 0.0001], yet this difference again disappeared after tasting [t(264.22) = 1.224, p = 0.22].

In summary, for each of the six scales, we observed a clear and consistent pattern. Wherever prior expectations about the flavor of the beer were apparent based on the visual appearance, such expectations-based differences did not affect the corresponding ratings after having tasted the beer, since the after-tasting ratings of both beer types tended to converge.

Finally, 43.5% of the participants reported that they expected the dark beer to be more expensive than the pale beer. On the other hand, 28.0% reported the opposite, and 28.5% reported that both beers should have the same price. A Pearson's chi-square goodness-of-fit test for distribution conducted between 'dark,' 'pale,' and 'same price' choices – with confidence of 95% – shows a weak, but significant difference between these three choices [χ 2 (2) = 6.191, p = 0.045].

In the following section, we report an exploratory analysis that was inspired by our main analysis. That is, we wanted to verify whether the convergence observed in the after-tasting ratings would generalize across countries (here, we mainly consider Belgian versus British versus 'other' participants for reasons outlined below)<sup>7</sup> . Because this analysis was conducted post hoc, we merely report it to inspire new research, and do not commit ourselves to strong conclusions based on these results.

<sup>7</sup>A similar analysis was carried out across gender. Similarly to the assessment across country groups, only a main effect of gender was observed for liking ratings [F(1,133) = 15.6, p = 0.0001, η 2 G = 0.05]. Here, men tended to give higher average liking ratings compared to women. In summary, the differences in rating across gender do not seem to affect the overall results, since men and women follow similar judgment trends. Refer to the Supplementary Material for the complete results of the analysis (see footnote 9).

#### Cross-Country Comparison

fpsyg-08-02205 December 17, 2017 Time: 16:35 # 6

Given the fact that dark Belgian beers are usually very different, in terms of their flavor profile, than dark British beers (for instance, some dark Belgian beers may be twice as strong as their British counterparts), and considering that Belgian and British participants were amongst the most well-represented nationalities within our population<sup>8</sup> – and both have a historical tradition of producing and consuming dark beers – the data was subdivided into three sub-groups (Belgians, with N = 39, British, with N = 25, and others, with N = 71).

To analyze if there were significant differences in the ratings across these sub-groups, a repeated measures ANOVA was calculated, with country as the between-participant variable (Belgium, United Kingdom, and others), type of beer/time as within-independent variables, and the rating scales as dependent variables (see **Figure 3**). For succinctness, we only report the effects that reached statistical significance (refer to the Supplementary Material<sup>9</sup> for the complete results of the analysis). For liking, a main effect of country group was observed [F(2,132) = 6.42, p = 0.002, η 2 G = 0.04]. Here, participants from Belgium gave lower ratings, on average, as compared to those from the UK and from other countries. For bitterness ratings, an interaction between country group and color was observed [F(2,132) = 8.88, p = 0.0002, η 2 G = 0.04]. This interaction effect largely emerged from the participants from 'other' countries expecting the dark beer to be more bitter prior to tasting, allowing them to regress more to the similar average ratings that were obtained for dark and pale beers after tasting. For all other scales, no pronounced effects of country group were observed.

The price judgment across the different subgroups of countries was also visualized. **Table 1** shows this comparison. Here, most Belgians and 'others' tend to expect the dark beer to be more expensive than the pale one (values in bold). On the other hand, the corresponding results for the British participants do not show such a trend. Nevertheless, these percentage differences did not reach statistical significance [χ 2 (4) = 2.33, p = 0.68].

TABLE 1 | Participants' price judgment for the beers, across the different three sub-groups of countries (Belgium, United Kingdom, and others).


#### Correlations

**Figure 4** shows the correlations across the ratings for the pale, and dark beer, respectively. The upper panel depicts the numerical correlations values, while the lower panel denotes the correlations that remain statistically significant after Holm correction. Among the significant results we can appreciate that, for both beer types, expected liking and after-tasting liking ratings are positively correlated. For both beers, bitterness is negatively associated with liking. For dark beers sweetness is positively related to liking, but negatively to bitterness. For both beers, sourness and bitterness, strength and bitterness, strength and sourness, and body and strength are all positively related. Last, for pale beers, body was positively associated with liking and bitterness.

#### DISCUSSION AND FUTURE WORK

The participants in the present study evaluated the flavor of two beers having distinctly different appearances (one very dark and another pale/amber), but indistinguishable in terms of their flavor (when tasted without visual cues). The sensory/hedonic evaluation of the two beers occurred both before and after tasting. Here, we hypothesized that by creating different color versions of the same drink (in this case, two different color versions of a light beer) it would be possible to significantly influence the tasting experience (Moir, 1936; Demattè et al., 2006; Spence et al., 2010). These results suggest that there can be differences between what the participants expect from both beers, and their judgment after tasting them, yet more evidence is needed in order to show that these expectations could, in fact, influence the overall tasting experience.

In terms of expectations, the participants expected to like the pale beer more than the dark one. The dark beer was expected to be more bitter, to taste stronger, and to have more body than the pale beer (see **Figure 2**). Even though blond/pale beers are most commonly regarded as more filtered, with stronger hoppiness, and more bitter, than dark ones, it has been previously shown that for certain beverages (including beers), darker brown colors are associated with stronger, or more bitter, tastes/flavors (Guinard et al., 1998).

When comparing these expectations with after-tasting ratings across both beer types, no significant differences were observed. The latter result suggests that the differences in color did not affect the way the participants judged the beers after tasting. Such a lack of crossmodal effects suggests that the visual expectations were discounted when it came to the tasting experience. Hence, we could somehow conclude that color differences in beers do not have a significant impact on a beer's flavor judgement. In other words, people do not seem to rely on beer color categories while denoting a beer's flavor, at least when the expectations set by eye (regarding a beer's taste/flavor attributes) turn out to be a long way from the actual tasting experience.

As a matter of fact, for our experiment in particular, the most salient flavor cues of the dark beer were mostly incongruent (bitterness, alcohol strength and body), since the formula of this dark beer was based on the brewing process of a blondlight beer. Here, we hypothesize that the contrast between what

<sup>8</sup> In this experiment, the French were also among the highest number of participants. The French, however, are not known for having a specific beer culture as far as dark beers are concerned.

<sup>9</sup>Complete data analysis, in .html and .pdf formats, are available through the following link: https://doi.org/10.6084/m9.figshare.5687830.v1 (retrieved on November, 2017).

was expected and what was experienced could have weakened any possible transference of sensations, from the process of expectations into the tasting experience itself (Cheskin, 1972). The latter would be in line with the theory of assimilationcontrast (Hovland et al., 1957). This theory claims that if the difference between expectation and reality is somehow within a person's limit of acceptance, it would be possible to, consciously or not, change the perceptual evaluation of a food/drink product in order to bring it in line with the expectations (see Piqueras-Fiszman and Spence, 2015, for a review). However, if such a difference is sufficiently great, a consumer would tend to exaggerate this difference between expectations and reality, shifting the product's evaluation toward the opposite direction, when compared to what was originally expected (Schifferstein, 2001; Yeomans et al., 2008; Wang et al., 2017).

Interestingly, the color of these beers affected their expected price. Here, the consensus was that the dark beer was expected to be more expensive than the pale one (see Expected versus After-Tasting Ratings) – this was reported by almost half of the sample. The subsequent cross-cultural comparison of Section "Cross-Country Comparison" suggested that the British residents did not show the same price judgment tendencies when compared to the other two groups of participants (Velasco et al., 2014, 2016a,b; Wan et al., 2014a,b; Jacquot et al., 2016). Yet due to the relatively low frequencies, no statistical evidence was obtained for this trend. Nevertheless, we believe that this cross-cultural assessment may inspire future analysis of correlation between the darkness of a beer and its price, across the markets of very different countries and/or regions.

In future research, it might be interesting to investigate whether a congruently darkened beer formula might actually lead to a different tasting experience, when compared to its blond/pale counterpart. For instance, by somehow matching the expectations, a darkened pale-strong beer<sup>10</sup> may be judged as even stronger, and so on (see Gottfried and Dolan, 2003; White and Prescott, 2007; Reinoso Carvalho et al., 2015, 2016a,b, 2017; see Spence, 2011, for an overview on how crossmodal congruency can lead to perceptual enhancement in different multisensory experiences). Breaking down the color characteristics of a beer in more detail (e.g., by comparing differences in hue spectrum, or differences in haziness/cloudiness) could also help better disentangling the perceptual influence that a beer's color can have on the tasting experience (cf. Barnett et al., 2017). Given recent work showing that the shape of glassware can influence the tasting experience as well (e.g., Spence et al., 2012; Spence and Piqueras-Fiszman, 2014; Spence, 2015a; see Spence and Van Doorn, 2017, for an overview), it is necessary to bear in mind that the color of a beer is never experienced in isolation, but is often affected by the glassware/material (e.g., plastic, glass) in which it is presented<sup>11</sup>. Concerning the limitations of the experiment reported here, we did not take into account the participants' initial beer preferences. Such differences may potentially bias the visual scoring and consequent taste ratings. We also chose to compare the expectations and the actual taste/flavor ratings of each of the participants while experiencing both beers (withinparticipant design). However, in our everyday experience of beer we do not necessarily compare two different beers in detail while choosing between them. We further ask ourselves if an experimental design where each participant would drink the same beer twice, blind versus sighted, would provide similar results (when compared to this method that uses triangle testing instead). Following the latter thought, future experiments may also consider comparing, for instance, a blind-tasting versus informed tasting conditions, in order to compare the effects

<sup>10</sup>For example, traditional Belgian brewers such as Duvel Moortgat (http://www.duvel.com/fr-be/la-biere/duvel), and Abdij der Trappisten Westmalle

<sup>(</sup>http://www.trappistwestmalle.be/nl/de-tripel-van-westmalle?agecheck=plus18) are well-known for offering strong pale beers (retrieved March, 2017).

<sup>11</sup>Several breweries customize their glassware in many different ways, as part of their branding strategies; i.e., http://www.telegraph.co.uk/men/thinkingman/11765323/Does-the-shape-of-your-glass-affect-how-a-beer-tastes.html (Retrieved 19th April, 2017).

of colour along with semantic-(in)congruency oriented analyses (e.g., Guinard et al., 2000, 2001).

#### ETHICS STATEMENT

fpsyg-08-02205 December 17, 2017 Time: 16:35 # 8

This study was carried out in accordance with the recommendations of 'The Social and Societal Ethics Committee at KU Leuven (SMEC),' with written informed consent from all participants. All participants gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by The Social and Societal Ethics Committee at KU Leuven (SMEC) – protocol registered as G-2015 09 339.

#### AUTHOR CONTRIBUTIONS

FR, JW, and CS designed the study. FR collected the data and participated in the design of the experimental stimuli. PM lead the data analysis, with the participation of FR. All authors participated in manuscript preparation and all authors revised the final version of the manuscript.

## REFERENCES


#### FUNDING

This research was supported by the Rethinking the Senses grant from the AHRC (United Kingdom) awarded to CS (AH/L007053/1). FR was partly funded by the CAPES Foundation, Brazil (BEX 3488/13-6). JW was supported by the Flemish Methusalem program (METH/14/02). FR and JW were also supported by KU Leuven IOF internal funding scheme (C32/17/005). PM was supported by the Flemish Organization for Scientific Research, FWO (12X8218N).

#### ACKNOWLEDGMENTS

We would like to specially thank Brecht De Causmaecker for producing the beers, and the Leuven Institute for Beer Research (LIBR, KUL) for donating them. We would also like to thank Prof. Raymond van Ee for his support during the design of the procedure and during the development of the triangle test. We would further like to thank François Nelissen, Elien Haentjens, Sarah Ahannach, Sam Van Broeck, Dr. Rebecca Chamberlain, Carlos Garcia, and a very special thanks to the MIM crew, for their collaboration during the experimental days.



crossmodal correspondences between colours and basic tastes. Flavour 4:23. doi: 10.1186/s13411-015-0033-1


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Reinoso Carvalho, Moors, Wagemans and Spence. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Influences of Product Temperature on Emotional Responses to, and Sensory Attributes of, Coffee and Green Tea Beverages

Ragita C. Pramudya and Han-Seok Seo\*

Department of Food Science, University of Arkansas, Fayetteville, AR, United States

#### Edited by:

Carlos Velasco, BI Norwegian Business School, Norway

#### Reviewed by:

Olivia Petit, Inseec Business School, France Qian Janice Wang, University of Oxford, United Kingdom

> \*Correspondence: Han-Seok Seo hanseok@uark.edu

#### Specialty section:

This article was submitted to Eating Behavior, a section of the journal Frontiers in Psychology

Received: 02 October 2017 Accepted: 13 December 2017 Published: 11 January 2018

#### Citation:

Pramudya RC and Seo H-S (2018) Influences of Product Temperature on Emotional Responses to, and Sensory Attributes of, Coffee and Green Tea Beverages. Front. Psychol. 8:2264. doi: 10.3389/fpsyg.2017.02264 Coffee and green tea are popular beverages consumed at both hot and cold temperatures. When people consume hot beverages concurrently with other activities, they may experience at different temperatures over the period of consumption. However, there has been limited research investigating the effects of product temperatures on emotional responses and sensory attributes of beverages. This study aimed to determine whether emotional responses to, and sensory attributes of, brewed coffee and green tea vary as a function of sample temperature. Using a check-all-that-apply (CATA) method, 157 participants (79 for coffee and 78 for green tea) were asked to evaluate either coffee or green tea samples served at cold (5◦C), ambient (25◦C), and hot (65◦C) temperatures with respect to emotional responses and sensory attributes. The results showed that sample temperature could have significant influences on emotional responses to, and sensory attributes of, coffee and green tea samples. More specifically, 6 and 18 sensory attributes of coffee and green tea samples, respectively, significantly differed with sample temperature. Beverage samples evaluated at 65◦C were characterized, regardless of activation/arousal level, by positive emotional responses terms and favorable sensory attributes. While beverages evaluated at 25◦C were associated more with negative emotional responses with low activation/arousal, those evaluated at 5◦C were more frequently characterized as having negative emotional responses with high activation/arousal. Sensory and emotional drivers of liking for both coffee and green tea differed both with sample temperature and gender. While both emotional responses and sensory attributes were identified as drivers of liking among females, only emotional responses were identified as drivers of liking among males. In conclusion, this study provides empirical evidence that both emotional responses to, and sensory attributes of, coffee and green tea beverages can vary with sample temperatures. To provide a better understanding of product characteristics, emotional responses to, and sensory attributes of, coffee or green tea beverages should be tested over a wider range of product temperatures.

Keywords: product temperature, sensory attribute, emotional response, coffee, green tea, gender

# INTRODUCTION

fpsyg-08-02264 January 9, 2018 Time: 17:49 # 2

Serving temperatures have been found to influence perceived intensities in basic taste solutions (Moskowitz, 1973; Bartoshuk et al., 1982; Lipscomb et al., 2016). Moreover, serving temperatures have been found to affect flavor/taste intensities and acceptances of various beverage products, including milk (Francis et al., 2005), wine (Zellner et al., 1988; Ross and Weller, 2008; Cliff and King, 2009), carbonated beverages (Cardello and Maller, 1982), and fruit-flavored beverages (Zellner et al., 1988). Those earlier studies, however, focused on quantification of intensity variation rather than qualification of sensory attributes. In other words, limited research has been done to examine whether detectability of certain sensory attributes can be affected by serving temperature of food or beverage products.

There has been no research regarding the effects of product temperatures on emotional responses to food or beverage products. Research investigating how product temperatures affect associations between emotional response and sensory perception of food or beverage products is also limited. However, three points are worth noting. First, food-evoked emotions play an important role in food acceptance and choice (King and Meiselman, 2010; Dalenberg et al., 2014; Gutjar et al., 2015). Furthermore, measuring both evoked emotions and sensory perception has been found to yield better understanding of consumer acceptance and preference toward foods or beverages (Samant et al., 2017). Second, thermal sensation (physical warmth or coldness) has been found to evoke emotional responses in humans (Kanosue et al., 2002; Sung et al., 2007; Williams and Bargh, 2008). Neuroimaging studies have revealed that when the body is exposed to different temperatures, significant changes of neural activations can be observed in the brain regions responsible for emotion processing, as well as thermal sensory perception (Kanosue et al., 2002; Sung et al., 2007; Rolls et al., 2008; Rolls, 2010). In a functional magnetic resonance imaging study conducted by Guest et al. (2007), liquid stimuli into the mouth at three different temperatures (5, 20, and 50◦C) increased neural activation in the brain regions associated with taste perception and reward, such as the insula, the somatosensory cortex, the orbitofrontal cortex, the anterior cingulate cortex, and the ventral striatum. In particular, pleasantness ratings of oral thermal stimuli were correlated with neural activations in the orbitofrontal cortex and the pregenual cingulate cortex. Finally, when people consume hot or cold meals concurrently with other activities like engaging in social conversations or performing office work, they may experience their meals over a wider range of food or beverage product temperatures because the temperatures decrease with time (Pramudya and Seo, 2018); it was reported that people generally consume a meal over a time interval between 10 and 60 min (Bell and Pliner, 2003).

This study aimed to determine whether and how temperatures of product samples affect emotional responses to, and sensory attributes of, brewed coffee and green tea beverages consumed at different temperatures: hot (65◦C), ambient (25◦C), and cold (5◦C) temperatures. These three values were chosen because those typically encounter during consumption of coffee and green tea beverages in daily life. More specifically, brewed coffee and green tea beverages are often consumed at hot temperatures; university students in the United States rated the range of 62.8–68.3◦C as ideal for consuming coffee beverages (Borchgrevinka et al., 1999). In addition, when people consume hot beverages while engaged in other activities (e.g., social conversation or office work) over a period of time, initially hot beverage temperature may fall to near ambient temperature (25◦C) during consumption. Finally, coffee and green tea continue to gain popularity as cold beverages (5◦C), e.g., iced coffee and iced matcha. Coffee and green tea beverages were specifically chosen as target products for this study because both are widely popular beverages consumed across numerous cultures worldwide, and are considered as "emotional" beverages that provide psychological comfort (Juneja, et al., 1999; Cooper, 2012; Bhumiratana et al., 2014; Labbe et al., 2015).

Four research propositions were tested in this study. First, it was to be determined whether specific sensory attributes of coffee or green tea samples would be more detectable or dominant at hot, ambient, or cold temperatures (Research proposition 1), based on previous research regarding the effects of serving temperatures on intensities of sensory attributes in basic taste solutions, foods, and beverages (Moskowitz, 1973; Zellner et al., 1988; Ross and Weller, 2008; Kim et al., 2015; Lipscomb et al., 2016; Stokes et al., 2016; Steen et al., 2017). Due to temperaturedependent variations with respect to perceived intensity, certain attributes may be more dominant in coffee or green tea samples at hot, ambient, or cold temperature. Steen et al. (2017) evaluated brewed coffee samples at six serving temperatures ranging from 62 to 31◦C by measuring volatile compound profiles using gas chromatography-mass spectrometry and eight flavor attributes (overall intensity, sour, bitter, sweet, tobacco, roasted, nutty, and chocolate) using descriptive sensory analysis. Intensities of four sensory attributes, i.e., overall intensity, bitter note, sweet note, and roasted flavor, were found to differ with sample temperatures. These attributes were especially associated with brewed coffee samples evaluated at temperatures of 50◦C or higher, possibly due to greater levels of aliphatic ketones, alkylpyrazines, some furans, and pyridines (Steen et al., 2017).

Second, it was to be determined whether specific emotions would be more highly evoked at hot, ambient, or cold temperature of coffee or green tea samples (Research proposition 2). Since sensory attributes have been found to be associated with emotional responses, temperature-induced variations in sensory attributes might be expected to affect emotional responses toward coffee or green tea beverage samples served at hot, ambient, and cold temperatures (Seo et al., 2009c; Ng et al., 2013; Chaya et al., 2015). Prior research has also demonstrated that warm or cold stimuli to the whole or partial body (e.g., hands or legs) can affect not only hedonic valence, but also emotional responses such as thermal comfort or discomfort (Kanosue et al., 2002; Guest et al., 2007; Sung et al., 2007; Rolls et al., 2008).

Third, based on previous research that found gender differences with respect to sensory perception (Doty et al., 1984; Larsson et al., 2003; Royet et al., 2003; Doty and Cameron, 2009; Ferdenzi and Roberts, 2013) and emotional processing (Wager et al., 2003; Seo et al., 2009c; Duerden et al., 2013), it was to be determined whether the effects of sample temperatures on sensory attributes and emotional responses would vary with gender (Research proposition 3). Females have been found to show better performances than males in odor memory, odor identification, and verbal fluency tasks (Larsson et al., 2003), possibly because of the greater number of neural activations of the left orbitofrontal cortex in females (Royet et al., 2003). Behavioral and neuroimaging studies have also demonstrated that males are more attentive to sensory aspects of emotional stimuli, while females are more attentive to subjective feelings of emotional stimuli (Orozco and Ehlers, 1998; Wager et al., 2003). A recent meta-analysis of neuroimaging studies associated with gender differences in emotional processing found that female processing of emotional stimuli occurs predominantly in the bilateral anterior insula as well as the mid and posterior insula on the left side, while males respond to emotional stimuli predominantly in the left anterior and mid insula as well as in the right posterior insula (Duerden et al., 2013). For this reason, it was anticipated that product temperature-dependent variation with respect to sensory attributes and emotional responses would be more pronounced in females than in males.

Finally, both sensory attributes and emotional responses have been found to play an important role in consumer acceptance of food or beverage products (Seo et al., 2009c; King and Meiselman, 2010; Piqueras-Fiszman and Spence, 2012; Dalenberg et al., 2014; Gutjar et al., 2015; Samant et al., 2017). It was therefore to be determined whether the impacts of sensory attributes and emotional responses on liking of coffee or green tea beverages would vary as a function of sample temperature (Research proposition 4a) and gender (Research proposition 4b). More specifically, if specific sensory attributes (Research proposition 1) and/or emotions (Research proposition 2) would be predominantly present at hot, ambient, or cold temperatures of coffee or green tea samples, the relative impact of individual sensory and emotional responses on liking of those samples may differ as a function of sample temperature. Moreover, if product temperature-induced sensory attributes and/or emotions differ by gender (Research proposition 3), the relative impacts of individual sensory and emotional responses on liking of coffee or green tea samples may differ between females and males.

The present study was designed to test the four research propositions for coffee (Study 1) and green tea (Study 2) beverages. This study was conducted in conformance with the Declaration of Helsinki for studies on human subjects. The protocol used in this study was approved by the Institutional Review Board of the University of Arkansas (Fayetteville, AR, United States). A written informed consent was obtained from each participant prior to the participation.

# STUDY 1: EFFECTS OF SAMPLE TEMPERATURES ON EMOTIONAL RESPONSES TO, AND SENSORY ATTRIBUTES OF, COFFEE BEVERAGE

## Materials and Methods Participants

Through the consumer profile database of the University of Arkansas Sensory Service Center (Fayetteville, AR, United States), 79 coffee consumers (51 females and 28 males) ranging in age from 19 to 76 years [mean ± standard deviation (SD) = 39 ± 16] were recruited. Using a pre-screening survey, all participants self-reported that they habitually drink one or more cups of coffee with no added condiments, e.g., sugar, milk, and creamer, etc., and they prefer black coffee [i.e., greater than 5-points on a 9-point hedonic scale ranging from 1 (dislike extremely) to 9 (like extremely)]. Participants also self-rated preferences for hot beverages (mean ± SD = 7.9 ± 1.0) on a 9-point hedonic scale ranging from 1 (dislike extremely) to 9 (like extremely) and cold beverages (mean ± SD = 7.8 ± 1.3). All participants were asked to refrain from eating, drinking (except water), and cigarette smoking for 2 h prior to their participation to avoid potential influences of such activities on sensory perception (Cho et al., 2017).

#### Sample Preparation and Presentation

Grounded roasted coffee beans (Sugar Skull blend, Onyx Coffee Lab, Fayetteville, AR, United States) were brewed for 20 min using commercial coffee makers (Model DCC-2900, Cuisinart, East Windsor, NJ, United States) using a proportion of 90 g of ground coffee per 1,800-mL of spring water. A warm-up coffee sample (Lidl Essentials Coffee Classic, Lidl, Arlington, VA, United States) was prepared in the same manner. Brewed coffee was poured into a 3,000-mL stainless steel dispenser (Bunn, Springfield, IL, United States) to maintain its high temperature. Brewed coffee was served at three different temperatures: 65, 25, and 5◦C. Sample preparation to achieve temperatures of 25 and 5 ◦C involved placing coffee samples in a water bath to facilitate the cooling process. Each sample (55-mL) was presented in a 118-mL white Styrofoam cup identified with a three-digit code. Styrofoam cups were used to (1) minimize exposure of hands to thermal stimulation and (2) maintain target temperatures of coffee samples, and because Styrofoam cups are commonly used for serving both hot and cold beverages in the United States.

#### CATA Questions of Emotion and Sensory Tests for Coffee Beverage

Since temperatures of brewed coffee samples can change quickly over time, rapid methods of emotion and sensory testing were used in this study. More specifically, participants were asked to check all appropriate terms, listed on either emotion check-allthat-apply (CATA) question or sensory CATA question. This method was found to be suitable for characterizing product temperature-dependent sensory-attribute variations in foods and beverages (Chapko and Seo, 2017; Pramudya and Seo, 2018). The emotion CATA question included 39 emotion terms from

the EsSense Profile <sup>R</sup> (King and Meiselman, 2010). The sensory CATA question included 49 sensory attribute terms of coffee beverages generated by a previous study (Chapko and Seo, 2017). The following attributes were included: 21 aroma attributes (ashy, berry, bitter, brown sugar, burnt, cereal, chemical, chocolate, cocoa, fruity, green/vegetative, metallic, musty/earthy, nutty, papery/cardboard, pungent, roasted, skunky, sour, sweet, and tobacco); three appearance attributes (cloudy, oily, and transparent); 22 taste/flavor attributes (ashy, berry, brown sugar, burnt, cereal, chemical, chocolate, cocoa, fruity, green/vegetative, metallic, musty/earthy, nutty, papery/cardboard, pungent, roasted, skunky, tobacco, bitter taste, salty taste, sour taste, and sweet taste); and three mouthfeel attributes (astringent, mouth coating, and viscous). For each sensory modality (i.e., aroma, appearance, flavor, taste, and mouthfeel), the terms were presented in alphabetical order to assist participants in quickly finding all attributes that they wanted to check. Lee et al. (2013) showed that consumer panelists took significantly less time to answer CATA questions when the terms were listed in a fixed order rather than in the Williams design presentation order. It was also found that the influence of CATA term order on consumer responses was minimal (Lee et al., 2013).

#### Procedure

This study was conducted at the University of Arkansas Sensory Service Center (Fayetteville, AR, United States). Prior to sample presentation, each participant was given a verbal introduction to the experimental protocol. Participants were then asked to taste brewed coffee (not that used in actual testing) as a warmup sample and select all appropriate terms from those listed on the emotion CATA question that characterized their emotional responses evoked by experiencing the sample (Varela and Ares, 2012). The warm-up session allowed participants to not only better understand both protocol and emotion CATA question, but also to minimize any carry-over effect.

Following the warm-up session, participants were asked to taste coffee samples at three different temperatures, i.e., 65, 25, and 5◦C in a monadic sequential fashion. Participants were asked to drink each sample as much as they wanted, then select (as in the warm-up session) all the terms on the emotion CATA question for characterizing their emotional responses to the sample. The presentation order of the three serving temperatures was randomized over a time interval of 5 min. Following the evaluation of the three samples with respect to emotional response, participants were given a 5-min break prior to a sensory testing session. During each break, spring water (Clear Mountain Spring Water, Taylor Distributing, Heber Springs, AR, United States) and unsalted crackers (Nabisco Premium Unsalted Tops Saltine Crackers, Mondelez Global LLC, East Hanover, NJ, United States) were provided as palate cleansers.

Prior to the main sensory testing session, participants were asked to taste and evaluate with respect to sensory attribute a warm-up sample of brewed coffee. They were asked to select all sensory terms listed on the sensory CATA question for characterizing sensory attributes of the sample. Participants were then asked to taste and evaluate coffee samples at the same three temperatures used in the emotion testing session. Participants were also asked to provide their overall liking of each sample on a 9-point hedonic scale ranging from 1 (dislike extremely) to 9 (like extremely).

#### Statistical Analysis

Data were analyzed using XLSTAT statistical software (Addinsoft, New York, NY, United States) and SPSS 24.0 for WindowsTM (IBM SPSS Inc., Chicago, IL, United States). As previously proposed by Meyners et al. (2013) for an overall test of CATA data, chi-square testing was performed to determine whether the proportion of selections by participants for all terms of either the emotion CATA question or the sensory CATA question differed as a function of sample temperature or gender. To measure an effect size (or strength of association between two nominal variables) for chi-square test (or contingency table), Cramér's V value was used. Cramér's V values, ranging from 0 (no association between the variables) to 1 (perfect association), of 0.1, 0.3, and 0.5 were considered small, medium, and large effect-sizes, respectively (Cohen, 1988; Kittler et al., 2007).

Cochran's Q-test (Cochran, 1950), using the exact probability and distribution of the Q statistic (Patil, 1975), was also performed to determine whether the proportions of selection by participants for individual terms of either the emotion CATA question or the sensory CATA question differed by sample temperature or gender. If significant differences were found among the variables, post hoc multiple pairwise comparisons were performed using the Marascuilo procedure (Marascuilo and McSweeney, 1967). Correspondence analysis, based on chisquare distance, was used to visualize relationships of sample temperatures to emotional responses and sensory attributes. Significant terms of the CATA questions, as determined by the Cochran's Q-test, were used for correspondence analysis.

A three-way analysis of variance (ANOVA) was performed treating "sample temperature" and "gender" as main effects and "participant" as a random effect. If a significant difference in means was indicated by the ANOVA, post hoc comparisons between independent variables were performed using Tukey's honest significant difference method. To measure an effect size for ANOVA, a partial eta squared (η 2 p ) value was used; the η 2 p values of 0.01, 0.06, and 0.14 are considered small, medium, and large effect-sizes, respectively (Kittler et al., 2007; Velasco et al., 2014). Penalty-lift analysis (Williams et al., 2011; Meyners et al., 2013) was also conducted to identify positive and negative drivers of overall liking among emotion and sensory attribute terms of coffee samples. Mean differences in overall liking between the selected and unselected cases for individual emotions and sensory attributes were then determined. A positive (or negative) value for a particular attribute indicates the mean liking of participants who selected that attribute was greater than the mean liking of those who did not (Meyners et al., 2013). A statistically significant difference was defined to exist when P < 0.05.

#### Results

#### Overall Effects of Sample Temperatures on Emotional Responses and Sensory Attributes

To determine whether the proportions of selection by participants for all terms of either the emotion CATA question

TABLE 1 | A contingency table of the proportions of selection by 79 participants for individual emotion terms among coffee samples evaluated at the three different temperatures.


Cochran's Q-test (Cochran, 1950), using the exact probability and distribution of the Q statistic (Patil, 1975), was performed to determine whether the proportions of participant selection for individual terms of emotion check-all-that-apply (CATA) question could differ by sample temperature. The proportions with different letters within each row represent a significant difference determined by post hoc multiple pairwise comparisons using the Marascuilo procedure (Marascuilo and McSweeney, 1967). Cramér's V value was used to measure strength of association between two nominal variables (sample temperature × selected/unselected case) for the contingency table. Cramér's V values, ranging from 0 (no association between the variables) to 1 (perfect association), of 0.1, 0.3, and 0.5 were considered small, medium, and large associations, respectively (Cohen, 1988). <sup>1</sup>Only significant terms, determined by Cochran's Q-test, among 39 emotion terms of the EsSense Profile <sup>R</sup> (King and Meiselman, 2010) were shown (P < 0.05).

or the sensory CATA question differed as a function of sample temperature, the data were collapsed into the three temperature conditions: 5, 25, and 65◦C. Chi-square testing revealed that the proportions of selection by participants for all emotion terms significantly differed among the three temperatures evaluated in this study (χ <sup>2</sup> = 65.24, P < 0.001, V = 0.08): 5◦C (12.3%), 25◦C (12.7%), and 65◦C (18.8%). More specifically, participants selected greater numbers of emotion terms when they evaluated coffee samples at 65◦C than at 5 or 25◦C, but the effect size (Cramér's V value) was low. In addition, the selection proportions for all sensory terms were not significantly different among the three temperature conditions (P = 0.91): 5◦C (18.6%), 25◦C (18.4%), and 65◦C (18.8%).

**Table 1** is a contingency table showing the proportions of selection by participants for individual emotion terms of coffee samples served at 5, 25, and 65◦C. A higher proportion, i.e., closer to 1.00, indicates that the term was more frequently chosen by participants. Cochran's Q-test revealed that 16 emotion terms of coffee samples significantly differed as a function of sample temperature: "active," "bored," "calm," "disgusted," "eager," "energetic," "glad," "good," "happy," "nostalgic," "peaceful," "pleasant," "pleased," "satisfied," "warm," and "wild." In addition, **Table 2** is a contingency table showing the proportions of selection for individual sensory-attribute terms of coffee samples served at the three temperatures. Cochran's Q-test revealed that six sensory attributes of coffee samples significantly differed with respect to sample temperature: "pungent aroma," "roasted aroma," "metallic flavor," "roasted flavor," "skunky flavor," and "bitter taste."

A bi-plot of correspondence analysis (**Figure 1**), drawn by the above 16 emotional responses and six sensory attributes, visualizes associations of sample temperatures with emotional responses and sensory attributes. More specifically, a coffee sample tasted and evaluated at 65◦C was characterized more with emotion terms, "happy," "pleased," "satisfied," "warm," as well as sensory term "roasted flavor." A coffee sample evaluated at 25◦C was characterized more by emotion terms "bored" and "wild," and by sensory attribute terms "roasted aroma" and "bitter taste." Finally, a coffee sample consumed at 5◦C was characterized more with not only sensory attribute terms "pungent aroma," "metallic flavor," and "skunky flavor," but also by emotion terms of "active," "disgusted," and "energetic." These results support the research propositions that specific sensory attributes (Research proposition 1) or emotional responses (Research proposition 2) can be variously dominant at hot, ambient, or cold temperature of coffee samples.

#### Gender Comparison with Respect to the Effects of Sample Temperatures on Emotional Responses and Sensory Attributes

To determine whether the proportions of participant selection for all terms of either the emotion CATA question or the sensory CATA question differed as a function of gender, the data were collapsed into two groups: females and males. Chi-square testing revealed that the proportions of selection by participants for all emotion terms were not significantly different between female (14.4%) and male (14.9%) participants (P = 0.51). In addition, the proportions of selection for all sensory terms were not


TABLE 2 | A contingency table of the proportions of selection by 79 participants for individual sensory attribute terms among coffee samples evaluated at the three different temperatures.

Cochran's Q-test (Cochran, 1950), using the exact probability and distribution of the Q statistic (Patil, 1975), was performed to determine whether the proportions of participant selection for individual terms of sensory check-all-that-apply (CATA) question could differ by sample temperature. The proportions with different letters within each row represent a significant difference determined by post hoc multiple pairwise comparisons using the Marascuilo procedure (Marascuilo and McSweeney, 1967). Cramér's V value was used to measure strength of association between two nominal variables (sample temperature × selected/unselected case) for the contingency table. Cramér's V values, ranging from 0 (no association between the variables) to 1 (perfect association), of 0.1, 0.3, and 0.5 were considered small, medium, and large associations, respectively (Cohen, 1988). <sup>1</sup>Only significant terms, determined by Cochran's Q-test, among 49 sensory attribute terms were shown (P < 0.05).

significantly different between female (18.9%) and male (18.1%) participants (P = 0.28).

Cochran's Q-test revealed that sample temperatures significantly affected six emotional responses ("disgusted," "happy," "pleased," "satisfied," "warm," and "wild") and one sensory attribute ("roasted flavor") of brewed coffee samples from both female and male participants. However, the effects of sample temperatures on emotional responses to, and sensory attributes of, coffee samples were found to be different for 10 emotions and six sensory attributes. More specifically, for female participants, but not male participants, sample temperatures were found to affect seven emotional responses ("active," "bored," "calm," "glad," "good," "mild," and "peaceful") and three sensory attributes ("skunky aroma," "skunky flavor," and "bitter taste") of coffee samples. In contrast, for male participants, but not female participants, sample temperatures were found to influence three emotional responses ("nostalgic," "pleasant," and "worried") and three sensory attributes ("burnt aroma," "sour taste," and "viscous") of coffee samples. These results support the research proposition that the effects of sample temperatures on sensory attributes and emotional responses vary with gender (Research proposition 3).

#### Impacts of Emotional Responses and Sensory Attributes on Liking of Coffee Samples as a Function of Sample Temperature and Gender

A three-way ANOVA, treating "sample temperature" and "gender" as main effects and "participant" as a random effect, revealed that participants liked coffee samples evaluated at 65◦C (mean ± SD = 6.0 ± 1.9) more than those evaluated at 25◦C (4.2 ± 2.0) or 5◦C (4.0 ± 2.3) (P < 0.001, η 2 <sup>p</sup> = 0.30). However, there was neither a significant effect related to gender (P = 0.83), nor interaction between sample temperature and gender (P = 0.70).

Penalty-lift analysis identified drivers of liking with respect to emotional responses and sensory attributes at three different coffee sample temperatures. Overall, when considering all coffee samples tasted at three different temperatures, "pleased," "satisfied," "pleasant," "warm," "calm," and "energetic" emotions, as well as "roasted flavor" attribute were identified as positive drivers of liking, while "disgusted" emotion, "bitter taste," and "metallic flavor" attributes were determined as negative drivers of liking (**Figure 2A**).

When coffee samples were consumed and evaluated at 65◦C, "pleasant," "pleased," and "satisfied" emotions as well as "roasted flavor" attribute were identified as positive drivers of liking, while a "sour taste" attribute was determined as a negative driver of

liking (**Figure 2B**). In addition, not only "calm" emotion, but also "sweet aroma," "oily," "roasted flavor," and "astringent mouthfeel" attributes were determined as positive drivers of liking, while both the "disgusted" emotion and the "pungent flavor" attribute were identified as negative drivers of liking for coffee sample evaluated at 25◦C (**Figure 2C**). Finally, when coffee samples were evaluated at 5◦C, "interested," "active," and "energetic" emotions, as well as the "roasted flavor" attribute were identified as positive drivers of liking, while both the "disgusted" emotion and the "burnt flavor" attribute were determined as negative drivers (**Figure 2D**). These results support the research proposition that the impact of sensory attributes and emotional responses on liking of coffee samples varies as a function of sample temperature (Research proposition 4a).

Gender was found to differ with respect to positive and negative drivers of liking for coffee samples tasted at three different temperatures. For female participants, not only "satisfied," "pleased," "calm," "good," "happy" emotions, but also the "roasted flavor" attribute was identified as positive drivers of liking, while the "disgusted" emotion and the "bitter taste" attribute were determined as negative drivers of liking (**Figure 3A**). For male participants, there were only emotionrelated drivers of liking, i.e., "pleased," "warm," and "satisfied" emotions as positive drivers and the "disgusted" emotion as a negative driver (**Figure 3B**). These results support the research proposition that the impacts of sensory attributes and emotional responses on liking of coffee samples vary as a function of gender (Research proposition 4b).

# STUDY 2: EFFECTS OF SAMPLE TEMPERATURES ON EMOTIONAL RESPONSES, AND SENSORY ATTRIBUTES OF, GREEN TEA

## Materials and Methods Participants

Seventy-eight green tea consumers (55 females and 23 males) ranging in age from 18 to 80 years (mean ± SD = 41 ± 17) were recruited. Through a pre-screening survey, all participants self-reported that they weekly drink one or more cups of green tea without any condiments and like green tea, i.e., higher than 5-point on a 9-point hedonic scale ranging from 1 (dislike extremely) to 9 (like extremely). In addition, participants selfrated that they like both hot beverages (mean ± SD = 7.9 ± 1.1) on a 9-point hedonic scale ranging from 1 (dislike extremely) to 9 (like extremely) and cold beverages (mean ± SD = 8.2 ± 0.9). All participants were asked to refrain from eating, drinking (except water), and cigarette smoking for 2 h prior to their participation.

#### Sample Preparation and Presentation

selected and unselected cases.

For green tea samples, green tea bags (Korean Organic Green Tea, Nokchawon Co. Ltd., Seoul, Korea) were steeped with boiled water in a proportion of two bags per 200-mL of spring water for 3 min. For a warm-up sample, another green tea product (Sun Nokcha, Haioreum, Lyndhurst, NJ, United States) was steeped in the same manner. After steeping, the green tea was poured into a 3,000-mL stainless steel dispenser (Bunn, Springfield, IL, United States) to maintain its high temperature. Green tea samples were randomly presented at three different temperatures: 65, 25, and 5◦C in a monadic sequential fashion. As for the coffee samples, sample preparation to temperatures of 25 and 5 ◦C required green tea samples to be placed in a water bath to facilitate the cooling process. Each green tea sample (55-mL) was presented in a 118-mL white Styrofoam cup identified with a three-digit code.

#### CATA Questions of Emotion and Sensory Tests for Green Tea

An emotion CATA question, including 39 emotion terms of the EsSense Profile <sup>R</sup> (King and Meiselman, 2010), was used for measuring emotional responses evoked by drinking green tea samples presented at the three serving temperatures. In addition, a sensory CATA question of green tea included 57 sensory attribute terms, based not only on previous research regarding descriptive sensory analyses of green tea (Ikeda et al., 2004; Ito and Kubota, 2005; Lee and Chambers, 2007; Lee et al., 2008; Castiglioni et al., 2015), but also on descriptions by consumer and descriptive panelists. The following attributes were included: 21 aroma attributes (animalic, ashy, beany, bitter/tannic, burnt, chemical, citrus, earthy/dirty, fermented, floral, fruity, grainy, grassy/cut grass, hay-like, herbal/herb-like, long lasting, metallic, mild/mellow, nutty, roasted, and pungent); six appearance attributes (brown color, clear, green color, sediment, turbid, and yellow color); 23 taste/flavor attributes (animalic, ashy, beany, burnt, chemical, citrus, earthy/dirty, fermented, floral, fruity, grainy, grassy/cut grass, hay-like, herbal/herb-like, long lasting, mild/mellow, nutty, roasted, pungent, bitter taste, salty taste, sour taste, and sweet taste); five mouthfeel attributes (astringent, metallic, mouth coating, smooth, and viscous); and two aftertaste attributes (bitter aftertaste and sour aftertaste).

#### Procedure

Both emotion and sensory tests of green tea samples were conducted in the same manner as described in Study 1 of coffee samples.

#### Statistical Analysis

Data was analyzed in the same manner as described in Study 1 of coffee samples.

#### Results

#### Overall Effects of Sample Temperatures on Emotional Responses and Sensory Attributes

To determine whether the proportions of selection by participants for all terms of either the emotion CATA question or the sensory CATA question differed as a function of sample temperature, the data were collapsed into the three temperature conditions: 5, 25, and 65◦C. Chi-square testing revealed that the proportions of selection by participants for all emotion terms significantly differed among green tea samples evaluated at the three temperatures (χ <sup>2</sup> = 27.81, P < 0.001, V = 0.06): 5◦C (12.8%), 25◦C (12.7%), and 65◦C (16.8%). More specifically, participants selected a greater number of emotion terms when evaluating green tea at 65◦C than at 5 or 25◦C, but the effect size (Cramér's V value) was low. In addition, the proportions of selection for all sensory terms did not significantly differ among green tea samples evaluated at the three temperatures (P = 0.90): 5 ◦C (17.1%), 25◦C (17.1%), and 65◦C (17.4%).

**Table 3** is a contingency table showing the proportions of participant selection for individual emotion terms of green tea samples tasted and evaluated at 5, 25, and 65◦C. Cochran's

TABLE 3 | A contingency table of the proportions of selection by 78 participants for individual emotion terms among green tea samples evaluated at the three different temperatures.


Cochran's Q-test (Cochran, 1950), using the exact probability and distribution of the Q statistic (Patil, 1975), was performed to determine whether the proportions of participant selection for individual terms of emotion check-all-that-apply (CATA) question could differ by sample temperature. The proportions with different letters within each row represent a significant difference determined by post hoc multiple pairwise comparisons using the Marascuilo procedure (Marascuilo and McSweeney, 1967). Cramér's V value was used to measure strength of association between two nominal variables (sample temperature × selected/unselected case) for the contingency table. Cramér's V values, ranging from 0 (no association between the variables) to 1 (perfect association), of 0.1, 0.3, and 0.5 were considered small, medium, and large associations, respectively (Cohen, 1988). <sup>1</sup>Only significant terms, determined by Cochran's Q-test, among 39 emotion terms of the EsSense Profile <sup>R</sup> (King and Meiselman, 2010) were shown (P < 0.05).

Q-test revealed that 19 emotion terms of green tea samples significantly differed as a function of sample temperature: "active," "adventurous," "affectionate," "bored," "calm," "darling," "disgusted," "energetic," "good," "joyful," "loving," "nostalgic," "peaceful," "pleasant," "polite," "satisfied," "secure," "warm," and "wild." In addition, **Table 4** is a contingency table that shows the proportions of selection for individual sensory terms of green tea samples evaluated at the three temperatures. Cochran's Q-test revealed that 18 sensory attributes of green tea samples significantly differed with respect to sample temperature: "animalic aroma," "floral aroma," "herbal/herb-like aroma," "roasted aroma," "pungent aroma," "brown color," "green color," "yellow color," "mild/mellow flavor," "nutty flavor," "roasted flavor," "pungent flavor," "bitter taste," "sour taste," "sweet taste," "astringent mouthfeel," "smooth mouthfeel," and "bitter aftertaste."

A bi-plot of correspondence analysis (**Figure 4**), drawn by the above 19 emotional responses and 18 sensory attributes, visualizes associations of sample temperatures with emotional responses and sensory attributes. More specifically, green tea sample tasted and evaluated at 65◦C was more characterized with emotion terms "affectionate," "calm," "good," "loving," "nostalgic," "peaceful," "pleasant," "satisfied," "secure," and "warm," as well as sensory terms "floral aroma," "herbal/herb-like aroma," "roasted aroma," "brown color," "mild/mellow flavor," "roasted flavor," and "sweet taste." Green tea samples evaluated at 25◦C were more characterized by emotion terms "bored," "disgusted," and "polite" and the sensory term "bitter taste." Finally, green tea samples evaluated at 5◦C were characterized by emotion terms "active," "adventurous," "energetic," "joyful," and "wild," as well as sensory terms "animalic aroma," "pungent aroma," "green color," "pungent flavor," "sour taste," "astringent mouthfeel," and "bitter aftertaste." These results support the research propositions that certain sensory attributes (Research proposition 1) or emotional responses (Research proposition 2) can be more dominant at hot, ambient, or cold temperature of green tea samples.

#### Gender Comparison with Respect to the Effects of Sample Temperatures on Emotional Responses and Sensory Attributes

To determine whether the proportions of participant selection for all terms of either the emotion CATA question or the sensory CATA question differed as a function of gender, the data were collapsed into two groups: females and males. Chi-square testing revealed that the proportions of selection by participants for all emotion terms were not significantly different between female (13.7%) and male (15.1%) participants (P = 0.07). The proportions of selection for all sensory terms were also not significantly different between female (16.8%) and male (18.0%) participants (P = 0.12).

TABLE 4 | A contingency table of the proportions of selection by 78 participants for individual sensory attribute terms among green tea samples evaluated at the three different temperatures.


Cochran's Q-test (Cochran, 1950), using the exact probability and distribution of the Q statistic (Patil, 1975), was performed to determine whether the proportions of participant selection for individual terms of sensory check-all-that-apply (CATA) question could differ by sample temperature. The proportions with different letters within each row represent a significant difference determined by post hoc multiple pairwise comparisons using the Marascuilo procedure (Marascuilo and McSweeney, 1967). Cramér's V value was used to measure strength of association between two nominal variables (sample temperature × selected/unselected case) for the contingency table. Cramér's V values, ranging from 0 (no association between the variables) to 1 (perfect association), of 0.1, 0.3, and 0.5 were considered small, medium, and large associations, respectively (Cohen, 1988). <sup>1</sup>Only significant terms, determined by Cochran's Q-test, among 57 sensory attribute terms were shown (P < 0.05).

Cochran's Q-test revealed that sample temperatures significantly affected two emotional responses ("disgusted" and "warm") and five sensory attributes ("animalic aroma," "roasted aroma," "brown color," "green color," and "yellow color") of green tea samples for both female and male participants. The effects of sample temperatures on emotional attributes and sensory attributes of green tea samples were found to differ with gender for 17 emotions and eight sensory attributes. More specifically, for female participants, but not male participants, sample temperatures were found to affect 16 emotional responses ("active," "adventurous," "affectionate," "bored," "calm," "daring," "energetic," "good," "loving," "nostalgic," "peaceful," "pleasant," "satisfied," "secure," "tame," and "whole") and seven sensory attributes ("herbal/herb-like aroma," "beany flavor," "mild/mellow flavor," "roasted flavor," "bitter taste," "smooth mouthfeel," and "bitter aftertaste") of green tea samples. In contrast, for male participants, but not female participants, sample temperatures were found to influence one emotional response ("understanding") and one sensory attribute ("sweet taste") for green tea samples. These results support the research proposition that the effects of sample temperatures on sensory attributes and emotional responses vary with gender (Research proposition 3).

#### Impacts of Emotional Responses and Sensory Attributes on Liking of Green Tea as a Function of Sample Temperature and Gender

A three-way ANOVA, treating "sample temperature" and "gender" as main effects and "participant" as a random effect, revealed that hedonic ratings of green tea samples differed significantly with respect to sample temperature (P < 0.001, η 2 <sup>p</sup> <sup>=</sup> 0.22): at 65◦C (mean <sup>±</sup> SD <sup>=</sup> 6.3 <sup>±</sup> 1.7) <sup>&</sup>gt; at 25◦<sup>C</sup> (5.3 ± 1.8) > at 5◦C (4.6 ± 2.1). However, there were no significant effects of gender (P = 0.33), or interaction between sample temperature and gender (P = 0.28).

Penalty-lift analysis identified drivers of liking with respect to emotional responses and sensory attributes at three different temperatures of green tea sample. Overall, when considering all green tea samples experienced at three different temperatures, "warm," "satisfied," "good," "peaceful," "pleasant," and "calm" emotions, as well as "mild/mellow flavor," "smooth mouthfeel," and "brown color" attributes were identified as positive drivers of liking. Additionally, "bitter aftertaste," "bitter taste," "astringent

mouthfeel," and "yellow color" attributes were identified as negative drivers of liking for green tea samples evaluated at different temperatures (**Figure 5A**).

Positive and negative drivers of liking with respect to emotional responses and sensory attributes were found at three different temperatures of green tea sample. When green tea samples were consumed and evaluated at 65◦C, not only "good" and "warm" emotions, but also "smooth mouthfeel," "roasted aroma," and "herbal/herb-like flavor" attributes were identified as positive drivers of liking, while the "bitter taste" attribute was determined as a negative driver of liking (**Figure 5B**). When green tea samples were evaluated at 25◦C, only sensory attributes were identified as positive and negative drivers of liking: i.e., "mild/mellow flavor" and "smooth mouthfeel" attributes as positive drivers and "bitter taste" and "bitter aftertaste" as negative drivers of liking (**Figure 5C**). In addition, when green tea samples were evaluated at 5◦C, not only the "free" emotion, but also "earthy aroma," "smooth mouthfeel," "herbal/herb-like flavor," "grass/cut grass aroma," and "mouth coating" attributes were identified as positive drivers of liking, while the "astringent mouthfeel" attribute was determined as a negative driver of liking (**Figure 5D**). These results support the research proposition that the impacts of sensory attributes and emotional responses on liking of green tea samples vary as a function of sample temperature (Research proposition 4a).

Positive and negative drivers of liking for green tea samples tasted at three different temperatures were found to differ between female and male participants. For female participants, "pleasant," "peaceful," "warm," "satisfied," "calm," and "good" emotions as well as "smooth mouthfeel," "mild/mellow flavor," and "brown color" attributes were identified as positive drivers of liking. In addition, four sensory attributes, i.e., "bitter taste," "bitter aftertaste," "astringent mouthfeel," and "yellow color," were determined as negative drivers of liking (**Figure 6A**). For male participants, only the "warm" emotion was identified as a positive driver of liking for green tea samples evaluated at different temperatures (**Figure 6B**). These results support the research proposition that the impacts of sensory attributes and emotional responses on liking of green tea samples vary as a function of gender (Research proposition 4b).

# DISCUSSION

# Variations with Respect to Emotional Responses and Sensory Attributes as a Function of Sample Temperature of Coffee and Green Tea (Research Propositions 1 and 2)

In most sensory studies of hot foods or beverages, samples have been evaluated in a temperature range at which those samples are typically consumed. For example, brewed coffee has been evaluated at one specific temperature between 75 and 55◦C (Nebesny and Budryn, 2006; Seo et al., 2009a,b; Bhumiratana et al., 2014; Di Donfrancesco et al., 2014). However, it is wellknown that people consume hot foods or beverages over a wider range of temperature in everyday life, and certain beverages such as coffee and tea are often consumed at both hot and cold temperatures. Nevertheless, little attention has been paid to whether and how emotional responses as well as sensory attributes of hot foods or beverages can change as a function of their product temperatures.

The results from this study showed the dynamics of sensory attributes in both coffee and green tea samples with respect to serving temperatures of 65, 25, and 5◦C. Six and 18 sensory attributes of coffee and green tea samples, respectively, significantly differed in terms of sample temperature. Sample temperature-induced changes in sensory attributes of brewed coffee have been also observed in other studies (Stokes et al., 2016; Steen et al., 2017). In a recent study conducted by Stokes et al. (2016), "coffee flavor," "roasted/burnt flavor," and "full body" attributes were more associated with brewed coffee samples evaluated at higher temperatures of 60.4, 70.8, and 74.4◦C, while "earthy flavor" and "sour/acidic taste" were more related to those evaluated at lower temperatures of 31.0 and 41.1◦C. The present study to some extent showed similar results, that "roasted flavor" attribute of brewed coffee was more often identified at higher temperature (65◦C), while "pungent aroma," "metallic flavor," and "skunky flavor" attributes were more often characterized at lower temperature (5◦C). To the authors' best knowledge, this study was the first to demonstrate that sensory attributes of green tea can vary with sample temperatures. Notably, green tea samples showed a greater number of significant sensory attributes affected by sample temperatures than did coffee samples,

FIGURE 5 | Mean drops in overall liking with respect to emotional responses and sensory attributes in green tea samples as a function of sample temperature: all temperatures (A), 65◦C (B), 25◦C (C), and 5◦C (D). "A," "F," "M," and "AT" next to sensory attribute term represent "aroma," "flavor," "mouthfeel," and "aftertaste," respectively. Numerical value of each emotion or sensory attribute term represents a mean difference in overall liking between the selected and unselected cases; a positive (or negative) value for each term indicates an increase (or decrease) of overall liking between the selected and unselected cases.

indicating that sensory attributes of green tea samples were more sensitive to temperature changes than those of coffee samples. Like coffee samples, green tea samples were more frequently characterized using desirable sensory attributes at higher temperature (65◦C), while those at lower temperatures (5 and 25◦C) were more often described using undesirable attributes.

Both coffee and green tea samples evaluated at higher temperature (65◦C) were more often characterized using emotions of positive valence with either high or low level of activation/arousal. In other words, beverage samples consumed at 65◦C more frequently evoked emotions of positive valence, such as "pleased," "happy," "satisfied," and "warm," etc., than did those consumed at either 25 or 5◦C. In addition,

while beverage samples consumed and evaluated at 25◦C were characterized with emotions of negative valence with low level of activation/arousal, those evaluated at 5◦C were described with emotions of negative valence with high level of activation/arousal. Sample temperature-induced variation with respect to emotional responses might be associated with the dynamics of sensory attributes among the three temperature conditions of coffee or green tea samples. More specifically, the tetrachoric correlation analysis (Divgi, 1979) of sensory attributes and emotional responses for coffee samples revealed that "roasted flavor" and "mouth-coating" attributes showed positive correlation with emotions of positive valence, such as "warm" (roasted flavor: +0.54, mouth-coating: +0.34), "pleasant" (+0.49, +0.18), "satisfied" (+0.37, +0.20), "pleased" (+0.32, +0.22), and "happy" (+0.22, +0.13). In addition, "pungent aroma," "chemical flavor," "metallic flavor," and "bitter taste" attributes positively correlated with emotions of high level of activation/arousal, such as "active" (pungent aroma: +0.08, chemical flavor: +0.18, metallic flavor: +0.17, and bitter taste: +0.22), "eager" (+0.14, +0.08, +0.21, and +0.09), and "energetic" (+0.28, +0.22, +0.18, and +0.15). These results were in agreement with previous studies that showed associations between sensory attributes and emotional responses (Seo et al., 2009c; Porcherot et al., 2010; Chaya et al., 2015). Furthermore, the tendency of warmer food or beverage products to evoke positive emotions illustrated the "temperature-premium effect," where exposure to warm temperatures can increase a consumer's evaluation of a product through the activation of the concept of positive emotional warmth in an individual, leading to greater positive reactions (Zwebner et al., 2013). Such an affective response of thermal stimuli could be explained by the increased neural activations in the brain regions associated with thermal sensation, sensory discrimination, emotional awareness and processing, and cognitive processing during direct exposure to warm stimulation (Sung et al., 2007).

# Gender Effects on the Sample Temperature-Induced Variations with Respect to Emotional Responses to, and Sensory Attributes of, Coffee and Green Tea (Research Proposition 3)

Influences of sample temperatures on emotional responses and sensory attributes were observed in both female and male participants. Even though female and male participants did not exhibit any significant differences in terms of the proportions of selection by participants for either all emotion terms or all sensory terms, the results for female participants showed a larger number of emotion and sensory terms that significantly varied as a function of sample temperature compared to those of male participants. In other words, female participants displayed more consensus and less variable responses toward coffee and green tea samples presented at three different temperatures. This result might be related to earlier findings that female participants outperformed male participants with respect to odor sensitivity, odor identification, odor memory, and verbal proficiency (Doty et al., 1984; Larsson et al., 2003; Doty and Cameron, 2009; Ferdenzi and Roberts, 2013) although gender differences were not always observed. Moreover, a functional neuroimaging study conducted by Royet et al. (2003) found that while males showed neural activations in their bilateral insula and left piriform–amygdala regions during hedonic judgment of odors, females showed neural activations not only in the same regions as male participants, but also in left orbitofrontal cortex related to odor identification, language, and emotion. Females have also been found to be more emotionally expressive toward foods and beverages than males (King et al., 2010; Jaeger and Hedderley, 2013).

# Impacts of Emotional Responses and Sensory Attributes on Likings of Coffee and Green Tea (Research Proposition 4)

Our findings support previous research suggesting that not only sensory attributes, but also emotional responses to some extent contribute to overall liking of foods and beverages (Seo et al., 2009c; Samant et al., 2017). Interestingly, drivers of liking with respect to emotional responses and sensory attributes were found to differ as a function of sample temperature in both coffee and green tea samples. For coffee samples, only the "roasted flavor" attribute was observed as a positive driver of liking at all three temperatures, while positive and negative drivers of liking changed at each temperature. "Roasted flavor" was more often identified at higher temperature (65◦C), possibly suggesting that participants increasingly like brewed coffee served at 65◦C the most. For green tea, the "smooth mouthfeel" attribute served as a positive driver of liking at all three temperatures. Like coffee samples, positive and negative drivers of liking for green tea samples varied with sample temperatures. Overall, among 57 sensory attributes of green tea samples, "mild/mellow flavor," "smooth mouthfeel," "brown color," and "herbal/herb-like aroma" were found to be positive drivers of liking, while "bitter aftertaste," "bitter taste," "astringent mouthfeel," and "yellow color" were negative drivers of liking. This result was in agreement with previous research where the United States consumers liked green tea samples with "mild flavor," "no aftertaste," "weak bitterness," "flowery or fruity flavor," and "brown flavor" notes (Lee et al., 2010). In addition, "sweet taste" and "roasted-related flavors" were considered to be drivers of liking for green tea samples (Lee et al., 2008). Building on previous research regarding sensory drivers of liking for green tea samples, this study added empirical evidence that emotions also serve as drivers of liking for green tea samples. Specifically, "warm," "satisfied," "good," "peaceful," "pleasant," and "calm" emotions were found to play important roles in modulating liking of green tea samples served at different temperatures.

It is worth noting that drivers of liking for coffee or green tea samples were found to differ between female and male participants. While both emotional responses and sensory attributes contributed to likings of beverage samples among

female participants, only emotional responses were considered as drivers of liking among male participants. This result might be related to previous findings that females outperformed males in odor sensitivity, odor identification, and odor memory tasks (Doty et al., 1984; Larsson et al., 2003; Doty and Cameron, 2009; Ferdenzi and Roberts, 2013). Females have also been found to perform better in taste sensitivity tasks than males (Michon et al., 2009). Since females could better detect sample temperature-induced changes in sensory attributes than males, sensory attributes might contribute to likings of coffee and green tea samples among female participants, but not among male participants.

# CONCLUSION

To summarize, the results of this study showed that both emotional responses to, and sensory attributes of, coffee or green tea samples can vary with sample temperature. In other words, people may experience different sensory attributes and emotions with decreasing temperature of brewed coffee or green tea beverages, affecting their likings of those beverages. In addition, sample temperature-induced variations with respect to emotional responses and sensory attributes differed between female and male participants. Furthermore, while sensory attributes as well as emotional responses were found to be drivers of liking among female participants, only emotional responses were identified as drivers of liking among male participants. In conclusion, our findings provide empirical

## REFERENCES


evidence that emotional responses to, and sensory attributes of, coffee and green tea beverages can vary as a function of sample temperature, and that such temperature-induced variations can differ by gender. Our findings emphasize the need to consider product temperature-induced dynamics of emotional responses and sensory attributes when evaluating food or beverage products that are temperature-sensitive. In other words, processors, manufacturers, sensory professionals, and marketers in the food industry should put more effort into exploring emotional responses to, and sensory attributes of, food or beverage products over the wider range of product temperatures that consumers may encounter in daily life. Such efforts may lead to both a better understanding of product characteristics and increases in consumer acceptance and purchase intent.

# AUTHOR CONTRIBUTIONS

RP and H-SS conceived and designed the study. RP collected the data, and RP and H-SS analyzed the data. RP and H-SS participated manuscript preparation and approved the final manuscript.

# FUNDING

This study was based upon work that is supported, in part, by the United States Department of Agriculture National Institute of Food and Agriculture Hatch Act funding to H-SS.




Zwebner, Y., Lee, L., and Goldenberg, J. (2013). The temperature premium: warm temperatures increase product valuation. J. Consumer Psychol. 24, 251–259. doi: 10.1016/j.jcps.2013.11.003

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Pramudya and Seo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# How Strong Is Your Coffee? The Influence of Visual Metaphors and Textual Claims on Consumers' Flavor Perception and Product Evaluation

#### Anna Fenko\*, Roxan de Vries and Thomas van Rompay

Department of Communication Science, Faculty of Behavioural, Management and Social Sciences, University of Twente, Enschede, Netherlands

This study investigates the relative impact of textual claims and visual metaphors displayed on the product's package on consumers' flavor experience and product evaluation. For consumers, strength is one of the most important sensory attributes of coffee. The 2 × 3 between-subjects experiment (N = 123) compared the effects of visual metaphor of strength (an image of a lion located either on top or on the bottom of the package of coffee beans) and the direct textual claim ("extra strong") on consumers' responses to coffee, including product expectation, flavor evaluation, strength perception and purchase intention. The results demonstrate that both the textual claim and the visual metaphor can be efficient in communicating the product attribute of strength. The presence of the image positively influenced consumers' product expectations before tasting. The textual claim increased the perception of strength of coffee and the purchase intention of the product. The location of the image also played an important role in flavor perception and purchase intention. The image located on the bottom of the package increased the perceived strength of coffee and purchase intention of the product compared to the image on top of the package. This result could be interpreted from the perspective of the grounded cognition theory, which suggests that a picture in the lower part of the package would automatically activate the "strong is heavy" metaphor. As heavy objects are usually associated with a position on the ground, this would explain why perceiving a visually heavy package would lead to the experience of a strong coffee. Further research is needed to better understand the relationships between a metaphorical image and its spatial position in food packaging design.

Keywords: packaging design, visual metaphor, grounded cognition, textual claim, flavor evaluation

# INTRODUCTION

Taste and flavor are the main factors people consider when buying food (Cardello, 1994; Steptoe et al., 1995; Lappalainen et al., 1998). However, in supermarkets and food stores consumers rarely have an opportunity to taste products before purchase. Therefore, when making a purchase decision people rely more on packaging cues and previous knowledge (Creusen and Schoormans, 2005; Schifferstein et al., 2013). Consumers form certain expectations about the taste and flavor

#### Edited by:

Carlos Velasco, BI Norwegian Business School, Norway

#### Reviewed by:

Felipe Reinoso Carvalho, KU Leuven, Belgium Franck Celhay, Montpellier Business School, France

> \*Correspondence: Anna Fenko a.fenko@utwente.nl

#### Specialty section:

This article was submitted to Eating Behavior, a section of the journal Frontiers in Psychology

Received: 31 October 2017 Accepted: 15 January 2018 Published: 05 February 2018

#### Citation:

Fenko A, de Vries R and van Rompay T (2018) How Strong Is Your Coffee? The Influence of Visual Metaphors and Textual Claims on Consumers' Flavor Perception and Product Evaluation. Front. Psychol. 9:53. doi: 10.3389/fpsyg.2018.00053

**37**

of food products by examining different attributes of packaging, such as its color, shape, material, text, typeface and images displayed on the packaging (Deliza et al., 2003; Jaeger, 2006). Expectations affect consumer judgments about food quality and its hedonic properties after tasting and are very robust against later disconfirmation (Cardello and Sawyer, 1992; Tuorila et al., 1994).

The framework of multisensory human–food interaction suggests that sensory attributes of packaging can influence not only the general food expectations, but can directly affect multisensory food experience, such as taste and flavor. Multiple studies within this framework have demonstrated that sensory packaging elements can communicate various attributes of food. For instance, hedonic and health benefits of food products can be successfully communicated by certain combinations of packaging material and color (Fenko et al., 2015b), packaging shape and the sound of product name (Fenko et al., 2016c). Taste intensity and evaluation have been shown to depend on packaging shape (Becker et al., 2011), material (Krishna and Morrin, 2008; Van Rompay et al., 2017) and color (Kauppinen-Räisänen, 2014; also see Spence, 2016 for the overview of the effects of packaging design).

We suggest that not only sensory elements of the package, such as color and material, can influence multisensory food experience, but also the informational elements such as text and images. Most food packaging designs comprise both textual elements (brand name, product category, nutrition information, attribute claims, etc.) and images. Some images are directly related to the contents of the product, depicting a product itself, its ingredients or a meal that can be prepared from it. The images can also be metaphorically associated with the brand and communicate certain brand values (like an image of Jolly Green Giant on a pack of canned vegetables or an image of Pillsbury Doughboy on a pack of baking powder). Textual claims and images communicate different product properties (e.g., a text can contain a health claim, while an image can communicate hedonic benefits) or the same product attribute (such as intensity, naturalness or freshness).

Metaphorical images are widely used in food packaging, but their effects on consumer responses have not been studied systematically. Furthermore, the relationships between metaphorical images and textual claims that can convey the same message remain unclear. The novelty of our study is determined by its focus on the relative effects of textual claims and metaphorical images on food package and their ability to change multisensory food experience, including flavor perception and product evaluation. The study aims to answer the following research question:

To what extent does a visual metaphor of strength compared to a textual claim of strong coffee depicted on a package of coffee beans influence consumers' product expectations, flavor perception and purchase intention?

#### Textual Claims

Textual claims about sensory and hedonic attributes of food, its ethnic origin, preparation process, various labels (e.g., organic, natural, or health labels) and nutritional information play an important role in consumers' perception of food and drink (ASAM and Bucklin, 1973; Dubé and Cantin, 2000; Spence and Piqueras-Fiszman, 2014; Piqueras-Fiszman and Spence, 2015). For instance, Wansink et al. (2004) served people a range of savory main courses that had been given either a basic name or a more evocative label (e.g., 'Seafood Filet' versus 'Succulent Italian Seafood Filet'). The use of descriptive food labels led to a doubling in the number of positive comments from participants, as well as the increase of hedonic ratings of food compared to basic food labels. Similarly, people rated M&Ms as having more intense chocolate taste when labeled as 'dark' rather than as 'milk' chocolate (Shankar et al., 2009).

Textual claims have different effects for specific product categories and specific consumer groups (Lähteenmäki, 2013). For instance, the hedonic label (as opposed to the health label) had a positive effect on consumer responses toward a hedonic product (a chocolate cookie) (Fenko et al., 2016b). The effect was opposite for a healthy product (apple juice), where the health label (as opposed to hedonic label) increased product evaluation and purchase intention. Another study (Fenko et al., 2015a) showed that familiarity and novelty claims can differently affect food neophobic and food neophilic consumers. For neophobic consumers, the familiarity claims positively influenced perceived product familiarity, while for the neophilic consumers, the novelty claims increased taste expectations and purchase intentions.

Most studies agree that textual claims can contribute to a higher perceived attractiveness, quality and purchase intention of products (Machiels and Karnal, 2016). Based on the results of previous findings, we propose the following hypothesis:

H1: A package of coffee beans with a textual claim indicating strong coffee will positively influence product expectations, flavor perception and purchase intention compared to a package without textual claim.

#### Visual Images

Imagery on a product's packaging can have a significant effect on people's sensory experiences (Bone and France, 2001; Jaeger and MacFie, 2001; Sakai et al., 2005; Cardello, 2007; Mizutani et al., 2010; Liang et al., 2013). For instance, Underwood and Klein (2002) reported that consumers rated products (bacon, margarine, and candy bars) as tasting better when presented with a picture rather than without a picture. Mizutani et al. (2010) have demonstrated that the type of image shown on a drink's container (whether it is pleasant or unpleasant, and related or unrelated to food) can influence consumers' perception of hedonic and sensory properties of the orange juice. Deliza et al. (2003) demonstrated that a picture on a product's packaging can influence consumers' sensory and hedonic expectations of orange juices. Participants were strongly influenced by the type of picture (either a drawing or a photo) of the product on the package.

In addition to food images, visual imagery may comprise nonfood depictions such as brand mascots and background imagery (i.e., a picture of a setting sun or a green field). These images are often used to create a product identity and to promote brand personality (Phillips, 1996). For instance, the Quaker Oats logo represents a figure of a Quaker man associated with the values of honesty, integrity and purity. The Nesquick's Quicky bunny evokes instant connotations with speed which implies that consumers can easily prepare the chocolate-flavored drink and quickly drink it. Cornelius Rooster by Kellogg's Cornflakes is a symbol of the early wakening and starting off the day with a healthy breakfast.

#### Visual Metaphors

fpsyg-09-00053 February 1, 2018 Time: 19:33 # 3

In many cases, usage of visual elements implies metaphor usage whereby the product or brand (target domain) is related to another domain (source domain; see Forceville, 2002). For example, the product attribute of strength (e.g., strength of coffee; target domain) can be metaphorically represented by an image of a strong animal, such as a lion ("strong as a lion"; source domain). In consumer research, usage of metaphors has been shown to enhance appreciation of product and brand (McQuarrie and Mick, 2003; Phillips and McQuarrie, 2009).

One reason for this is that metaphors present consumers with a 'puzzle' to be solved. For instance, when visualizing a lion's head on a coffee package to convey strength, consumers have to figure out how 'coffee' (the target domain) and a 'lion' (the source domain) are related. Arguably, solving this 'puzzle' is rewarding and hence may inspire positive evaluations (Heckler and Childers, 1992; Ortony, 1993; McQuarrie and Mick, 2003).

We suggest that an image of a strong animal (such as a lion) can serve as a relevant visual metaphor for a strength of coffee. Therefore, we propose the following hypothesis:

H2: A coffee package with an image of a lion as a metaphor for strength will positively influence consumers' product expectations, flavor perception and purchase intention compared to a package without an image of a lion.

Several studies have shown that explanatory information may enhance visual metaphor effects (e.g., Millis, 2001; Leder et al., 2006), especially when the metaphor is difficult to understand without additional information (Phillips, 2000; Van Rompay and Veltkamp, 2014). Similar idea that verbal texts could support and clarify the implicit meaning of images has been earlier proposed by Barthes (1977) who referred to this phenomenon as "verbal anchoring." We expect that the effects of metaphor usage will be stronger when accompanied by a textual claim accentuating the metaphor, and thus guiding participants in metaphor 'resolution.'

Hence, we further propose the following hypothesis:

H3: The positive effects of the metaphorical image of strong coffee on consumers' product expectations, flavor evaluation and purchase intention will be enhanced when accompanied by a textual claim guiding consumers in interpreting the metaphor.

#### Metaphors and Grounded Cognition

Grounded cognition theory argues that so called embodied metaphors represent abstract concepts such as exclusion, power, and intimacy in terms of 'image-schemas' (Lakoff and Johnson, 1980, 1999; Van Rompay and Ludden, 2015). For instance, the verticality schema as apparent from utterances such as 'looking up to someone' or 'looking down on others.' In such expressions, being powerful is linked to looking 'down' from a position up high, and being weak is associated with 'looking up' from a position down below (Van Rompay et al., 2005, 2012). Sundar and Noseworthy (2014) found similar effects for the position of brand logos, with a positioning up high generally associated with more powerful brands.

However, depending on the context, verticality may also inspire associations with lightness or heaviness. For instance, Van Rompay et al. (2014) showed that washing powder was experienced as more heavyweight (literally) when visuals were presented on the lower part of the package, a finding in line with earlier research on 'heavy' and 'light' locations in product packaging (e.g., Kahn and Deng, 2010). Furthermore, smell was experienced as lighter and fresher when imagery was presented up high. Jostmann et al. (2009) have demonstrated that people tend to equate heaviness with importance, an association also apparent in language use, for example, "a weighty issue" or "an issue not to be taken light-heartedly." Participants that held a heavy clipboard judged monetary value of a product higher that those who held a light clipboard.

Such findings concur with the skepticism people may feel when holding lightweight devices such as a mobile phone. A recent study (Ludden and Van Rompay, 2015) shows that lightweight mobile phones might lower price expectations. Similarly, Fenko et al. (2016a) have demonstrated symbolic congruence between the weight of beer bottles and brand values. Heavy bottles were perceived as representing excellence, authenticity and authority, while light bottles were associated with dynamic and accessible brands.

These studies illustrate the embodied associations between spatial location and sensory constructs related to weight and heaviness, which in the case of coffee might inspire perceptions of a strong taste and aroma. Based on this argumentation, we formulate the following hypothesis (H4):

H4: An image of a lion presented on the bottom of a coffee package will positively influence consumers' product expectations, flavor perception and purchase intention compared to an image of a lion presented on top of a coffee package.

## MATERIALS AND METHODS

To look at the relative impact of text and metaphorical images on consumer responses to coffee, we conducted the experiment with the 3 (image on top of the package vs. image on the bottom of the package vs. no image) × 2 (text claim vs. no text claim) between-subjects design. The field experiment took place at one of the Starbucks coffee houses in a medium-sized city in the Netherlands. Participants were exposed to one of the six packaging designs for a fictional brand of coffee beans and were offered a sample cup of coffee allegedly prepared from these beans. The coffee they tasted was a regular coffee from a wellknown Dutch brand prepared with a French press and served in a simple plastic cup. Before tasting coffee, participants were asked to look at the package and to evaluate their product expectations. After sampling coffee, participants were asked to evaluate the taste of coffee, its perceived intensity, its expected physiological effects and their purchase intention.

#### Participants

fpsyg-09-00053 February 1, 2018 Time: 19:33 # 4

To ensure that the experimental sample represented the target population of regular coffee consumers, all participants were recruited among the visitors of a local Starbucks. In total 131 people agreed to participate in the study. All participants were residents of the Netherlands, at least 18 years old, mean age 30 years, 68% were females. A few people who indicated that they did not drink coffee were excluded from the study, resulting in a final number of 123 participants. More than half of the participants (52%) were drinking at least one cup of coffee per day, 33% were drinking a few cups a week, and 10% were drinking just one or two cups a month, and 6% were drinking coffee less than once a year.

Participants characteristics per condition are presented in **Table 1**. A Chi-square test showed that there were no differences in gender distribution between the experimental conditions [χ 2 (5) = 5.29, p = 0.38]. A one-way ANOVA confirmed that there were no age differences between conditions [F(5,117) = 1.11, p = 0.36, non-sign].

#### Stimulus Material

In order to design stimuli for the main study, a focus group pre-study was conducted with professional coffee experts. The participants in the focus group were six baristas from Starbucks, both male and females, between 21 and 29 years old. Participants were asked to come up with a text claim that communicated the product attribute of strong coffee. All participants agreed that the text claim "Extra strong" was the best text claim to communicate the strength of coffee.

Participants were also asked to evaluate 10 different visual metaphors and to select the image most closely associated with the experience of "strong coffee". The stimuli contained the images of athletes, strong animals (a lion, a wolf, an elephant and a horse) and strong materials (a stone, a rope). The image that received the highest rating was the image of a lion.

Based on the results of the pre-study, the text claim 'Extra strong' ('extra sterk' in Dutch) and the visual metaphor of a lion were selected for the main study. The final packages for the experimental study are presented in **Figures 1**–**6**.

The final scales, the items and reliability coefficients are presented in **Table 2**. All items were measured on a 7-point Likert scale from "not at all" to "very." All scales demonstrated sufficient


FIGURE 1 | Image on top; the text claim.

FIGURE 2 | Image on the bottom; the text claim.

or high reliability (Cronbach's α above 0.70, according to Spector, 1991).

#### Measures

Recent studies have developed extensive lexicon to describe coffee drinking experience, including long list of emotions

(Bhumiratana et al., 2014), pleasantness, emotions and perceptions induced by coffee beverage (Labbe et al., 2015). In our study, we were focused on the more specific sensory experience of "strength" of coffee and hedonic evaluation of flavor. Therefore, we used focus group discussion to extract the most common description of coffee strength and taste evaluation.

Participants in the group discussion agreed that the strength of coffee represents one of its most important attributes. However, coffee experts pointed out that the strength of coffee could be experienced in two different ways: (1) as the intensity of taste and aroma; and (2) as the physiological effects of coffee, such as an increased arousal, energy and alertness. Therefore, two different

FIGURE 6 | No image, no text claim.

FIGURE 4 | Image on top; no text claim.

scales were constructed to measure these two aspects: Perceived Strength of coffee and Physiological Effects of coffee. When asked to describe physiological effects of coffee, most participants agreed that coffee makes them more awake, energetic, focused, alert and gives them a boost but also a relaxed feeling. This input was used to design a measurement scale for Physiological Effects of coffee. Coffee experts were also asked to describe the taste experience of "good" coffee, which resulted in a range of descriptive words, such as sweet, bitter, sour, nutty, smoky and spicy. The descriptions of good or bad coffee differed greatly per person. For instance, "bitter" and "nutty" flavors were evaluated positively by some experts and negatively by others. Therefore, in the final scale only simple descriptions such as "good" or "bad" were used for the Flavor Evaluation scale, and a separate scale was constructed for Intensity, or the Perceived Strength of coffee.

The final scales of Product Expectations, Flavor Evaluation of coffee, Perceived Strength of coffee, Physiological Effects of drinking coffee, and Purchase Intention, the items and reliability coefficients are presented in **Table 2**. All items were measured on point Likert scales from 1 ("Totally disagree") to 7 ("Totally agree"). All scales demonstrated sufficient or high reliability (Cronbach's α above 0.70, according to Spector, 1991).

#### Procedure

The experiment took place during four working days in April 2017 between 11 am and 5 pm in a Starbucks coffee house in one of the cities in the Netherlands. During the time of the experiment no changes in the environment took place, no additional promotional materials were placed, and the amount of customers remained relatively constant. Customers were approached while they were waiting to order their drinks and were asked to participate in an experiment commissioned by the University of Twente. Participants were guided through the experiment by written instructions. If any of the participants had questions, they were answered by the experimenter. The written instruction to participants contained the following note: "Important: The products that are used in this study have by no means a relation with Starbucks products. The brand Starbucks has no relation with this study."

The questionnaire was designed in Qualtrics program that was also used to randomized the conditions. Participants were randomly assigned to one of the six conditions. They were shown a package of a fictional brand of coffee beans and were asked to fill out the questions about their Product Expectations electronically.

In the second part of the experiment participants were provided with a sample of a coffee supposedly prepared from the contents of the package they had seen. In fact, the coffee they sampled was regular coffee of a popular Dutch coffee brand prepared with a French press and served in a simple plastic cup.

To make sure that the drinks did not differ in sensory properties between participants, the preparation was standardized. Each cup was prepared using 54 g of ground coffee grinded for French press in the Starbucks grinder. The water quality and temperature as well as the time of brewing (4 min) were controlled. The coffee was served to participants immediately after brewing.

After sampling coffee, participants filled out the questions about their Flavor Evaluation, Perception of Strength, Physiological Effects of coffee and their Purchase Intention. Finally, demographic information and the frequency of participants' coffee consumption was recorded. The experiment lasted from 5 to 15 min per participant.

# RESULTS

Two-way MANOVA with Image (image on top, image on the bottom, or no image) and Text (yes or no) as independent factors and Product Expectations, Flavor Evaluation, Perception of Strength, Physiological Effects of coffee and Purchase Intention as dependent variables was performed. The frequency of coffee consumption was used as the covariate in the model. Post hoctests using the Bonferroni correction were used for post-hoc pairwise comparisons.

TABLE 2 | The measurement scales and the reliability of the constructs.


The results have demonstrated that Image significantly influenced Product Expectation [F(2,117) = 9.35, p < 0.001, η <sup>2</sup> = 0.14]; Perceived Strength of coffee [F(2,117) = 4.13, p < 0.05, η <sup>2</sup> = 0.03] and Purchase Intention of the product [F(2,117) = 5.63, p < 0.05, η <sup>2</sup> = 0.09]. These effects are shown in **Figure 7**. Post hoc tests revealed significant differences between both packages with the image and the package without the image (both p's < 0.05). The differences between the images on top and on the bottom of the package were non-significant (p > 0.05). Participants had significantly more positive expectations of the product when they saw packages with an image of a lion on top of the package (M = 5.59; SD = 0.68) or on the bottom of the package (M = 5.48; SD = 0.69) compared to the package without an image (M = 4.80; SD = 1.19; p < 0.05). However, participants perceived coffee as significantly stronger when they saw the image on the bottom of the package (M = 4.71; SD = 0.79) compared to image on top (M = 4.36; SD = 0.96; p < 0.05) or the package without an image (M = 4.22; SD = 1.21; p < 0.05). The results for Purchase Intention showed a similar pattern: consumers were more likely to buy the product when the image of a lion was positioned on the bottom (M = 4.85; SD = 0.91) compared to the packages with the image on top (M = 4.23; SD = 1.00; p < 0.05) and no image (M = 4.19; SD = 1.12; p < 0.05). The effects of Image on Flavor Evaluation and Physiological Effects of coffee were non-significant (see **Table 3** for details).

Text significantly influenced Perceived Strength of coffee [F(1,117) = 4.42, p < 0.05, η <sup>2</sup> = 0.08] and Purchase Intention [F(1,117) = 7.53, p < 0.05, η <sup>2</sup> = 0.12]. Participants perceived coffee as significantly stronger (M = 4.79; SD = 1.13) when they

TABLE 3 | Main MANOVA results.


∗∗The effect is significant at 0.01 level; <sup>∗</sup> the effect is significant at 0.05 level.

saw the textual claim "Extra strong" compared to the package without a claim (M = 4.23; SD = 0.93). The presence of the claim also significantly increased Purchase Intention (M = 4.76; SD = 1.14) compared to the package without a claim (M = 4.12; SD = 0.86). Both effects are shown in **Figure 8**. The effects of Text on other dependent variables were non-significant. Furthermore, no interaction effects between Text and Image were found (see **Table 3** for details).

# DISCUSSION

fpsyg-09-00053 February 1, 2018 Time: 19:33 # 8

This study was focused on the relative effects of textual claims and metaphorical images presented on the packaging of coffee beans on the consumers' responses, including product expectations, flavor evaluation, perceived strength and physiological effects of coffee and purchase intention. The results demonstrated that both visual elements and textual claims depicted on the package are able to influence consumers' responses to the product.

The effects of the textual claim in our study were in line with our first hypothesis (H1). As expected, the textual claim "Extra strong" increased consumers' perception of the strength of coffee and their intention to purchase the product. However, the textual claim did not seem to influence product expectations, flavor evaluation and the perception of the physiological effects of coffee. These results suggest that a text claim is more efficient in communicating specific product attributes, but not a hedonic experience or general product expectations. A simple and clear message can convince consumers that a product indeed has the attribute claimed and thus result in the higher purchase intention. Therefore, a text claim can be a simple and effective tool for communicating sensory product attributes on food packaging.

In line with our second hypothesis (H2), the presence of the metaphorical image on the package positively influenced consumers' product expectations before tasting. This result is in line with previous findings that suggested that the attractiveness of images contributes to the evaluation of a product (Jeong, 2008). However, the presence of an image only influenced product expectations, but did not affect actual taste experience and purchase intention.

In our study, no interactions were found between the effects of a textual claim and a metaphorical image. Therefore, our hypothesis that textual claims and metaphorical images can reinforce each other (H3) was not confirmed. These results are in line with the study by Machiels and Karnal (2016) who did not found any interaction effects of text and image on the evaluation of orange juice. Together, these results suggest that images and texts do not merge into an integrated cognitive response, but could be processed in two different systems of information processing (Petty and Cacioppo, 1986; Paivio, 1990; Smith and DeCoster, 2000; Kahneman and Frederick, 2002; Evans, 2008). Textual claims represent informational cues that can be processed rationally. For instance, food labels (such as health, organic or fair trade labels) can evoke consumer skepticism (Tootelian and Ross, 2000; Grunert, 2002; Grunert and Wills, 2007; Sirieix et al., 2013; Fenko et al., 2016b). The fact that consumers express concern about the truthfulness of the information presented on labels and perceive some of the claims as misleading (Szykman et al., 1997; Hamilton et al., 2000; Chan et al., 2005) suggests that textual claims are likely to be processed deliberately and evaluated critically. Images, on the other hand, are perceived in a more holistic way and elicit more emotional responses than textual claims (Messaris, 1997). Pictures are often considered as heuristic cues that lead to peripheral or heuristic processing (Petty et al., 1991). Visual metaphors tend to be implicit and they do not require rational judgment about the credibility of the message (McQuarrie and Mick, 1999; Phillips, 2000). In general, visual images seem to be processed more intuitively and affectively compared to textual claims (McQuarrie and Mick, 1992; Mick, 1992; Peracchio and Meyers-Levy, 1994). Further research is needed to investigate how visual and verbal elements work together and how consumers process textual information and imagery to form integrated responses to products.

In line with our fourth hypothesis (H4), the results demonstrate that the image presented on the bottom of the package positively influenced the perceived strength of coffee and the product's purchase intention compared to the image presented on top of the package and the package without an image. This result is in line with previous studies that found an association between a lower position of an image on the package and the idea of heaviness (Van Rompay and Veltkamp, 2014). This result can be further explained by the comments made by the participants in our pre-study focus group. They described strong taste as being "heavy" on the tongue. Therefore, our results confirm that the experience of strength can be related to the idea of heaviness and can be therefore communicated by lower spatial position of an image on product packaging.

This result could be interpreted from the perspective of the grounded cognition theory. From this perspective, a picture in the lower part of the package would (automatically) activate the "strong is heavy" metaphor. As heavy objects are in our everyday interactions associated with a position on the ground, this would explain why perceiving a visually heavy package (i.e., with the lion positioned in the bottom part) would lead to the experience of a strong coffee.

It is interesting to note that in our study the product expectations were influenced by the presence of the image, while the perception of strength of coffee and purchase intention are influenced by the position of the image. The reason may be that the expectations are based on the packaging evaluation, which depends on conscious esthetic judgment (how beautiful, how balanced and elegant it looks), while the effects on perceived strength and purchase intention depend on the unconscious, automatic processing of metaphorical meaning, which is reinforced by the lower position of the figure.

Dual information processing (Petty and Cacioppo, 1986; Paivio, 1990; Kahneman and Frederick, 2002) can explain why

in certain cases cross-modal correspondences in food experience can be explained by sensory metaphors grounded in embodied experience (such as "warm is kind," "white is morally pure," or "strong is heavy"), while in other cases color, shape, image, typeface and other packaging details become "sophisticated form of visual rhetoric" (Scott, 1994) or semiotic codes that can be "decoded" logically by knowledge and historical connotations (Celhay and Remaud, in press).

We believe that in our experiment two types of metaphors were investigated: "strong as a lion" is a conventional metaphor that requires elaborate processing, while "strong is heavy" is a cognitive metaphor that is unconscious by nature. Traditional theories of metaphorical thought suggest conscious and elaborate processing of metaphorical meaning (Mick, 1992). However, grounded cognition theory (Lakoff and Johnson, 1999) challenged this approach, highlighting the systematic patterns underlying metaphorical expressions and their pervasive, mostly unconscious use in everyday language. A rapidly growing body of experimental research have demonstrated the role of cognitive metaphors in human thought (Barsalou, 2008; Williams et al., 2009; Landau et al., 2010). Social and moral concerns tend to be comprehended metaphorically in terms of sensory experiences, such as morality (dirty behavior), sociability (warm person), fairness (even-handed or balanced judgments), importance (weighty matter). These phenomena have been described as "cognitive unconscious": "It is not merely that we occasionally do not notice these processes"; rather, they are inaccessible to conscious awareness and control (Lakoff and Johnson, 1999, p. 11). Moreover, experiments suggest that metaphorical effects of embodied experiences can be eliminated when people become aware of their nature. For instance, awareness that a weight has been inserted in a book eliminates its metaphorical effect on judgments of the book's importance (Reinhard et al., 2012).

## Limitations and Suggestions for Future Research

The results of the present study are difficult to generalize, because the study was performed with a single product (coffee beans) and was focused on the single product attribute (strength of coffee). In future research, it is interesting to test whether the results would show similar pattern for a broad range of food products and for different product attributes, including hedonic attributes, healthiness, freshness, naturalness, authenticity, and general quality.

The study also used the limited amount of packaging manipulations. For instance, the visual metaphor was depicted in a rather sketchy manner (only a head of a lion was visible). Previous studies suggest that the style of the visual image (either a drawing or a more realistic photograph) can strongly influence consumers' sensory and hedonic expectations (Deliza et al., 2003). It is possible that the drawing of an animal's head alone without a body and the ground on which an animal is standing creates an impression of lightness instead of heaviness. Therefore, it is interesting to test whether the effects of the position will be different if the lion is depicted in full body standing on a high mountain.

The experimental manipulation included only one metaphorical image (a lion) that was congruent with the product attribute (strength of coffee). In the future studies it would be interesting to test whether the effects found in our study could be also found if the image is not metaphorically related to the sensory product property.

Besides, the image of a lion could convey not only a meaning of strong animal suggested by our informants, but also a meaning of royalty (Lion King) or an indication of the origin of coffee beans (an African country). In the future studies on metaphorical images it might be useful to combine quantitative methods (such as scales) with open-ended questions, asking participants directly what kind of meaning the image has to them. Such combination of qualitative and quantitative data might enrich the results of the study and provide more insight into the cognitive mechanisms of metaphorical thinking.

Grounded cognition framework suggests that embodied metaphors are universal (Lakoff and Johnson, 1980). They are grounded in human physical interactions with environment and show little variability across cultures due to universality of bodily constraints (Lakoff and Johnson, 1999). In this study we were focusing at the embodied metaphor grounded in human interactions with objects varying in weight and gravitational forces keeping heavy objects down on the ground. Arguably, such bodily metaphors are stable across cultures compared to culturally informed metaphors.

However, it is also interesting to look at the meaning of visual metaphors from the cross-cultural perspective. Some empirical studies within grounded cognition framework have discovered the effects of culturally specific metaphors. For instance, Lee and Schwarz (2012) found the effect of the smell of fish on adverse effects on cooperative behavior, highlighting the effect of metaphor "fishy is suspicious" which is present only in a limited amount of languages. The meaning of visual metaphors can differ between cultures and even within one culture due to gender and generational differences (Piqueras-Fiszman et al., 2011). In future studies, it is important to take a cross-cultural perspective into account while studying effects of visual metaphors on multisensory food experience.

It is also important to look at the possible moderating variables, such as the product involvement or price sensitivity. Some studies indicate that the response to packaging attributes can depend on consumers' characteristics, such as design sensitivity (Becker et al., 2011), general health interest (Fenko et al., 2016c), health promotion focus (Karnal et al., 2016) and skepticism toward textual claims (Fenko et al., 2016b). Consumers that are more involved in the product category or specific product attribute (e.g., health benefits) tend to be less sensitive to peripheral cues, such as a packaging shape or the sound of the brand name. It is possible that for these consumers a text claim would provide a more useful information and have a higher influence on the purchase intention. On the other hand, highly involved consumers who consider themselves experts in a product category tend to be more skeptical toward textual claims. Therefore, it is important to further study the influence of text and visual metaphors on consumers with low and high product or attribute involvement.

In our study, no significant effects of either text or image on physiological effects of drinking coffee (arousal) were found. We did not expect to find such effects, because the experiment was too short for each individual participant to notice such an effect. The measure of physiological effects was included in order to separate two meanings of "strengths" of coffee that we had found in the pre-study focus group. Asking separate questions about the strengths of taste and the expected strengths of arousal allowed us to measure the effect of packaging on flavor perception ("the strengths of taste") more precisely. However, in future studies it would be interesting to test the effects of packaging information (textual and pictorial) on physiological arousal measured both subjectively and by using objective methods such as hart rate and skin conductivity.

#### CONCLUSION

This study demonstrates that the use of textual claims and metaphorical images depicted on a package of coffee beans can significantly change consumers' product expectations, the strength perception of coffee and purchase intention. Textual claims seem to be more efficient in directly communicating clear and simple product attributes, such as strength and thus positively influence purchase intention. A metaphorical image depicted on the package is able to communicate the same

#### REFERENCES



attribute in an indirect way. In addition, an image can enhance the esthetic quality of the package and increase general product expectations. The location of the image on the package is also important for consumer experience. The results of our study suggest that placing an image on the bottom of the package can metaphorically communicate certain product attributes (such as intensity, heaviness and strength) and thus increase consumers' purchase intention for consumers who value these attributes.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of Ethical Committee of the Faculty of Behavioural Science of the University of Twente. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Ethical Committee of the Faculty of Behavioural Science of the University of Twente (COMMISSIE ETHIEK (CE) FACULTEIT GEDRAGSWETENSCHAPPEN; protocol # 17077).

#### AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.


Barthes, R. (1977). Image Music Text. New York, NY: Hill and Wang.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Fenko, de Vries and van Rompay. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Understanding Freshness Perception from the Cognitive Mechanisms of Flavor: The Case of Beverages

Jérémy Roque1,2,3 \*, Malika Auvray<sup>3</sup> and Jérémie Lafraire2,4

<sup>1</sup> Centre de Recherche Pernod Ricard, Créteil, France, <sup>2</sup> Center for Food and Hospitality Research, Institut Paul Bocuse, Écully, France, <sup>3</sup> Institute of Intelligent Systems and Robotics, Université Pierre et Marie Curie, Paris, France, <sup>4</sup> Institut Jean Nicod, Ecole Normale Supérieure, Paris, France

Freshness perception has received recent consideration in the field of consumer science mainly because of its hedonic dimension, which is assumed to influence consumers' preference and behavior. However, most studies have considered freshness as a multisensory attribute of food and beverage products without investigating the cognitive mechanisms at hand. In the present review, we endorse a slightly different perspective on freshness. We focus on (i) the multisensory integration processes that underpin freshness perception, and (ii) the top–down factors that influence the explicit attribution of freshness to a product by consumers. To do so, we exploit the recent literature on the cognitive underpinnings of flavor perception as a heuristic to better characterize the mechanisms of freshness perception in the particular case of beverages. We argue that the lack of consideration of particular instances of flavor, such as freshness, has resulted in a lack of consensus about the content and structure of different types of flavor representations. We then enrich these theoretical analyses, with a review of the cognitive mechanisms of flavor perception: from multisensory integration processes to the influence of top–down factors (e.g., attentional and semantic). We conclude that similarly to flavor, freshness perception is characterized by hybrid content, both perceptual and semantic, but that freshness has a higher-degree of specificity than flavor. In particular, contrary to flavor, freshness is characterized by specific functions (e.g., alleviation of oropharyngeal symptoms) and likely differs from flavor with respect to the weighting of each sensory contributor, as well as to its subjective location. Finally, we provide a comprehensive model of the cognitive mechanisms that underlie freshness perception. This model paves the way for further empirical research on particular instances of flavor, and will enable advances in the field of food and beverage cognition.

Keywords: freshness, flavor, multisensory perception, crossmodal correspondences, beverages

# INTRODUCTION

The perceived freshness of food and beverages has been generally considered as the result of several sensory contributions: olfactory, gustatory, tactile, trigeminal, visual, and auditory (Westerink and Kozlov, 2003; Labbe et al., 2009a,b; Saint-Eve et al., 2010). Databases—Google Scholar and Science Direct—were searched over all years of records using pre-specified terms (freshness, refreshing,

#### Edited by:

Carlos Velasco, BI Norwegian Business School, Norway

> Reviewed by: Anna Fenko,

University of Twente, Netherlands Carlos Alberto Arce-Lopera, ICESI University, Colombia

\*Correspondence: Jérémy Roque jeremy.roque@pernod-ricard.com

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 29 August 2017 Accepted: 26 December 2017 Published: 11 January 2018

#### Citation:

Roque J, Auvray M and Lafraire J (2018) Understanding Freshness Perception from the Cognitive Mechanisms of Flavor: The Case of Beverages. Front. Psychol. 8:2360. doi: 10.3389/fpsyg.2017.02360

**49**

perception, consumers) in the consumer science literature. Our searches identified more than one hundred studies dealing with freshness, with a large fraction of the studies focusing on freshness of fruits and vegetables (see Péneau, 2005 for a review) and other food products like bread and biscuits (e.g., Heenan et al., 2008). However, most have considered freshness as a multisensory attribute of food and beverage products without investigating the multisensory integration processes and cognitive mechanisms<sup>1</sup> at hand. Furthermore, the consumer science literature clearly embeds two distinct types of freshness whose cognitive underpinnings are probably very different. To better characterize the subject of the present review, it is thus necessary to first address the sematic ambiguity of the term "freshness".

The semantic ambiguity of "freshness" is straightforward. When experiencing products containing both liquid and solid components as well as several tastants and odorants (e.g., cocktails), consumers can characterize them as fresh based on either the overall multisensory experience (involving for instance coldness, sourness, or a menthol odor), or on the aging of the organic ingredients it contains (e.g., the age of the mint leaves in a mojito). For instance, Péneau (2005, p. 6) defined freshness of fruits and vegetables as the "level of closeness to the original product, in terms of distance, time and treatment". Thus, it seems that at least two meanings may be conveyed by the word "freshness". In this review, we will focus on the first meaning. We will also focus on the multisensory integration processes at the perceptual level and on the top–down factors that may influence those processes. We will consider the case of beverages, by reviewing the perceptual and cognitive mechanisms that cause a particular beverage to be perceived as fresh.

According to Labbe et al. (2009a), the concepts of refreshing and freshness (in the first sense mentioned above) seem closely linked because they have common sensory drivers such as coldness and mint flavor. The notion of refreshment in beverages was also shown to strongly correlate with a thirst-quenching sensation (Labbe et al., 2009b). In the present review, we address the concept of freshness instead of refreshing because the term refreshing is more directed toward the post-consumption phase, together with the impact fresh beverages have on arousal or thirst-quenching sensation (Labbe et al., 2009a). In the following sections, we will thus refer to "perceived freshness" because it covers both the expected and the actual sensation of a given product.

Freshness, in that restricted sense, has been analyzed at various levels of description. From the physiological level of analysis, freshness perception seems linked to the alleviation of unpleasant physical symptoms, such as elevated body temperatures or mouth dryness, during the consumption of a cool or fresh beverage (see Labbe et al., 2009a for a review). With respect to the sensory level, several sensory descriptors have been shown to positively or negatively influence freshness perception. Some of them have also been associated with the term "refreshing", as well as with psychophysiological factors involved in freshness perception: "thirst-quenching" and "mouth-wetting" (Labbe et al., 2009a,b, 2011). According to consumers, coldness and sourness are the common sensory properties that enhance freshness perception in food and beverages (Zellner and Durlach, 2002), waters (Labbe et al., 2011), and soft drinks (McEwan and Colwill, 1996; Fenko et al., 2009; Saint-Eve et al., 2010; Zhang et al., 2016). On the other hand, sweetness (McEwan and Colwill, 1996; Guinard et al., 1998; Labbe et al., 2009b) and thickness (Scriven et al., 1989; McEwan and Colwill, 1996; Guinard et al., 1998; Labbe et al., 2009b; McCrickerd et al., 2015) were associated with decreased freshness perception. Regarding colors, a clear color was found to enhance thirst-quenching and refreshing perceptions of soft drinks (Clydesdale et al., 1992; Zellner and Durlach, 2003). Red and orange colors were also associated with an increase in thirstquenching perception of fruit-based drinks (Zellner and Durlach, 2002). According to different studies, smells like mint, orange (Zellner and Durlach, 2002; Labbe et al., 2009b), peppermint, lemon (Fenko et al., 2009), citrus, and peach (Martin et al., 2005) were judged to be the most refreshing aromas for food and beverages, whereas chocolate was most commonly listed as the least refreshing (Zellner and Durlach, 2002; Labbe et al., 2009b). Other kinds of perceptual features such as carbonation in beverages have also been shown to positively influence freshness perception (McEwan and Colwill, 1996).

However, to go beyond simply listing the perceptual features that contribute to freshness perception, it is necessary to identify the mechanisms that integrate them into a unified freshness perception. The binding of multisensory inputs into one unified percept has been investigated and extensively discussed in the case of flavor (see Auvray and Spence, 2008 for a review). So why does the literature lack of information on the mechanisms of freshness perception?

One possible, simple explanation of this gap is that it would be redundant with the literature on flavor. We will show that this answer should be qualified: even if some redundancies will be unavoidable, freshness differs from flavor in some important respects. Indeed, flavor is more generic than freshness, which in turn seems more generic than particular flavors (e.g., mint flavor). All freshness instances are systematically confounded with particular flavors (e.g., mint or citrus flavors) but all flavors are not necessarily confounded with perceived freshness. This type of relationship corresponds to what is called a "taxonomic hierarchy" between categories or concepts (Murphy, 2002, p. 325). Specifying at which level of the hierarchy freshness is located (superordinate, basic-level, or subordinate) would lead us beyond the scope of the present review. Rather, we aim to determine what the higher degree of specificity of freshness implies, regarding the multisensory mechanisms at hand. Furthermore, we will argue that thoroughly considering a particular instance of flavor such as freshness may shed light on some theoretical debates on flavor perception.

First of all, competing theses regarding the representational nature of flavor and freshness will be reviewed. It will be discussed whether flavor and freshness have to be considered as cases of synesthetic experiences, object representations, or rather as perceptual categories (see Section "The Representational

<sup>1</sup> In the present review, we use "cognitive mechanisms" or "cognitive factors" to refer to the set of mechanisms involved in the extraction and integration of information that constrains, influences, and interacts with the everyday experience of food and drink (Verhagen and Engelen, 2006).

Nature of Flavor and Freshness"). This conceptual clarification will allow us to move to the central question of the present review which is the characterization of the perceptual and cognitive mechanisms that underlie the experience of freshness. The existing literature on the cognitive underpinnings of flavor perception will be used as a heuristic to determine whether or not the same cognitive underpinnings apply mutatis mutandis to the perception of freshness. We will focus on the cases of crossmodal interactions and correspondences, which contribute to the experiences of flavor and likely to freshness (see Section "Crossmodal Interactions and Correspondences"). Then, the memory and attentional aspects impacting the perceptual and cognitive processes mentioned above will be described (see Sections "Memory, Expectations, and Knowledge" and "Attention"). Finally, the particular conditions that influence the degree of integration of a multisensory perception, such as congruency and the unity assumption (see Section "Particular Conditions that Influence the Degree of Integration"), will be reviewed. This will allow us to build a preliminary model of freshness perception and provide specific empirical research hypotheses about the putative crossmodal interactions and correspondences that influence the experience of freshness.

# THE REPRESENTATIONAL NATURE OF FLAVOR AND FRESHNESS

In this section, we will consider both the computational processes underpinning flavor and freshness perception and the types of representations for which flavor and freshness are eligible. The processes will be addressed first by introducing a distinction between multisensory integration and sensory fusion. Then, the representational nature of flavor will be questioned by considering various types of representations: synesthetic experiences, object representations, and perceptual categories. Each time, the representational nature (structure and content) of freshness will be compared to that of flavor.

#### Sensory Fusion

During the consumption of a food or beverage, multiple sensory inputs are, under certain circumstances (see Section "Particular Conditions that Influence the Degree of Integration"), integrated by the central nervous system to potentially give rise to a unified flavor perception (e.g., Auvray and Spence, 2008). One of the main functions of the cognitive system is to resolve certain conflicts that occur during the integration process. Sometimes, the conflict resolution between two or more perceptual features can be such that each sensory contributor, or at least one of them, will lose its individual sensory characteristics. This phenomenon has been named sensory fusion (Verhagen and Engelen, 2006). Sensory fusion corresponds to a higher degree of sensory integration and to a particular level of unification between sensory features that can either belong to the same sensory channel (intramodal case) or to two distinct sensory modalities (crossmodal case). An intramodal instance of fusion occurs for instance when two colors (e.g., red and yellow) combine and consequently lose their initial individual sensory qualities to form a new sensation (i.e., red and yellow are merged into orange). The fusion process is specific to particular dimensions of the sensory signal, as it does not seem to occur between basic tastes for instance (e.g., McBurney and Gent, 1979). Indeed, sweetness and bitterness cannot be merged into a third basic taste. A crossmodal instance of fusion occurs when two or more sensory features belonging to distinct sensory modalities are combined and result in a distinct percept. In this case, at least one of the initial sensory inputs loses its respective modal identity (i.e., the perceptual format that specifies the type of sensory modality at the origin of the representation) and the fusion of the two forms one perceptual unit (see Verhagen and Engelen, 2006; Auvray and Spence, 2008 for reviews). For instance, it is generally assumed that a conflict occurs systematically between the spatial information conveyed by the taste signal and the spatial information conveyed by the smell signal, especially the retronasal one (Small and Prescott, 2005). The perceptual consequence of the conflict resolution by the cognitive system is that sensations originating in the nose are "referred to" or experienced as if they were transduced by the receptors in the mouth (Spence et al., 2014). Different terms have been used to describe this subjective phenomenon resulting from a sensory fusion between taste and smell such as the "location binding" (Stevenson et al., 1995), the "olfactory illusion" (Prescott, 1999), or the "mouth capture" (Shepherd, 2012). It is worth noticing that the strong claim that flavor perception is systematically underpinned by sensory fusion conflicts with the idea that humans still have the capacity to analyze a flavor into its parts (e.g., Stevenson, 2014).

In the case of freshness, it is likely that the modal identity of each sensory contributor is not always systematically accessible to the subject. Two non-exclusive hypotheses can be formulated to explain this phenomenon: i) the modal identity of the sensory contributor is not accessible because of a fusion occurring at some point during the multisensory integration and therefore the modal identity of certain contributors is lost, and/or ii) the modal identity of some contributors to freshness is intrinsically ambiguous. In fact, it has been shown that coldness can enhance the perception of freshness (Labbe et al., 2009a) and coldness is mediated by specific receptors found in trigeminal cold-sensing neurons. However, these receptors are widely distributed (on the tongue, in the nasal cavity, and in the peripheral nervous system) and can be activated either by cold temperatures or by different organic compounds with cooling properties such as menthone and menthol (for a review of mechanisms of temperature perception, see Patapoutian et al., 2003). Moreover, it has been shown that some people can experience the "thermaltaste illusion" in which a particular temperature experienced on the tongue will induce other taste sensations (Bajec and Pickering, 2008). Thus, in a nutshell, the mere fact of not being conscious of all the distinct sensory contributors to freshness may be due to a fusion occurring between two or more sensory features during the multisensory integration step, or to ambiguous activations of certain receptors, or, as is likely to both.

# Synesthetic Experience

fpsyg-08-02360 January 9, 2018 Time: 17:50 # 4

The reasons why some people are often unable to tease apart the relative contributions of smell and taste to flavor perception (Spence and Piqueras-Fiszman, 2014) have been discussed in the previous section. In this section, one step further will be taken to consider why certain odors can elicit changes in the perception of a particular tastant (e.g., Stevenson et al., 1999). For instance, the sensory fusion of taste and smell described earlier can induce a crossmodal enhancement under certain circumstances (congruency dependent, see Section "Congruency" as well as task dependent and attention dependent, see Section "Attention"). The sweetness enhancement that can result from the sensory fusion between a sweet taste and a strawberry odor has been widely studied in flavor perception. When these two perceptual features are experienced together, the strawberry flavor stored in memory will induce a sweetness enhancement: people will tend to perceive the sweetness as more intense for a product with the strawberry odor as compared to a control odorless stimulus with the same concentration of sweet taste<sup>2</sup> . The fact that this pairing can lead to perceptual changes of the taste (i.e., higher sweetness ratings) have led some researchers to argue that flavor perception could result from a synesthetic experience (Stevenson and Boakes, 2004; Verhagen and Engelen, 2006).

Synesthesia has been defined as a neurological condition in which the stimulation of one sensory or cognitive pathway leads to an automatic, involuntary experience in a second sensory or cognitive pathway (Cytowic, 2002). It can be characterized by four criteria that are jointly sufficient for individuating genuine forms of synesthesia: (a) the existence of a conscious pairing between an inducer (e.g., a number) and a concurrent (e.g., a color) perceived at the same time; (b) the relative idiosyncrasy of the pairing; (c) the automaticity of the process which causes an inevitable and involuntary experience of the concurrent when the inducer is present; and d) the consistency of the occurrence over time. Auvray and Farina (in press, see also Deroy and Spence, 2013) have suggested several arguments that question the appropriateness of considering flavor perception as the result of a synesthetic experience. The first criterion of a conscious pairing is not satisfied in the case of flavor perception. In fact, considering the sensory fusion occurring in flavor perception, it is the overall perceptual unit which is perceived by the participants and not the two individual sensory properties at the same time (see **Figure 1**). Regarding the second criterion, it has been shown that the multisensory processes underpinning flavor perception are experienced by everyone, at least intra-culturally (see section "Culture and Expertise"), thus it does not involve idiosyncrasy. Regarding the third criterion of automaticity of the process, controversial results have been found for flavor perception since the sensory fusion

is dependent on the task and the attention of the participants (see Section "Attention"). Only the fourth criterion of consistency over time is verified regarding flavor perception. On the basis of the above-mentioned arguments, even if some arguments can be discussed, we argue that they are enough to consider the burden of proof to be on the proponents of the idea that flavor represents a genuine case of synesthetic experience. As freshness reasonably corresponds to a particular instance of flavor, it is then likely that the sensory contributors to freshness can also be bound in one perceptual unit. Moreover, people may often be unable to tease apart the relative contributions of each sensory contributor. Thus, similarly to flavor, we can reasonably argue that freshness does not result from a synesthetic experience.

#### Object Representation and Categories

It has been put forward that "Object perception could represent a basic processing strategy applied to any sensory system that has to extract meaning from a complex stimulus array" (Stevenson, 2014, p. 1360). Hence, in order to further understand the mechanisms responsible for flavor perception, several authors have considered the object concept (e.g., Auvray and Spence, 2008; Small, 2012). Even though the object concept has been applied to vision, audition, and olfaction (see Stevenson, 2014), it has not been considered for other chemosensory modalities such as taste and chemesthesis (i.e., sensations due to chemical compounds). By listing the criteria that seem to characterize "object-hood" and making a distinction between the various sensory modalities that constitute flavor, Stevenson (2014) discussed whether flavors are integrated by default as objects (i.e. holistically), as a series of perceptual features, or both at different times. He described eight criteria for object-hood, taking into account the structural aspects of the stimulus and memory dependence that reflect respectively the bottom–up and the top– down processes contributing to the constitution of an object representation. One of the criteria that the notion of object-hood depends on is figure-ground segregation. The figure refers to the object itself and the background corresponds to the other perceptual features experienced at the same time and detected by

<sup>2</sup> In addition to the crossmodal enhancement that can result from the sensory fusion between a conscious taste and an unconscious smell, another kind of crossmodal enhancement can also be obtained for instance when one odor and one taste at subthreshold are presented together; Dalton et al. (2000) have shown that the threshold for detecting the odor was then lowered and the participants were sometimes able to detect the odor even if the individual perceptual features were weak.

the same receptor system. To illustrate that notion, let's consider an example: when someone chews gum just before lunch time, the ingredients of the dish in the mouth will be experienced against a flavor background that consists of the remaining perceptual compounds of the gum. However, these persisting gum compounds will also impact the perception of the dish itself. According to Stevenson (2014), there is no evidence that a flavor object can be perfectly differentiated from its background, considering flavor as an object in the mouth. The figure-ground segregation criterion would not be satisfied.

Another reason that leads Stevenson to reject the idea that flavor is eligible for the status of object representation is that flavor-object-representations would have no additional function beyond that of olfactory or visual objects. Three potential functions are considered: detection, identification, and violation of expectancies. According to Stevenson (2009), foods are identified visually and via the orthonasal olfactory system before being placed in the mouth. Thus, forming flavorobject-representations in order to identify and detect foods would carry a useless cost for the cognitive system since these functions are achieved before the oral step. However, Stevenson acknowledges that the violation of expectancies hypothesis is more promising since flavor-object-representations could be compared to representations in memory. This comparison could then help the cognitive system to tease apart the expected properties from the unexpected properties of a given food or beverage. This comparison would then serve as a protective system since the perception of unexpected properties could help the individual to avoid ingesting poisonous substances. However, once again according to Stevenson this hypothesis is redundant since it is plausible that the violation of expectancies does not necessarily imply a comparison process between what is stored in memory and what is actually experienced.

Stevenson's redundancy argument is clearly debatable since it is likely that the system based on taste is the precursor of a more fine grained protective system involving multiple sources of information conveyed by flavor objects. This is at least as plausible as Stevenson's idea, from an evolutionary point of view. Furthermore, Stevenson's argument seems quite inconsistent with the idea that flavor actually represents a functional sensory system with inputs from somatosensation, gustation, and olfaction where the meaning of the sensation would be more involved in the final representation than its precise organ or site of origin (Small and Prescott, 2005). This concept of flavor that focuses on the meaning of the sensation recalls Gibson's (1966) ecological approach in which perceptual systems are defined by their function, rather than tied to the organ of transduction. Moreover, even if we concede that flavor objects have no obvious additional functions beyond those of gustatory, visual, or olfactory objects, this can be due to the generic character of the term flavor. Let us consider a more specific instance of flavor such as freshness. According to Labbe et al. (2009a), one possible function of freshness in the mouth is the alleviation of oropharyngeal symptoms (e.g., mouth dryness). Thus, by considering a particular instance of flavor such as freshness, it is possible to highlight specific functions of the flavor object experienced. Moreover, the different perceptual attributes and functions of a particular group of objects are generally contained in categories which have been defined by Rosch (1978) as groups of objects sharing common attributes that are most representative of items inside and less representative of items outside the corresponding category. Rosch (1978) has highlighted that one of the main functions of categories is to structure information and support recognition, discriminability, and inductive reasoning. Therefore, claiming that flavor does not have any specific function seems to imply that it is not eligible for the status of being a category. This is indeed what Stevenson's analysis tends to suggest. According to Stevenson (2014), categorization is a semantic process which occurs mainly after the perceptual process (i.e., the construction of the object). However, we will see in the next paragraph how categorization has also been defined on the basis of perceptual processes.

A perceptual theory of knowledge was proposed by Barsalou (1999, p. 578) that highlights the convergence of cognition and perception. He described the nature of perceptual symbol systems as follows: "Subsets of perceptual states in sensory-motor systems are extracted and stored in long-term memory to function as symbols. As a result, the internal structure of these symbols is modal, and they are analogically related to the perceptual states that produced them.". In other words, perceptual symbol systems represent schematic components of the multisensory experience and are intimately intertwined with higher order cognitive levels linked to semantics and memory, which help to structure information into categories. Thus, contrary to Stevenson's idea that categorization is a semantic process that occurs after the perceptual process, according to Barsalou (1999) there is no real distinction between the structure of categories and the perceived features that belong to these categories.

To summarize, considering a specific instance of flavor such as freshness enables us to discuss the multisensory processes that can lead to its specific representational nature and more clearly characterize the perceptual and cognitive mechanisms at stake. For instance, the above-mentioned debate on categorization highlights the fact that a freshness representation could be considered as a category structured into a perceptual symbol system that consistently maintains the junction between the perceptual level and other higher levels of processing (e.g., semantic content). Thus, in order to further characterize the ordinary perceptual experience of freshness, it appears necessary to consider the interactions between the various perceptual features that contribute to freshness and other higher levels of processing. To do so, in the next section, the notion of crossmodal correspondences will be described and distinguished from the notion of crosslevel correspondences.

# CROSSMODAL INTERACTIONS AND CORRESPONDENCES

#### The Crossmodal Perception of Flavor

Several crossmodal interactions have been reported in the case of flavor and the relative contributions of each sensory modality have already been partially documented (see Verhagen and

Engelen, 2006; Shepherd, 2012, for reviews). A large body of research has also explored the evidence regarding the existence, and the consequences for human information processing, of a particular form of crossmodal interactions named crossmodal correspondences. Crossmodal correspondences have been defined as "the many nonarbitrary associations that appear to exist between different basic physical stimulus attributes, or features, in different sensory modalities" (see Spence, 2011, p. 972). The literature on flavor perception has highlighted the existence of crossmodal correspondences between many pairs of perceptual features belonging to different modalities constitutive of flavor such as tastes and sounds (Knöferle and Spence, 2012), auditory pitch and smell (Belkin et al., 1997), colors and odors/tastes (Spence et al., 2010, 2015), smells and shapes (Seo et al., 2010), and even shapes and tastes (Velasco et al., 2015).

The aim of this section is to go beyond the mere description of these correspondences in order to bring conceptual clarification. Indeed, crossmodal correspondence mechanisms refer to several phenomena occurring at distinct levels of information processing. Four principal types of crossmodal correspondences have been distinguished (see Spence, 2011 for a review): Structural correspondences are possibly innate, but may also depend on the maturation of neural structures for stimulus coding (e.g., between the loudness of a sound and the brightness of a light). Statistical correspondences are learned and result from the extraction of certain environmental regularities by the cognitive system. For instance, the correlation between the size, or mass, of an object and its resonant frequency (i.e., the larger the object, the lower the frequency). Semantically mediated correspondences may occur when common linguistic terms are used to describe two different stimuli. For instance, when a semantically ambiguous term refers to distinct sets of stimuli depending on the context, such as high and low, which can describe both pitch and elevation. Emotionally mediated correspondences consist of associations between basic perceptual features that seem to be mediated by certain dimensions of emotion such as the valence, or the level of arousal induced by the perceptual features.

On the basis of the above-mentioned typology of crossmodal correspondences, two fundamental types of correspondences appear: those occurring only at a perceptual level (e.g., between the size of an object and its resonant frequency), and those involving other levels of processing such as the semantically mediated correspondences and the emotionally mediated correspondences. In the next sections, the expression "crossmodal correspondences" will be used only to refer to the former. In other words, we will stick to the literal meaning of crossmodal which implies that at least two sensory modalities are involved and that the correspondence occurs at a perceptual level. By contrast, we will use "crosslevel correspondences" to refer to the associations involving another level of processing (e.g., linguistic or emotional).

Beyond the theoretical level, the existence of crossmodal correspondences can be inferred from their effects. For instance, Knöferle and Spence (2012) have reported that stimuli sharing a crossmodal correspondence can induce shorter RTs (reaction times) in a particular task. However, according to the authors, there is little evidence that such stimuli also have perceptual consequences. In line with this hypothesis, Gallace and Spence (2006) have shown that the presentation of either a crossmodally congruent or incongruent sound did not actually change the perceived size of a circle that was presented with it, despite the fact that participants' RTs changed significantly. Two noticeable exceptions are worth mentioning though. Liang et al. (2013) have shown that rounded shapes enhanced sweetness sensitivity (at least at near-threshold levels), whereas angular shapes did not. Another study has highlighted that low-pitched notes played by brass instruments can enhance the perceived intensity of the bitter taste of caffeine, and high-pitched notes played by the piano can enhance the perceived intensity of sucrose (Crisinel et al., 2012). Thus, some crossmodal correspondences seem to lead to a crossmodal enhancement effect, a phenomenon that is well-known in flavor perception (see Section "Synesthetic Experience"). We will argue in the next section that knowing how to trigger the mechanisms of crossmodal correspondences involved in freshness (if any) could facilitate consumers' categorization of a given product as fresh or even lead to freshness enhancement. However, if we want to enhance the freshness perceptual experience by triggering specific crossmodal correspondences, it appears necessary to first determine the type of correspondence likely at hand in freshness perception.

# The Crossmodal Perception of Freshness in Beverages

The literature on the sensory contributors to freshness perception can feed hypotheses about the crossmodal correspondences that could exist and potentially lead to a freshness enhancement effect (see Labbe et al., 2009a for a review). Several studies have highlighted that carbonation was part of consumers' expectations for fresh beverages (e.g., McEwan and Colwill, 1996; Guinard et al., 1998). Moreover, the perception of the carbonation of a beverage may also be influenced by auditory cues provided by the bubbles (Yau and McDaniel, 1992; Guinard and Mazzucchelli, 1996). Zampini and Spence (2005) conducted several experiments to investigate the role of auditory cues in the perception of carbonation in beverages. In a first experiment, the carbonated water samples were judged to be more carbonated when the overall sound level was increased and/or when the high frequency components (2–20 kHz) of the water sound were amplified. They were also evaluated as being more carbonated when they were held close to the ear rather than further away. Another experiment in which the participants had to assess the level of carbonation and the oral irritation of water samples, in the mouth, revealed that neither the perceived carbonation nor the perceived oral irritation were influenced by variations in the level of auditory feedback. Overall, Zampini, and Spence's results highlight the significant role that auditory cues play in modulating perception of the carbonation of beverages. However, for the perception of carbonation in the mouth, oralsomatosensory and nociceptive cues dominate over auditory cues (see Dessirier et al., 2000; Carstens et al., 2002, for more details on the mode of action of carbonation). Then, a crossmodal correspondence occurring between visual carbonation and the intensity of the trigeminal stimulation due to carbonation can

also be expected. Other results have been obtained by recent studies regarding the influence of auditory cues corresponding to the pouring of a beverage on the perception of its temperature (Velasco et al., 2013) or corresponding to the opening of the packaging on the perception of freshness in terms of a new and not-tampered-with product (see Spence and Wang, 2015 for a review). The results obtained by Velasco et al. (2013) have highlighted a crosslevel correspondence between word attributes and sounds that were congruent ("Hot Drink" and the sound of the hot pouring water, and "Cold Drink" and the sound of the cold pouring water).

Regarding flavor, it seems to be difficult to identify semantic ambiguity due to its generic character. Correspondences with a non-perceptual cognitive level (e.g., semantic content) that we characterized as crosslevel correspondences have not been investigated yet in freshness perception. However, in line with the semantically mediated correspondence in which low and high describe stimuli varying in pitch and visual elevation, the semantic ambiguity relative to freshness is more easily identifiable (i.e., multisensory stimulations versus aging of fruits and vegetables, see Introduction). It is thus reasonable to suspect potential interactions between the perceptual features themselves that influence freshness perception (e.g., crossmodal correspondences), as well as between the perceptual features and the various meanings assigned to freshness (e.g., crosslevel correspondences).

The perceptual and semantic information is stored in memory and is thus strongly dependent on the participants' background knowledge. If there is a need to better identify the respective contributions and functions of each sensory modality in the case of freshness perception, it is also important to take into account the influence of the cognitive factors such as attention and memory in studies' elaboration and analysis of their limitations.

# MEMORY, EXPECTATIONS, AND KNOWLEDGE

#### Learned Associations and Expectations

Human beings, at least in western countries, are prone to be in contact with a rich food and drink environment in which several associations between various perceptual features (e.g., colors, texture) can be repeatedly encountered. In the case of a negative post-ingestive effect, people will store this information in memory with respect to the poisonous food they ingested and they will subsequently adapt their dietary behavior. Such past experience will drive long-term flavor preference formation and intake (Myers and Sclafani, 2006).

In the case of freshness perception, Labbe et al. (2009a) have argued that the positive experience of alleviation of unpleasant symptoms (thirst, mouth dryness, mental fatigue, feeling too hot) following consumption of a given beverage leads to a learned association of these positive experiences (e.g., between the coldness of a drink and the relief from the sensation of feeling too hot) with freshness perception. This is in line with what Zellner and Durlach (2002) found among a group of American students. When asked to list foods, beverages, and sensory characteristics they considered to be refreshing, the American students mentioned water most of the time (90% of respondents) as well as cold temperature (see also Eccles et al., 2013). Prior learning may explain other associations with freshness, such as the positive association with clear appearance (Zellner and Durlach, 2003) and the negative associations with sweetness, thickness (McEwan and Colwill, 1996; Guinard et al., 1998; Labbe et al., 2009b), intense flavor, and after-taste (Guinard et al., 1998). Labbe et al. (2009a) concluded that further work is needed before assuming that flavor-refreshing learning is as robust as other types of associative learning such as odorant-sweet taste learning.

Moreover, it should be noted that prior learning can modulate the kinetic aspects of food or drink consumption, contributing to an increase or decrease in the perceived intensity of a particular sensory compound (Blissett et al., 2007). The actual moment of swallowing seems particularly important since it has been shown that this is when major aroma pulses are induced (Buettner et al., 2002; Hodgson et al., 2003), thereby making a major contribution to the final percept of the beverage. However, McCrickerd et al. (2014) have highlighted the notion of product dependence by suggesting that the strength of associations formed between a drink's sensory characteristics and its post-ingestive effect would be weak compared to that for solid foods. McCrickerd et al. (2014) think that it could potentially be the case because beverages are consumed rapidly, and this reduced oral exposure time may limit the strength of its oro-sensory signal and subsequent learning. Moreover, a high degree of variability in swallowing patterns has been found between individuals, with some individuals performing simple swallowing actions whereas others incorporate learned tasting behaviors into their everyday consumption routines to adapt their consumption habits to their physiological requirements (Blissett et al., 2007; Buettner and Beauchamp, 2010). It is possible to go further, hypothesizing that some people adapt their consumption habits not only at a physiological level, but also to obtain the most pleasant experience, when consuming a drink that they expect to be fresh. Concerning freshness, Westerink and Kozlov (2003) have highlighted that people tend to value temporary sensory input during the actual freshness experience (e.g., temperature), whereas they tend to keep in long-term memory other sensory features related to the freshness percept (e.g., particular compounds such as menthol).

Beyond the perceptual level, we can also wonder how the semantic properties related to freshness perception influence subsequent experiences of freshness. According to Mathis (2002), semantic properties elicit consumers' knowledge and beliefs (e.g., mental representations built from previous experiences). These mental representations help for stimuli identification, building knowledge of their properties, and the adaptation of consumers' behavior according to their expectations relative to the product experienced. Regarding the concept of expectations it is worth refining the notion to distinguish two levels of expectations: (i) expectations as perceptual priors, what Seriès and Seitz (2013) named the "structural" expectations resulting from the integration of the various perceptual features of a given stimulus, and (ii) expectations as beliefs (implicit or explicit at a doxastic level, see Dretske, 1988, p. 117 for this distinction) that can be manipulated through instructions, sensory cues, or contextual variables (see Seriès and Seitz, 2013). It should be noted that the generation of different taste/flavor expectations can be a function of the participants' background and/or culture (Zellner and Durlach, 2003), expertise/experience with particular food or beverage domains (Parr et al., 2003; Smith, 2007), and age (Philipsen et al., 1995).

## Culture and Expertise

fpsyg-08-02360 January 9, 2018 Time: 17:50 # 8

The influence of participants' background and culture has been investigated by Nguyen et al. (2002), who conducted a study highlighting that the odors of vanilla, caramel, strawberry, and mint induced sweetness enhancement in western countries where people often experience those odors with sucrose. By contrast, non-western participants did not describe some of these odors as sweet, probably due to a less frequent pairing of these odors with sweetness in their food culture. In another cross-cultural study (Wan et al., 2014), the same seven drinks were presented in three different types of glass to participants from mainland China, the United States, the United Kingdom, South Korea, and India. The results revealed that the same beverage color sometimes set up distinctly different flavor expectations depending on both the type of receptacle and the cultural background of the participants. These sources of variability relative to background knowledge and culture probably have major influences on the lack of consensus concerning the constitutive properties of freshness in food and beverages (Cardello and Schutz, 2003; Heenan et al., 2008; Zhang et al., 2016).

Regarding the influence of expertise, conflicting results have been reported. A handful of studies have highlighted that the level of expertise can help an individual categorize certain flavor components more easily and may improve perceptual capacities (e.g., the capacity to identify the perceptual similarities between two different products; Ballester et al., 2008). This has been nicely illustrated in the case of wine expertise: based on common mental representations, experts are able to efficiently categorize different odors of two types of wine. It has been argued that this allows them to both organize and use their perceptual knowledge more efficiently (see Hughson and Boakes, 2001 for a review). However, some studies have highlighted that expertise did not necessarily result in enhanced perceptual capacities. For instance, Pangborn et al. (1963) investigated the effect of color on sweetness perception. The addition of color to a solution to give the appearance of a rose wine caused wine experts but not novices to judge the solution as sweeter than colorless controls. We may reasonably hypothesize that level of expertise may impact freshness perception as well, since it has been shown that the color, as well as the sweetness of beverages influence the perceived freshness (e.g., Guinard et al., 1998; Zellner and Durlach, 2003). Moreover, different sensory expectations might be triggered by different perceptual features depending on the consumer's background knowledge (Zellner and Durlach, 2003).

Another experiment conducted by Morrot et al. (2001) revealed that expertise did not necessarily improve the perceptual capacities of experts. They investigated whether wine experts could consistently associate olfactory descriptors with different types of wines. In the first session, the participants were asked to draw up a list of olfactory descriptors for a white wine and a red wine (based on a list of descriptors that was supplied to them or their own descriptors). Then, the participants had to indicate which of the two wines most intensely presented the character of each descriptor. In the second session, one week later, the same white wine previously presented in session 1 was artificially colored red with an odorless dye and the participants were asked to do the same task of comparison between the red wine and the white wine (colored red). Morrot et al. (2001)results revealed that the white wine artificially colored red in session 2 was described as a red wine in terms of olfactory descriptors by the panel of 54 experts. They suggested the existence of a "perceptual illusion" due to the color change that would influence the pairing between the different wines and their appropriate sensory descriptors.

Regarding freshness, clear color has been reported by consumers to be the most expected color for fresh food or beverages (e.g., Zellner and Durlach, 2002). Thus, it might be that white wines would be considered fresher than red wines. However, it remains an empirical question whether a change in the color of the wines would be sufficient to impact participants' perception of their relative freshness.

An alternative explanation of Morrot et al. (2001) results has been proposed by Shepherd (2012, p. 139). According to Shepherd, the participants' attention in Morrot et al.'s (2001) study was biased toward using the same descriptors for what they believed to be the same purpose. However, an important source of inter-individual variability can also partially result from the way the participants attend to a specific task (Stevenson, 2012, see Section "Attention"). The influence of attentional factors on both flavor and freshness is considered in more detail in the next section.

# ATTENTION

Two different modes of attention have been described by Posner (1980): exogenous and endogenous modes of attention. The exogenous mode of attention is underpinned by the salience of certain environmental features, whereas the endogenous mode corresponds to relatively voluntary mechanisms sensitive to learned events. Alleged influences of exogenous, and to a lesser extent endogenous, attentional processes have been suggested for the mouth capture phenomenon occurring in flavor perception. Further investigation is needed to conclude whether the mouth capture phenomenon can be generalized to all instances of flavor such as freshness.

#### Exogenous Attentional Processes

Stevenson et al. (2011) manipulated exogenous attention by varying the stimulus characteristics so that either oral or nasal cues became more salient. Their results showed that the participants were prone to shift the localization of the percept toward the physical locus of the more salient cue. Furthermore, Stevenson (2012) has highlighted that the contribution of

olfaction to flavor goes largely unnoticed, compared to that of taste and somatosensation, even when participants are asked to detect its presence. Hence, he suggested that exogenous attentional processes may be more important in inducing oral localization than endogenous attentional processes. Therefore, because exogenous attention in turn depends on the stimulus characteristics, we may reasonably expect differences between freshness and other instances of flavor, regarding their subjective location. In the case of beverages, the trigeminal component is likely one of the main contributors to freshness, due to the processing of coldness by the trigeminal cold-sensing neurons for instance (Labbe et al., 2009a). This is not necessarily the case, or at least not to a similar extent, for all types of flavors (e.g., strawberry). We can thus wonder whether the trigeminal component may bias the subjective location of freshness.

Moreover, it has been shown that exogenous attentional processes can also have perceptual consequences regarding the way participants pay attention to the task (e.g., according to the instructions they receive). For instance, while a sweetness enhancement phenomenon can occur after a single co-exposure between a sweet taste and a novel odor, Prescott et al. (2004) have observed that the adoption of a synthetic perceptual strategy during the co-exposure would also be necessary to produce sweetness enhancement. They observed that participants generally used a synthetic perceptual strategy when they were asked to rate the overall flavor intensity of the stimulus. By contrast, when the participants were asked to rate the intensity of several perceptual features of the stimulus separately, they tended to rely more on an analytical perceptual strategy. Sweetness enhancement would thus be dependent on the number and type of rating scales provided to participants. This phenomenon has been called the "halo-dumping" effect; in the case of few rating scales provided, participants tend to dump their ratings for a perceptual feature for which no response scale has been provided (e.g., the intensity of a fruity odor) onto another perceptual feature for which a response scale has been provided (e.g., the sweetness of the fruity odor). However, some studies have provided data that discredit the hypothesis of a systematic halo-dumping effect. For instance, Nguyen et al. (2002) have shown that the odor-taste enhancement effect could occur even when multiple scales are presented.

#### Endogenous Attentional Processes

Endogenous attention enables us to extract relevant information relatively early from a rich and complex stimulus environment. That is, stimuli are better processed, in terms of response time and accuracy, when they are anticipated. For instance, the results obtained by Ashkenazi and Marks (2004) have highlighted that endogenous attention can improve the detectability of the gustatory flavorant "sucrose" but not the olfactory flavorant "vanillin". The authors suggested that olfactory stimuli would be already fully processed before attention is directed toward them, reducing the functional consequence of endogenous attention. When consuming a beverage that consumers expect to be fresh, endogenous attention may be shifted toward a specific locus in the mouth or nose where consumers expect a freshness experience.

To summarize, it clearly appears that the subjective location of a final percept and the respective contributions of each of its sensory contributors is dependent on several parameters that belong to both the perceptual level (e.g., modulating the intensity of some perceptual features) and higher order cognitive levels such as memory and both exogenous and endogenous attentional processes. Starting by investigating the particular influence of crossmodal interactions and correspondences that can occur in freshness perception will allow us to generate new empirical evidence in terms of the multisensory processes at stake. Nevertheless, it is important to bear in mind that these complex multisensory processes are also dependent on conditions that influence the degree of integration. Two major documented conditions are reviewed in the next section.

# PARTICULAR CONDITIONS THAT INFLUENCE THE DEGREE OF INTEGRATION

#### Congruency

The term congruency was previously evoked in Section "Synesthetic Experience" to highlight some cognitive processes that occur in flavor representations, giving rise in some cases to crossmodal enhancement phenomena. The existing literature on congruency suggests that several types of congruency can occur at different levels but also that the characterization of congruency is generally vague and even sometimes circular. For instance, Schifferstein and Verlegh (1996) defined congruency as the extent to which two stimuli are appropriate for combination in a food product. According to Shepherd (2012), congruency can be defined as the extent to which two stimuli complement each other. Lim and Johnson (2012) suggested a statistical account of congruency according to which congruency would correspond to the extent to which two stimuli commonly appear together and thus are highly associated in a food. Small and Prescott (2005)suggested a distinction using the term "perceptual congruency" that can be observed, according to the authors, when sniffed odors elicit descriptions of qualities that are more usually associated with basic taste qualities and so may arise as a result of repeated pairing (the terms "contiguity" and "synchrony" are also used by Small and Prescott, 2005). This definition of perceptual congruency is close to that of perceptual similarity used by some researchers when referring to perceptual qualities that are alternatively attributed to different compounds. Lim and Green (2007) conducted several studies based on spatial discrimination between capsaicin and QSO4, which can both induce bitterness and burning sensations on the tongue. They concluded that perceptual similarity is a notion that also refers to functions shared by the different compounds in terms of the perceptual consequences they induce.

Another distinction has been suggested by Spence (2011, p. 972), who argued that crossmodal correspondences may be used by humans "along with spatiotemporal and semantic

congruency to help solve the crossmodal binding problem" (i.e., determining which of the many simultaneous afferent stimuli in different modalities should be bound together). The use of "along with" suggests that congruency is something different from, though related to, crossmodal correspondences. According to Spence (2011), perceptual congruency can refer to spatial and/or temporal co-occurrence during multisensory integration, in contrast with another type of congruency, "semantic congruency" that refers to the situations in which pairs of stimuli presented vary in terms of their identity and/or meaning. This definition of semantic congruency has also been used together with alternative terms such as appropriateness and/or compatibility effects between the stimuli (e.g., Piqueras-Fiszman and Spence, 2011) as well as the notion of consistency (see Spence, 2011).

These various ways to define congruency have created some confusion between the very notion of congruency and the proper mechanism of crossmodal correspondences. In particular, it has led some researchers to interpret their results in terms of a certain degree of congruency between pairs of stimuli without considering the cognitive mechanisms at hand. For instance, White and Prescott (2001) concluded that simultaneous orthonasal presentation of "congruent" odors (strawberry) was found to shorten RTs for sweetness taste recognition, relative to "incongruent" odors (grapefruit). However, sweet taste and strawberry odor are often co-experienced from childhood and this association is actually learned through repeated exposure. We suggest that the effect on RTs observed by White and Prescott (2001) seems to result from a typical case of crossmodal correspondence, built from past experience of the regular association of sweet taste and strawberry odor, rather than only congruency between these two perceptual features.

In order to clarify the difference between the phenomenon of crossmodal correspondences and the condition of congruency, two types of congruency that appear to occur at two different levels will be distinguished: (i) the perceptual congruency that refers to the spatial and/or temporal co-occurrence of two or more stimuli during multisensory integration, and (ii) the semantic congruency occurring at a higher cognitive level that helps to determine whether or not two or more stimuli are compatible or consistent in terms of identity and/or meaning.

For instance, in the case of freshness perception in beverages, consistency effects could be assessed by measuring the consequences of presenting the sound of a liquid containing bubbles poured in a glass with either a low or high pitch, and a picture of a glass containing either small or big bubbles. If, as we hypothesize, high-pitched sound is consistent with small bubbles and low-pitched sound is consistent with big bubbles, measurable effects can be obtained such as shorter RTs in consistent blocks, highlighting a positive consistency effect. This type of result can provide evidence of a crossmodal correspondence phenomenon occurring between particular perceptual features that influence freshness perception.

#### Unity Assumption

Besides congruency, "the unity assumption" is another condition that the degree of multisensory integration depends on. As for congruency, the very notion of the unity assumption remains vague and ambiguous. Various ways of understanding the unity assumption have been proposed. According to research on this concept (Welch and Warren, 1980; Welch, 1999), the unity assumption corresponds to the observer's assumption that two or more sensory inputs refer to a single distal event. According to Welch and Warren (1980), the strength of the unity assumption is a function of the number of physical properties (e.g., time, space, temporal patterning, number, shape, size) that are redundantly represented in the stimulus situation, as well as the relative weighting assigned by the observer to these properties. Based on Welch and Warren's (1980) definition of the unity assumption, Vatakis and Spence (2007) have proposed that whenever two or more sensory inputs are highly consistent in one or more dimension(s) (linked to the physical properties of the stimulus as well as influenced by top–down factors such as the semantic content), observers will be more likely to process them as referring to the same underlying multisensory event rather than as referring to separate unimodal events.

There are at least two ways to characterize the unity assumption: on the one hand, the bottom–up approach includes the physical properties linked to the stimulus, such as spatial and/or temporal co-occurrence which actually overlaps with a certain way of understanding congruency (Spence, 2011). On the other hand, some authors have highlighted that it remains unclear whether the unity assumption refers more to a top– down (i.e., more cognitively mediated) or rather to a bottom–up (i.e., more stimulus-driven) process (see Welch and Warren, 1980; Welch, 1999; Spence, 2007, on this point). The top– down approach to the unity assumption also overlaps with a certain way of understanding congruency since the higher cognitive level of congruency (i.e., semantic congruency) would also serve to solve the crossmodal binding problem (Spence, 2011). It should be noted that it also remains unclear whether the process of unification occurs consciously or unconsciously (Bertelson and de Gelder, 2004; Spence, 2007). Then, two main types of unity assumption will be distinguished: (i) the unity assumption as a bottom–up influence in terms of perceptual priors and (ii) the unity assumption as a top– down influence in terms of beliefs. Both types of unity assumption will influence the integration process and will in some cases lead the observer to consider different afferent sensory inputs to refer to the same multisensory event or object.

As was mentioned earlier, flavor is generally presented as a unified percept resulting from the integration of all senses. This implies that each flavor representation will depend on the unity assumption of the observer concerning the multimodal object experienced. Even though the particular influence of such an assumption has not been empirically explored yet for the perceptions of flavor and freshness, it is reasonable to think that it can modulate the degree of integration in the case of such multisensory perceptions. Considering flavor or freshness as unified percepts or not has various consequences regarding the overall product experienced such as different intensity judgments or different subjective locations of the final percept.

## CONCLUSION

The present review aimed at exploiting the existing literature on flavor to characterize the perceptual and cognitive mechanisms that underlie the multisensory perception of freshness, in the case of beverages. We have hypothesized that some questions about flavor remained open until now because flavor is a generic term, and in some circumstances its genericity blurs the representational nature of particular instances of flavor such as freshness, as well as the corresponding cognitive mechanisms at hand. In fact, flavor has been characterized in various ways during the past twenty years: it has been suggested that flavor representations could correspond to synesthetic experiences or result from different processes leading to object representations or categories. Given the complexity of the various perceptual and cognitive mechanisms that underpin flavor perception, the main purpose of researchers has been to obtain empirical evidence regarding the respective contributions of each sensory modality. There is a general consensus on the fact that (i) retronasal smell is the main sensory contributor to the experience of flavor even if its implication is generally not consciously perceived, and (ii) the localization of flavor perception has an illusory component due to the mouth capture phenomenon.

Freshness perception has also been described as the result of a multisensory integration and it reasonably corresponds to a particular instance of flavor, underpinned by similar cognitive mechanisms. Similarly to flavor, it appears that freshness perception is characterized by a hybrid content, both perceptual and semantic. In fact, a semantic ambiguity is present regarding freshness since it can alternatively refer to particular sensory stimulations (e.g., coldness, sourness, menthol odor) as well as different characteristics linked to the age or crispness of fruits or vegetables. Due to these different meanings to which freshness can directly refer, it supports a higher degree of specificity compared to flavor. However, the question of the function of flavor objects in the mouth has appeared as a tipping point which has highlighted that considering a particular instance of flavor facilitates the identification of particular functions (e.g., the alleviation of mouth dryness in the case of freshness). Moreover, it is important to bear in mind that freshness potentially differs from flavor with respect to (i) the sensory modality that constitutes its main contributor, (ii) its subjective location (e.g., mouth capture in the case of flavor), and (iii) the typology of its modulating factors.

In order to characterize the perceptual and cognitive mechanisms that underlie the experience of freshness, we focused on the case of crossmodal interactions and correspondences

(see Section "Crossmodal Interactions and Correspondences") which can induce measurable effects such as shorter RTs (e.g., faster product categorization) and crossmodal enhancement phenomena resulting from the interaction between two or more sensory inputs under certain circumstances. A distinction has been introduced between the crossmodal correspondences that can occur between two sensory modalities at a perceptual level and the correspondences that involve higher order cognitive levels, in the case of interactions between perceptual and semantic features that we characterized as crosslevel correspondences. This is of particular interest from an applied perspective since freshness features are part of the consumers' sensory expectations and likely determine food and beverage acceptance and appreciation. Indeed, knowing how to trigger the mechanisms of crossmodal or crosslevel correspondences regarding freshness could facilitate consumers' categorization of a given product as being fresh or even lead to freshness enhancement. However, crossmodal correspondence mechanisms still remain to be explored in freshness perception. To enhance the freshness perceptual experience, it is important (i) to identify the specific perceptual features contributing to freshness perception and to specify their respective weights, according to the stimulus context considered (e.g., beverages), and (ii) to obtain empirical evidence of the different types of correspondences that occur regarding freshness perception.

This approach must also consider the impact of top–down influences such as memory, expectations, and background knowledge on freshness, similarly to what has been thoroughly investigated in flavor perception (see Sections "Memory, Expectations, and Knowledge" and "Attention"). For instance, regarding freshness in the case of beverages, the positive learned associations following a particular drink consumption will be stored in memory and will influence the subsequent experiences.

#### REFERENCES


The multisensory processes potentially leading to crossmodal enhancement or crossmodal correspondences are function of particular conditions such as the different forms of congruency. Two different types of congruency have been distinguished in this review: on the one hand, the perceptual congruency that refers to the spatial and/or temporal co-occurrence between two or more stimuli during multisensory integration and on the other hand the semantic congruency occurring at a higher cognitive level that helps to determine whether or not two or more stimuli are consistent in terms of identity and/or meaning. The concept of the unity assumption, which has been defined as the condition under which different afferent sensory inputs are processed as referring to the same multisensory event or object, has been analyzed. The fact that there are two ways of interpreting the unity assumption in the literature has been underlined: on the one hand the unity assumption as a bottom–up influence in terms of perceptual priors and on the other hand, the unity assumption as a top–down influence in terms of beliefs.

Although the majority of these particular multisensory processes have been reported in flavor perception, they still remain to be investigated regarding particular instances of flavor such as freshness. From our analyses and the conceptual distinctions that have been introduced, we propose a model of freshness perception that will pave the way for further empirical research in the food and beverage domain, and more precisely on flavor and freshness perception (**Figure 2**).

### AUTHOR CONTRIBUTIONS

JR, JL, and MA contributed to the theoretical elaboration, organization of the structure of the arguments, reviewing of the full article. JR and JL wrote most parts of the article.

Driver (Oxford: Oxford University Press), 141–178. doi: 10.1093/acprof:oso/ 9780198524861.003.0007


changing the sonic properties of the soundtrack playing in the background. Food Qual. Prefer. 24, 201–204. doi: 10.1016/j.foodqual.2011.08.009


Dretske, F. (1988). Explaining Behavior. Cambridge, MA: MIT Press.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Roque, Auvray and Lafraire. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Visual Search for Wines with a Triangle on the Label in a Virtual Store

Hui Zhao<sup>1</sup> , Fuxing Huang<sup>1</sup> , Charles Spence<sup>2</sup> and Xiaoang Wan<sup>1</sup> \*

<sup>1</sup> Department of Psychology, School of Social Sciences, Tsinghua University, Beijing, China, <sup>2</sup> Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom

Two experiments were conducted in a virtual reality (VR) environment in order to investigate participants' in-store visual search for bottles of wines displaying a prominent triangular shape on their label. The experimental task involved virtually moving along a wine aisle in a virtual supermarket while searching for the wine bottle on the shelf that had a different triangle on its label from the other bottles. The results of Experiment 1 revealed that the participants identified the bottle with a downward-pointing triangle on its label more rapidly than when looking for an upward-pointing triangle on the label instead. This finding replicates the downward-pointing triangle superiority (DPTS) effect, though the magnitude of this effect was more pronounced in the first as compared to the second half of the experiment, suggesting a modulating role of practice. The results of Experiment 2 revealed that the DPTS effect was also modulated by the location of the target on the shelf. Interestingly, however, the results of a follow-up survey demonstrate that the orientation of the triangle did not influence the participants' evaluation of the wine bottles. Taken together, these findings reveal how in-store the attention of consumers might be influenced by the design elements in product packaging. These results therefore suggest that shopping in a virtual supermarket might offer a practical means of assessing the shelf standout of product packaging, which has important implications for food marketing.

#### Edited by:

Anton Nijholt, University of Twente, Netherlands

#### Reviewed by:

Alan M. Wing, University of Birmingham, United Kingdom Alejandro Catala, University of Twente, Netherlands

#### \*Correspondence:

Xiaoang Wan wanxa@mail.tsinghua.edu.cn

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 21 August 2017 Accepted: 29 November 2017 Published: 13 December 2017

#### Citation:

Zhao H, Huang F, Spence C and Wan X (2017) Visual Search for Wines with a Triangle on the Label in a Virtual Store. Front. Psychol. 8:2173. doi: 10.3389/fpsyg.2017.02173

Keywords: visual search, virtual reality, triangle, wine labels, DPTS effect

# INTRODUCTION

Product packaging constitutes a powerful marketing tool (see Spence, 2016, for a recent review). It allows manufacturers and marketers to deliver important information regarding the product, attract consumers' attention, and ultimately influence their evaluation of a product via various specific features, including color, shape, curvature, label, typeface, and so on (for comprehensive reviews, see Hine, 1995; Simmonds and Spence, 2017). As for food and drink purchases, consumers have their first sensory contact with the products via what they see. Visual cues also provide a dominant sense as far as the generation of product expectations are concerned (Schifferstein et al., 2013; Piqueras-Fiszman and Spence, 2015). Packaging not only affects how food and drinks are perceived at the point of sale (e.g., Becker et al., 2011; van Rompay et al., 2016), but can also influence how they are experienced at the point of consumption (Schifferstein et al., 2013; Spence, 2016).

**63**

Over the years, many studies have been designed to examine how the attention of consumers in-store is affected by the design elements in product packaging (e.g., Schoormans and Robben, 1997; Underwood et al., 2001; Pieters et al., 2010; Otterbring et al., 2013). In the store setting, of course, each and every product on the shelves competes for the consumer's limited visual attention. Nevertheless, according to Hoyer (1984), it only takes a few seconds for a customer to make their decisions when buying fastmoving consumer goods (FMCGs). Perhaps unsurprisingly, it has been argued that the more attention a customer pays to a product, the greater the likelihood that they will choose it (see Orquin and Loose, 2013, for a comprehensive review). Compared to changing the appearance of the product itself, changing the packaging and labeling of the product may well constitute a more costefficient, but no less effective means of attracting the attention of customers.

Purchasing wine constitutes a particular challenge (relative to other product categories). In fact, searching for a specific brand/vintage in the wine aisle can be both difficult and challenging, given the often complex and ever-changing range on products one finds for sale on the wine shelves. The appearance of the bottles, especially the design of the wine labels, might be expected to influence a consumer's liking of, or preference for, the product (Cutler, 2006; Boudreaux and Palmer, 2007; Labroo et al., 2008; De Mello and Pires, 2009; Westerman et al., 2013; Gmuer et al., 2015). What is more, it can also guide customers' attention in searching for the product presented on the store shelves (Thomas and Pickering, 2003; Elliot and Barth, 2012).<sup>1</sup>

The visual search task has been one of the most common experimental paradigms used to study attention (e.g., Nothdurft, 1999; Hopf et al., 2000). In visual search, certain shapes are easier to find than others. For example, a circle with a straight line like a "Q" is easier to find amongst regular circles than vice versa (Treisman and Souther, 1985; Treisman and Gormican, 1988). Such research show that it is easier to search for the presence of a feature than for its absence. Searching for a downwardpointing triangle among upward-pointing distractor triangles has also been shown to be faster than vice versa (Larson et al., 2007). The latter effect is known as the Downward-Pointing Triangle Superiority (DPTS) effect. The existence of this effect has been attributed to the fact that downward-pointing triangles are more likely to convey threat-related information than are upwardpointing triangles. It has been suggested that this is because they resemble angry faces in which the muscles are pulling down to form a "V" shape (Larson et al., 2012; Toet and Tak, 2013), and therefore capture attention more readily than do neutral stimuli such as upward-pointing triangles (see Larson et al., 2007, 2012; Watson et al., 2012).

It is widely acknowledged that the rapid detection of threatrelated stimuli conveys an evolutionary advantage and is thus vital to human survival. The "Shape of Threat" account of the DPTS effect is also supported by neuroscience evidence showing that viewing downward-pointing triangles increases the neural response seen in the amygdala (Larson et al., 2009). This neural structure plays a crucial role in emotional processing, especially in the detection of potential threats (e.g., Morris et al., 1996; LeDoux, 2000).

It is important to note, however, that this threat-related explanation of the DPTS effect might not easily be used to account for the comparable pattern of results documented with non-threatening images of triangular-shaped foods and pizza packaging (Shen et al., 2015). Considering the global precedence in visual perception (e.g., Navon, 1977; Pomerantz, 1983), one possibility is that the global outline shape of the stimuli (i.e., the downward-pointing triangle which could be threat-related) might be processed before the meaning of the stimuli (food or food packaging which are not threat-related). Nevertheless, the DPTS effect has also emerged with wine bottles that have triangular shapes on their labels whereby the triangles were only local features (Zhao et al., 2016). What is more, compared to foods presented in triangular form, or food packaging which is often stacked horizontally, the wine labels attached to the front of wine bottles and typically presented vertically. This means that the triangles appearing on the wine labels may actually point downward or upward in both daily life and experimental research (see Shen et al., 2015, for an example of a downward-pointing triangle actually being used in a commercially successful wine label). By contrast, foods or food packaging in triangular form usually point toward or away from the observer in daily life, but they were presented vertically in Shen et al.'s (2015) experiments. Nevertheless, it remains unclear whether a similar DPTS effect would also emerge with consumers in a store setting. Here it is important to note that conducting field studies (e.g., in a supermarket) can be both expensive and time-consuming. By contrast, using virtual reality (VR) to simulate a store experience offers a potentially efficient and effective means of keeping the costs of research down and can be argued to increase the external validity of one's research relative to other computer-monitor based lab experiments.

A computer-mediated three-dimensional environment can be created in VR in which users can experience a sense of presence, and which provides realistic visual scenes, sense of movement, and so on (e.g., Biocca, 1992; Steuer, 1992; though see also Gallace et al., 2012). In particular, VR can simulate naturalistic environments via an immersive human–computer interaction (Berneburg, 2007; Lee and Chung, 2008; Bressoud, 2013). It can also be used to efficiently deliver realistic food cues to elicit craving (at least as indexed by self-report and salivation) just like actual food does (Ledoux et al., 2013; though see also Spence, 2011). Intriguingly, virtual stores can elicit similar consumer choice of food products such as milk and cookies as those that have been documented in a physical store (van Herpen et al., 2016; see also Waterlander et al., 2015), and have been used to examine purchases of alcoholic drinks, such as beer (Bigné et al., 2016).

In the present study, we wanted to examine whether the DPTS effect would emerge in the visual search for bottles of wine in a virtual supermarket. Specifically, three questions were addressed by the research reported here: First, is the visual search for a wine bottle with a downward-pointing triangle on its label in a virtual supermarket significantly more efficient (that is, faster

<sup>1</sup>Here it is worth noting that many wine producers also use the weight of the bottle to try and convey product quality too (see Piqueras-Fiszman and Spence, 2012).

and/or more accurate) than when the target bottle had a triangle point upward? Second, considering that products presented on the top shelf typically receive more attention than those placed lower down (e.g., Chandon et al., 2009; though, see van Herpen et al., 2016, for the opposite pattern in consumers' choice of milk and cookies), we also examined whether the DPTS effect in the visual search for wines might be modulated by the position of the target bottle on the store shelf. Third, how does the orientation of the triangular shape on the wine label influence people's rating of the bottles themselves, and is their preference for the item related to the efficiency of their visual search performance for it?

# EXPERIMENT 1

Experiment 1 attempted to replicate Zhao et al.'s (2016) recent results using pictures of wine shelves in a virtual supermarket. The participants were invited to come to a psychology laboratory that was equipped with a head-mounted display (HMD) VR system, and to immerse themselves in a virtual supermarket to perform the visual search in a wine aisle.

# Methods

#### Participants

Twenty-four Chinese participants (mean age = 21.08 ± 1.95 years, ranging from 18 to 27 years; 12 females and 12 males) were recruited from the subject pool of the Applied Cognition Laboratory of Tsinghua University, Beijing, China.<sup>2</sup> All of the participants reported having normal or corrected-to-normal vision. Each participant was compensated with 35 Chinese Yuan (CNY) for taking part in the study. The present study was approved by the ethics committee of the Psychology Department of Tsinghua University, and conformed to the ethical standards for conducting research established by the American Psychological Association.

#### Apparatus and Virtual Displays

Both of the experiments reported in the present study were conducted at the VR Laboratory at the Department of Psychology of Tsinghua University. An NVIS nVisor SX60 HMD VR system was used, which produces stereoscopic imagery at 60 frames per second per eye. The display for each eye had a resolution of 1280 (horizontal) × 1024 (vertical) pixels, and the optical field for each eye was 44◦ (horizontal) × 35◦ (vertical). **Figure 1** shows the screenshots from the left eye of the HMD. The participants used a Logitech F710 wireless gamepad to interact with the virtual environment, while the Vizard software (by WorldViz) was used to conduct the experiment and record the data.

The immersive virtual environment (IVE) in this study consisted of a supermarket created with the Autodesk 3Ds Max 2009 software. The whole supermarket was 20 m long, 15 m wide, and 5 m high, though the participants in the present study

were only allowed to move along one of the aisles (7.6 m long and 1.6 m wide) with 4-shelf grayish white rack on both sides. There were a total of four racks on each side of the aisle, each of which consisted of four shelves with punch backboard. Each rack had a depth of 0.45 m. The base of the top, second, third, and bottom shelves were 1.77, 1.36, 0.94, and 0.5 m away from the ground, respectively. As can be seen in **Figure 1**, all of the bottles were presented on the top shelf. In order to make sure that all of the participants were able to see the same displays, the eye height in the virtual supermarket was set to 1.85 m regardless of the actual height of the participants. Therefore, when the participants looked at the rack, they were able to see all of the bottles (approximately 1.65 m away from where they were standing), without having to lift their heads.

Within each display, there were a total of 8 bottles, 0.15 m apart from each other. As can be seen in **Figure 2**, the bottles were made of clear glass in order to reveal the color of the liquid within (approximately crimson, beige, brown, or clear), implying that the bottles contained red wine (0.41 m in height and 0.09 m in diameter), white wine (0.41 m in height and 0.09 m in diameter), whiskey (0.32 m in height and 0.11 m in diameter), or Chinese baijiu<sup>3</sup> (0.27 m in height and 0.07 m in diameter), respectively. Each bottle had a downward- or upward-pointing triangle on its label, with fictional brand names. All of the bottles in each display presented the same wine, though one of the bottles was randomly determined to have a different label from the others. In other words, all of the trials in this experiment were the target-present trials.

In this virtual supermarket, the participants pressed a button on the gamepad in order to move along the aisle at a constant speed of 1.5 m/s, while keeping their bodies physically still.

<sup>2</sup>Zhao et al. (2016) used a within-participants design with a sample of 20 participants in their Experiment 1, and documented a significant DPTS effect of 164 ms (SD = 131 ms) with triangles on wine labels. While we set the Type I error probability of the present study at 0.05, we need at least nine participants (resulting in nine pairs of data) to obtain a power of 0.9 for the present study.

<sup>3</sup>Note that baijiu is a traditional Chinese distilled alcohol which has a clear appearance. It is often referred to as "Chinese liquor/spirits" in English or "white wine" in Chinese (see Wan et al., 2015).

By contrast, they were instructed to physically turn their body around when needed. Thus, the information regarding transition was purely based on optic flow, whereas the information regarding rotation was based on both optic flow and body senses. Such a combination of cues has been shown to efficiently provide self-motion information to the participants (e.g., Wan et al., 2010, 2012).

#### Design and Procedure

The experimental task involved trying to find the bottle with a label that was different from the others, i.e., finding the one having a downward-pointing triangle on its label among others having upward-pointing triangles, or vice versa. After finishing a practice block of eight trials, each participant completed eight blocks of eight trials each. Each type of bottle was shown in 16 trials, with half with a downward-pointing target and an upwardpointing target in the remainder. All of the trials were mixed and presented in a random order. Within each trial, the location of the target was randomly determined.

At the beginning of each experimental block, the participants stood at one end of the aisle, adjacent to the rack on their left and facing the other end of the aisle. They were instructed to press a button on the gamepad to move forward along the aisle until a red arrow pointing to the right appeared in front of them, which instructed them to turn around to face the rack on their right where a display of eight bottles was presented. The red arrow disappeared when facing the display, and then a blue rod extending horizontally from their body to the display appeared (see **Figure 1** for an illustration). When the participants were searching for the target, they rotated their head to direct the blue rod to point to the target, and then pressed another button on the gamepad to pick it up. If they made a correct response, the target bottle disappeared; if not, a red cross appeared, and a short beep was played via the helmet they were wearing in order to alert them of the mistake. After that, a red arrow pointing to the left appeared to instruct them to turn left to face the aisle and then to proceed to the next trial. This procedure was repeated until the participants reached the end of the aisle where they were instructed to turn around to face the end of the aisle from which they had started and now had to perform the task with the displays on their left. When they arrived back at the start of the aisle, which means they had finished a total of eight trials, the block ended.

#### Data Recording

During each trial, the participant's response (to select the target) was recorded. We used two dependent variables to assess the participants' visual search performance, including: (1) the responses times (namely the RTs), consisting of the response latencies (from seeing the display to starting moving), the time durations to start moving to pointing the target, and the time durations to press a button on the game pad to confirm the selection, and (2) accuracy to indicate whether the target was correctly selected.

# Results and Discussion

In this experiment, the mean accuracy of participants' responses was high (95.9%). RT data lower than 150 ms or in excess of three standard deviation of the group mean were excluded from the subsequent data analyses, which resulted in 0.91% of the data being discarded. Preliminary analyses revealed that searching for a downward-pointing target (4282 ms, 95.2%) was comparable to searching for an upward-pointing one (4402 ms, 96.5%), both Fs < 2.13, ps > 0.15. However, when we further broke down the data, we noticed that the participants' responses speeded-up as a function of the number of experimental blocks that they had completed. Therefore, we divided the eight experimental blocks into two halves. Mean RTs calculated based on correct trials (in which the participants correctly selected the target) and accuracy data for each type of target are shown in **Figure 3**.

The 2 (Target Orientation: downward- or upwardpointing) × 2 (Block Order: the first or last 4 blocks) Analyses of Variances (ANOVAs) on the RT and accuracy data revealed a significant main effect of Block Order on the RTs, F(1,23) = 13.92, p < 0.01, η 2 <sup>p</sup> = 0.38, but not on the accuracy data, F(1,23) = 0.03, p = 0.87. These results suggested that the participants responded more rapidly in the last four blocks (4232 ms, 95.9%) than in the first four blocks (4709 ms, 96.0%) with comparable accuracy in the two halves of the study. The results also revealed a marginally significant main effect on the RTs, F(1,23) = 3.14, p = 0.09, η 2 <sup>p</sup> = 0.12, that was qualified by a significant interaction between Target Type and Block Order, F(1,23) = 5.29, p = 0.03, η 2 <sup>p</sup> = 0.19. Post hoc pairwise comparisons revealed that in the first four blocks, searching for a downward-pointing target (4523 ms, 94.7%) was faster than searching for an upward-pointing target (4894 ms, 97.4%), t(23) = 2.16, p = 0.04, Cohen's d = 0.45, with comparable accuracy, t(23) = 1.26, p = 0.22, thus indicating a significant DPTS effect of 371 ms. By contrast, in the last four blocks, searching for a downward-pointing target (4228 ms, 95.4%) was comparable to searching for an upward-pointing target (4237 ms, 96.3%), both ts < 1.26, ps > 0.22.

Considering the high (i.e., near-ceiling) accuracy in this experiment, we also log-transformed the accuracy data and performed the Target Orientation × Block Order ANOVA. None of the main or interaction effects were significant, all Fs < 1.90, ps > 0.18.

Taken together, these results revealed that the DPTS effect emerged when the participants started to search for a wine bottle having a triangle in a different orientation from the others on the front label, though such an effect might be eliminated by the practice effect when the task was performed repeatedly. The emergence of the DPTS effect in the visual search for wine labels in a virtual supermarket is consistent with what has been documented in similar search performed with pictures of store shelves previously (Zhao et al., 2016), thus suggesting that wine labels, as a local packaging feature, can effectively attract consumers' attention in visual search for a product presented on the store shelves (Thomas and Pickering, 2003; Elliot and Barth, 2012). However, the attenuation of the DPTS effect with practice was not unusual given the published visual search literature (e.g., see Hoffman et al., 1983; Hock et al., 1985; Menneer et al., 2012).

It should be noted that all 8 of the bottles within each display were presented on the top shelf on one side of the aisle in our first experiment. This is relevant given that previous research has revealed that the products presented on the top shelf receive more attention than those at the bottom (Chandon et al., 2009; see also Sunaga et al., 2016). Thus, it remains unclear whether the DPTS effect observed with the bottles presented on the top shelf would also be generalized to products that happened to be presented on other shelves. Nevertheless, we were worried that it might introduce too many confounding factors (e.g., head movement, and display size, etc.) to present the bottles on the bottom shelf in the virtual supermarket. Thus, in Experiment 2, we chose to present the eight bottles within each display on two adjacent shelves, and examine the influence of target position. What is more, we also presented the displays on both sides of the aisle to simulate a somewhat more natural shopping situation. Thus far, it remains unclear whether the orientation of the triangle on the wine labels influence consumers' subjective ratings of and/or preference on the bottles of liquid. In order to address this issue, we also had the participants rate the stimuli after the visual search task of Experiment 2.

## EXPERIMENT 2

#### Methods

Twenty-four Chinese participants (mean age = 21.04 ± 2.96 years, ranging from 18 to 28 years; 12 females and 12 males) were recruited from the same subject pool as in Experiment 1. None of the participants had taken part in Experiment 1. All aspects of the methods of this experiment were identical to those reported in Experiment 1 with the following exceptions.

First, in order to examine the position effect, the eight bottles of liquid were presented on two adjacent shelves (i.e., the top and second shelves), which we refer to as the upper and lower shelves, respectively. Thus, there were 4 bottles on each shelf, 0.2 m apart from each other. In order to make sure that all of the participants saw the same displays across the two adjacent shelves, the eye height in this experiment was set to 1.72 m, which was lower than that in Experiment 1. Thus, a 2 (Target Orientation: downwardor upward-pointing) × 2 (Target Location: the upper or lower shelf) within-participants experimental design was used. In order to more naturally simulate an everyday shopping experience, we also used a virtual hand to replace the blue rod in Experiment 1 (see **Figure 4** for an illustration).

Second, in this experiment, the bottles were presented on both sides of the aisle. At the beginning of each block, the participants stood at the center of the aisle, and then moved forward to respond to the displays on their right. When they had finished with all the displays on the right and arrived at the end of the aisle, they were instructed to turn around in order to be able to respond to all the displays on the other side. Therefore, the shelf displays were always on the participant's right side in this experiment. As the participants stood at the center of the aisle, the displays were approximately 1.1 m away from where they were standing.

Third, in order to eliminate or at least reduce the practice effect observed in Experiment 1, the task in this experiment was designed to be somewhat more complicated. Specifically, the participants first had to press a button on the gamepad to identify whether the target (which had a different wine labels from all seven other bottles) was on the upper or lower shelf, then to direct the virtual hand (by turning the head) to choose the target on the selected shelf. The participant's responses to identify the shelf where the target was located and to choose the target were both recorded. Therefore, a total of four dependent variables were used to assess the participants visual search performance, including: (1) the RTs of identification (from seeing the display to pressing a button on the game pad to indicate where the target was located on the upper or lower shelf; (2) accuracy to indicate whether the shelf where target was located was correctly identified; (3) the RTs of choosing the target, consisting of response latencies, the time durations to start moving to pointing to the target, and the time durations to press a button on the game pad to confirm the selection; and (4) accuracy to indicate whether the target was correctly chosen.

Last but by no means the least, after finishing the visual search task, all of the participants completed a survey at Unipark<sup>4</sup> in order to rate the bottles (with a downward- or upward-pointing triangle in its label) presented in the visual search task as well as simple shapes of triangle (pointing downward or upward). During each trial, a picture was shown, and the participants rated the picture on the same 7-point scales as Shen et al. (2015) used, including (1) valence scale (ranging from very unpleasant to very pleasant), (2) arousal scale (ranging from very relaxing to very exciting), (3) familiarity scale (ranging from extremely unfamiliar to extremely familiar), and (4) liking scale (ranging from extreme dislike to extreme liking). At the end of the experiment, all of the participants were also asked how often they consumed and purchased alcoholic drinks in daily life (i.e., never, occasionally, sometimes, or often).

#### Results

#### The DPTS Effect in the Visual Search Task

In this experiment, the mean accuracy of participants' responses to identify whether the target was located on the upper or lower

```
4www.unipark.de
```
shelf was high (98.4% correct). After the participants had made a correct identification response, they had a high accuracy of 97.0% in responses of choosing the target. RT data (for both identification and choosing) lower than 150 ms or in excess of three standard deviation of the group means were excluded from the subsequent data analyses. This resulted in 2.7 and 2.4% of the data being discarded, respectively. Mean RTs calculated based on correct trials and accuracy data for each type of target are shown in **Figure 5**.

Considering the observed influence of practice on the DPTS effect reported in Experiment 1, a repeated-measures ANOVA – 2 (Target Orientation: downward- or upward-pointing) × 2 (Target Location: the upper or lower shelf) × 2 (Block Order: the first or last 4 blocks) was first performed on the data of Experiment 2. The results revealed a significant main effect of Block Order on the identification RTs, F(1,23) = 18.56, p < 0.001, η 2 <sup>p</sup> = 0.45, suggesting that the participants identified the shelf where the target was located more rapidly during the last four blocks (2595 ms) than during the first four blocks (2931 ms). Nevertheless, none of other main effects of Block Order or any interaction terms between Block Order and Target Orientation/Location was significant, all Fs < 2.93, ps > 0.10. Therefore, Block Order was not included in the following analyses.

Next, 2 (Target Orientation: downward- or upwardpointing) × 2 (Target Location: the upper or lower shelf) repeated-measures ANOVAs were performed on the RT and accuracy data of the identification responses. The results revealed a significant main effect of Target Location on the RTs, F(1,23) = 26.20, p < 0.001, η 2 <sup>p</sup> = 0.53, thus suggesting that the participants identified the target more rapidly when it was presented on the upper shelf (2516 ms) than when it was presented on the lower shelf (3018 ms). None of other main or interaction effects was significant, all Fs < 0.80, ps > 0.38, indicative of the absence of the DPTS effect in the identification responses.

After that, analogous analyses were performed on the RT and accuracy data of the choosing responses. The results revealed a significant main effect of Target Orientation on the RTs, F(1,23) = 7.00, p = 0.01, η 2 <sup>p</sup> = 0.23, and on the accuracy data, F(1,23) = 5.25, p = 0.03, η 2 <sup>p</sup> = 0.19. However, these main effects were qualified by the marginally significant interaction terms between Target Orientation and Target Location on the RTs, F(1,23) = 3.90, p = 0.06, η 2 <sup>p</sup> = 0.15, and on the accuracy data, F(1,23) = 4.01, p = 0.06, η 2 <sup>p</sup> = 0.15. What is more, there was also a significant main effect of Target Location on the accuracy data, F(1,23) = 4.26, p = 0.05, η 2 <sup>p</sup> = 0.16, but not on the RTs, F(1,23) = 0.60, p = 0.45. In order to interpret these interaction terms, the data were further broken down and pairwise comparisons were performed for each target location. The results revealed that choosing a downward-pointing target on the upper shelf (2223 ms, 96.1%) was faster than choosing an upward-pointing target on the same shelf (2492 ms, 96.3%), t(23) = 3.68, p < 0.01, Cohen's d = 0.76, with comparable accuracy, t(23) = 0.13, p = 0.90, indicative of a significant DPTS effect of 269 ms. By contrast, a downward-pointing target was chosen less accurately on the lower shelf (2376 ms, 96.0%) than an upward-pointing target on the same shelf (2453 ms, 99.7%), t(23) = 3.21, p < 0.01, Cohen's d = 0.18, with comparable RTs, t(23) = 0.87, p = 0.40, indicative of the absence of the DPTS effect.

Similar to Experiment 1, we once again log-transformed the accuracy data in Experiment 2, and performed the Target Orientation × Target Location ANOVAs. None of the main or interaction effects were significant on the transformed accuracy data of the identification responses, all Fs < 1.90, ps > 0.18. As for the transformed accuracy data of choosing responses, the results revealed a significant main effect of Target Orientation, F(1,23) = 5.10, p = 0.03, η 2 <sup>p</sup> = 0.18, a marginally significant main effect of Target Location, F(1,23) = 3.70, p = 0.07, η 2 <sup>p</sup> = 0.14, and a marginally significant interaction term, F(1,23) = 4.03, p = 0.06, η 2 <sup>p</sup> = 0.15. Therefore, the patterns of results we obtained with transformed accuracy data were consistent with those with raw data.

#### Ratings of Stimuli in the Downward- and Upward-Pointing Orientations

The mean ratings for each stimulus in the downwardand upward-pointing orientations were summarized (see **Table 1**). The 2 (Triangle Orientation: downward- or upwardpointing) × 4 (Stimulus Type, red wine, white wine, whiskey, or baijiu) ANOVAs on these scores revealed a significant main effect of Triangle Orientation on familiarity scores, F(1,23) = 4.26, p = 0.05, η 2 <sup>p</sup> = 0.16, but not on any of the other three scores, all Fs < 2.06, ps > 0.16. These results therefore suggest that


TABLE 1 | Ratings of valence, arousal levels, familiarity, and liking (on 7-point scales) for downward- and upward-pointing stimuli (with SDs in parentheses) in Experiment 2.

the same bottles were rated as more familiar when the triangle in its label was oriented in the downward-pointing direction than when they were shown in the upward-pointing orientation instead, with comparable ratings of pleasantness, arousal, and liking scores. The results also revealed significant main effects of Stimulus Type on valence, F(1,23) = 5.16, p < 0.01, η 2 <sup>p</sup> = 0.18, familiarity, F(1,23) = 4.93, p < 0.01, η 2 <sup>p</sup> = 0.18, and liking scores, F(1,23) = 10.80, p < 0.001, η 2 <sup>p</sup> = 0.32. None of other main effects or interaction terms was significant on any of the scores, all Fs < 1.67, ps > 0.18. By contrast, the simple triangle, when presented by itself, received lower valence scores when oriented in the downward-pointing direction than when it was presented in the upward-pointing orientation instead, F(1,23) = 4.18, p = 0.05, η 2 <sup>p</sup> = 0.15, with comparable arousal, familiarity, and liking scores, all Fs < 2.83, ps > 0.10. Taken together, these results therefore suggest that the downward-pointing triangle was considered to be less pleasant than the same stimulus oriented in the upward direction, whereas such ratings did not influence how pleasant people considered the bottles having these triangles on the labels.

Last, but by no means the least, the participants reported that they occasionally (83%) or never (17%) consumed alcoholic drinks in daily life, and they occasionally (46%) or never (54%) purchased alcoholic drinks on their own.

### Discussion

The results of Experiment 2 revealed a significant DPTS effect when the participants chose the target bottle on the top shelf, whereas no such effect was observed with the target bottle presented on the second shelf or in their responses of identifying the shelf on which the target was located. Taken together with the results of Experiment 1, we replicated the DPTS effect with the visual search for bottles of wine in a virtual shopping situation, thus suggesting that incorporating a downward-pointing triangle on a wine label might influence people's attention in situations that are more complex and realistic than laboratory-based experiments.

On the other hand, even though the participants considered the downward-pointing triangle to be less pleasant than the upward-pointing one (see also Shen et al., 2015), we found no significant difference in the pleasantness rating scores of the same bottles of wine with a triangle pointing downward vs. upward. It is worth noting that these results are inconsistent with Westerman et al.'s (2013) findings that bottles of water or vodka with several downward-pointing triangles on their labels were rated as being less liked, less appealing, and less likely to be purchased than those with upward-pointing ones. It should be noted that their study and ours used different types of drinks, different label designs, and different groups of participants, all of which might contribute to the discrepancy between the rating results (see also Spence, 2012). Most importantly, several small triangles were presented on the left or right of the label in their study, which makes the triangles a more salient feature of the packaging, whereas we only presented one distinctive triangle on the center of the wine labels. Taken together, these results also suggest that people's ratings of the bottles with triangles on the labels might be modulated by many other factors, such as the contextual information.

#### GENERAL DISCUSSION

In the present study, we ran two VR-based experiments to examine the influence of having triangles (downward or upward oriented) on wine labels on people's visual search for wines. The participants walked along the wine aisle in a virtual supermarket and searched for the wine bottle on the shelf that had a different triangle on its label from the other bottles. Generally speaking, the results of both experiments revealed that, if anything, choosing a bottle with a downward-pointing triangle on its label was faster than when it had an upward-pointing triangle on the label instead. It should be noted that these results are consistent with Zhao et al.'s (2016) findings with images of store shelves. Once again, we replicate the DPTS effect with naturalistic stimuli where the triangle was only a local feature in a simulated shopping situation.

These results cannot easily be interpreted by the "Shape of Threat" account proposed by Larson et al. (2007, see also Larson et al., 2012; Watson et al., 2012), for at least two reasons. For one, the bottles of wine with downward-pointing triangles on their labels were not rated as being any more unpleasant than those with upward-pointing triangles. Due to the lack of the

associations between the bottles of wines with a downwardpointing triangle in their labels and negative subjective emotional ratings, the facilitated search for these bottles of wines may not be simply attributed to the affective features of the stimuli. For another, these wine bottles that the participants were asked to choose in a virtual store has no relation to threat of any kind whatsoever. Alternatively, the DPTS effect observed with non-threatening images (Shen et al., 2015; Zhao et al., 2016) and objects, as in the present study, are more in line with the possibility that the downward-pointing triangles more readily capture people's visual attention because of particular perceptual features, such as the lack of stability and people's expectations of the consequences of that perceived instability. Previous studies have shown that unstable shapes might be ascribed feelings of fear (Pavlova et al., 2005), while unstable-looking logos might be used to infer the presence of unsafe conditions (Rahinel and Nelson, 2016).

The results of the present study also revealed a position effect in visual search for the wines. On the one hand, the participants were faster to identify the target when it was presented on the top shelf than when it was presented on the second shelf. On the other hand, the position of the target shelf also modulated the DPTS effect. These results are in line with other position effects whereby, for instance, products presented at a higher location in the rack tend to attract more attention (Chandon et al., 2009; though see also Sunaga et al., 2016). What is more, it should be noted that consumers might use in-store displays as a source of information when trying to assess the quality and likely price of the products (Valenzuela and Raghubir, 2009). Here it is worth noting that products on higher shelves are generally assumed to be of better quality and have higher price than those placed on lower shelves (Valenzuela et al., 2013; Valenzuela and Raghubir, 2015). Therefore, in the present study, wines presented on the top shelf might attract more attention and are considered to be better, which might be important factors for the DPTS effect to emerge. It is worth noting that the vertical location of an object might also be associated with stimulus valence (for a recent review, see Cian, 2016). For example, negative words are recognized more rapidly when placed at the bottom of a computer screen, whereas positive words are recognized more rapidly when they placed at the top of the screen, suggesting an association between "being higher" and "being better" (Meier and Robinson, 2004). Taken together, these results suggest that having the downwardpointing triangle on the label of products which are assumed to be good might guide people's attention more efficiently, whereas it might not work if the products are not considered in this way.

There are also some straightforward limitations in the present study that should be acknowledged. First, as for a lab experiment conducted on a university campus, the participants were Chinese college students (see Henrich et al., 2010, for the discussion of biased samples) who at most, only occasionally drink or purchase alcoholic drinks. Previous research has revealed that older frequent wine consumers' evaluation of wine was more strongly influenced by brand and packaging (Mueller and Szolnoki, 2010), while the present study conducted with young inexperienced consumers might even underestimate the influence of wine labels on consumers' visual attention. What is more, it will be also interesting in future research to examine the possible crosscultural differences in the participants' ratings of the stimuli. On the other hand, it is also worth bearing in mind that those customers who purchase alcoholic drinks in stores are not necessarily the ones who drink it. For example, back in the 1950's, Cheskin (1957, see also Cheskin, 1981) reported that rounding the corners of the labels on the front of one brand of gin bottle made these more appealing to these female customers who purchase gins for their husbands to drink.

Second, in addition to the triangles on the wine labels shown in the present study, each bottle also has an upward-pointing V form in its shape because the neck of the bottle is narrower than the body, which might introduce an additional confounding factor of "global/local congruency." That is, when a bottle has a downward-pointing triangle on its label, the orientations of the triangle on the wine label and the bottle are inconsistent with each other; whereas the orientation of the upward-pointing triangle on the wine label is consistent with the bottle. Therefore, future research is called for in which the shape of the product is better controlled for. What is more, it will be also interesting to use heterogeneous distractors in other shapes on the wine labels such as circles and squares, which might be a more natural situation in daily life.

# CONCLUSION

Once again, our results demonstrate how the appearance of the bottles, in particular, the design of the wine labels, might guide consumers' attention when searching for the product presented on the store shelves (Thomas and Pickering, 2003; Elliot and Barth, 2012; Zhao et al., 2016). Changing the labels of the product might be a more efficient and effective way to attract consumers' attention than changing the appearance of the product, while increased attention might lead to greater likelihood of it being chosen (Orquin and Loose, 2013).

# ETHICS STATEMENT

This study was carried out in accordance with the ethical standards for conducting research established by the American Psychological Association with written informed consent from all subjects. The protocol was approved by the ethics committee of the Psychology Department of Tsinghua University.

# AUTHOR CONTRIBUTIONS

Each of the listing co-authors made the following contributions to the paper: HZ, FH, CS, and XW co-developed the idea for the study. HZ, FH, and XW collaboratively designed the study. HZ collected the data and conducted the data analysis. HZ, FH, CS, and XW conducted the interpretation of the data, and drafted the manuscript. All of the authors have read and approved the final version of the manuscript.

# FUNDING

fpsyg-08-02173 December 12, 2017 Time: 16:57 # 10

This research was supported by the National Natural Science Foundation of China (Grant No. 71472106) awarded to XW and Tsinghua University Initiative Scientific Research Program.

### REFERENCES


Cheskin, L. (1957). How to Predict What People will Buy. New York, NY: Liveright.


### ACKNOWLEDGMENT

The authors would also like to thank Jichengsi Guo, Hanxiang Zhou, and Kaung Myat for their assistance in the preparation of the experimental stimuli and procedure.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer AC and handling Editor declared their shared affiliation.

Copyright © 2017 Zhao, Huang, Spence and Wan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Galvanic Tongue Stimulation Inhibits Five Basic Tastes Induced by Aqueous Electrolyte Solutions

Kazuma Aoyama<sup>1</sup> \*, Kenta Sakurai<sup>2</sup> \*, Satoru Sakurai<sup>3</sup> , Makoto Mizukami<sup>4</sup> , Taro Maeda2,5 and Hideyuki Ando2,5

<sup>1</sup> School of Interdisciplinary Mathematical Sciences, Meiji University, Tokyo, Japan, <sup>2</sup> Graduate School of Information Science and Technology, Osaka University, Suita, Japan, <sup>3</sup> Lighting Business Division Lighting Equipment Business Unit Development Group, Panasonic Corporation Eco Solutions Company, Kadoma, Japan, <sup>4</sup> Microsoft Japan Co., Ltd., Tokyo, Japan, <sup>5</sup> Center for Information and Neural Networks, National Institute of Information and Communications Technology, Suita, Japan

#### Edited by:

Anton Nijholt, University of Twente, Netherlands

#### Reviewed by:

Kasun Karunanayaka, Imagineering Institute, Malaysia Takafumi Koike, Hosei University, Japan Nimesha Ranasinghe, National University of Singapore, Singapore

#### \*Correspondence:

Kazuma Aoyama aoyama@meiji.ac.jp Kenta Sakurai kenta-sakurai@hiel.ist.osaka-u.ac.jp

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 30 August 2017 Accepted: 20 November 2017 Published: 05 December 2017

#### Citation:

Aoyama K, Sakurai K, Sakurai S, Mizukami M, Maeda T and Ando H (2017) Galvanic Tongue Stimulation Inhibits Five Basic Tastes Induced by Aqueous Electrolyte Solutions. Front. Psychol. 8:2112. doi: 10.3389/fpsyg.2017.02112 Galvanic tongue stimulation (GTS) modulates taste sensation. However, the effect of GTS is contingent on the electrode polarity in the proximity of the tongue. If an anodal electrode is attached in the proximity of the tongue, an electrical or metallic taste is elicited. On the other hand, if only cathodal electrode is attached in the proximity of the tongue, the salty taste, which is induced by electrolyte materials, is inhibited. The mechanism of this taste inhibition is not adequately understood. In this study, we aim to demonstrate that the inhibition is cause by ions, which elicit taste and which migrate from the taste sensors on the tongue by GTS. We verified the inhibitory effect of GTS on all five basic tastes induced by electrolyte materials. This technology is effective for virtual reality systems and interfaces to support dietary restrictions. Our findings demonstrate that cathodal-GTS inhibits all the five basic tastes. The results also support our hypothesis that the effects of cathodal-GTS are caused by migrating tasting ions in the mouth.

Keywords: taste, electrical stimulation, electrical tongue stimulation, taste inhibition, virtual taste

# INTRODUCTION

Galvanic tongue stimulation (GTS) induces an electric or metallic taste (Stevens et al., 2008). This fact was first discovered by Sulzer in the 18th century. The technology has been used for gustatory testing in medical research and as a tool in neural science (Krarup, 1958; Von Békésy, 1964; Cardello, 1981). As GTS can induce taste without consumption of solids or liquids, this technology would be effective in providing a virtual consumption experience and in supporting dietary restrictions. Therefore, GTS is likely to be employed for virtual reality and health engineering purposes (Vazquez-Buenosaires et al., 2003; Aruga and Koike, 2015; Ranasinghe and Do, 2017). For such engineering applications, GTS should be able to preferably regulate all the five taste sensations, i.e., sweetness, bitterness, saltiness, sourness, and umami. However, the electrical taste induced by GTS is extraordinarily complex and it has been generally known as a method to induce electrical taste (Lawless et al., 2005). Conventional studies about GTS reported that an electrical taste is induced when the anode is in the proximity of the tongue (anodal-GTS) (Platttig and Innitzer, 1976; Ranasinghe et al., 2013). On the contrary, Hettinger and Frank (2009) demonstrated that GTS inhibits the taste of salt solutions when the cathodal electrode alone is placed in the subject's mouth

(cathodal-GTS). Although their studies demonstrate the inhibitory effects of cathodal-GTS on salty and bitter-salty tastes, the group did not investigate whether other basic tastes were affected. Moreover, the inhibitory mechanism was not adequately investigated.

Currently, two tentative hypothesis of the cathodal-GTS inhibitory mechanism exist: the ionic migration hypothesis and the nervous stimulation hypothesis. The ionic migration hypothesis postulates that the electrical field formed by the GTS current migrates and removes the tasting ions from the tongue (Hettinger and Frank, 2009) (**Figure 1**). In contrast, the nervous stimulation hypothesis, which is the simplest considerable alternate hypothesis, states that the electrical current disrupts nerve activation or habituates nerves. In anodal-GTS, Volta and Bujas proposed that stimulation current directly affects (may depolarize) taste cells or taste nerves (Bujas, 1971; Nakamura and Miyashita, 2016). Therefore, it is rational to consider that in cathodal-GTS, the cathodal current directly deactivates nerves or brings a negative effect. The objective of the present study is to determine the inhibitory mechanism of cathodal-GTS, and to investigate whether it influences all five basic tastes.

The gustatory sensors, referred to as the taste buds, detect chemicals in the mouth, which constitute the sense of taste. Each taste bud contains taste cells, which have receptors at the surface of the cell (Chandrashekar et al., 2006) that detect materials in aqueous solutions (Kandel et al., 2014). From this perspective, materials that generate tastes can be divided into two categories: electrolytes and non-electrolytes. If the nervous stimulation hypothesis is valid, the cathodal-GTS should inhibit the taste induced by both electrolytes and nonelectrolytes. However, if the ionic migration hypothesis is valid, then the cathodal-GTS should inhibit only the taste induced by electrolytes. As it has not been demonstrated that cathodal-GTS inhibits tastes apart from saltiness, we support the ionic migration hypothesis as the mechanism of taste inhibition by cathodal-GTS.

To demonstrate the validity of the ionic migration hypothesis, we conducted an experiment in which subjects were exposed to four types of aqueous solutions, i.e., sucrose, glycine, caffeine, and MgCl2. Sucrose and glycine exhibit sweet tastes, and caffeine and MgCl<sup>2</sup> exhibit bitter tastes. Sucrose and caffeine are non-electrolytes, while glycine and MgCl<sup>2</sup> are electrolytes (Stone and Oliver, 1969). We used only those materials that exhibit sweetness and bitterness because non-electrolytes that elicit umami, acidity, or salty tastes are unknown. For this experiment, subjects regulated concentrations of sucrose and MgCl<sup>2</sup> solutions so that the strengths of the two become equivalent to the strength of glycine and caffeine solutions, respectively. The effect of cathodal-GTS on the subject's capability to taste electrolyte and non-electrolyte solutions was investigated using these fixed concentrations. To analyze their capability to taste sweetness, subjects tested the following pairs: glycine with GTS vs. without GTS, sucrose with GTS vs. without GTS, glycine without GTS vs. sucrose without GTS, and glycine with GTS vs. sucrose with GTS. To analyze their capability to taste bitterness, subjects tested the following pairs: MgCl<sup>2</sup> with GTS vs. without GTS, caffeine with GTS vs. without GTS, MgCl<sup>2</sup> without GTS vs. caffeine without GTS, and MgCl<sup>2</sup> with GTS vs. caffeine with GTS. Subjects verbally answered whether one of the samples had a stronger taste than the other or not.

Subsequently, we investigated whether cathodal-GTS could inhibit all five basic tastes induced by aqueous electrolyte water solutions, and the relation between the strength of cathodal-GTS and the inhibitory effect on taste. Subjects were provided with five types of aqueous electrolyte solutions, namely, NaCl, MgCl2, glycine, glutamic sodium, and citric acid, with salty, bitter, sweet, umami, and sour tastes, respectively. Each type of solution was prepared in five concentrations (**Table 1**). The solution with the highest concentration in each type was termed the "adjusting sample," while the others were termed "comparable samples." In each trial, a subject first tasted the comparable sample. Then, the subject attempted to adjust the taste intensity of the adjusting sample to match it to that of the comparable sample by altering the current intensity of the cathodal-GTS.

Therefore, the contributions of this work to the field of electrical taste, taste physiology, or taste psychology are the following:


# MATERIALS AND METHODS

All the experiments complied with the safety standards approved by the local ethics research committee at the Graduate School of Information Science and Technology, Osaka University, Japan. All the participants had the experiments explained to them and signed a letter of consent. The study protocol was performed in accordance with the ethical standards laid down in the Declaration of Helsinki.

In terms of safety thresholds for electrical stimulation, we followed the safety guidelines for transcranial direct current stimulation (tDCS). Since there are no standard limitations for GTS, various stimulation strengths and durations were used in previous studies, e.g., up to approximately 55 mA but some studies described the voltage instead of the current strength (Aruga and Koike, 2015; Khan et al., 2016). tDCS is a technique for modulating brain and neuronal excitability by stimulating the head electrically. The paper published by Bikson et al. (2016) indicated that the use of conventional tDCS protocols in human trials for up to 40 min and at 4 mA has not produced any reports of serious adverse effects or irreversible injury in over 1000 subjects. GTS is similar to tDCS in that it stimulates the head electrically. Therefore, we believe that the stimulation limitations for tDCS should be employed for GTS. In this work, we employed safer limitations than that of tDCS, i.e., the stimulation current is only up to 2.5 mA and the longest stimulation duration is 30 s to ensure the safety of subjects. In fact, all subjects used up to 1 mA of current except for one subject who used a maximum current of approximately 2.3 mA.

In all experiments, we used an in-house constant current stimulator, which consists of an operation amplifier (LMC6482), MOSFET (2SK3113), and transistor (2SA1413). This circuit is driven by a microcontroller (18F2620) that receives commands from a PC.

# Effect of Cathodal-GTS on Sweet and Bitter Tasting Electrolyte and Non-electrolytes

Eight male participants enrolled in this experiment. An anodal electrode (NIPLODE, Fukudadenshi, Inc., Tokyo, Japan) was attached to the back of the neck, and a conductive wire that passed through a drinking straw was used for the cathode (**Figure 1**). The reason why we attached the anodal electrodes to the back of the neck is because this position induces white flash visual sensations and ticking sensations on the skin vanishingly compared with electrode positions in previous works (Hettinger and Frank, 2009; Aruga and Koike, 2015). During the preliminary phase of this experiment, subjects adjusted the concentrations of sucrose and MgCl<sup>2</sup> aqueous solutions so that they exhibit taste strengths identical to those of aqueous solutions of 5.0% glycine and 0.3% caffeine, respectively. Using these fixed sucrose and MgCl<sup>2</sup> samples and the 5.0% glycine and 0.3% caffeine samples, subjects were presented four types of conditional sample-pairs for each taste quality. The following pairs were presented for the

TABLE 1 | Consistency of adjusting material and comparable materials in the second experiment.


TABLE 2 | Experimental conditions.


sensation of sweetness: glycine with GTS vs. without GTS, sucrose with GTS vs. without GTS, glycine without GTS vs. sucrose without GTS, and glycine with GTS vs. sucrose with GTS. The following were presented for the sensation of bitterness: MgCl<sup>2</sup> with GTS vs. without GTS, caffeine with GTS vs. without GTS, MgCl<sup>2</sup> without GTS vs. caffeine without GTS, and MgCl<sup>2</sup> with GTS vs. caffeine with GTS. Cathodal-GTS was applied using a 1.0-mA and 2000-ms square current. In each trial, the subject tasted each of the two comparable solutions once. Each condition was repeated four times, and the 16 trials were conducted in a random sequence for each taste quality.

# Relation between the Strength of Cathodal-GTS and Inhibitory Influences on the Five Basic Tastes

Six adult male participants enrolled in this experiment. An anodal electrode (NIPLODE, Fukudadenshi, Inc., Tokyo, Japan) was attached to the back of the neck, and a conductive wire that passed through a drinking straw was used as the cathode. Solutions of glycine, MgCl2, NaCl, glutamic sodium, and citric acid were used as the taste samples.

Each solution was prepared in five concentrations (**Table 1**); the solution with the highest concentration was termed "adjusting sample," while the others were termed "comparable samples." In each trial, a subject first tasted the comparable sample. Then, the subject was asked to adjust the strength of the

taste of the "adjusting sample" to that of the "comparable sample" by using a sliding volume controller connected to the stimulator. This sliding volume was connected to a microcontroller (NXP mbed LPC 1768) which sends commands to a PC, which then sends commands to the constant current stimulator. The current strength ranged from 0 to 2.5 mA and longest current duration was 30 s. However, all subjects finished adjusting the slide volume within 30 s. Each combination of the adjusting and comparable samples for each taste quality was tested six times. Thus, each subject completed 120 trials. Owing to safety limitations, the trials for only one tasting sample were conducted in 1 day. The sequence of the trials was randomized. The sampled current intensity data were normalized for each subject by dividing the data by the maximum current intensity recorded for the subject.

The differences in conditions between the first and second experiments are shown in **Table 2**.

#### RESULTS

# Examining the Influence of Cathodal-GTS on Sweet and Bitter Tasting Electrolyte and Non-electrolytes

**Figures 2** and **3** illustrate the results for the sensations of sweet and bitter tastes, respectively.

**Figures 2A** and **3A** compare the average selection percentage between the electrolytes and non-electrolytes in eliciting a sensation of sweetness and bitterness, respectively. Statistical analyses, performed with Kruskal–Wallis analysis of variance (ANOVA) and multiple comparisons (Scheffe's method) test, on the response percentages demonstrated that the percentage of "similar level" responses were significantly higher than the others (**Figure 2A**: F2, 21 = 8.55, p < 0.05; **Figure 3A**: F2, 21 = 11.81, p < 0.05). This indicates that the strength of taste of the electrolyte and non-electrolyte samples were sensed as being similar.

**Figures 2B** and **3B** demonstrate the effect of cathodal-GTS on the sensations of sweetness and bitterness, respectively, which were elicited by the electrolytes. Statistical analyses performed using the Kruskal–Wallis ANOVA and Scheffe's method discovered that Cathodal-GTS weakened both the taste sensations (**Figure 2B**: F2, 21 = 16.28, p < 0.05: **Figure 3B**: F2, 21 = 18.01, p < 0.05). However, cathodal-GTS did not demonstrate an effect on the sweet and bitter tastes that were elicited by the non-electrolytes, i.e., sucrose and caffeine, as the percentages of "similar level" responses were significantly higher in comparison with the other conditions (**Figure 2C**: F2, 21 = 19.92, p < 0.05; **Figure 3C**: F2, 21 = 9.51, p < 0.05).

Finally, the effect of cathodal-GTS on the electrolyte and nonelectrolyte, eliciting sweetness or bitterness, are illustrated in **Figures 2D** and **3D**, respectively. Cathodal-GTS affected only the taste sensation of the electrolytes, i.e., glycine and MgCl<sup>2</sup> (**Figure 2D**: F2, 21 = 17.28, p < 0.05; **Figure 3D**: F2, 21 = 16.15, p < 0.05).

## Investigating the Relation between the Strength of Cathodal-GTS and Inhibitory Effects on the Five Basic Tastes

**Figure 4** illustrates the normalized and averaged current strength that was required to inhibit the taste of the adjusting sample so that the taste strength was similar to the strength of comparable sample. The solutions used were glycine, MgCl2, citric acid,

FIGURE 3 | Influence of cathodal-GTS on the bitterness of electrolyte and non-electrolyte aqueous solutions. These figures indicate the percentage of selection in each condition: (A) Caffeine vs. MgCl<sup>2</sup> without GTS, (B) MgCl<sup>2</sup> without GTS vs. with GTS, (C) Caffeine without GTS vs. with GTS, (D) Caffeine with GTS vs. MgCl<sup>2</sup> with GTS. Data reported as mean ± standard error of mean. Asterisks (<sup>∗</sup> ) in these figures reveal the significant differences (p < 0.05) detected by statistical analysis using Kruskal–Wallis ANOVA and multiple comparisons test (Scheffe's method).

NaCl, and glutamic sodium, which elicit sweet, bitter, sour, salty, and umami tastes, respectively. The concentration of the comparable sample was negatively correlated with the strength of the current for all the five types of electrolyte solutions. Statistical analyses performed using the Kruskal–Wallis ANOVA and Scheffe's method for each graph demonstrated significant variations between the conditions (**Figure 4A**: F3, 140 = 72.54, p < 0.05; **Figure 4B**: F3, 140 = 72.82, p < 0.05; **Figure 4B**: F3, 140 = 81.47, p < 0.05; **Figure 4B**: F3, 140 = 98.42, p < 0.05; **Figure 4B**: F3, 140 = 75.52, p < 0.05).

## DISCUSSION

The results of the first experiment demonstrate that, while cathodal-GTS does not effectively inhibit tastes induced by

non-electrolyte solutions, it inhibits those induced by electrolyte solutions. **Figures 2A** and **3A** indicate that the electrolytes and non-electrolytes elicited tastes with similar strengths. However, when cathodal-GTS was applied, it weakened the sensation of the taste elicited by the electrolytes: glycine and MgCl<sup>2</sup> (**Figures 2B**, **3B**); however, it did not adequately influence the non-electrolytes: sucrose and caffeine (**Figures 2C**, **3C**). The fact is further emphasized by the results illustrated in **Figures 2D** and **3D**.

Cathodal-GTS inhibited the sensation of sweet and bitter tastes elicited by the electrolytes, but did not effectively for nonelectrolytes. These results support the ionic migration hypothesis as the mechanism of the taste inhibition. The major dissimilarity between electrolyte and non-electrolyte aqueous solutions is the production of ions. Electrolytes become ions in solution, while non-electrolytes do not. Therefore, ionic migration occurs when a current stimulation is applied to the electrolyte solution. During ionic migration, the electrical field formed by the GTS current impels cations and anions toward the cathode and anode, respectively. The ionic migration hypothesis matches with our experimental results that only the tastes induced by the electrolyte solutions were inhibited by the electrical field.

**Figure 4** indicates that the five basic tastes induced by electrolytes were inhibited by cathodal-GTS, and that this effect is strongly correlated with the strength of the current. The correlation coefficient for each condition was 0.9 or higher. Therefore, the strength of taste sensation is highly likely to be regulated by the strength of the current.

All subjects reported that the taste strength they experienced immediately after the stimulation offset was stronger than that they experienced prior to stimulation for all the electrolyte solutions. Hettinger and Frank (2009) reported a similar phenomenon with salty and bitter-salty solutions. However, in the first experiment in our study, the subjects did not experience taste enhancement with the non-electrolyte solutions. This indicates that the taste enhancement is caused by the preceding taste inhibitory effect. We have two descriptions of the mechanism of taste enhancement. The first is the nerve adaptation hypothesis, which postulates that taste receptors and nerves adapt to the low taste-intensity environment while GTS was applied, and then, is more strongly activated by a normal taste-intensity when GTS was stopped, resulting in the taste enhancement. Another description is that the taste ions that were attracted to each pole became diffuse when GTS ceased. The taste receptors on the tongue would be stimulated at a high frequency, resulting in a temporarily stronger taste sensation. The current study, however, is not sufficient to reveal which description is appropriate.

During the first experiment, one subject for MgCl<sup>2</sup> and two subjects for glycine reported that cathodal-GTS did not inhibit taste. However, they reported the inhibition with a similar configuration on another day. On the other hand, those who continuously reported the effect reported that the strengths of the effects differ day to day. We recognize that these occurrences are results of alterations in the physical conditions of the subjects and the precise positions of the electrodes and the subjects' tongues. Especially, since glycine is zwitter ion, the state of the inner mouth (e.g., pH and so on) may hugely affect the state of ions of glycine (Tse et al., 1978).

Moreover, some subjects verbally reported that during trials for citric acid, they felt electrifying sensations on their tongues and, at the end of the sessions, they had difficulty distinguishing between the effects of GTS and the masking effect caused by the electrifying sensation. Some also reported that taste was inhibited by the ticking sensation induced by the combination of citric acid and GTS.

This study validated the hypothesis that ionic migration is the mechanism of cathodal-GTS, which inhibits all five basic tastes. Our observations contribute to the physiological understanding of how cathodal-GTS functions, and could be applied as tools for dietary restrictions by altering meal experience.

In our preparatory experiment, we confirmed that cathodal-GTS does not affect the taste elicited by Na-saccharin. The fact is also reported in the previous works conducted by Hettinger and Frank (2009). They reported the taste inhibition for saltiness but they did not consider the state of electrolytic dissociation in the mouth. For use of cathodal-GTS in VR or diet restriction, it is preferable to control all five basic taste in the meal. To achieve this, the migration of all taste materials should be controlled. It would be our future work to model the ionic migration in the mouth, which considers degree of electrolytic dissociation, molar weight, and ionic polar character.

# CONCLUSION

In this work, we demonstrated that the ionic migration hypothesis is valid explanation of mechanisms for the taste inhibitory by cathodal-GTS. We also revealed that cathodal-GTS is able to inhibit all five basic tastes which is elicited by electrolyte water solutions.

# AUTHOR CONTRIBUTIONS

KA and SS proposed the hypothesis and conducted the experiments. KA, SS, and HA designed the experiments. KS, MM, and TM assisted with the experimental setup. SS and KA collected and analyzed the data. KA, MM, and HA wrote the manuscript. All the authors discussed the results and commented on the manuscript. The works by MM and SS have done at Osaka University when they were students and are not to represent interests of Microsoft and Panasonic Corporation Eco Solutions Company.

# ACKNOWLEDGMENT

This work was supported by JSPS KAKENHI Grant-in-Aid for Young Scientists (A) Grant Number 17H04690.

# REFERENCES

fpsyg-08-02112 December 2, 2017 Time: 15:56 # 7


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Aoyama, Sakurai, Sakurai, Mizukami, Maeda and Ando. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Development and Testing of a Small-Size Olfactometer for the Perception of Food and Beverages in Humans

#### Paola Risso<sup>1</sup> \*, Mario Covarrubias Rodriguez <sup>2</sup> , Monica Bordegoni <sup>2</sup> and Alberto Gallace<sup>1</sup>

<sup>1</sup> Department of Psychology, Milan Center for Neuroscience, University of Milano-Bicocca, Milan, Italy, <sup>2</sup> Department of Mechanical Engineering, Politecnico di Milano, Milan, Italy

Studies of olfactory perception and about the way humans interact with, and perceive food and beverages require appropriate olfactory devices. Moreover, small size, and portable interfaces are needed within the context of Human Computer Interaction (HCI), to enrich and complete the design of different mediated experiences. In this paper, the authors tested a new portable olfactory device for the orthonasal administration of smells. The main aim was to verify if the experience generated by the odors delivered through such device can affect people's taste perception. Once established that people could perceive odors using the olfactory device, a group of participants was asked to taste two different types of food (Experiment 1) and three types of beverages (Experiment 2) and to evaluate them on a number of perceptual-dimensions (such as pleasantness, freshness, sweetness, saltiness, and bitterness). The participants could taste the food and the beverage without the presence of additional olfactory stimuli, or under conditions where olfactory stimuli (the smell of chocolate or citrus) were also presented using the device. The results showed that the participants' evaluation of food and beverages was significantly modulated by the concurrently presented odors. The experimental results suggest that: (1) the device is effective in controlling the delivery of odors to human participants without the complexity of management that often affect larger odors delivery systems; (2) odors administered by means of such device can have an effects on food and beverage perception, without the need to change their chemical properties.

Keywords: olfaction, taste, human machine interfaces, multisensory interaction, cognitive neuroscience, gustatory perception

# INTRODUCTION

In Human-Computer Interaction (HCI), olfaction is rarely included as interaction modality. This is, at least in part, because: (1) most of the available devices for odor presentation are extremely costly, cumbersome and not easily manageable by non-expert technicians; (2) scientific knowledge of olfactory perception is less developed as compared to other sensory modalities. The implementation of devices able to simulate a real flavor experience by means of trigeminal and tactile taste stimulation has proven to be a difficult task. Technical difficulties associated with the digitization of flavors and odors have likely contributed to the reduced commercial success

#### Edited by:

Carlos Velasco, BI Norwegian Business School, Norway

#### Reviewed by:

Maria G. Veldhuizen, John B. Pierce Laboratory, United States Marianna Obrist, University of Sussex, United Kingdom Massimiliano Zampini, Università di Trento, Italy

> \*Correspondence: Paola Risso

p.risso@campus.unimib.it

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Digital Humanities

> Received: 31 May 2017 Accepted: 05 April 2018 Published: 24 April 2018

#### Citation:

Risso P, Covarrubias Rodriguez M, Bordegoni M and Gallace A (2018) Development and Testing of a Small-Size Olfactometer for the Perception of Food and Beverages in Humans. Front. Digit. Humanit. 5:7. doi: 10.3389/fdigh.2018.00007

of the devices that have been developed so far for this purpose (see Spence et al., 2017, for a detailed review). As consequence, it is still hard nowadays to effectively use smell by means of an olfactometer device, especially under conditions of multisensory stimulus presentation (Sanders and McCormick, 1993; see Barfield and Danas, 1995; Nakaizumi et al., 2006; Nakamoto, 2012). In this work, we propose and test a new and innovative olfactory display able to present multiple odors to human participants.

A brief review follows on the state of the art of olfactory devices. In particular, the focus of the review will be on the description of the most recent miniaturized and portable olfactory devices.

# State of the Art of Portable Olfactory Devices

Generally, the term "olfactometer" is used to describe complex devices, mainly used in a laboratory setting in a standardized computer-controlled manner with determined air flow, odor concentration, odor duration, onset, and offset (Al Aïn and Frasnelli, 2017). Such devices are often custom-built prototypes, although a few commercial versions do exist. Usually, these devices are very expensive and exceed the financial budget available for the largest part of research laboratories. Moreover, because of their technical characteristics, the majority of these olfactometers are cumbersome, not practical to handle and often not easily transportable or wearable.

Recently, a series of olfactory devices designed for administering odors in less controlled and more ecological situations have been developed (see **Table 1**). For example, jewelry or parts of clothing controlled by smartphones and pc accessories can now be used for odor delivery purposes. However, all these different approaches to olfactory stimulation have to face the challenge of using invisible, extremely volatile, and persistent stimuli. Moreover, they require a sophisticated miniaturization system to adequately spread the smells in the environment.

In those cases where the olfactory devices have been miniaturized, their potential use has been rather limited. For example, Nakaizumi et al. (2006) developed a device that makes use of two air cannons to project rings of odors directly to the nose of the participants. The authors implemented four prototypes of the device and the last, presented at the SIGGRAPH'2005, is capable of launching four different odorants. However, the size of the two cannons was not specified in the original reports. Nevertheless, despite of its high level of technological implementation, such device is certainly not easily transportable, at least in its current form (also note that precise distances and positions from the cannons are needed for the device to deliver the smell correctly). By contrast, an example of a small wearable and transportable device is the Essence necklace, designed to control odor intensity, and frequency of release through a smartphone (Amores and Maes, 2017). The scent delivery is controlled by an encapsulated piezoelectric transducer, which vibrates by emitting ultrasounds. Such transducer is immersed in the odorant liquid contained in a 3D printed cover, hang on the participant's neck. The ultrasonic sounds transform the scented liquid into vapor-like particles, which immediately are distributed around the necklace. The 3D print cover can contain 7 ml of an essence, sufficient to provide 27–28 h of continuous odor volatilization. Despite of its small size, it is important to note that this prototype allows delivering only one essence at a time, limiting its field of use to a very few situations (and without considering the problem of perceptual and neural adaptation to the same odor by humans).

A portable device that allows the delivery up to six scents through the digital integration of odors is Aroma Shooter (Aromajoin Corporation, Kyoto). Such a device delivers essences up to a distance of 600 mm by using a gas ejection method that allows the control of the directional air flow. The system uses a series of scented cartridges, which can be automatically changed at intervals of 1 s and can be remotely controlled by computers and mobile devices. Also, a wearable version of the same device, with the approximate size of a small pendant (the Aroma Shooter Mini), has been developed to deliver only one essence at a time.

A small olfactory device designed, once again, in the form of a piece of jewelery is the Mist Shine, produced by Indiegogo (Indiegogo). Mist Shine is a perfume spray bottle, which can be worn by the user. Such a device allows the liquid odorant to be transformed into a micron mist using an ultrasonic unit and be delivered by simply pressing a bottom. A version of this apparatus can be connected to the mobile devices through Bluetooth technology.

The largest part of the above mentioned olfactory devices are conceived as a part of a new trend in clothing, smartphone, and pc accessories and targeted to a wide range of fashion consumers. In the same vein, the Scentsory Design <sup>R</sup> & Technology Ltd., (Scentsory Design) designed the 'Aroma Rainbow Sensitive Dress'. The idea of this conceptual dress was born from a multi-disciplinary research project, with the aim of exploring the emotional fashion through the release of odorous molecules. In practice, several colored tubes corresponding to different scents are embedded in the dress. The odors are delivered to the user and to the environment using nanotechnology-based release systems. The tubes are activated by touch screen monitors according to their different color labels.

An extreme application of the odor delivery devices is the Swallowable Parfum <sup>R</sup> by Lucy McRae (Swallowable Parfum). In this case, the olfactory device is a digestible scented capsule that after the ingestion allows the body skin to emanate a unique fragrance. The odor is excreted by the skin in the form of droplets, which emanate a fragrance determined by the enzymes that metabolize fat in the human body. In this case, no change of the delivered odors would seem to be possible after the ingestion of the capsule. However, this new technology is based on chemical rather than electromechanical mechanisms.

Some years ago, another interesting transportable/wearable olfactory device was developed by Yamada et al. (2006). The aim of the authors was at enabling a person to identify the placement of an odor source in a virtual environment by changing its strength. Their device includes: (1) an Odor Presenting Unit worn by the user; (2) a backpack containing the odor generator unit that is used to modulate the strength of the essences

#### TABLE 1 | Key features of the reviewed devices.


according to the position of the carrier, (3) a neckband connected with a tube to the odor generator unit that conveys the essence to the user's nose, (4) a little bag attached to the waist containing the system to detect the position of the carrier by means a tag reader, (5) a breath detecting sensor to ensure that odors are administered to the user during the breath inhalation phase. All the wearable components have been miniaturized to give the minimum clutter and annoyance to the wearer. In particular, the Odor Presenting Unit carries smells to the nose using a piezotype inkjet head device. The piezo element component applies a pressure pulse to the tubule filled with the odorant liquid that directly reaches the nose in the form of droplets. The response time of the inkjet head device is 300 µs. Moreover, the wearable device is monitored by an Odor-Controlling Unit composed by a microcomputer that controls the air pump and a notebook PC that calculates the strength of odor presentation according to the data sent by the user's positioning system. Besides its very high technological level and the miniaturizations of its components, this olfactory device allows only a limited series of odors to be presented and still produces some degree of clutter for the wearer.

As recently pointed out by a number of authors (Obrist et al., 2016; Spence et al., 2017), in order to obtain good results in chemosensory digitization, it is necessary to take into account both technical and psychological limitations of an olfactory display. Moreover, the effectiveness of these devices should depend on the similarity of the human responses to the stimuli elicited by the device, concerning those occurring under natural emissions of odors (e.g., see the Smell Synthesizer of the Museum of Food and Drink's MOFAD Lab; Berenstein, 2015). That would seem to suggest that novel olfactory devices will need to be tested also using perceptual and neuroscientific experiments and paradigms.

#### Olfactory Crossmodal Integration

In the last decades, psychological and neuroscientific research has shown that the different features of a product, such as its color, shape, odor, taste, tactile feel, sound and so on, are rarely processed in isolation by our neural system. Some interactions occur among them, and human's final perception is much more than a mere sum of these characteristics (see Spence, 2011, 2017). This consideration also applies to food evaluation. In this context, the perception of taste should be considered a multisensory experience, rather than a unisensory experience (e.g., Spence et al., 2013). In fact, psychological and neuroscientific research has shown that visual, auditory, olfactory and tactile aspects of food (see Spence and Piqueras-Fiszman, 2014, for a recent review) can all modulate our gustatory perception. For example, Michel et al. (2014) reported that the participants in their experiment provided higher tastiness ratings when asked to taste a salad dish visually arranged in an artistic way (similar to a Kandinsky's paint), than when the same dish was presented without any artistic arrangement. As far as beverages are concerned, Zampini and Spence (2005) reported that participants' perception of carbonation intensity of sparkling mineral water is affected by an alteration of the frequency of the auditory feedback emitted by the liquid just before consumption. In particular, amplification of the higher frequencies emitted by the beverage when served, results in the participants reporting a higher intensity of carbonation while tasting the water.

Interestingly, several studies have also demonstrated that food perception is not only affected by the different sensory qualities of the food itself, but also by those of the context where food is presented. For example, the color of a container can affect the taste of food and beverage presented in it (e.g., Zampini and Spence, 2005; Krishna and Morrin, 2008; Piqueras-Fiszman et al., 2012; Spence and Wan, 2015; Biggs et al., 2016). In particular, Risso et al. (2015) recently reported that people perceive mineral water as more carbonated when contained in a red or blue plastic cup, than when contained in a white cup. Similarly, Stewart and Goss (2013) showed that different combinations of colors and shapes in a plate could significantly modify the sweetness, flavor intensity, quality, and enjoyment perception of the dessert served in them (see also Bruno et al., 2013). Even the weight of the container would seem to affect the perception of the beverage presented in it, where heavier cups make the participant perceive the mineral water less pleasant than when served in lighter cups (Maggioni et al., 2015).

As far as the crossmodal interaction between odors and food is concerned, it was demonstrated that an odor could also elicit a taste response in a tasteless solution, besides the expected olfactory response (Burdach et al., 1984). In a series of pioneering experiments, it was observed that strawberry odor enhanced the sweetness of a sucrose whipped cream, while peanut butter odor did not enhance such perception. Moreover, strawberry odor did not enhance the saltiness of sodium chloride. Interestingly, it was also shown that the color red, differently from the strawberry odor, did not modulate the perception of sweetness (Frank and Biram, 1988; Frank et al., 1989). It was also demonstrated that "cut grass smell" defined as a "green odor," enhanced bitterness perception (Caporale et al., 2004). Similarly, it was shown that the sardine aroma enhanced salt intensity in tasteless solutions or in low-salt content solutions (Nasri et al., 2012). That is, under natural conditions of stimulus presentations, certain odors can affect the perception of food and beverages. An effective olfactometer device should then lead to similar interactive effects.

Olfaction is certainly one of the most important sensory system used by humans to evaluate gustatory stimuli. In fact, it has been shown that our taste perception decreases when olfaction is not available (e.g., Murphy and Cain, 1980). By these observations, if one's aim is at modulating people's experience of food also under ecologically valid conditions of stimulus presentation, olfactory stimuli need to be presented effectively and in a highly controlled way. In particular, smallsize transportable olfactory interfaces would allow us to present in a timely precise context specific odors, which might modulate people's food experiences and choices.

# The Multi-Fragrance Olfactory Display (MFOD)

In this paper, a novel Multi-Fragrance Olfactory Display (MFOD), which is light, small and able to release multiple fragrances, was used for the presentation of odor stimuli in two behavioral experiments. The MFOD is a multi-odor dispenser consisting of small cases that release up to eight fragrances in a precisely controlled manner.

The MFOD is based on the SFR (Solid Fragrance Release) method for the generation and the release of fragrances. The SFR consists of delivering scents through the modulation of an airflow striking a tablet of solid fragrance. This method differs from that adopted in those devices where liquid or vaporous fragrances are delivered (which in turn are more "invasive" and permeating into the environment). Also, the use of solid particles delivered through airflow allows a more precise control of the flow.

The same principle can be used for the implementation of a wearable configuration and of a desktop configuration (Covarrubias et al., 2016). In this specific research, a desktop version of the MFOD was used (see **Figure 1**). It consists of an actuated dispenser able to store up to eight fragrances and to control timing, intensity, and duration of the fragrance release. The MFOD (see **Figure 1**) includes a centrifugal fan (1), which provides and controls the airflow, a servo-motor (2), a cylindrical repository (3), and eight small tubes including compact powder fragrances (4). The servo-assisted cylindrical repository is controlled through an Arduino board (arduino.cc), which is connected to the E-Prime tool (E-Prime). E-Prime is a software environment used for design, implementation, data collection, and analysis of computer-based behavioral experiments. In this work, it was used for the selection and delivery of the fragrances. Finally, a pipe (5) releases the selected odor to the user's nostrils.

The MFOD performs three main functions.


3. Odor Delivery– The airflow generated by the fan passes through the selected tube in the fragrance repository, and the odorous airflow, generated by an erosion process, goes through the flexible plastic pipe (600 mm length and 10 mm internal diameter) and is delivered close to the user's nose (about 3 cm from the pipe).

The fragrance intensity can be adjusted by modifying the airflow generated by the centrifugal fan. The latency from generation to the perception of a selected odor is <0.5 s. This time interval was calculated on the basis of a series of pre-tests performed by setting two characteristics of the device, i.e., the fan speed and the servo motor speed.

By using a small anemometer, which is a device for measuring the air speed with a fan, some measurements were taken at several points of the prototype when varying the input voltage of the DC fan from 5 to 20 Volt, as shown in **Figures 2**, **3**.

A further test was carried out to measure the time that the servo-motor takes to rotate the cylinder at specific angles. In particular, a rotation of 60◦ is performed in 0.16∼0.13 s. The configuration with 12 Volts was adopted to produce a perception of synchrony between the delivery of the fragrance and the presentation of the stimuli.

The device used in this study extends the functionalities of previous devices and reduces their limitations (i.e., technical complexity, use complexity, and size). In particular, the device can be used to deliver up to eight fragrances. Note, however, that its current design allows up to twenty odors to be easily delivered. Moreover, the olfactometer was designed to be easily interconnected with the software for behavioral research EPrime 2.0. Therefore, the device can be integrated into complex psychological and neurocognitive studies where a high level of

delivery tube 600 mm long.

control on timing, randomization of conditions, reaction times of participants is required. Additionally, by means EPrime 2 the olfactometer can be interconnected to other scientific devices (i.e., eye tracker, skin conductance response recorders, etc.).

The device might be used in a large number of applied contexts. In fact, the prototype used in the present study has been designed to be adapted both in experimental contexts, and in less controlled and more ecological contexts. For example, it might be used to deliver odors in virtual reality environments, where it would help to enhance the sense of presence (Bordegoni and Carulli, 2016; Carulli et al., 2016). The device might also be adopted to improve the effect of exposure therapies in posttraumatic stress disorders patients (see a detailed review of Aiken and Berry, 2015 on this topic), to alleviate the effect of pain in a number of clinical conditions (Marchand and Arsenault, 2002) or to provide alerting cues to drivers (Ho and Spence, 2017). Given the large numbers of odors which can be presented with this new device, one additional application of our system is also to provide a valid alternative to the odor kits (manually administered), that are used at the moment for testing olfactory capability in different populations of participants (e.g., see the small bottles of the kit for sommelier's training 'Le Nez du vin <sup>R</sup> , Sniffin' Sticks, University Pennsylvania Smell Identification Test).

In this paper, the authors used the olfactory display in order to determine if the sensory experience generated by the device affects people's evaluation of some food and beverage characteristics (just as it occurs for odors naturally present in the environment). In particular, in the experiments presented here, the olfactory device was used to modulate people's perception of four different qualities of food (Experiment 1; pleasantness, sweetness, saltiness, and bitterness) and of beverages (Experiment 2; pleasantness, sweetness, saltiness, bitterness, and carbonation intensity).

The main aim of the present research was to investigate whether the device might be used to deliver odors under conditions of multisensory stimulus presentation and to generate perceptual multisensory interactions between the concurrently presented stimuli (food and odors). It is important to highlight that in the present experimental design, the crossmodal interaction among foods, beverages, and odors was tested by only using two odors (despite the possibility of administering up to eight odors). Adding more odor-food combinations could have affected negatively the participants' evaluations due to the multiple interactive effects likely occurring during the presentation of a large number of foods and odors in a reduced amount of time.

#### EXPERIMENT 1

The aim of the experiment is to test the effect on food perception of the orthonasal presentation of odors administered by means the new small-size olfactory device. In particular, Experiment 1 aims at verifying if the experience generated by the odors delivered through such device can affect people's taste perception of two different types of food on a number of hedonic and perceptual-dimensions (such as pleasantness, freshness, sweetness, saltiness and bitterness).

#### Materials and Methods Participants

Twenty-two participants with a mean age of 24.5 years (SD = 6.84, 12 female), took part in the experiment. All the participants gave written informed consent before their participation. The experiments described here were all performed in accordance with the ethical standards laid down in the 2008 Declaration of Helsinki and approved by the local ethical committee. The experiment lasted for approximately 20–30 min. People who claimed to be affected by any permanent or temporary olfactory or taste dysfunction, were excluded from taking part in the study.

#### Stimuli

Two different types of food were used: salty crackers (Carrefour) and lemon sugar candies (Perugina). The stimuli have been selected in order to have opposite taste characteristics (salt and sweet), without changing their organoleptic quality during the experiment (such as fresh food). Common white plastic dishes (produced by Bibo Italy S.p.A. for Carrefour) were used to serve the food. The two odors administered were chocolate and a mixture of citrus fruit (Oikos Fragrances <sup>R</sup> ). The odor selection was focused on finding two completely different and easily classifiable smells. A very important characteristic of the fragrances used is to be made up of scented powder. This important innovation adopted in our device does not require the dilution of fragrances, thus making the delivery of odors much easier under experimental procedures.

#### Procedures

Before starting the experiment, a short pre-test was performed in order to evaluate people's ability to perceive the odors by means of the experimental device. The study was conducted in an experimental booth with the participants sitting comfortably on a chair in front of a desk. The participants were instructed to grasp and eat the food presented on the desk at a signal of the experimenter, while at the same time they smelled an odor or clear air administered to both nostrils for about 15 s. The odor was administered by means of a small pipe connected to the olfactometer and held attached to the participants' nose, just between their nostrils (at about 3 cm). The method for releasing the odorant molecules, based on SFR-Solid Fragrance Release, ensured that the flow of odorant molecules effectively reached the participant's nostrils. This method, as compared to the volatilization of liquid compounds, results to be more effective in preventing the dispersion of part of the odor molecules in the surrounding environment. Given that the process that leads to olfactory perception of the airflows dynamics and allows peripheral processing of the signal, is extremely complex and delicate, the experimenter avoided inserting the pipe directly into the participant's nostrils. In fact, the insertion of the tube into the participants' nostrils could have prevented an effective perception of the odor presented. As a consequence of this presentation modality, some odorant molecules administered through the MFOD in our experiment may likely have reached people's mouth as well as the nostrils, and thus be also processed by retronasal route. However, it should be considered that this is the natural condition in which human perceive orthonasal olfactory stimulation in everyday situations.

The administration modality and timing of the stimuli in Experiment 1 were defined using some pre-tests. This procedure assured that all participants started to smell the odor before grasping the food and bring it to their mouth. Such a precaution was introduced to be sure that people chewed and swallowed the foods, while they were smelling the odor. The device was contained in a box that was hidden from the participant's view.

The participants were informed that after tasting the food they had to rate it along four dimensions (pleasantness, sweetness, saltiness, and bitterness) by means of a 150 mm long visual analog scale (VAS), anchored with the terms "not at all" and "very much." The VAS was presented at the center of a 17′′ PC screen. The participants used the mouse to select the point on the scale that best represented their evaluation. Each food was presented three times for each olfactory condition, for a total of 18 (2 foods × 3 odors × 3 repetitions) samples of food to be evaluated by each participant. The presentation of the food and the odors were completely randomized.

#### Results

The mean participants' judgments along the VASs were submitted to a series of repeated measures ANOVAs for the hedonic dimension of Pleasantness, as well as for the dimensions of Sweetness, Saltiness and Bitterness (belonging to the so-called "basic tastes" of the flavor network; see Spence et al., 2014; Spence, 2016) with the within-subjects factors of Odor (chocolate vs. citrus), Repetition (evaluation 1, evaluation 2, evaluation 3) and Food (sugar candy vs. cracker).

On the dimension of Pleasantness, the analysis revealed significant main effect of Food [F(1, 21) = 7.26; p = 0.014; d = 0.26]. The factor of Repetition [F(2, 42) = 0.70; p = 0.50] and Odor [F(2, 42) = 0.35; p = 0.70], as well as the interaction between Repetition and Odor [F(4, 84) = 0.22; p = 0.93], Repetition and Food [F(2, 42) = 0.07; p = 0.94], and Odor and Food [F(2, 42) = 0.76; p = 0.47], and the interaction between Repetition, Odor and Food [F(4, 84) = 2.10; p = 0.089] did not revealed any significant effect. A post**-**hoc test (Newman–Keuls corrected) on the main effect of Food showed that participants perceived crackers as more pleasant than candies (p = 0.01).

The data analysis on the dimension of Sweetness showed a significant main effect of Odor [F(2, 42) = 5.27; p = 0.01; d = 0.20] and Food [F(1, 21) = 163.28; p < 0.0001; d = 0.87], the interaction between Repetition and Odor [F(4, 84) = 3.10; p = 0.02; d = 0.13] effect size and Repetition and Food [F(2, 42) = 4.27; p = 0.21; d = 0.17]. The main effect of Repetition [F(2,42) = 0.58; p = 0.56], as well as the interaction between Odor and Food [F(2, 42) = 0.93; p = 0.40] and Repetition, Odor and Food [F(4, 84) = 0.23; p = 0.92] did not revealed any significant effect. A Newman-Keuls post-hoc test on the dimension of Odor revealed that partecipants perceived both foods as sweeter when tasted together with chocolate odor, as compared to citrus odor (p = 0.21) and air (p = 0.01; **Figure 4**). A Newman-Keuls posthoc test on the dimension of Food showed that participants, as expected evaluated candies as sweeter than crackers (p = 0.0001).

A Newman-Keuls post-hoc test on the interaction between Repetition and Odor showed that people perceived both foods as sweeter when tasted together with chocolate odor as compared to air (p = 0.001) and citrus odor (p = 0.05) during the third evaluation but not during the previous two (**Figure 5**). The comparison of the first and second evaluations did not show any significant effect. A Newman-Keuls post-hoc test on the interaction between Repetition and Food showed that participants perceived candies as sweeter during the third evaluation, as compared to the first evaluation (p = 0.2). No other significant differences were found.

On the dimension of Saltiness, the analysis revealed a significant main effect of the Food dimension [F(1, 21) = 237.37; p < 0.0001; d = 0.92]. The effect of Repetition [F(2, 42) = 3.13; p = 0.13], Odor [F(2, 42) = 0.92; p = 0.41], as well as the interaction between Repetition and Odor [F(4, 84) = 0.16; p = 0.96], Repetition and Food [F(2, 42) = 1.83; p = 0.17], odor and Food [F(2, 42) = 0.37; p = 0.69] and Repetition, Odor and Food [F(4, 84) = 0.13; p = 0.97] did not show any significant effect. A Newman-Keuls post-hoc test on the Food dimension showed, as expected, that crackers were evaluated as more salty than candies (p < 0.0001).

The data analysis on the dimension of Bitterness, revealed a significant main effect of Food [F(1, 21) = 4.84; p = 0.04; d = 0.19]. The dimension of Repetition [F(2, 42) =1.44; p = 0.25], Odor [F(2, 42) = 0.60; p = 0.55], as well as the interaction between Repetition and Odor [F(4, 84) = 0.39; p = 0.82], Repetition and Food [F(2, 42) = 1.85; p = 0.83], Odor and Food [F(2, 42) = 1.33; p = 0.27], and Repetition, Odor and Food [F(4, 84) = 0.54; p = 0.70] did not showed any significant effect. A Newman-Keuls post-hoc test on the dimension of Food demonstrated, as expected that crackers were evaluated as more bitter than candies (p < 0.04).

tasted regardless of the evaluation sequence, as a function of the different odor conditions. Error bars represent the standard errors of the means. Asterisks represent Newman-Keuls significant differences at \*p < 0.01.

# EXPERIMENT 2

In summary, Experiment 1 revealed that that participants perceived both foods (crackers and candies), as sweeter when presented together with the chocolate odor, as compared to when presented with air or citrus odors. Based on that result, we proceeded with the second experiment to extend our investigation to liquid taste perception.

The aim of Experiment 2 was to test if the experience generated by the odors delivered through our olfactory device could affect people's taste perception of three different types of beverages on a number of hedonic and perceptual-dimensions (such as pleasantness, freshness, sweetness, saltiness, bitterness and carbonation intensity).

#### Materials and Methods Participants

Twenty-seven participants with a mean age of 24.19 years (SD = 3.53, 21 female), took part in the Experiment. All the participants gave informed written consent prior to their participation. The Experiment described here were all performed in accordance with the ethical standards laid down in the 2008 Declaration of Helsinki and approved by the local ethical committee. The Experiment lasted for approximately 20–30 min. People who claimed to be affected by any permanent or temporary olfactory or taste dysfunction were excluded from taking part in the Experiment.

#### Stimuli

Three different types of beverages were used: sparkling mineral water, tonic water and Sprite <sup>R</sup> . Common transparent plastic cups were used to serve the liquids. The beverages were selected on the basis of the same rationale of Experiment 1 with the additional characteristic to be all three transparent (to avoid the colorrelated effect on taste perception). The two odors administered were the same of Experiment 1 (chocolate and a mixture of citrus fruit, manufactured by Oikos Fragrances <sup>R</sup> ).

#### Procedures

Before the beginning of the experiment, a short pre-test was performed to evaluate people's ability to perceive the odors by means the experimental device. The study was conducted in an experimental booth. The participants sat comfortably on a chair in front of a desk. The participants were instructed to grasp the plastic cup and taste the beverage served in it at a signal of the experimenter, while at the same time they smelled an odor or clear air administered to both nostrils for about 6 s. The timing of Experiment 2 differed from that adopted in Experiment 1 (<9 s) given that the act of drinking a sip of a beverage is very fast, as compared to the act of taste a solid food. Therefore, if the experimenter had asked to the participants to hold the liquid in their mouth for a time that exceeded the normal act of tasting a beverage, the results could have been distorted. Specifically, following a number of pre-tests a time of 6 s was given to the participants to keep the liquid in their mouth. Longer intervals were in fact considered excessive (and lead to boredom) for the participants. The procedure of odor delivery was completely identical to that of Experiment 1, as well as the fact that the olfactometer was not visible to the participants. The participants were also informed that after tasting the liquid they had to rate it along five dimensions (pleasantness, sweetness, saltiness, bitterness and carbonation intensity) by means of a 150 mm long visual analog scale (VAS), anchored with the terms "not at all" and "very much." The VAS was presented at the center of a 17′′ PC screen. The participants used the mouse to select the point on the scale that best represented

their evaluation. Each liquid was presented three times for each olfactory condition, for a total of 27 (3 beverages × 3 odors × 3 repetitions) samples of liquid to be evaluated by each participant. The presentation of the beverages and the odors was completely randomized.

#### Results

The mean participants' judgments along the VASs were submitted to a series of repeated measures ANOVA with the within-subjects factors of Odor (chocolate vs. citrus), Repetition (evaluation 1, evaluation 2, evaluation 3) and Beverage (Sparkling Water, Tonic Water, Sprite <sup>R</sup> ) for the hedonic dimension of Pleasantness, as well as for the dimensions of Sweetness, Saltiness, Bitterness and Carbonation Intensity. The results of the analysis on the dimension of Pleasantness revealed a significant effect of Beverage [F(2, 52) = 16,06; p < 0.0001; d = 0.38] and Repetition [F(2, 52) = 6.16; p = 0.00; d = 0.19]. The main effect of Odor [F(2, 52) = 0.32; p = 0.73], as well as the interaction between Odor and Beverage [F(4, 104) = 0.91, p = 0.46] the interaction between Odor and Repetition [F(4, 104) = 0.65; p = 0.63], the interaction between Beverage and Repetition [F(4, 104) = 0.58, p = 0.68] and the interaction between Repetition, Odor and Beverage [F(8,208) = 1.58, p = 0.13] did not result to be significant. That is, the odors presented did not modulate the participants' perception of pleasantness. A Newman–Keuls corrected post-hoc test on the main effect of Beverage revealed that participants evaluated Sprite <sup>R</sup> as more pleasant, as compared to Tonic Water (p < 0.000) and sparkling mineral water (p < 0.000). A Newman– Keuls post-hoc test on the main effect of Repetition showed that people regardless of the odors administered found all beverages as more pleasant during the first evaluation, as compared to the second evaluation.

The analysis on the dimension of Sweetness revealed a significant main effect of Odor [F(2, 52) = 3.99; P = 0.24; d = 0.13], Beverage [F(2, 52) = 51.82; p < 0.000; d = 0.67], and Repetition [F(2, 52) = 6.27; p < 0.01; d = 0.19]. A posthoc test (Newman–Keuls) on the main effect of the Odor showed that participants found all the beverages as sweeter when tasted simultaneously with the chocolate odor than when tasted together with the citrus odor (p = 0.03; **Figure 6**). A Newman– Keuls post-hoc test on the main effect of Beverage revealed, as expected, that people perceived Sprite <sup>R</sup> as sweeter with respect Tonic Water (p < 0.000) and sparkling water (p < 0.000). A Newman–Keuls post-hoc test on the main effect of Repetition showed that participants found all the beverages as sweeter during first evaluation, as compared to the second (p = 0.00) and third (p = 0.01) evaluation.

The results of the analysis of Saltiness showed a significant main effect of Beverage [F(2, 52) = 6.34; p = 0.00; d = 0.20] and of Repetition [F(2, 52) = 3.11; p = 0.05; d = 0.11]. The main effect of odor [F(2, 52) = 2.37; p = 0.10], the interaction between Odor and Beverage [F(4, 104) = 1.42; p = 0.23], the interaction between Odor and Repetition [F(4, 104) = 0.77; p = 0.54], the interaction between Beverage and repetition [F(4, 104) = 0.21; p = 0.93] and the interaction between Odor, Beverage and Repetition [F(8, 208) = 1.92; p = 0.25] did not showed any significant effect. A post-hoc test (Newman–Keuls) on the main effect of

the Beverage revealed that as expected, people perceived Sprite <sup>R</sup> as less salty than Tonic Water (p = 0.01) and sparkling water (p = 0.01). A post-hoc test (Newman–Keuls) on the main effect of the repetition showed that all the beverages were perceived as more salty during the first evaluation, as compared to the third evaluation (p = 0.04).

The analysis performed on the Bitterness dimension revealed a significant main effect of Beverage [F(2, 52) = 38.60; p < 0.0001; d = 0.60]. The dimension of Odor [F(2, 52) = 0.82; p = 0.44], Repetition [F(2, 52) = 1.25; p = 0.30], as well as, the interaction between Odor and Beverage [F(4, 104) = 0.86; p = 0.49], Odor and Repetition [F(4, 104) = 0.50; p = 0.74], Beverage and Repetition [F(4, 104) = 1.02; p 0.40] and the interaction between Odor, Beverage and Repetition [F(8,208) = 1.37; p = 0.21] did not show any significant effect. A newman-Keuls post-hoc test on the main effect of Beverage revealed that as expected, participants perceived Tonic Water as more bitter than sparkling water (p < 0.001) and Sprite <sup>R</sup> (p < 0.001).

The results regarding the Carbonation Intensity dimension revealed a significant main effect of Beverage [F(2, 52) = 8.30; p = 0.00; d = 0.24]. The main effect of Odor [F(2, 52) = 0.27; p = 0.77], Repetition [F(2, 52) = 0.74; p = 0.48], as well as, the interaction between Odor and Beverage [F(4, 104) = 0.83; p = 0.51], Odor and Repetition [F(4, 104) = 0.46; p = 0.71], Beverage and Repetition [F(4, 104) = 0.41; p = 0.80 and the interaction between Odor, Beverage and Repetition [F(8,208) = 0.83; p = 0.57] did not revealed any significant effect. A newman-Keuls post-hoc test on the main effect of Beverage revealed that people perceived sparkling water as less carbonated than Tonic Water (p = 0.16) and Sprite <sup>R</sup> (p = 0.001).

#### DISCUSSION

The results of the present study suggest that odors administered by means our olfactory device can alter people's experience of food, just as it occurs in natural multisensory environments. Moreover, the present study also suggests that the device used to deliver the essences is highly manageable in a laboratory setting. In fact, the possibility to interconnect the olfactometer with the behavioral research software EPrime 2 and to interact with other devices connected to it, allows the implementation of sophisticated experiments in the field of psychology and cognitive neuroscience. With the aid of such device we were able to show multisensory interactions on food perception that are similar to those occurring without the mediation of a HCI device (e.g., when an odor perceived orthonasally affect the taste of the food in the mouth; Borowsky, 1987).

In particular, the results of Experiment 1 showed that participants perceived both foods (crackers and candies), as sweetest when presented together with the chocolate odor, as compared to when presented with air or citrus odors. In Experiment 2, the participants perceived all the beverages (Sparkling Water, Tonic Water, and Sprite), as sweetest when presented together with the chocolate odor, as compared to when presented with citrus odor regardless of the evaluation sequence. That is, the olfactory stimuli delivered by means of the new olfactory display showed to be effective in modulating people's evaluation of food and beverages.

The experimental results demonstrated for the first time that the orthonasal integration of an odor by means our device could affect people's food taste evaluations without adding any additional ingredient to the food. Intriguingly, this is an unusual result concerning a number of claims regarding orthonasal crossmodal integration between food and odors. In fact, it has been previously suggested that, while retronasal integration (e.g., when the odorant contained in food molecules reaches the back of the nose through eating or drinking) should modulate food flavor perception (i.e., sweetness or saltiness), orthonasal integration (e.g., when odor is smelled directly by the nostrils) should only modulate the hedonic dimension (i.e., pleasantness) of food evaluation (see Spence, 2016, for a detailed review). It is, however, worth noting that, despite the fact that during the experiments the olfactory stimulation was orthonasal, some chemical molecules may have somehow reached the oral cavity, and may have been processed also via the retronasal route. Such a possibility, raise the question of how to obtain a pure measure of orthonasal integration between odors and foods under experimental conditions. Future research should be directed to shed light on this issue.

The experimental results reported here are fully consistent with those by Stevenson et al. (2011). These authors demonstrated that the taste intensity ratings of a sweet solution could be significantly enhanced by the concurrent orthonasal presentation of an odor. Interestingly, Stevenson et al.'s results were found while participants tasted a solution, while we showed similar results both for foods and beverages. It is possible that the high precision of the computerized control of the odors administration timing of the MFOD (the latency from generation to perception of a selected odor is <0.5 s, with a length of the delivery pipe of 600 mm) is at the basis of the result found (the presence of orthonasal crossmodal integration). That is, the device used in this paper might have allowed better management of the technical procedures and precision of stimulus delivery, necessary for this kind of studies (although our previous considerations about a retronasal interaction may also be applied here; see the previous paragraph).

Intriguingly, our results have shown that people perceived both foods as sweetest when tasted together with the simultaneous administration of chocolate odor, as compared to air or citrus odors only during the third evaluation and not during the first or the second one. One possible interpretation of this result is that after the first two tasting sessions, food habituation may have occurred. That is, a reduced activation of taste receptors due to habituation (Poellinger et al., 2000) could have amplified the modulatory effect exert by the olfactory stimuli presented by means of the device (i.e., the weaker the taste, the stronger the effect produced by olfaction). This hypothesis should certainly be verified by means of further studies.

As far as the modulation of beverage perception by the device is concerned, it is worth noting that the majority of previous works failed to create the sensation of tasting "virtual beverages" (via the internet) by means of electrical and thermal stimulation (Lant and Norman, 2017; Ranasinghe et al., 2017). One of the reasons for these failures might be due to the lack of a simultaneous chemical stimulation by means odors (using olfactory devices). Since odors represent a pivotal component of taste perception, it would be then interesting to try to use our device also under similar conditions of stimulus presentation (i.e., virtual tasting).

Importantly, no modulatory effects were found on the dimensions of Pleasantness, Bitterness, Saltiness, and Carbonation Intensity. As far as this negative result is concerned, it should be considered that the more or less explicit perception by the participants that the context of stimulus presentation was artificially manipulated (e.g., that the odor did not arise from the beverage itself), could have made more difficult the integration between the olfactory and gustative qualities of the drinks (cf. Dietrich, 2006). Alternatively, it is possible that during Experiment 2 participants experienced a greater difficulty to achieve the temporal and spatial coincidence (Spence, 2011) needed to integrate taste and olfactory stimulation. In fact, also because of the risk of choke, people cannot swallow and sniff from the nostrils at the same time. Interestingly, in Stevenson et al. (2011) experiments, participants held the solution to be tasted in their mouth while smelling the odors through the nostrils, perhaps with the specific aim to achieve this important condition of sensory integration. Note, however, that this procedure likely reduces the ecological validity of the study, given that it is not a natural procedure adopted by people when they are drinking liquids.

In Experiment 2, differently from Experiment 1, no interactive effects were found among repetitions and odors. This result suggests that the tastings sequence did not modulate people's beverages evaluations. This result would seem to suggest that orthonasal crossmodal integration of odors and taste at the basis of people evaluation, might somehow differ, as a function of the gustatory experience involved (foods or beverages). Future research should be addressed to better comprehend the mechanisms at the basis of this difference.

Importantly, the small size and weight of the device presented in this paper (size Height 40 cm, Width 40 cm, Length 50 cm, and Weight <2 kilos) makes it easily adaptable in laboratory and even in more ecological contexts than other olfactory systems. The olfactometers used so far in research laboratories are difficult to be handled, extremely complex during the experimental managing, rather cumbersome and extremely expensive.

The results of Experiment 1 and 2 are in line with previous scientific evidence showing that people's perception of food and beverages can be altered by the multisensory context where the stimuli are presented (Zampini and Spence, 2005; Stevenson et al., 2011; Piqueras-Fiszman et al., 2012; Maggioni et al., 2015; Risso et al., 2015). In fact, despite of the fact that the participants knew that the odor did not originate from the food itself, their evaluation was still significantly affected by it. As far as this point is concerned, our results are the first to show an orthonasal interaction between taste and odor in people's evaluation of food and beverages sweetness by using a small size olfactory interface.

Our results may also be interpreted in terms of a "halo dumping" enhancement effect caused by the sweet quality of chocolate odors. That is, the sweetness of the odors "drove" people's judgments of taste. However, such an evaluative bias was previously found by retronasal perception only and in an unaware condition of stimulus presentation (Green et al., 2012). By contrast, in the present study, odors presentation was orthonasal, and people were perfectly aware of the nature of odor presentation. Finally, halo dumping is generally a homogeneous effect that involves all the characteristics to be evaluated (Thorndike, 1920). By contrast, our results showed a significant modulation only on the scale of sweetness.

Another possible interpretation concerning to the effect found in our experiments could be referred to the fact that participants could have rated the chocolate odor sweetness, instead of food sweetness. However, it is important to highlight that the presentation of the olfactory stimuli here was orthonasal. For this very reason, people were well aware that beverage and food taste perceptions were separated from odor perception. That is, it is unluckily that the olfactory stimuli delivered by our device were confused with the retronasal perception of the odor molecules contained in the stimuli to be evaluated. That said, it is still possible that the sweet quality of chocolate odor was unconsciously transferred and integrated to food perception, thus influencing the participants in their evaluation. Indeed, often people use verbal expressions that are generally used in taste perception to describe an odor. For example, smells sweet is often used concerning vanilla odor (Auvray and Spence, 2007). It is possible that the confusion generated by the use of this kind of semantic metaphor is at the base of the effects found in the present study. Future studies are needed to further clarify this point (see Gallace et al., 2010, for the semantic associations between words and food).

Further studies should also be addressed at investigating the effects of delivering olfactory stimuli by means of devices, such as that presented here, on the perception of different food and beverages, also under ecologically valid condition of stimulus presentation (e.g., in restaurants). Finally, a series of studies will need to be performed in order to study the effects of delivering odors by means olfactory devices within virtual reality simulations. In particular, it will be interesting to observe how olfactory information contribute to our sense of presence, and what aspects of the stimulation are more relevant to improve this perception (e.g., temporal visuo-olfactory synchrony, spatial position, etc.).

It is important to highlight here that the olfactory stimuli delivered by means of the MFOD consisted of compressed powders. Differently from the solution adopted by the majority of olfactory device commercialized nowadays, such a technical arrangement avoids the use of liquid odorants that requires complex systems in order to balance their dilutions.

It should be mentioned that the device adopted in the present study is a prototype and that further devices (which will allow to deliver up to 20 odors and with a higher degree of miniaturization) are now under development. These new generation of portable olfactometers, will certainly allow a large number of applications, comprising those related to enhancing the sense of presence in virtual reality environments. These devices will also contribute to investigating perceptual, emotional, and retention-related (memory) aspects of olfaction in humans, as well as be part of multisensory rehabilitations techniques for a number of neuropsychological disorders (Atanasova et al., 2008; Baba et al., 2012).

# WEB LINKS

Aroma Shooter, https://aromajoin.com/en/hardware/shooters/ aroma-shooter-mini-1

Scentsory Design https://link.springer.com/content/pdf/10. 1007/978-3-540-79486-8\_32.pdf

Aroma Rainbow Sensitive Dress http://www.escent.ai/ smartsecondskin

Mist Shine Indiegogo, https://www.indiegogo.com/projects/ mist-shine-wearable-smart-scent#/

Swallable Parfum, https://www.lucymcrae.net/swallowableparfum/

E-Prime, www.pstnet.com/eprime.cfm

# AUTHOR CONTRIBUTIONS

PR: Experiment design, Experiment implementation, olfactometer interface with ePrime2, data collection, data analysis, manuscript writing; MC: Olfactometer design, Olfactometer implementation, Olfactometer interface with ePrime2, manuscript writing; MB: Olfactometer design, Olfactometer implementation, Olfactometer interface with ePrime2, manuscript writing, manuscript supervision; AG: Experiment design, Experiment implementation, data analysis, data analysis supervision, manuscript writing, manuscript writing supervision.

## ACKNOWLEDGMENTS

The authors would like to thank the apprentices of the Department of Psychology of Bicocca University for the experimental data collection and the Bicocca University students for their participation in the study.

Frontiers in Digital Humanities | www.frontiersin.org

#### REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Risso, Covarrubias Rodriguez, Bordegoni and Gallace. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Multisensory Technology for Flavor Augmentation: A Mini Review

Carlos Velasco<sup>1</sup> \*, Marianna Obrist<sup>2</sup> , Olivia Petit<sup>3</sup> and Charles Spence<sup>4</sup>

<sup>1</sup> Center for Multisensory Marketing, Department of Marketing, BI Norwegian Business School, Oslo, Norway, <sup>2</sup> Sussex Computer Human Interaction Lab, Creative Technology Research Group, School of Engineering and Informatics, University of Sussex, Brighton, United Kingdom, <sup>3</sup> INSEEC Business School, Paris, France, <sup>4</sup> Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom

There is growing interest in the development of new technologies that capitalize on our emerging understanding of the multisensory influences on flavor perception in order to enhance human–food interaction design. This review focuses on the role of (extrinsic) visual, auditory, and haptic/tactile elements in modulating flavor perception and more generally, our food and drink experiences. We review some of the most exciting examples of recent multisensory technologies for augmenting such experiences. Here, we discuss applications for these technologies, for example, in the field of food experience design, in the support of healthy eating, and in the rapidly growing world of sensory marketing. However, as the review makes clear, while there are many opportunities for novel human–food interaction design, there are also a number of challenges that will need to be tackled before new technologies can be meaningfully integrated into our everyday food and drink experiences.

#### Edited by:

Francesco Ferrise, Politecnico di Milano, Italy

#### Reviewed by:

Takuji Narumi, The University of Tokyo, Japan Mehdi Ammi, Université Paris-Sud, France

# \*Correspondence:

Carlos Velasco carlos.velasco@bi.no

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 06 August 2017 Accepted: 09 January 2018 Published: 30 January 2018

#### Citation:

Velasco C, Obrist M, Petit O and Spence C (2018) Multisensory Technology for Flavor Augmentation: A Mini Review. Front. Psychol. 9:26. doi: 10.3389/fpsyg.2018.00026 Keywords: food, multisensory, experience, augmentation, vision, audition, touch, haptics

# INTRODUCTION

Interest in multisensory perception is growing rapidly in the fields of Human–Computer Interaction (HCI, Obrist et al., 2016), sensory marketing (e.g., Petit et al., 2015), and the arts (e.g., Haverkamp, 2013; Vi et al., 2017). This places knowledge concerning the human senses, and their interactions, at the center of design processes (Obrist et al., 2017). In the context of Human-Food Interaction (HFI, Choi et al., 2014; Comber et al., 2014), there has been an increasing interest in how multisensory technologies can augment/modify multisensory flavor perception<sup>1</sup> , and food and drink experiences more generally and possibly also to sensorially nudge people toward healthier food behaviors (Nijholt et al., 2016; Petit et al., 2016). The key idea here is that flavor is a multisensory construct (involving taste, or gustation, olfaction, and possibly also trigeminal components; see Kakutani et al., 2017) and all the senses can potentially influence the way in which we experience it (Spence, 2015a). Hence, multisensory technologies, that is, technologies that are designed to stimulate the human senses, allow researchers to control the different inputs that accompany a given multisensory flavor, or food experience.

Why, it can be asked, use multisensory technologies in order to augment our flavor experiences? Given that technology is already ubiquitous in our everyday experiences, such technologies in the context of HFI hold the potential to transform how we will eat in the future (Spence and Piqueras-Fiszman, 2013). More specifically, we want to argue that a meaningful marriage of multisensory

<sup>1</sup>By flavor augmentation, we refer to the process of modifying, boosting, or enhancing, a given flavor experience, be it perceived or imagined, using technology.

science (e.g., considering the guiding principles of multisensory flavor perception, e.g., Prescott, 2015; Spence, 2015a) and technology in systems capable of augmenting flavor perception can impact what people choose to eat and drink, how they perceive the ensuing flavor experience, and how much they ultimately end-up consuming.

In this mini-review, we present an overview of multisensory technologies for flavor augmentation that have been developed recently. Importantly, we follow Prescott's (2015) distinction between core intrinsic (taste, smell, and some elements of touch) and extrinsic (e.g., color, shape, atmospheric sound – which can modulate the experience of flavor but might not be constitutive) elements of the flavor experience and focus on the role of the latter in flavor augmentation<sup>2</sup> . Our aim is to make researchers working in different fields aware of the various ways in which multisensory technologies that target extrinsic elements of flavor experiences are starting to transform how we interact with and experience what we consume. As such, we expect this manuscript to provide a first point of contact for those interested in multisensory technologies and flavor augmentation. Additionally, we hope that this review will contribute to bridging the gap between researchers working in the fields of HCI/HFI on the one hand, and food science, marketing, and psychology, on the other. It is our view that the latter disciplines would benefit from an increased awareness of the different technologies that are currently available to those working in HCI/HFI. These latter, in turn, would realize some of the potential uses that their technologies have, as well as the financial gains that may derive from such applications. We conclude by presenting challenges that face those wanting to augment flavor perception and experiences.

### FLAVOR PERCEPTION AND AUGMENTATION

Here, we present some key concepts and technologies associated with flavor augmentation on the basis of flavor extrinsic cues (see **Table 1** for a summary of some representative examples). People rarely put something in their mouth without first having made a prediction about what it will taste like. These expectations, set primarily by what we see and smell (orthonasally), but also sometimes by what we hear and feel/touch, anchor the experience when we come to taste something (see Verhagen and Engelen, 2006). For example, visual cues such as color or shape can be used to guide food and drink expectations, search, and augmentation based on semantic knowledge (learned associations as a function of a common identity or meaning such as between the color red and tomato flavor) and crossmodal correspondences (feature compatibility across the senses, such as between sweetness and curvature; e.g., Shermer and Levitan, 2014; Velasco et al., 2015, 2016b; Sawada et al., 2017).

### Visual Augmentation

Vision is critical when it comes to setting our flavor expectations and hence modifying our flavor experiences (Piqueras-Fiszman and Spence, 2015; Spence et al., 2016). Current technologies allow one to go beyond traditional means of food enhancement, based on vision (e.g., just matching or mismatching visual information with a given flavor), and to create novel HFIs that dynamically modulate our flavor experiences, and perhaps also more broadly, our consumption behaviors.

For instance, Nishizawa et al. (2016) developed an augmented reality (AR) system using a projector and a camera in order to transform the visual characteristics (e.g., texture, color) of foods or plates digitally, in real-time, based on the evidence showing that these factors influence people's perception of what they eat (e.g., Okajima et al., 2013). In a similar vein, and as a more specific example, Okajima and Spence (2011) modified the texture of tomato ketchup by changing the skewness of the luminance histogram whilst not changing the chromaticity of the video feed. Modifying such visual features, among others, was found to influence sensory attributes such as the ketchup's perceived consistency and taste such that different skewness led participants think they were tasting different ketchups (see also Narumi et al., 2011; Huisman et al., 2016).<sup>3</sup>

Augmented reality systems build on mixed reality (MR) interactions (i.e., incorporating both virtual and real inputs, see Narumi, 2016). AR would appear to have been adopted more rapidly than virtual reality (VR) in flavor- and food-related technology research and practice. For instance, Kabaq<sup>4</sup> is an AR food program that offers restaurants the option of presenting their customers with 3D visions of the food that they serve, before ordering. As for VR, whilst some researchers are exploring the possibility of virtual flavors via digitally controller electric and thermal taste sensations, such systems are currently of very limited use/potential (see Spence et al., 2017, for a critique). That being said, there is potential to design experiences in VR that target the user's flavor expectations (e.g., before going to a restaurant or buying a product). There are currently many ongoing research initiatives that have been designed to further our knowledge on the applications of VR systems to flavor/food experience design<sup>5</sup> (e.g., Bruijnes et al., 2016). One such initiative involves using VR to expose (virtually) people with food-related medical conditions to obesogenic environments (Schroeder et al., 2016; Wiederhold et al., 2016).

Companies are now exploring product packaging that, together with a smartphone, can be turned into inexpensive VR headsets (e.g., as in the case of some of Coca Cola's cardboard packaging). Such headsets might enable brands to deliver targeted experiences in VR. Whilst, at present, this approach appears more as a curiosity than anything else, we anticipate that it might 1 day become an extension of the total

<sup>2</sup>Whilst we do not focus on olfactory interfaces, devices based on orthonasal olfaction (and its interaction with other senses) have also been proposed by researchers working in the topic of flavor augmentation (e.g., Nambu et al., 2010; Hashimoto and Nakamoto, 2016).

<sup>3</sup> See also Okajima's Laboratory website (https://goo.gl/kH1S9Q) for some examples.

<sup>4</sup>http://www.kabaq.io/

<sup>5</sup>The ACM International Conference on Multimodal Interaction workshop on "Multisensory Human-Food Interaction" (https://goo.gl/HRRdVs) or the Special Issue on "Virtual reality and food: Applications in sensory and consumer science" in The Journal of Computers and Graphics (https://goo.gl/FKwWjF).


TABLE 1 | Examples of multisensory technologies for flavor augmentation based on extrinsic cues associated with the flavor and food/drink experiences.

product experience, in that any given product might have its own customized multisensory experience(s) in VR (Lingle, 2017; Michail, 2017). Such experiences may be designed based on research showing the influence of visual atmospheric cues (e.g., lightning, environment) on flavor perception (Stroebele and De Castro, 2004; Spence et al., 2014).

#### Auditory Augmentation

Often described as the forgotten flavor sense, research on auditory contributions to the experience of eating and drinking has grown rapidly in recent years. The evidence currently suggests that audition is critical to the perception of attributes such as crunchiness, crispiness, and crackliness (Spence, 2015b). What is more, the sounds associated with eating and drinking such as chewing, gulping, or lip-smacking (Zampini and Spence, 2004; Youssef et al., 2017), environmental noise (Woods et al., 2011), and soundscapes/music (Crisinel et al., 2012; Kantono et al., 2016) can all influence food perception (e.g., tastes, odors, textures, flavors). For instance, noise can enhance the perception of umami and diminish perceived sweetness (Yan and Dando, 2015). Based on these kinds of findings, there is growing interest in developing technologies that can capitalize on the sense of audition for flavor augmentation (Velasco et al., 2016a; see also a reference to "EverCrisp app" by Kayac Inc in Choi et al., 2014, designed to enhance food-biting sounds).

Systems that build on the role of mastication sounds on flavor perception constitute one example of flavor augmentation based on audition. The "Chewing Jockey," for example, is a device that uses a bone-conduction speaker, a microphone, jaw movement tracking sensor, and a computer, to allow one to monitor mastication and use such movements to synchronize and control sound-delivery (Koizumi et al., 2011). Based on such a concept, researchers are now interested in the modulation of texture perception (e.g., in the elderly who find it difficult to chew solid foods, see Endo et al., 2016), consumption monitoring (Elder and Mohr, 2016), and the creation of novel and fun food interactions (e.g., mapping unexpected sounds such as screaming sounds to gummies chewing, Koizumi et al., 2011), by modifying, or replacing the actual sounds of mastication.

The role of audition goes beyond mastication sounds though, as there are many other auditory cues that we may hear at more or less the same time as we eat (Velasco et al., 2016a). These include those sounds directly associated with our interaction with the food, but also atmospheric sounds. In terms of the former, Kadomura et al. (2013) introduced "EducaTableware," which include a fork and a cup that use food's (electrical) resistance values, and eating times and intervals to emit sounds while a user consumes a given food (see also Kadomura et al., 2011). This device creates a novel interaction between the user and the food (e.g., for entertainment). In terms of atmospheric sounds, although music devices are ever-present, there is much room for development. For example, based on the idea that sounds can influence taste/flavor perception and enjoyment (i.e., hedonics; Spence, 2017), MR systems that combine real food and audiovisual virtual environments may be developed (e.g., what would it be like to eat a cheesecake, via VR, on Mars? see Project Nourished<sup>6</sup> ).

#### Tactile/Haptic Augmentation

What we feel/touch can also influence the perception of flavor while eating and drinking (e.g., Krishna and Morrin, 2008; Biggs et al., 2016; Slocombe et al., 2016). Researchers have demonstrated that elements such as the weight, size, shape of cutlery and tableware can influence flavor expectations and perception (Spence, 2017; van Rompay et al., 2017). An example of this comes from Michel et al. (2015), who reported that relatively heavy cutlery can lead to tastier food perception. Notably, similar to systems that build on vision and audition, most of the potential of touch-related devices for flavor augmentation so far has been in terms of MR solutions.

For instance, Hirose et al. (2015) developed a fork-type device that involves an accelerometer, a photo reflector sensor, and motor slider, to digitally control the center of gravity, and therefore the perceived weight, of the eating utensil. The intention behind "Gravitamine spice" is to modify the felt weight of the food/cutlery before eating. Another example comes from Ikeno et al. (2015) who showed that different patterns of vibrations accompanying the action of pouring a beverage can influence how much is poured. These technologies might potentially be used to nudge people to consume a little less, to

<sup>6</sup>http://projectnourished.com/

create novel human–food interactions, and to augment flavor. Meanwhile, Iwata et al. (2004) developed a haptic device for biting, known as the "Food simulator." This interface generates a force on the user's teeth, which is based on the force profile of people biting a given food, in order to stimulate the sensation of biting such a food.

There are also multiple emerging haptic/tactile technologies that can be used for flavor augmentation or innovative HFI design. For instance, Tsutsui et al. (2016) developed a high resolution tactile interface for the lips, a part of the body that is often stimulated while eating and drinking, which created a new interaction design space. There might also be opportunities when it comes to MR scenarios where people eat and receive haptic feedback on their body either associated with the food they eat (e.g., Choi et al., 2014) or remote dining with touch-related signals from co-diners (e.g., Wei et al., 2011). Of course, in many cases, there may be no specific need for haptic/tactile interfaces be technology-based. Nevertheless, what technology can potentially offer is a new way of stimulating the skin (e.g., contingently) and therefore opens-up a space for novel interactions and flavor experiences.

# Combining Multiple Extrinsic Flavor Elements for Flavor Augmentation

Visual, auditory, and tactile/haptic flavor augmentation systems have, in general, focused on allowing the integration of one property (e.g., color) or series of properties (e.g., color and shape), in a given sense (e.g., vision) with specific flavor experiences. Importantly, though, eating and drinking constitute some of life's most multisensory experiences (e.g., involving color, shape, sound, vibration, texture roughness, etc., Spence, 2015a). It is perhaps little wonder, then, that those trying to emulate more real-life experiences have focused on designing technologies that allow the integration and controllability of inputs associated with multiple sensory modalities (e.g., Kita and Rekimoto, 2013). For example, the "Straw-like User Interface (SUI)" augments the user's drinking experiences based on multisensory inputs (e.g., using pressure, vibration, and sound, see Hashimoto et al., 2006; see also Ranasinghe et al., 2014). Another example comes from Ikeno et al. (2013) who developed a system that combines vibrations and sounds (e.g., an auditory "glug" characteristic of a Sake bottle when a drink is poured) to influence the subjective impression of a liquid. Whilst there is certainly no need to stimulate all of the senses, for a given flavor augmentation, solutions that allow the delivery of multiple cues at a given time might broaden the scope for multisensory design.

# DISCUSSION AND CONCLUSION

This review presents flavor, and more general, food and drink augmentation in the context of multisensory experience design. In particular, we provide an overview of both older and more recent efforts around flavor augmentation in HFI. In addition to psychologists and sensory/food scientists, those researchers involved in HCI are increasingly exploring new ways of transforming our eating and drinking experiences. The proliferation of VR and MR systems provide the most promising platforms for new (multisensory) flavor experiences in the near future.

We have concentrated on exemplar systems that have capitalized on flavor extrinsic elements from vision, audition, and touch/haptics for flavor augmentation. Whilst such systems are still far from ubiquitous, they are nevertheless increasingly being considered by some of the key players/influencers of the food and drink industry – such as, for example, chefs, culinary artists, experiential brand event managers, and so on (Spence, 2017).

Importantly, however, there are multiple challenges ahead for both researchers and practitioners who may be interested in using multisensory technologies for flavor augmentation. First, a vast gap often exists between technology, as showcased in HCI research, and what actually ends-up in more commercial settings relevant to those working in the food and drink industry (e.g., in a fancy modernist restaurant on in a branded experiential event). Second, there is a need for long-term follow-up investigations, as most of the research examples that have been reported to date have been based on one-off, small scale studies (e.g., small sample sizes with limited experimental designs; for example, Nishizawa et al., 2016, conducted two studies with four and six participants, respectively). There is a need to control for variables such as novelty and habituation, something that will undoubtedly be required in order to know whether the brain adapts to the multisensory flavor experiences designed with new technologies, or whether instead the benefits may last into the medium/longer-term. In other words, there needs to be a consistent added value for flavor and food augmentation to become more than a one-time curiosity or gimmick.

The aforesaid challenges might be addressed (at least in part), by the meaningful integration of scientific insights concerning multisensory flavor perception with new technologies. Whilst research on the principles governing multisensory integration during flavor perception is ongoing (see Prescott, 2015; Spence, 2015a), design guidelines have nevertheless been suggested (Schifferstein and Desmet, 2008; Velasco et al., 2016a). Taking a full-scale, evidence-based approach to the design of multisensory flavor experiences that incorporates technology is not an easy task and therefore will require both time and a fundamentally multidisciplinary approach.

However, the hope is that multisensory technologies might inspire tomorrow's practitioners to: (1) modify flavor perception and experiences; (2) nudge people toward healthier food behaviors; (3) facilitate food choice before ordering/buying; (4) make dining more entertaining. For example, TeamLab, an art collective, collaborated recently with the Sagaya restaurant in Tokyo to develop a dining experience described as follows: "when a dish is placed on the table, the scenic world contained within the dish is unleashed, unfolding onto the table and into the surrounding space. For example, a bird painted on a ceramic dish is released from the dish and can perch on the branch of a tree that has been unleashed from a different dish" (cited in

Stewart, 2017, p. 1<sup>7</sup> ). Other examples include the oft-mentioned Michelin-starred modernist restaurant Ultraviolet by Paul Pairet in Shanghai. There, diners are guided through a multisensory dining experience that is accompanied by changing lights, projections, and soundscapes (Yap, 2016; Spence, 2017). Technology in the context of multisensory flavor experience design is a means to transform sensory information into ingredients/raw materials for our future flavor experiences. In that sense, we foresee more applications and novel design spaces being explored in the wider food and drink world.

7 See also a report in DesignBoom with visual documentation: https://www.design boom.com/design/teamlab-interactive-saga-beef-restaurant-sagaya-ginza-tokyo-04-13-2017/.

#### REFERENCES


# AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

# FUNDING

MO contribution to this review is partially supported by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program under grant agreement no. 638605.

(New York, NY: Springer-Verlag), 548–551. doi: 10.1007/978-3-319-031 61-3\_50



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Velasco, Obrist, Petit and Spence. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.