Crossmodal correspondences between color and flavor identification/perception have been well-documented in research (see Spence et al., 2010, and Spence, 2015 for reviews). Over the past 50 years, studies have shown that people consistently associate each of the basic tastes with specific colors; for instance, sweetness with red/pink, sourness with green/yellow, saltiness with blue/white, and bitterness with dark colors like black and deep blue (e.g., Favre and November, 1979; Spence, 2015; Woods and Spence, 2016). Such color–taste/flavor pairs are arguably the result of associative learning, when one learns to associate a specific color with a specific gustatory cue through repeated exposure. This would explain cross-cultural differences in color-taste/flavor pairings, for instance, light blue being associated with mint flavor for Taiwanese participants but raspberry for United States (US) participants (Shankar et al., 2010). Interestingly, new color-taste associations can be learned even with relatively brief exposure periods (e.g., four training sessions, Higgins and Hayes, 2019).

An extensive body of research has demonstrated how food color can drive perceptual changes in the tasting experience. On a basic level, increasing the color intensity of foods and drinks has led to higher taste/flavor intensity ratings (e.g., Johnson and Clydesdale, 1982; Norton and Johnson, 1987; Calvo et al., 2001). When it comes to specific tastes, red-colored drinks have been found to enhance sweetness detection (Johnson and Clydesdale, 1982), expectations (Wei et al., 2012), and perceived intensity (Pangborn, 1960; Clydesdale et al., 1992; Lavin and Lawless, 1998; Hidaka and Shimoda, 2014). Food color can also interfere with flavor identification, such as shown by Zampini and colleagues with lime‐ and orange-flavored beverages colored green or orange (Zampini et al., 2007). With a more complex beverage, Wang and colleagues have demonstrated recently that participants used rosé wine aroma and flavor terms to describe a white wine that had been dyed pink to match the color of an actual rosé wine (Wang and Spence, 2019).

More recently, human-computer interaction research has incorporated technology to alter the sensory input of given foods. For instance, Ranasinghe and colleagues created two such beverage delivery systems, the Virtual Lemonade and the Vocktail. The Virtual Lemonade uses colored LED lights to overlay a color on plain water, combined with electrically stimulated sour taste sensations on the user’s tongue (Ranasinghe et al., 2017a). The Vocktail goes one step further, by combining colored lights and electrically induced tastes with scent delivery, to digitally alter the flavor of a given drink (Ranasinghe et al., 2017b).

Virtual Reality can be defined as a type of human-computer interaction where one interacts with a three-dimensional computer-generated environment presented via a stereoscopic head mounted display which tracks the location of the user’s head (Crofton et al., 2019). Given the increasing popularity of VR, several studies have been conducted in the last decade to understand whether VR is suitable for use in food research.

The potential usefulness of this kind of research is supported by emerging evidence that the food-related decisions that people make in VR appear to replicate decisions that they make in the real-world. For example, Ung and colleagues exposed participants to both real-world and VR buffets, in order to investigate whether the nutritional value of foods selected by participants in VR paralleled the value of the foods that they selected in real-world contexts (Ung et al., 2018). There was a strong correlation between the nutritional values of food that participants served themselves in VR and real-world environments (r ≥ 0.75). Similarly, participants exposed to both real-world and VR store environments made similar cereal product selection decisions in VR as in the physical environment (Siegrist et al., 2019). In the same vein, researchers placed parents in VR and real-world environments, in order to investigate whether the portions of a pasta-based meal that they chose for their children in VR were similar to the portions that they selected in the real-world (Persky et al., 2018). Results showed that there was a strong relationship between the portions that parents served in VR, and in the real-world. For example, a correlation of r = 0.822 was observed between the amount of real apple juice that a parent served and the amount of VR apple juice that they served. In terms of consumer testing, a comparison of beer tasted in a laboratory, in a real pub, or in a variety of immersive environments ranging from projection walls to VR headsets demonstrated that repeatability of hedonic scores for beer was better both in the real pub and in the immersive environments, compared to the central lab testing site (Sinesio et al., 2019). These findings suggest that the use of VR in food research is a useful and meaningful method as participants have been found to treat food the same way in both VR and the real-world. This is especially important given the increasing use of VR in food disorder therapy, either by altering participants’ self-image or by introducing virtual food cues (see Clus et al., 2018, for a review).

Furthermore, a nascent body of literature examines how digitally introduced colors may influence taste perception. For example, Huang and colleagues examined how displaying different colors of tea (red or green) in VR influenced individuals’ subsequent taste ratings of drinks in the real-world (Huang et al., 2019). Results of this study largely did not find any overall effect of color. However, the authors did suggest that there were some color-induced taste rating differences driven by individual color–food associations. More recently, Ammann et al. (2020) assessed whether changing the color of foods (two juices and a piece of cake) shown in VR influenced flavor identification. They demonstrated that seeing modified product colors in VR did in fact negatively impact flavor identification, and that flavor identification performance was not significantly different when participants did the study in VR vs. in real life, using food-coloring-modified products. Finally, even food-extrinsic changes in virtual environment have been shown to influence taste evaluations of different foods, such as a grenadine-based beverage tasting sweeter when consumed in a sweet-congruent environment compared to a bitter-congruent environment (Chen et al., 2020), or when cold brew coffee tastes sweeter when consumed in an environment with pleasant color and music, compared to one with unpleasant color and music (Nivedhan et al., 2020).

Color-induced gustatory effects have been found in studies using augmented reality (AR) technology. In contrast to VR, AR projects digital information (typically visual imagery) onto the physical world (Crofton et al., 2019), therefore plausibly providing a greater degree of realism in the entire eating scenario. For instance, Okajima and colleagues have constructed an AR “food changer” system to identify and modify the appearance of food via a sophisticated computer vision algorithm, either using a projector (Nishizawa et al., 2016) or a head-mounted display (Ueda and Okajima, 2019; Ueda et al., 2020). The findings of their experiments, which changed the color, saturation, and visual texture of various foods, found that there was a correlation between the color saturation and the rated sweetness in cake (Nishizawa et al., 2016); the mouthfeel, greasiness, and deliciousness of sashimi can be altered by its visual texture and color (Ueda and Okajima, 2019) and the moistness and deliciousness of sponge cake, as well as the watery taste in ketchup, can be altered by dynamically modifying the luminance distribution of the foods. However, the studies are limited by small sample sizes (four participants in the cake study, 12 participants in the sashimi study, and 13 participants each in the sponge cake and ketchup study).

In a demonstration of altering both color and smell, Narumi and colleagues created a MetaCookie+ system consisting of headset and aroma delivery system, which was able to track and alter the appearance and scent of a cookie in real time (Narumi et al., 2011). The researchers showed that, without changing the chemical composition of the food itself, 79% of participants experienced a change in the cookie taste using the pseudo-gustatory display. Going beyond taste perception, AR has also been shown to modify people’s level of perceived satiety by altering the apparent size of the food consumed using real time-shape deformation (Narumi et al., 2012).

However, such custom AR technology requires a high level of technical expertise in computer vision, making it relatively infeasible in most sensory testing situations. In the current study, we created a simple yet convincing mixed-reality VR setup which allows for the simultaneous presentation of food stimulus in both virtual and physical reality.

To demonstrate how VR could be incorporated in flavor perception research, we decided to investigate a scenario which would be difficult to study in the real-world. Namely, we investigated the influence of visual appearance on the perceived flavor of black coffee. Since people have learned to associate a light brown color with milky coffee, we hypothesized that a black coffee which appears to be light brown would be rated as tasting sweeter and creamier than one that appears dark brown. If the hypothesis holds, then we could develop future applications where people can add “virtual creamer” to their coffee to cut down on calories while still maintaining the perception of creaminess. Moreover, this study is uniquely suited for VR because, in the real-world, it would be difficult¹ to make black coffee appear light brown without altering its taste.

From a theoretical perspective, this study is a proof of concept for using VR as a way to study the merging of virtual and actual sensory cues in the formation of our eating experience. Notably, unlike previous VR studies which separated virtual and real sensory cues (e.g., Huang et al., 2019, where participants first saw a color cue and then tasted the samples in a black screen), the present study enhances the realism of the situation by enabling the participants to simultaneously interact with the same object in both the physical and virtual environment. From an industrial perspective, this study demonstrates the possibility of performing rapid product testing with a consumer panel, in situations when it may be time-intensive or costly to produce products with the same range of visual features in the real-world.

Thirty-two participants (5 women, 27 men) aged 18–38 years (M = 23.1, SD = 3.1) were recruited for the study from York St. John University. Participants were recruited at computer science labs on campus by word of mouth. The studies took place between 10 a.m and 6 p.m (13 participants took part before 1 p.m, 19 participants after 1 p.m). No further selection criteria were applied other than having normal senses of vision, smell, and taste. All participants gave their informed consent to take part in the study. The study was approved by the Research Ethics Committee of York St. John University.

Black cold brew coffee (Califa Farms) were used in the study. To test sweetness level as a possible moderator in color–taste influences, two sweetness levels were created with the addition of either 4% or 8% of sucrose by volume. The coffee was served in 30 ml samples.

A room (4 m × 4 m × 2.3 m) with a table (0.9 m in diameter), mug, and straw was modeled in the Unity game engine (Unity Technologies) to mirror the physical setup in the experiment room (Figure 1). The color of the coffee was manipulated in VR such that it either appeared light brown or dark brown. A straw was placed in the mug so participants could consume the coffee while wearing the VR headset (HTC Vive Pro).

In order to create a mixed-reality environment where the mug of coffee would move synchronously in VR and the physical world, a VR tracker (HTC Vive) was placed under the physical mug. This way, the movement of the mug was aligned such that when participants lifted the mug in the physical world, the mug also appeared lifted in VR. such that the mug could be tracked in the VR environment as well.

A within-subject study with two virtually presented coffee colors (light brown, and dark brown) and two levels of sucrose in the physical coffee product (4, and 8%) was carried out. The study took place at a research laboratory at the Department of Computer Science at York St John University. Each participant was seated in front of a table. First, they read an information sheet and gave consent to partake in the study. Next, they rinsed their mouths with water and were instructed to wear the VR headset.

At the onset of each trial, the participant was instructed to pick up the mug from the table and take a sip from it. After tasting, participants were asked to orally evaluate their liking for the drink as well as its sweetness and creaminess on 1–9 scales (with 1 being not at all, and 9 being very much). These evaluations were reported verbally by the participants, and then recorded by the experimenter via a Qualtrics questionnaire. Participants then placed the mug back onto the table after giving their evaluation. In between trials, the experimenter switched out the coffee sample inside the mug. They also handed a cup of water to the participants, who rinsed their mouths with water between each trial. It should be stressed that participants wore the headset during the entire tasting session and never saw any of the actual coffee samples.

Each participant tasted all four color (light brown, and dark brown) and sweetness (4, and 8%) combinations. The order of conditions was counterbalanced across participants using a Latin Square Williams Design.

After tasting all four samples, participants removed their headsets and completed the rest of the Qualtrics questionnaire on a laptop. They answered questions regarding their VR experience (never used it before, < 1 h, 1–5 h, > 5 h), frequencies of drinking black coffee and of drinking milky coffee (daily, 4–6 times a week, 2–3 times a week, once a week, never), and preference for sweet and bitter foods (scale from 1 to 9).

Each experimental session lasted approximately 15 min and participants were debriefed afterwards.

Data from all 32 participants were included in the analysis. After checking for data normality and multicollinearity between the measured variables (i.e., liking, sweetness, and creaminess), a repeated-measures multivariate analysis of variance (RM-MANOVA) was conducted with coffee color (light brown, and dark brown) and sugar level (4, and 8%) as within participant factors (SPSS, version 25). The model included liking, sweetness, and creaminess as measures. While our sample size is too small for in-depth demographic analyses, we reported summary statistics of the participants’ VR familiarity, coffee drinking frequency, and preference for sweet and bitter tastes to communicate more information about the background of the participants.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.