Modulating taste perception through color and shape: a mixed reality study on solid foods

The study explores the cross-modal correspondence between color and taste in a mixed reality (MR) context. Using dehydrated apple snacks, we examined how product and context color influence the perception of sweetness and sourness. Three experiments were tested: the effect of product color, the effect of shape and color of context, the effect of congruent product and context color and shape. A 2-alternative forced choice (2-AFC) test was conducted with 102 participants to assess taste perception. Results showed no significant differences for product or context color alone. However, in the third experiment, participants selected consistently the green snack in the green angular context as the sourest, demonstrating the importance of sensory congruence. Conceptually, participants associated red and pink with sweetness (70%), green and yellow with sourness (68%). Additionally, over 90% linked rounded shapes to sweetness and angular shapes to sourness. The study highlights how visual stimuli and shape can influence solid food taste perception, with MR providing a valuable tool for controlling sensory inputs.

1 Introduction

Cross-modal correspondence refers to the phenomenon where information from one sensory modality influences or is linked to perception in another modality (Small, 2012). This interplay between senses occurs when sensory inputs from different channels interact and affect stimuli perception. For example, sounds might influence how we perceive the texture of food (Zampini and Spence, 2005; Dematte et al., 2014), or smells might impact our perception of color (Nehmé et al., 2016). The study of cross-modal correspondence explores how multisensory integration shapes our overall sensory experience and perception, acknowledging that our senses do not operate in isolation but in a dynamic and interconnected manner.

One intriguing area of research in this field is the relationship between color and taste, where specific colors are often conceptually associated with particular taste qualities. Research on cross-modal interactions has shown that color can significantly influence taste perception, with certain hues being linked to particular flavors (Spence et al., 2015; Ammann et al., 2020; Carvalho and Spence, 2019). The selection of red and pink for sweetness, and green and yellow for sourness, is grounded in well-established cross-modal correspondence research. Studies have shown that these color-taste associations are deeply ingrained, often linked to learned environmental cues and cultural factors. For instance, red and pink are frequently associated with sweetness due to their connection with ripe fruits like strawberries or raspberries, which are commonly perceived as sweet (Spence and Levitan, 2021). On the other hand, green and yellow are commonly associated with sourness, likely due to their prevalence in unripe or acidic fruits, such as limes, lemons, and green apples, which are known for their tart and sour flavors (Woods and Spence, 2016). These associations are thought to stem from evolutionary adaptations and cognitive biases, where humans have developed expectations about taste based on visual cues in their environment (Velasco et al., 2015). In addition to color, shape has also been shown to influence taste perception through cross-modal correspondences. Specifically, round shapes are often associated with sweetness, while angular shapes tend to be linked with sourness or bitterness. These associations may stem from shared emotional valence or ecological factors—smooth, round shapes being perceived as more pleasant and thus aligning with sweet tastes, and sharp, angular shapes evoking alertness or caution, corresponding with sour or bitter tastes (Velasco et al., 2015; Ngo et al., 2011; Deroy and Valentin, 2011). Such shape-taste correspondences have been observed across cultures and are relevant in packaging design, plateware, and immersive environments where visual context can subtly alter flavor perception. Understanding these relationships has practical implications in areas such as food marketing and product design. However, there is not enough evidence to assert that a change in color alone is sufficient to alter the perception of sweetness or sourness.

Traditional methods of studying these correspondences can be limited by real-world constraints of food colors. Virtual reality (VR) technology has emerged as a powerful tool for investigating these complex sensory phenomena, including cross-modal correspondence (Ares, 2019, Cornelio et al., 2022) modifying only visual aspects of foods and beverages. VR offers a unique advantage by allowing researchers to precisely control and manipulate sensory inputs in a highly immersive and customizable context. This flexibility enables the study of cross-modal interactions, such as the influence of visual stimuli on taste perception, with a level of engagement and precision that would be challenging in real physical settings. Prior studies have demonstrated that VR can enhance user engagement, provide consistent sensory experiences, and offer cost-effective means for conducting complex sensory studies (Halabi and Saleh, 2021). For instance, VR has been used to explore how changing context color can modify taste perception, such as increasing sweetness in beverages; moreover, VR offers potential applications in promoting healthier eating habits by altering perceived taste without changing the actual content of the food or beverage (Wu et al., 2022; Mesz et al., 2023).

VR encompasses a range of technologies designed to create immersive digital environments that simulate physical presence in virtual or augmented spaces. These technologies vary in complexity and application and can be categorized into several distinct types: Immersive VR; Augmented Reality (AR); and Mixed Reality (MR) (Crofton et al., 2019). MR combines elements of both VR and AR to create contexts where physical and virtual objects coexist and interact in real-time. MR systems use advanced sensors and spatial mapping technologies to seamlessly integrate digital content with the physical world (Fuentes et al., 2021).

In cross-modal correspondence studies, controlling the stimuli of one sense while varying the other is crucial. MR offers a unique solution by allowing manipulation of visual factors like color without altering other sensory attributes, like taste and smell. Adding color to a food’s formula can alter its taste, potentially affecting the validity of results. With MR, color can be changed without impacting flavor, ensuring more reliable evaluations. Although MR has been extensively explored for its potential to manipulate visual stimuli, particularly in the context of beverages and simple food items, research on how MR can be used to influence taste perception in solid foods remains scarce. The complexity of solid food sensory interactions, such as texture and flavor, requires further investigation, especially when visual manipulations like color changes are applied. While Mixed Reality (MR) offers promising avenues for such investigations, its application to studying taste perception with solid foods, employing advanced interaction techniques like precise hand tracking and real-world object anchoring, remains an emerging area within Human-Computer Interaction (HCI). This study leverages novel MR capabilities to create a highly interactive and ecologically valid experimental setup, contributing to HCI methodologies for sensory research. The present study addresses this gap by exploring the impact of color manipulation in solid foods using MR technology, offering new insights into cross-modal correspondence research.

Therefore, the aim of this study was to use a novel MR setup, incorporating Meta Spatial Anchors for precise food localization and synchronized hand tracking for natural interaction, to alter the color of an apple snack and the color and shapes of the surrounding context. This allowed us to investigate whether changes in intrinsic color of the product and the extrinsic color of the surrounding influence the perception of the product’s taste, while also exploring the potential of these HCI techniques in sensory science. For this purpose, three experiments were conducted:

• Effect of the product color on the perception of sweet and sour taste.

• Effect of the context color and shape on the perception of sweet and sour taste.

• Effect of product color together with context color and shape on the perception of sweet and sour taste (congruence effect).

In designing these experiments, specific colors were chosen based on established cross-modal correspondences between color and taste. Red and pink were selected for their strong association with sweetness, a linkage that has been repeatedly demonstrated in studies where these colors are conceptually connected to ripe, sweet fruits such as berries (Spence et al., 2010; Wang et al., 2020). Similarly, green and yellow were chosen to represent sourness, reflecting their frequent association with unripe or acidic fruits like limes and green apples, which are often perceived as tart (Woods and Spence, 2016). These colors provided a clear framework to explore the influence of both product color and contextual color on taste perception.

Neutral grays were used in some contexts as a control to ensure that any changes in taste perception could be attributed to the color manipulations, rather than an inherent bias linked to the colors themselves (Ammann et al., 2020). This allowed for a more reliable assessment of how congruence between the product and its surrounding context, both in terms of color and shape, influenced the perceived sweetness or sourness of the snack. By selecting these colors, the study aimed to build upon previous research while providing new insights into how visual stimuli in a mixed reality context can modulate taste perception in solid foods.

2 Materials and methods

2.1 Samples

To test all experiments a commercial dehydrated apple snack was used (Fruch™, Argentina), directly purchased from the manufacturer (Fruch Patagonia, SAS). The snack consists of dehydrated red apple slices. According to the label, the only ingredients were red apple, sugar and vitamin C as antioxidant.

This snack was selected due to its relatively mild flavor, which provides a neutral sweet and acid base. We wanted to ensure that any changes in perceived taste would arise primarily from the visual modifications rather than from inherent flavor differences.

2.2 Mixed reality scenario

Participants were provided with a Meta Quest 2 VR headset featuring a 6 degrees of freedom tracking system, a horizontal field of view (FOV) of 97°, a vertical FOV of 93°, and a resolution of 1832 × 1920 pixels per eye. The experimental setup was developed using the Unity3D game engine, with support from the AD2RV non-profit organization led by Jean-Christophe Sakdavong. A key innovation in this setup was the use of Meta Spatial Anchors (Meta, 2024) to precisely track the real-world position of the food dispenser and the individual apple snacks. This ensured that virtual augmentations, such as color changes, were accurately co-located with the physical items throughout the experiment.

The development incorporated mixed reality elements, enabling the VR headset to track the position of real snacks within the laboratory environment. The headset dynamically adjusted the color of the snacks, the surrounding context, or both (depending on the experiment), based on their position derived from the spatial anchors. Hand tracking was accomplished using the headset’s integrated cameras and the Meta SDK for Unity, with interaction logic programmed in C#. This allowed for the display of virtual hands that mirrored the participants’ actual hand movements, enabling them to naturally interact with the real snacks (while maintaining their virtual colors). The system was designed to ensure that the virtual food representation was displayed in the exact physical location of the real food, synchronizing visual cues with the participants’ tactile experience. Participants had the option to rinse their mouth, prompted by a virtual window overlayed onto the real world, through which they could perceive a real glass of water and their actual hand (Figure 1).

Figure 1

Figure 1. Picture of a participant performing the experiment, interacting with the real product, while viewing the inserted virtual image through the Oculus headset.

To ensure precise alignment and a stable experience, a specific calibration and setup procedure was followed. The calibration process was initiated by positioning two Meta Spatial Anchors in the physical space. These anchors served to digitally locate a custom, 3D-printed dispenser that held the apple snacks (the 3D model for the dispenser is available upon request). Once the anchors were initially placed, the experimenter’s view would switch to a semi-transparent overlay, allowing for meticulous visual verification and adjustment to ensure the virtual representation of the dispenser perfectly matched its real-world location.

Maintaining the integrity of the spatial anchors’ positions between the calibration and the participant’s session was critical. To prevent the headset’s tracking cameras from entering sleep mode, which would risk losing the anchor data, the headset’s proximity sensor was covered with opaque adhesive tape. This simple modification effectively disabled the automatic standby function, ensuring that the tracking remained active and consistent throughout the experiment’s duration. Furthermore, the experimental room was illuminated with sufficient, diffuse, non-fluorescent lighting to guarantee that the participants’ hands were always clearly and evenly lit, thereby optimizing the performance and reliability of the integrated hand-tracking system.

The hand-tracking itself, powered by the Meta SDK, managed the complex challenge of occlusion. The system could accurately render the virtual hands even when fingers were partially obscured by other parts of the hand or by the physical snacks. The dehydrated apple snacks were intentionally small to minimize the surface area causing occlusion, which further contributed to the stability and fluidity of the hand-tracking during the grasping and eating motions. This comprehensive approach to setup and calibration ensured that the interaction felt natural and that the alignment between physical and virtual elements remained robust.

2.3 Experimental design

The study involved 102 participants, recruited via email. They were students and employees of Universidad Católica Argentina, with a gender distribution of 65% female and 35% male. Participants’ ages ranged from 18 to 62 years, with the majority (76%) being between 18 and 30 years old. Participants were informed that all data would remain confidential and would only be reported in aggregate. Informed consent was obtained from all participants.

Each participant completed two out of three experiments (either Experiments 1 and 3 or Experiments 2 and 3) to minimize session duration and prevent sensory fatigue. Assignment to groups was randomized.

All participants first completed either Experiment 1 or Experiment 2, followed by Experiment 3. Initially, 51 participants were recruited to complete Experiments 1 and 3; subsequently, another independent group of 51 participants was recruited to complete Experiments 2 and 3. In all experiments, a 2-alternative forced choice (2-AFC; Ennis, 1990) test was used, where participants had to choose between two samples: the one they perceived as sweeter (question a) and the one they perceived as sourer (question b). The 2-AFC method was employed in all experiments to assess whether visual manipulation influenced participants’ perception of sweetness and sourness without using scales, which are prone to noisy data and individual differences in interpretation. This binary, forced-choice structure reduces ambiguity, lowers cognitive load, and improves discrimination sensitivity, making it especially suitable for studying subtle cross-modal effects. Furthermore, the 2-AFC method aligns with the controlled nature of the mixed reality (MR) environment, where only visual attributes were manipulated while all other sensory properties of the samples remained constant. By focusing on relative judgments rather than absolute ratings, this method allowed for a more reliable assessment of the influence of visual cues on taste perception. To mitigate potential order effects and response biases, the order of presentation of the stimuli was counterbalanced across participants. Nonetheless, we acknowledge known limitations of this method, such as potential perceptual biases and the binary nature of the responses, which may reduce sensitivity to more nuanced perceptual variations. The MR environment, with its precise tracking and interactive capabilities, allowed controlled manipulation where only visual attributes were altered while participants could naturally interact with the physical snacks.

In each experiment, participants were presented with two snacks (one red and one green) either simultaneously or sequentially, depending on the experiment. The presentation order of the contexts (red/pink context and green/yellow context) was counterbalanced across participants to prevent order effects. Participants were randomly assigned to different conditions to ensure no systematic bias in sample presentation.

After the tasting session, participants completed a survey via Google Forms. The survey gathered demographic data and asked about their comfort during the virtual experience [using a 9-point scale, ranging from 1 (very uncomfortable) to 9 (very comfortable)]; whether they believed the sensations would differ if the experiment was performed in real life [using a 9-point scale, ranging from 1 (it would feel the same) to 9 (it would feel totally different)]; their familiarity with the concept of cross-modal correspondence with three options (not familiar at all, kind of familiarity, and very familiar), and their associations between basic tastes and specific colors and shapes (allowing the same color or shape to be selected for more than one taste).

2.3.1 Experiment 1: effect of the product color on the perception of sweet and sour taste

A neutral-colored context (white, gray) was used. The products were colored red and green in the MR scenario (Figure 2). This experiment involved 51 participants; each tasted two pairs of snacks (one red and one green) and were forced to select which snack from each pair was (i) sweetest and (ii) sourest.

Figure 2

Figure 2. Image of screen in experiment 1 scenario.

2.3.2 Experiment 2: effect of the context color on the perception of sweet and sour taste

Two contexts used were one with shades of red and pink colors with rounded lines (red context) and another one with different shades of green with angular lines (green context). In this experiment, the snacks in the MR scenario were colorless (light gray) (Figures 3, 4). This experiment involved 51 participants; each tasted two pairs of colorless snacks in different contexts and were forced to select which snack from each context was the (i) sweetest or (ii) sourest.

Figure 3

Figure 3. Experiment 2 scenario, red context.

Figure 4

Figure 4. Experiment 2 scenario, green context.

2.3.3 Experiment 3: effect of product color together with context color and shape on the perception of sweet and sour taste (congruence effect)

The same red and green contexts used in experiment 2 were used, but the snacks were colored red and green in the MR scenario (Figures 5, 6). This experiment involved 102 participants; each tasted two pairs of snacks (one red and one green) across different contexts. In the red context, they were forced to choose which snack was the sweetest, while in the green context, they were forced to choose which was the sourest. The aim was to determine whether product-context congruence had an additive effect.

Figure 5

Figure 5. Experiment 3 scenario, red context.

Figure 6

Figure 6. Experiment 3 scenario, green context.

2.4 Statistical analysis

Data from the 2-AFC tests were analyzed using exact binomial tests, following the procedure described by Bi et al. (1997). For each comparison, we used a binomial distribution table to determine the minimum number of correct responses (out of n observations) required to reject the null hypothesis that participants were guessing (i.e., performance at chance level of 50%). A result was considered statistically significant if the observed number of correct responses exceeded this threshold, corresponding to a p-value less than 0.05.

To account for inter-individual variability and the potential overdispersion present when aggregating responses across participants, a beta-binomial approach was used. This method is particularly well-suited for replicated 2-AFC tests with categorical outcomes and limited repetitions per subject, as it offers a robust estimation of significance while controlling for response variability. Although other statistical models could be applied, they often require more complex assumptions or larger datasets. In contrast, the beta-binomial test is widely adopted in sensory science due to its simplicity, interpretability, and reliability in detecting perceptual differences under controlled experimental conditions.

3 Results

3.1 Perceptual results

For Experiment 1, which investigated the effect of product color on taste perception, no significant differences were observed. Participants did not consistently associate red-colored apple slices with sweetness or green-colored slices with sourness. The beta-binomial analysis indicated that the observed differences were not statistically significant (p > 0.05), suggesting that product color alone was insufficient to influence taste perception above chance in this context (Figure 7).

Figure 7

Figure 7. Results of the experiments tests. Each pair of bars represents the participants’ selections for each question. “H1 Sour” corresponds to Experiment 1, where participants indicated which sample (green or red) they perceived as sourer, while “H2 Sweet” refers to Experiment 2, where participants chose the sweeter sample, and so on. Statistically significant differences (p < 0.05) were observed only in the “H3 Sour” condition, as determined by the beta-binomial distribution statistics.

For Experiment 2, which examined the effect of context color and shape on taste perception, no significant differences were found. Participants did not perceive a consistent variation in sweetness or sourness based on the color of the surrounding context (red or green). The beta-binomial analysis also confirmed that the color of the context alone did not significantly alter taste perception (p > 0.05; Figure 7).

In Experiment 3, which tested the combined effect of product and context color (congruence effect), a significant result was observed exclusively in the green congruence condition (green product in a green and angular context) for sourness perception. In this condition, participants significantly selected the green snack as being sourer than the red one compared to other conditions. This difference was statistically significant (p < 0.05) with 95% CI [0.502, 1.0], indicating that the congruence between the green product and green and angular context heightened the perception of sourness. No significant effects were observed for sweetness in the red congruence condition; however, a noticeable trend was presented, suggesting a potential influence of color congruence on sweetness perception that warrants further investigation.

3.2 Conceptual results

Regarding prior knowledge of cross-modal correspondence, 80% of participants reported no familiarity with the concept, 18% said that they had some understanding, while only 2% stated to be fully familiar with it.

Figure 8 shows the answers participants gave to the question: “when considering basic tastes, which color do you associate with each one?” It can be observed that participants conceptually associated the colors red and pink with sweetness in 70% of the cases, while the colors green and yellow were linked to sourness by 68% of the participants.

Figure 8

Figure 8. Results from the survey question: “When considering basic tastes, which color do you associate with each one?.” The figure illustrates participants’ associations between specific colors and basic taste perceptions.

Figure 9 shows the relationship, according to the participants, between shapes and basic tastes. More than 90% of the participants associated sweet flavors with rounded shapes, while sour flavors were predominantly associated with sharp or angular shapes. The answers for these two tastes were more consistent among participants than for bitter and salty tastes.

Figure 9

Figure 9. Results from the survey question: “When considering basic tastes, which type of shape do you associate with each one?.” The figure presents participants’ associations between sharp and rounded shapes and basic taste perceptions.

Moreover, the survey also revealed that participants reported moderate to high comfort levels in the virtual eating environment, an average rating of 7.4 on a 9-point scale. When participants were asked whether they believed the experience would be the same if conducted in a real-world setting, 48% said that it would be perceived differently, 30% were uncertain, and 22% believed it would feel the same.

4 Discussion

The findings of this study contribute to the understanding of how the combination of color and shapes influence taste perception, with mixed results across the tested experiments. Experiments 1 and 2 did not yield significant effects, indicating that neither the color of the product nor that of the surrounding context, when considered independently, were sufficient to consistently alter the perception of sweetness or sourness. This contrasts with previous studies where both product and context colors were reported to significantly impact sensory evaluations (Woods and Spence, 2016; Spence and Levitan, 2021), likely due to smaller sample sizes and the use of subjective rating scales, which are more prone to bias. In contrast, our study employed a two-alternative forced-choice (2-AFC) test with a larger sample size (102 participants), providing a more robust and objective measure of taste perception. The use of a comparative method, rather than quantification with scales, reduces bias and increases the reliability of our findings, even when no significant differences were observed.

The subtle flavor profile of the dehydrated apple snack and the specific color manipulations may have made it harder for visual cues alone to produce detectable changes in taste perception, leading to the lack of significant effects in Experiments 1 and 2. However, the significant result in Experiment 3—where the congruence between the green product and the green angular context heightened the perception of sourness—supports the notion that cross-modal congruence can significantly modulate sensory experiences. This finding aligns with existing theories suggesting that congruent sensory stimuli intensify specific flavor perceptions (Saluja and Stevenson, 2018). The amplification of sourness in the green congruence condition is likely tied to learned associations between green colors and sour flavors, particularly in the context of fruits and acidic foods.

It is worth considering whether the saturation and brightness of the colors used were sufficient to trigger strong crossmodal associations. We chose hues (red/pink for sweetness and green/yellow for sourness) that were clearly identifiable yet kept within a natural-looking range of brightness and saturation. If the visual cues were not vivid enough, they might not fully engage the expected flavor associations. Prior research suggests that more intense color saturation can enhance perceived flavor intensity (Spence, 2019), so using even more vivid colors might have yielded stronger crossmodal effects. On the other hand, overly neon or unrealistic colors could reduce the credibility of the experience for participants. In our case, the color intensity was a compromise: noticeable but still plausible, which may have limited its impact. This point highlights a possible reason why color alone did not alter taste perception – a higher color vividness might be needed to produce a measurable effect.

The conceptual results further elucidate how participants mentally associate colors and shapes with specific tastes. As illustrated in Figure 8, the majority of participants (70%) associated red and pink with sweetness, while 68% linked green and yellow to sourness. These findings suggest a strong cognitive bias aligned with cultural and environmental cues about color-taste correspondences, consistent with prior research linking warm colors (e.g., red and pink) to sweetness and cool, acidic colors (e.g., green and yellow) to sourness (Velasco et al., 2015). However, this might not be enough to alter the actual sensory perception of the product.

Moreover, the strong associations between rounded shapes and sweetness (over 90%), and between angular shapes and sourness align with established cross-modal correspondence literature. Rounded forms are generally perceived as more pleasant and are linked to sweetness, whereas sharp, angular forms evoke more intense reactions associated with sour or bitter flavors (Spence and Levitan, 2021). These findings reinforce the notion that both color and shape significantly influence taste perceptions, even in the absence of direct sensory input.

Interestingly, nearly half of the participants (48%) believed the tasting experience would feel different in a real-world setting as opposed to the VR scenario. This feedback highlights a limitation in ecological validity: despite the high level of control and realism provided by the MR setup, some participants felt that a virtual context is not entirely equivalent to a real one. One possible reason is that the MR environment, while immersive, lacked certain sensory cues present in reality (e.g., genuine ambient lighting, aromas, or the natural look of real food), which may have led participants to be more skeptical or less influenced by the virtual cues. In a real setting, those visual cues might interact with other senses and context factors to produce a stronger effect on taste perception. This interpretation aligns with recent findings suggesting that multisensory taste effects observed in XR can be weaker than in real life (Dawes et al., 2023).

Unlike studies relying exclusively on visual cues or labeling (Piqueras-Fiszman and Spence, 2015; Huang et al., 2019), this research allowed participants to taste the product using an advanced mixed reality (MR) setup that included Meta Spatial Anchors for accurate object localization and synchronized hand tracking for natural interaction. This multimodal and interactive approach provides deeper insights into how expectations formed by visual cues are either confirmed or challenged during actual consumption. It also enhances our understanding of how visual, tactile, and taste modalities integrate during the eating experience, particularly in solid food samples, a notable contribution to HCI methodologies in sensory evaluation. This multimodal approach provides deeper insights into how expectations formed by visual cues are either confirmed or challenged during actual consumption. It also enhances our understanding of how visual, tactile, and taste modalities integrate during the eating experience, particularly in solid food samples.

The use of solid food, as opposed to liquids or mere visual stimuli, adds a layer of realism and complexity to the sensory experience. Solid foods introduce additional sensory elements, such as texture and mouthfeel, which substantially impact overall flavor perception. Consequently, the findings of this study possess a higher degree of complexity compared to studies focusing solely on liquids (Wang et al., 2020).

MR enabled precise manipulation of both the food’s appearance (via dynamic color changes to the tracked physical snacks) and the surrounding context. The use of spatial anchors was crucial for maintaining the alignment between the real food and its virtual representation, while hand tracking facilitated a more intuitive and ecologically valid interaction than traditional screen-based or less interactive VR setups. This precise control over visual elements, without altering the food’s intrinsic properties, is a significant advantage for cross-modal research (Spence and Levitan, 2021). Despite some challenges related to cognitive load while navigating a MR setting during eating, most participants reported feeling comfortable within the virtual context, indicating that VR is a feasible and effective tool for sensory testing (Wang et al., 2021).

These findings, and the innovative methodology employing Meta Spatial Anchors and integrated hand tracking within Unity (C#), hold significant implications for the food industry, particularly in product development and marketing. By leveraging MR technology, it is possible to manipulate visual elements, such as color, to enhance or modify taste perceptions without altering the actual formulation of the product. This approach could be applied to create more engaging consumer experiences, promote different food choices, or reduce the need for additives by enhancing perceived taste through visual manipulations. However, it should be noted that the study is highly context-dependent and that it should be further evaluated using similar colors but other scenarios.

The successful implementation of this experimental design demonstrates the feasibility of using consumer-grade MR headsets, enhanced with sophisticated tracking and interaction mechanisms like Meta Spatial Anchors and SDK-based hand tracking, for complex sensory studies, paving the way for more accessible and versatile HCI research in this domain.

5 Conclusion

In conclusion, this study found limited but meaningful evidence for the role of color and shape in taste perception, particularly when considering the congruence between product and context. Although product and context colors alone did not significantly alter perceptions of sweetness or sourness, the congruence between green product and green context amplified sourness perception, highlighting the importance of sensory congruence in cross-modal correspondences.

Conceptually, participants consistently associated red and pink with sweetness and green and yellow with sourness, confirming existing theories on color-taste associations. Furthermore, the strong association between rounded shapes and sweetness and angular shapes with sourness underscores the role of cross-modal correspondences between visual shapes and taste perception. These results suggest that color, context, and shape all interact to influence how we perceive flavors, even when those sensory elements are presented in isolation or in a conceptual format.

Virtual reality and mixed reality technologies offer promising opportunities for exploring these interactions, providing controlled contexts for manipulating sensory inputs, and investigating how these correspondences affect consumer experiences. Future research should continue to explore the potential of these specific MR interaction paradigms—such as refining hand-based interactions with augmented food items and leveraging persistent spatial anchors for longitudinal studies—to manipulate additional sensory dimensions, such as texture and aroma, to create even more immersive and informative sensory experiences. This could further advance the development of immersive and informative sensory experiences, contributing new tools and techniques to the field of Human-Computer Interaction. These insights could inform product development and sensory marketing strategies aimed at enhancing consumer engagement through multisensory design.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Comité de Etica en Investigación de la Universidad Católica Argentina. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

MG: Formal analysis, Funding acquisition, Investigation, Writing – original draft, Writing – review & editing. J-CS: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Supervision, Writing – review & editing. MVG: Conceptualization, Investigation, Methodology, Supervision, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This research was funded by Instituto Nacional de Tecnología Agropecuaria (INTA).

Acknowledgments

The authors would like to thank all the consumers who participated in the test and the SIED laboratory (Sistema Institucional de Educación a Distancia) from UCA, for providing the oculus for the complete experiment.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ammann, J., Stucki, M., and Siegrist, M. (2020). True colours: advantages and challenges of virtual reality in a sensory science experiment on the influence of colour on flavour identification. Food Qual. Prefer. 86:103998. doi: 10.1016/j.foodqual.2020.103998

PubMed Abstract | Crossref Full Text | Google Scholar

Ares, G. (2019). Special issue on “virtual reality and food: applications in sensory and consumer science”. Food Res. Int. 117:1. doi: 10.1016/j.foodres.2019.02.001

PubMed Abstract | Crossref Full Text | Google Scholar

Bi, J., Ennis, D. M., and O’mahony, M. (1997). How to estimate and use the variance of d’ from difference tests. J. Sens. Stud. 12, 87–104. doi: 10.1111/j.1745-459X.1997.tb00055.x

Crossref Full Text | Google Scholar

Carvalho, F. M., and Spence, C. (2019). Cup colour influences consumers’ expectations and experience on tasting specialty coffee. Food Qual. Prefer. 75, 157–169.

Google Scholar

Cornelio, P., Dawes, C., Maggioni, E., Bernardo, F., Schwalk, M., Mai, M., et al. (2022). Virtually tasty: An investigation of the effect of ambient lighting and 3D-shaped taste stimuli on taste perception in virtual reality. Int. J. Gastron. Food Sci. 30:100626. doi: 10.1016/j.ijgfs.2022.100626

Crossref Full Text | Google Scholar

Crofton, E. C., Botinestean, C., Fenelon, M., and Gallagher, E. (2019). Potential applications for virtual and augmented reality technologies in sensory science. Innov. Food Sci. Emerg. Technol. 56:102178. doi: 10.1016/j.ifset.2019.102178

PubMed Abstract | Crossref Full Text | Google Scholar

Dawes, C., Beganovic, S., Schwalk, M., Mai, M., Pawlizak, S., Zhang, J., et al. (2023). Reality bites: highlighting the potential discrepancies between multisensory taste perception in extended and physical reality. Front. Comput. Sci. 5:849. doi: 10.3389/fcomp.2023.1268849

Crossref Full Text | Google Scholar

Dematte, M. L., Pojer, N., Endrizzi, I., Corollaro, M. L., Betta, E., Aprea, E., et al. (2014). Effects of the sound of the bite on apple perceived crispness and hardness. Food Qual. Prefer. 38, 58–64.

Google Scholar

Deroy, O., and Valentin, D. (2011). Tasting liquid shapes: Investigating the sensory basis of cross-modal correspondences. Chem Percept. 4, 80–90.

Google Scholar

Ennis, D. M. (1990). Relative power of difference testing methods in sensory evaluation. Food Technol. 44, 114–117.

Google Scholar

Fuentes, S., Tongson, E., and Gonzalez Viejo, C. (2021). Novel digital technologies implemented in sensory science and consumer perception. Curr. Opin. Food Sci. 41, 99–106. doi: 10.1016/j.cofs.2021.03.014

Crossref Full Text | Google Scholar

Halabi, O., and Saleh, M. (2021). Augmented reality flavor: cross-modal mapping across gustation, olfaction, and vision. Multimed. Tools Appl. 80, 36423–36441. doi: 10.1007/s11042-021-11321-0

PubMed Abstract | Crossref Full Text | Google Scholar

Huang, F., Huang, J., and Wan, X. (2019). Influence of virtual color on taste: Multisensory integration between virtual and real worlds. Comput. Hum. Behav, 95, 168–174. doi: 10.1016/J.CHB.2019.01.027

Crossref Full Text | Google Scholar

Mesz, B., Sakdavong, J.-C., Silén, S., and Hopia, A. (2023). Aesthetic emotions in a mixed reality gastrosonic experience: an exploratory study. Digit. Creat. 35, 74–89. doi: 10.1080/14626268.2023.2287189

Crossref Full Text | Google Scholar

Nehmé, L., Barbar, R., Maric, Y., and Jacquot, M. (2016). Influence of odor function and color symbolism in odor–color associations: a French–Lebanese–Taiwanese cross-cultural study. Food Qual. Prefer. 49, 33–41. doi: 10.1016/j.foodqual.2015.11.002

Crossref Full Text | Google Scholar

Ngo, M. K., Misra, R., and Spence, C. (2011). Assessing the shapes and speech sounds that people associate with chocolate samples varying in cocoa content. Food Qual. Prefer. 22, 567–572.

Google Scholar

Piqueras-Fiszman, B., and Spence, C. (2015). Sensory expectations based on product-extrinsic food cues: An interdisciplinary review of the empirical evidence and theoretical accounts. Food Quality and Preference, 40, 165–179. doi: 10.1016/j.foodqual.2014.09.013

Crossref Full Text | Google Scholar

Saluja, S., and Stevenson, R. J. (2018). Cross-modal associations between real tastes and colors. Chem. Senses 43, 475–480. doi: 10.1093/chemse/bjy033

PubMed Abstract | Crossref Full Text | Google Scholar

Small, D. M. (2012). Flavor is in the brain. Physiol. Behav. 107, 540–552. doi: 10.1016/j.physbeh.2012.04.011

PubMed Abstract | Crossref Full Text | Google Scholar

Spence, C. (2015). On the psychological impact of food colour. Flavour, 4, 1–16. doi: 10.1186/s13411-015-0031-3

Crossref Full Text | Google Scholar

Spence, C. (2019). On the relationship (s) between color and taste/flavor. Exp. Psychol. 66, 99–111. doi: 10.1027/1618-3169/a000439

PubMed Abstract | Crossref Full Text | Google Scholar

Spence, C., and Levitan, C. A. (2021). Explaining crossmodal correspondences between colours and tastes. Iperception 12, 1–28. doi: 10.1177/20416695211018223

Crossref Full Text | Google Scholar

Spence, C., Levitan, C. A., Shankar, M. U., and Zampini, M. (2010). Does food color influence taste and flavor perception in humans? Chemosens. Percept. 3, 68–84. doi: 10.1007/s12078-010-9067-z

Crossref Full Text | Google Scholar

Velasco, C., Woods, A. T., Deroy, O., and Spence, C. (2015). Hedonic mediation of the crossmodal correspondence between taste and shape. Food Qual. Prefer. 41, 151–158. doi: 10.1016/j.foodqual.2014.11.010

Crossref Full Text | Google Scholar

Wang, Q. J., Barbosa Escobar, F., Da Alves Mota, P., and Velasco, C. (2021). Getting started with virtual reality for sensory and consumer science: current practices and future perspectives. Food Res. Int. 145:410. doi: 10.1016/j.foodres.2021.110410

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, Q., Wang, Q. J., Meyer, R., Waters, S., and Zendle, D. (2020). A dash of virtual milk: altering product color in virtual reality influences flavor perception of cold-brew coffee. Front. Psychol. 11:788. doi: 10.3389/fpsyg.2020.595788

PubMed Abstract | Crossref Full Text | Google Scholar

Woods, A. T., and Spence, C. (2016). Using single colors and color pairs to communicate basic tastes. Iperception 7, 1–15. doi: 10.1177/2041669516658817

Crossref Full Text | Google Scholar

Wu, Z., Shi, R., Li, Z., Jiang, M., Li, Y., Yu, L., et al. (2022). Examining cross-modal correspondence between ambient color and taste perception in virtual reality. Front. Virtual Real. 3, 1–19. doi: 10.3389/frvir.2022.1056782

PubMed Abstract | Crossref Full Text | Google Scholar

Zampini, M., and Spence, C. (2005). The role of auditory cus in modulaiton the perceived crispness and staleness of potato chips. J. Sens. Stud. 19, 347–363. doi: 10.1111/j.1745-459x.2004.080403.x

Crossref Full Text | Google Scholar