Dynamics of autonomic nervous system responses and facial expressions to odors

Why we like or dislike certain products may be better captured by physiological and behavioral measures of the autonomic nervous system (ANS) than by conscious or classical sensory tests. Responses to pleasant and unpleasant food odors presented in varying concentrations were assessed continuously using facial expressions and responses of the ANS. Results of 26 young and healthy female participants showed that the unpleasant fish odor triggered higher heart rates and skin conductance responses, lower skin temperature, fewer neutral facial expressions and more disgusted and angry expressions (p < 0.05). Neutral facial expressions differentiated between odors within 100 ms, after the start of the odor presentation followed by expressions of disgust (180 ms), anger (500 ms), surprised (580 ms), sadness (820 ms), scared (1020 ms), and happy (1780 ms) (all p-values < 0.05). Heart rate differentiated between odors after 400 ms, whereas skin conductance responses differentiated between odors after 3920 ms. At shorter intervals (between 520 and 1000 ms and between 2690 and 3880 ms) skin temperature for fish was higher than that for orange, but became considerable lower after 5440 ms. This temporal unfolding of emotions in reactions to odors, as seen in facial expressions and physiological measurements supports sequential appraisal theories.


INTRODUCTION
Up to 80% of all new food products fail in the marketplace, despite the fact that they are typically subjected to a large number of sensory and consumer tests before their market introduction (Crawford, 1977). This suggests that the "standard" sensory and consumer tests, which typically include sensory analytical profiling and liking tests, have a low predictive validity with respect to general product performance. Possibly, consumer food choice outside the laboratory may be less based on cognitive information processing and rational reasoning, and more on unarticulated/unconscious motives and associations (Wansink, 2004). Reasons for likes or dislikes of specific foods are typically difficult to articulate but may determine much of our food choice. Unarticulated/unconscious motives and associations are not very well captured by traditional tests based on conscious cognitive processes, and may be better captured by physiological and behavioral measures (e.g., facial expressions) of the autonomic nervous system (ANS) which do not require conscious processes (Greenwald, 2009).
Physiological measures have been used extensively to capture responses of the ANS to various types of stimuli such as film clips, personalized recall of specific situations, and odors. In a previous study, Alaoui-Ismaïli et al. (1997) related various autonomic parameters to the pleasantness of five odorants, and found that unpleasant odors were associated with increased heart rate (HR) and longer skin conductance responses (SCR) compared to pleasant odors. Bensafi et al. (2002) related ANS measures to rated pleasantness, arousal, intensity, and familiarity for a set of six odorants and found that their results could be explained by two main factors: pleasantness, inversely related to HR (similar to Alaoui-Ismaïli et al., 1997) and arousal, positively related to skin conductance and rated intensity. Delplanque et al. (2009) found stronger SCR and higher HR for unpleasant compared to pleasant odors. They also established that HR differences between pleasant and unpleasant odors occurred relatively late in the deceleration phase, approximately 5-8 s after odor presentation. Considerable faster odor-specific responses were found for facial expressions; facial muscle activity associated with positive and negative facial expressions showed different activities for pleasant and unpleasant odors as soon as 400-500 ms after odor presentation (Delplanque et al., 2009).
Facial expressions have also been used extensively by others to measure emotional responses to food-related stimuli. Wellknown are the positive facial expressions of new-borns toward liked (sweet) and the negative expressions toward disliked (bitter) basic tastes, extensively documented by Steiner (1973). More recently, an automated tool, FaceReader, has been developed and used to analyze more diverse, universal facial expressions. Using different food stimuli, it was found that happy expressions were not systematically related to liking scores, in contrast to neutral, angry, and disgusted expressions (Danner et al., 2014), and that stronger facial expressions to disliked foods compared to liked foods were already detected at the first visual encounter with the food (De Wijk et al., 2012).
The dynamic features over time of physiological responses and facial expressions have typical been outside the scope of most studies, even though they play a key role in several modern theories on emotion, the so-called componential appraisal models (see Ellsworth and Scherer, 2003 for an overview). The models assume that the elicitation and the differentiation of emotions are determined by appraisals, the continuous, recursive evaluations of events, Delplanque et al. (2009) investigated the appraisal of odor novelty and pleasantness and consequent emotional responses by measuring facial muscle activity and HR. They demonstrated that odors were detected as novel or familiar before being evaluated as pleasant or unpleasant (Distel et al., 1991;Royet et al., 1999). In addition, their results also argued in favor of a dynamic construction of facial expressions providing support for sequential appraisal theories (Ellsworth and Scherer, 2003). For example, early reactions, such as raising the eyebrows and opening the eyes, were related to the detection of a novel or unexpected stimulus, which is associated with increased alertness and attention. After this novelty detection, assessment of pleasantness may lead to avoidance when the stimulus is aversive or threatening, or approach when a pleasant response is activated.
The present study will expand on previous studies by using (food) odors delivered by an olfactometer, offering a high degree of control over timing and concentrations, and by incorporating additional ANS measures [skin temperature (ST)] and other types of facial expressions. Similar to Delplanque et al. (2009) the present study will also focus on the temporal development of each measure instead of the more commonly used timeaveraged means (e.g., De Wijk et al., 2012;Danner et al., 2014). Physiological responses and facial expressions will be measured continuously and analyses will be based on time-averaged means (similar to most of the previous studies) as well as on their temporal development. It is hypothesized that the results based on time-averaged means will replicate the findings of similar studies by others, i.e., higher HR and skin conductance, lower ST and more negative facial expressions after exposure to the unpleasant odor compared to exposure to the pleasant odor. It is further hypothesized that ANS responses are slower than facial expressions, but that both follow sequential appraisal processes of evaluating the stimuli.

PARTICIPANTS
Twenty-six young healthy female participants (mean age: 22.6 ± 1.5 years, range: 20-25 years, 18.5 < BMI < 25 kg/m 2 ) were recruited from the subject pool of Food and Biobased Research, part of Wageningen University and Research Center. Participants self-reported their BMI and if they had actual/previous history of smell or taste disorders known to affect chemosensory function. Detailed information regarding the experiment was given and an informed consent form was signed by all participants prior to testing. The study was approved by the Medical Ethical Committee of the Wageningen University.

ODOR STIMULI AND PRESENTATION
As described elsewhere (He et al., under review), two food odors were selected on the basis of their relatively negative (fish odor) or positive (orange odor) valence (Boesveldt et al., 2010). The orange (cold-pressed Californian orange oil, Sigma Aldrich, St. Louis, MO, USA) and fish (Fish flavor oil, Givaudan Inc., Geneva, Switzerland) odors were diluted with mineral oil (to 70%, v/v) and 1,2-propanediol (to 27%, v/v), respectively. With a dynamic olfactometer based on air-dilution (OM2s, Burghart instruments, Wedel, Germany), each odor was delivered in three different concentrations (low, medium, or high), correspondingly perceived at different intensities in a pilot study. The olfactometer allows the presentation of odorous stimuli within a continuous humidified (80%) and warmed (37 • C) airstream of 8 L/min, which does not alter the mechanical or thermal conditions at the nasal mucosa (Kobal and Hummel, 1988). These stimuli were delivered through a nosepiece for 1 s with an inter stimulus interval of 60 s. Each block of six stimuli (i.e., orange odor in three concentrations and fish odor in three concentrations) was randomized and presented five times, for a total of 30 stimuli.

PROCEDURE
The experimental sessions took place in the physiological laboratory of the Restaurant of the Future located in Wageningen, the Netherlands. The experiment leader explained the experiment to the participant, allowed ample time for questions and asked the participant to sign the inform consent form (which they had received by e-mail prior to the experimental session) after which the electrodes were placed. Participants were seated in a comfortable chair, fitted with the olfactometer nosepiece, and oriented toward an adjustable computer monitor set with a webcam at eye-level (1 m viewing distance). They were asked to look directly toward the camera while receiving the odor stimulus to ensure recognition by the FaceReader software. Each trial started with an auditory attention signal to remind the participant to pay attention to the upcoming odor. The pleasantness and intensity of each odor was rated subsequently on a paper questionnaire 10 s after stimulation. The procedure is also shown schematically in Figure 1. The whole experiment lasted 45 min in total. Photograph 1 shows the set-up as used in this study.

Physiological ANS measures
Physiological measures included: 1. Skin conductance response (SCR) measured in µSiemens with electrodes placed on the palm of the non-dominant hand of the participant. 2. Heart rate (HR) measured in beats per minute with electrodes placed on the chest. 3. Skin temperature (ST) measured in degrees Celsius with an electrode placed on the palm of the non-dominant hand of the participant.
The physiological data were collected at 200 Hz via a MindWare Acquisition data acquisition system (MindWare Technologies, Inc.) with separate filter settings for the electrocardiogram, finger temperature and electrodermal (SCR) activity. Filter settings were low-pass 0.5 Hz, high-pass 45 Hz for HR frequency, low-pass 1 Hz, high-pass 45 Hz for SCR, and low-pass 10 Hz, high-pass 45 Hz for ST. Electrodes were used with a surface of 4.1 cm 2 and filled with 1% Chloride wet gel. Signals were transferred to the Acquisition Unit (16-bit A/D conversion) and stored on computer hard disk (sampling rate 500 Hz/s). Electrocardiographic R waves were detected offline, and intervals between heartbeats were converted to HR, expressed in beats per minute (BPM). SCR activity was recorded (high-pass filter: 0.025 Hz.) by the constant voltage method (0.5 V). The signal was amplified by 1000 and low-pass filtered (30 Hz).

Facial expressions
Facial expressions were automatically analyzed using FaceReader software version 4.0 (Noldus Information Technology B.V.). FaceReader works in three steps: (1) face finding, (2) face modeling, and (3) face classification. During face finding an accurate position of the face is found using the Active Template Method. During modeling, the Active Appearance Model is used to synthesize an artificial face model, which describes the location of 491 key points as well as the texture of the face. The actual classification of the facial expressions is done by training an artificial neural network as training material nearly 2000 manually annotated images were used. The network was trained to classify the six basic or universal emotions described by Ekman (1992): happy, sad, angry, surprised, scared, and disgusted and a neutral state. FaceReader analyzed the facial expressions on a frame-by-frame basis, i.e., at 25 Hz. Previous studies showed that FaceReader results corresponded between 71% (angry) to 99% (neutral) of all cases, with an average of 87%, with results from human observers (Terzis et al., 2012). FaceReader happiness scores correlated significantly (r = 0.79) with objectively measured activity in the zygomaticus supercilli or cheek muscle, a muscle that is activated during expressions of happiness (D'Arcey et al., 2012). A more detailed description of the science behind FaceReader can be found at: http://info.noldus.com/ free-white-paper-on-facereader-methodology/.

DATA ANALYSIS
The processed images with the facial expressions were combined with raw physiological data in Observer XT 10.5 software (Noldus Information Technology) for further analyses. The moments that odors were presented to the participants were marked automatically using the "trigger-out" signal from the olfactometer that signals the start of each odor presentation. The physiological measures SCR, HR, and ST were analyzed per odor presentation. The video images of the facial expressions were processed per odor presentation with FaceReader 4.0 software (Noldus Information Technology). Due to a technical malfunction, absolute ST values were not recorded, but the results can still be used to assess changes over time in ST per odor presentation. Results from some participants had to be removed from the analysis due to a large number of artifacts. The number of participants that is included in the analysis is 21 (  facial expressions become odor-specific (i.e., differ significantly between odors), absolute deltas between orange and fish odors were calculated together with the standard deviations for the 2.5 s interval preceding odor presentation to establish a pre-odor baseline. Subsequently, post-odor times were identified at which the absolute delta between the odors exceeded the pre-odor average plus three times the pre-odor standard deviation.

Facial expressions
Time-averaged means of facial expressions to the fish compared to the orange odor were significantly less neutral [F (1, 23) = 21.25, p < 0.001; Figure 2D] and more disgusted Associations between physiological measures, facial expressions, and ratings are summarized by correlational analysis based on 24 stimuli (two odors × three concentrations × four replicates) averaged across participants ( Table 2).

Physiological measures
Prior to the odor presentation, but after the warning signal is given, ANS measures show gradual changes that are independent of the odor valence whereby skin conductance and HR gradually increase and ST gradually decreases (Figure 3). Skin conductance continues to increase for seconds after odor presentation independent of the specific odor. After approximately 3 s, SCR for orange decreases whereas that for fish continues to increase. The difference in SCR becomes significant after 3920 ms ( Figure 3B and Table 3). Heart rate for the unpleasant fish odor increases almost instantaneously after the odor is presented whereas HR for the pleasant orange odor shows much smaller effects ( Figure 3A and Table 3). The difference in HR response between the odors becomes significant after 400 ms. Skin temperature follows a different, irregular pattern with higher temperatures for fish odor at shorter intervals (between 520 and 1000 ms and between 2690 and 3880 ms) and lower temperature at longer intervals (after 5440 ms) ( Figure 3C and Table 3) compared to orange odor.

Facial expressions
Neutral expressions become odor-specific after less than 100 ms. Disgusted expressions take approximately another 100 ms to become odor-specific. Angry, surprised, sad, and scared become after 500-1000 ms odor-specific, whereas happy expression become odor-specific after more than 1700 ms (Figure 4 and Table 3). Table 3 summarizes the times at which ANS responses and facial expressions significantly differentiate between the unpleasant fish and pleasant orange odor.

DISCUSSION
Human responses to pleasant and unpleasant food odors presented in varying concentrations were assessed with facial expressions and responses of the ANS. Analysis were carried out on results with and without averaging over time, and showed partly overlapping and partly different results.
ANOVAs on time-averaged results showed that the unpleasant fish odor triggered higher HR and SCR, lower ST, fewer neutral facial expressions and more disgusted and angry expressions  compared to the pleasant orange odor. Overall, our results were similar to the ones found in studies by others for HR (Alaoui-Ismaïli et al., 1997;Bensafi et al., 2002;Delplanque et al., 2009), skin conductance (Alaoui-Ismaïli et al., 1997;Delplanque et al., 2009), and ST (see Köster, 2009), indicating that these averaged physiological measurements are mainly responsive to the valence of a stimulus, and less to intensity, whereas facial expressions appear to demonstrate more concentration-specific effects.  facial expressions, and negative associations between odor liking and all other facial expressions, including happiness. Negative associations between odor liking and happy facial expressions have also been reported previously by others (Zeinstra et al., 2009;Danner et al., 2014;He et al., under review) suggesting that happy expressions cannot discriminate liked or disliked foods implicitly. Facial expressions of happiness are rarely displayed when one is alone and social interactions are absent suggesting that these expressions serve a social function (see also Gilbert et al., 1987 andParkinson, 2005). The fact that they did occur in this study in the presence of experimental staff suggests that the happy facial expressions may serve some kind of social signaling function, e.g., to signal the staff that one is OK despite the previous display of negative expressions associated with disliked odors. When results are not averaged across time, analyses demonstrate that facial expressions and physiological responses become rapidly odor-specific and are dynamic in nature. Responses such as skin conductance already start before the actual odor presentation These responses are obviously odor non-specific and probably reflect anticipatory processes, Almost immediately after the onset of the odor presentation, neutral facial expressions decrease followed after 100 ms by an increase in facial expressions of disgust. Within 400 ms HR for the unpleasant odor increase (similar to rapid acceleration in HR observed for negative emotions by Levenson, 1988), ST briefly increases, followed between 500 and 1000 ms by facial expressions of angry, surprised, sad, and scared, and after 1700 ms by happy expressions. During all this time, skin conductance gradually increases for both odors until approximately 3 s when skin conductance for the pleasant odor starts to decrease whereas that for the unpleasant odor continues to increase. Finally, after more than 4 s, skin conductance for the unpleasant odor decreases together with ST for the unpleasant odor. Combined these time-related results show that most facial expressions and physiological responses are fast reacting and odor-specific.

FIGURE 2 | Effects of odor and concentration on (A) skin conductance responses, (B) heart rate, (C) skin temperature, and (D) neutral facial expressions (averaged across time and bars indicate standard errors).
Our results correspond well with those found in previous studies; Delplanque et al. (2009) found odor-specific activities in two types of facial muscle activities 400-500 ms after odor presentation, which coincides approximately with sad, angry, and surprised expressions in the present study. These values also concur with the values found for other stimulus modalities such as vision; Dimberg et al. (2002) found facial responses to positive or negative visual stimuli after approximately 400-500 ms. In addition, we found other expressions that were triggered even faster, such as disgust, or slower, such as happy.
Response times for HR and for most of the facial expressions are well within 1 s after the odor is presented, and are often shorter than for example response time for odor detection (approximately 800 ms, De Wijk, 1989) or response time to decide whether or not an odor is more pleasant than a previous one (approximately 850 ms, Olofsson et al., 2012), where conscious action is needed. These differences in timing are possibly related to automated vs. conscious processes in the central nervous system. Facial expressions and ANS responses probably reflect automated processing of the central nervous system (see Dimberg et al., 2002 for automated processes and facial expressions), whereas decisions regarding detection and pleasantness/unpleasantness require also time-consuming conscious processing. The fact that automated emotional odor-response times may be as fast as response times in the visual domain despite the relatively slow peripheral and peri-peripheral processing of odors may reflect the anatomical overlap between CNS structures involved in olfaction and emotions; the peripheral and central olfactory system are only separated by one relay (glomerulus of the olfactory bulb) after the odor interacts with the primary olfactory neurons. Next, olfactory information is conducted to other olfactory structures, some of which are also involved in emotions (hippocampus, anterior cingulate cortex, orbitofrontal cortex and parts of the amygdala and insula Lundström et al., 2011;Soudry et al., 2011). Given the close correspondence of CNS structures involved in olfaction and emotions and the fact that these structures are activated simultaneously to when information becomes available for conscious, higher order cognitive processing in the cortex, it is no longer surprising that automated emotional odor response times are often faster than odor response times that involve conscious processing.
Combined the time-series responses found in this study show that most facial expressions and physiological responses are fast reacting and odor-specific. Moreover, different facial expressions and physiological measures develop at their own specific rate over time. Consequently, responses to the same stimulus may produce very different patterns of results depending on the time at which they are assessed. For example, fast responses around 500 ms, may be dominated by negative facial expressions such as disgust, increased HR and increased ST, whereas slower responses may be dominated by positive facial expressions, lower HR and decreased ST. The fast responses may be automated reflexes to novel and potentially dangerous stimuli, as observed by Delplanque et al. (2009), whereas the later responses may reflect a conscious processing of a sequence of different emotions, each resulting from a different appraisal of the stimulus by the observer (e.g., Ellsworth and Scherer, 2003). Results from the same laboratory indicate that conscious evaluative ratings of participants are associated with ANS responses and facial expressions between one and three seconds after stimulation (He et al., under review). This supports the notion that the fast responses, with response times of less than one second, are automated and relatively independent of evaluative ratings, whereas slower responses reflect conscious processing that form the basis for evaluative ratings and facial expressions of happiness for communicative purposes.
The present study has its obvious limitations; only a small number of odors were investigated, and their effects were investigated under controlled laboratory conditions with female participants. Nevertheless, the results may have some implications for consumer behavior in the real world. For example, visitors to supermarkets may have approximately 45 min to select their weekly groceries from up to 30,000 products. This task becomes even more daunting considering the fact that many of these selections are not planned but made in the supermarket. Given this abundance of choices consumers need a fast and partly www.frontiersin.org February 2014 | Volume 5 | Article 110 | 7 automated selection mechanism that combines affect, appraisal, action readiness and autonomic arousal. This fast selection mechanism may be based on fast and probably automated ANS responses and facial expressions similar to the ones found in the present study. These fast responses may not only be triggered by odors, but also product packages and brand names. To explore real-life applications, future studies will measure ANS responses and facial expressions in relation to consumer choice behavior. Initially, consumer behavior will be assessed in the semireal-life test environment of a virtual supermarket, followed by real-life assessment in an actual supermarket. Such studies will allow a proper evaluation of ANS measures and facial expressions as tools for marketing (research) because their associations with consumer product interactions and purchasing behaviors will be tested directly.
In summary, physiological and facial responses to odors prove to be fast and dynamic and the balance between these responses is continuously changing depending on their timing. This changing balance may reflect different sequential appraisals of emotions. This study along with other recent studies (e.g., Delplanque et al., 2009) shows the necessity of taking the time dimension into account and future studies should further explore the relation between dynamic responses and appraisals.