# EMOTION AND BEHAVIOR

EDITED BY: Fritz Strack, Paul Pauli and Peter Weyers PUBLISHED IN: Frontiers in Psychology

#### *Frontiers Copyright Statement*

*© Copyright 2007-2016 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

*The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88919-965-5 DOI 10.3389/978-2-88919-965-5

# About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **EMOTION AND BEHAVIOR**

Topic Editors: **Fritz Strack,** University of Würzburg, Germany **Paul Pauli,** University of Würzburg, Germany **Peter Weyers,** University of Würzburg, Germany

Emotions by L. K. Pauli

**Citation:** Strack, F., Pauli, P., Weyers, P., eds. (2016). Emotion and Behavior. Lausanne: Frontiers Media. doi: 10.3389/978-2-88919-965-5

# Table of Contents


# **Basic aspects of emotional-impulsive processing**


Antje B. M. Gerdes, Matthias J. Wieser and Georg W. Alpers

# **Emotional processes underlying approach**


Roland Deutsch, Kevin J. M. Smith, Robert Kordts-Freudinger and Regina Reichardt

# **Emotional processes underlying avoidance**

*85 The impact of perception and presence on emotional reactions: a review of research in virtual reality*

Julia Diemer, Georg W. Alpers, Henrik M. Peperkorn, Youssef Shiban and Andreas Mühlberger


Andre Pittig, Mirko Pawlikowski, Michelle G. Craske and Georg W. Alpers

# **Other emotional processes**

*119 Being watched by others eliminates the effect of emotional arousal on inhibitory control*

Jiaxin Yu, Philip Tseng, Neil G. Muggleton and Chi-Hung Juan

*124 Musical activity and emotional competence – a twin study* Töres P. Theorell, Anna-Karin Lennartsson, Miriam A. Mosing and Fredrik Ullén

# Editorial: Emotion and Behavior

Fritz Strack \*, Paul Pauli\* and Peter Weyers

Department of Psychology, University of Würzburg, Würzburg, Germany

Keywords: emotions, behavior, approach, avoidance, impulsive behavior

**The Editorial on the Research topic**

## **Emotion and Behavior**

Within the field of psychology, there exists a strict division between disciplines that focus on "normal" vs. "dysfunctional" behavior. Despite its practical value, this distinction may be an obstacle that prevents cross-fertilization. While seemingly dysfunctional mechanisms have long been used to understand the regularities of human perception and cognition, behavioral aberrations have rarely been employed to explain the normality of human action.

The current collection of research reports is meant to overcome this obstacle in the domain of emotion. In particular, the contributors to this special issue share the conviction, that human behavior is not solely determined by the reflective evaluation of its anticipated consequences. Instead, affective impulses are also relevant in fueling approach or avoidance. This is particularly obvious when it comes to phenomena of temptation or addiction when impulsive mechanisms gain the upper hand. In a related fashion, a phobic person typically knows that a phobic situation may not be dangerous although he feels afraid and acts accordingly.

These observations, along with a great number of experimental results, suggest that emotion and behavior may be linked in at least two ways that can be described as reflective and impulsive. In a more systematic fashion, the two psychological mechanisms have been described in the context of the Reflective-Impulsive Model (RIM; Strack and Deutsch, 2004, 2015), which provides a conceptual orientation for the reported research.

A second denominator of the collected papers is the attempt to merge different levels of analysis. Specifically, they range from basic neuronal to social approaches with the aim to understand the different ways of interaction between emotion and behavior.

The basic themes to which the selected papers contribute are: (a) basic aspects of emotionalimpulsive processing, (b) emotional processes underlying approach, and (c) emotional processes underlying avoidance.

**Basic aspects of emotional-impulsive processing** are addressed by Kozlik, Neumann, and Lozo who summarize findings suggesting that mechanisms of evaluative coding may better account for approach-avoidance behaviors than the principles of motivational orientation. Seibt, Mühlberger, Likowski, and Weyers address the phenomenon of facial mimicry in social situations, i.e., in response to emotional facial expressions of others. Based on results from EMG and fMRI, the authors identify the basic psychological processes that explain congruent and incongruent facial responses. Findings on multimodal interactions between pictures and sounds in their impact on emotional experiences are reviewed by Gerdes, Wieser, and Alpers. They conclude on basic interaction mechanisms leading to a congruent emotional experience.

**Emotional processes underlying approach** are discussed by Seibt, Häfner, and Deutsch with regard to sexual approach as a function of gender and both objective and subjective deprivation. Their results demonstrate greater approach for men than for women and show that subjective desire mediates objective deprivation as a determinant. Wu, Winkler, Wieser, Andreatta, Li, and Pauli focus on the approach motivation related to craving in smokers. They report that heavy

#### Edited and reviewed by: Luiz Pessoa,

University of Maryland, USA

#### \*Correspondence:

Fritz Strack strack@psychologie.uni-wuerzburg.de; Paul Pauli pauli@psychologie.uni-wuerzburg.de

#### Specialty section:

This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology

Received: 08 January 2016 Accepted: 18 February 2016 Published: 07 March 2016

#### Citation:

Strack F, Pauli P and Weyers P (2016) Editorial: Emotion and Behavior. Front. Psychol. 7:313. doi: 10.3389/fpsyg.2016.00313 smokers—against common assumptions—are capable to regulate emotion via deliberate reappraisal and suggest that such emotion regulation strategies might be used in treatment to learn to regulate craving too. Deutsch, Smith, Kordts-Freudinger, and Reichardt focus on relief, which is assumed to elicit approach behavior, and propose an integrative relief model that links affect, emotion, and motivational systems.

**Emotional processes underlying avoidance** are reviewed by Diemer, Alpers, Peperkorn, Shiban, and Mühlberger with a focus on the domain of virtual reality as a means to study the impact of perception and presence on emotional reactions, here fear and anxiety. Wieser, Gerdes, Reicherts, and Pauli present results on how pain and the resulting avoidance motivation influence the processing of emotional expressions and show an asymmetry in its impact on the processing of pleasant and unpleasant faces. The emotion of surprise is the topic of the contribution by Topolinski

#### and Strack. Their work focusses on the accompanying facial activities and the role of negative affect.

Most of the contributing authors collaborated during their time at the University of Würzburg, Germany, as members of the research group "Emotion and Behavior" funded by the German Research Foundation (DFG, FOR 605). The editors thank all contributors for excellent collaborations during the last years.

# AUTHOR CONTRIBUTIONS

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

# FUNDING

Funded by DFG FOR-605.

# REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Strack, Pauli and Weyers. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# **Contrasting motivational orientation and evaluative coding accounts: on the need to differentiate the effectors of approach/avoidance responses**

*Julia Kozlik <sup>1</sup> \*, Roland Neumann <sup>1</sup> and Ljubica Lozo <sup>2</sup>*

*<sup>1</sup> Department of Psychology, University of Trier, Trier, Germany, <sup>2</sup> Department of Psychology, University of Würzburg, Würzburg, Germany*

Several emotion theorists suggest that valenced stimuli automatically trigger motivational orientations and thereby facilitate corresponding behavior. Positive stimuli were thought to activate approach motivational circuits which in turn primed approach-related behavioral tendencies whereas negative stimuli were supposed to activate avoidance motivational circuits so that avoidance-related behavioral tendencies were primed (*motivational orientation account*). However, recent research suggests that typically observed affective stimulus–response compatibility phenomena might be entirely explained in terms of theories accounting for mechanisms of general action control instead of assuming motivational orientations to mediate the effects (*evaluative coding account*). In what follows, we explore to what extent this notion is applicable. We present literature suggesting that evaluative coding mechanisms indeed influence a wide variety of affective stimulus–response compatibility phenomena. However, the evaluative coding account does not seem to be sufficient to explain affective S–R compatibility effects. Instead, several studies provide clear evidence in favor of the motivational orientation account that seems to operate independently of evaluative coding mechanisms. Implications for theoretical developments and future research designs are discussed.

**Keywords: emotional responses, approach and avoidance, affective S–R compatibility, facial muscle contractions, theory of event coding**

A fundamental assumption in emotion research is that emotions predispose the organism to act adaptively in a frequently changing environment. Therefore, many emotion theorists postulate a close link between emotion and action tendencies (cf. Darwin, 1872; Frijda, 1986). In order to meet the requirements of survival, appropriate responses to significant stimuli in the environment must be selected. From an evolutionary perspective, proper response selection should enhance rather than diminish the organism's fitness. Therefore, the detection of basic principles that could explain how the cognitive apparatus meets the challenge to respond properly has a long tradition in psychological research.

In this sense a vast body of researchers postulated that significant stimuli activate specific motivational orientations that prepare the organism to act adaptively: positive stimuli were assumed to activate approach motivational circuits which in turn trigger approach-related behavioral tendencies, whereas negative stimuli should activate avoidance motivational circuits which trigger avoidance-related behavioral tendencies (Davidson et al., 1990; Gray, 1990; Lang et al., 1990; Strack and Deutsch, 2004). However, recent studies seriously questioned this assumption

#### *Edited by:*

*Fritz Strack, University of Würzburg, Germany*

#### *Reviewed by:*

*Seung-Lark Lim, University of Missouri - Kansas City, USA Regina Reichardt, Regensburg University, Germany*

*\*Correspondence:*

*Julia Kozlik, Department of Psychology, University of Trier, Universitätsring, 54286 Trier, Germany kozlik@uni-trier.de*

#### *Specialty section:*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology*

*Received: 21 August 2014 Accepted: 19 April 2015 Published: 01 May 2015*

#### *Citation:*

*Kozlik J, Neumann R and Lozo L (2015) Contrasting motivational orientation and evaluative coding accounts: on the need to differentiate the effectors of approach/avoidance responses. Front. Psychol. 6:563. doi: 10.3389/fpsyg.2015.00563* (Lavender and Hommel, 2007; Eder and Rothermund, 2008). According to Lavender and Hommel (2007) it is not the motivational orientation that mediates the link between evaluative processes and the activation of approach/avoidance responses but rather a simple evaluative coding mechanism. Hence, Lavender and Hommel argued that models accounting for mechanisms of general action control more economically explain why positive (negative) stimuli seem to activate approach-related (avoidancerelated) behavior. In what follows, we first present these two alternative theoretical models—the motivational orientation versus the evaluative coding account—in detail. Afterward, we review the literature of different types of approach/avoidance behavior in order to discuss to what extent the recommendation of Lavender and Hommel is applicable.

# **Motivational Orientation Account**

Emanating from the principle of hedonism, approaching pleasure and avoiding pain are conceived as the most fundamental motivators of human beings (Davidson et al., 1990; Gray, 1990; Lang et al., 1990). In line with this consideration, it is assumed that evaluative processes and approach/avoidance behavior are closely connected (Neumann et al., 2003). Specifically, the Reflective-Impulsive Model (RIM, Strack and Deutsch, 2004), which postulates that behavior is influenced by both impulsive and reflective mechanisms, predicts that the link between evaluation and response activation would be mediated by motivational approach or avoidance orientations. Positive stimuli were assumed to activate the appetitive motivational system and thereby facilitate any kinds of approach behavior, whereas negative stimuli should activate the defensive motivational system and thereby facilitate any kinds of avoidance behavior<sup>1</sup> .

How can this link between evaluation and response activation be described? Within the domain of motivational orientation accounts two different theoretical sub-accounts have been discussed. According to the *specific muscle activation hypothesis* there is a hard-wired link between stimulus evaluations and specific motor responses. Considering arm movements positive stimuli should automatically activate arm flexion whereas negative stimuli should activate arm extension (Cacioppo et al., 1993; Chen and Bargh, 1999). On the contrary, there has also been proposed a *distance regulation hypothesis* to describe the link between evaluation and behavior which assumes that it is not a particular muscle movement that is activated by valenced stimuli but rather a certain motor response that contextually effectuates approach or avoidance (Markman and Brendl, 2005; Seibt et al., 2008). In this sense, the distance regulation hypothesis implies that the link between evaluation and behavior is flexible as a function of contextual factors whereas the specific muscle activation hypothesis assumes the link to be inflexible.

# **Empirical Evidence**

First, evidence for the assumption that positive stimuli would trigger motivational approach and negative stimuli would trigger motivational avoidance came from a seminal study by Solarz (1960). In this study participants were required to respond to the valence of stimulus words fixed on a movable stage by pushing the stage away from (i.e., avoidance movement), or pulling it toward (i.e., approach movement), them. Overall participants were faster in approaching positive compared to negative words and avoiding negative compared to positive words. This basic affective stimulus–response compatibility effect was later replicated by Chen and Bargh (1999) who reasoned that "approach-like muscle movements" were faster in response to positive stimuli whereas "avoidance-like muscle movements" were faster in response to negative stimuli. Therefore, it has been assumed that the appetitive motivational system would automatically trigger arm flexion whereas the defensive motivational system would trigger arm extension (see also Cacioppo et al., 1993).

However, the idea that specific muscle movements were hardwired or inflexibly linked to different motivational circuits has been challenged. Markman and Brendl (2005) as well as Seibt et al. (2008) convincingly demonstrated that the direction of the affective S–R compatibility effect is affected by the reference frame induced via task instruction. In both the Solarz (1960) and the Chen and Bargh (1999) studies participants were required to move a lever toward or away from themselves (i.e., subject-based instructions). In doing so, lever movements toward (away from) their own body were faster in response to positive (negative) stimuli. Conversely, when an object-based instruction (*Move the lever toward or away from the stimulus!*) was given (as for example in Seibt et al., 2008) opposite compatibility effects could be observed: lever movements toward (away from) their own body were faster in response to negative (positive) stimuli (see also Laham et al., 2014). Therefore, different motivational circuits do not seem to directly trigger specific approach- or avoidance-like muscle movements (e.g., arm flexion or extension) but rather those movements that are interpreted as approach or avoidance. In sum, the evidence so far seems to favor the distance regulation hypothesis as a specification of the motivational orientation account.

# **Automaticity of the Link between Evaluative Processes and Approach/Avoidance Tendencies**

A vast body of literature has repeatedly shown that humans automatically<sup>2</sup> evaluate stimuli they face (see Bargh, 1997, for an

<sup>1</sup>We are aware of the fact that the categorical distinction of positive versus negative stimuli could be regarded as oversimplified. Considering affective stimuli in more detail one might rather characterize them beyond valence, for example, in terms of discrete emotion concepts. In doing so, it becomes obvious that certain stimuli are at least somewhat beyond the scope of approach/avoidance models. Although anger can be viewed as a negatively valenced affect (Carver and Harmon-Jones, 2009) in some studies anger-related stimuli triggered avoidance responses (Marsh et al., 2005; Roelofs et al., 2008), whereas other researchers demonstrated that approach responses were activated by stimuli related to anger (Wilkowski and Meier, 2010).Krieglmeyer and Deutsch (2013) tried to solve this inconsistency by arguing that superordinate goals determine the link between stimulus valence and response activation. Thus, negative stimuli might automatically elicit a motivation to ultimately increase the physical distance irrespective of instantaneous consequences (see also Krieglmeyer et al., 2011). In the case of anger-related stimuli, this goal can be achieved by flight (i.e., a concrete avoidance response) or even by fight (i.e., a concrete approach response) because attacking an opponent likely puts the opponent to flight.

<sup>2</sup>Due to Moors and De Houwer (2006) cognitive operations can be described as automatic processes if they are executed unintentional, uncontrolled, goal independent, autonomous, purely stimulus driven, unconscious, efficient, and fast. We were mostly interested in the question of whether the activation

overview; Zajonc, 1980). Thus, the affective meaning of a wide variety of stimulus classes (pictures, facial expressions, words, odors etc.) seems to be extracted automatically. Acting on the assumption that evaluative processes and approach/avoidance action tendencies were closely linked to each other one would expect that valenced stimuli automatically prime specific action tendencies. Therefore, any positive stimulus is supposed to automatically trigger approach behavior whereas any negative stimulus should automatically trigger avoidance responses. Indeed, a few studies demonstrated that affective stimulus–response compatibility effects also occur in experimental settings where participants were either instructed to respond to a non-valence stimulus feature (cf. Seibt et al., 2008) or to respond by pushing versus pulling a lever whenever a stimulus appeared on the screen (cf. Chen and Bargh, 1999, Experiment 2; Duckworth et al., 2002, Experiment 3). On the other hand, several other studies failed to show affective stimulus–response compatibility effects when valence was not a task-relevant feature (Rotteveel and Phaf, 2004; Lavender and Hommel, 2007). However, comparability of these studies is limited due to overall differences in their experimental parameters. Therefore, Phaf et al. (2014) reported results of a meta-analysis conducted on numerous manual reaction time studies where approach/avoidance movements were to be made in response to valenced stimuli. Interestingly, the authors observed a medium-sized affective S–R compatibility effect in studies where participants explicitly had to categorize stimulus valence. However, there was no such affective compatibility effect at all when valence was task-irrelevant. This finding might question the notion that affective stimuli automatically trigger action tendencies of approach or avoidance. However, Laham et al. (2014) recently published another meta-analysis based on other criteria for selecting studies to integrate in the analysis. Remarkably, this study did not reveal any significant influence of evaluation goals on the affective compatibility effect (although their results tended in the same direction as those reported by Phaf et al., 2014). Hence, there seem to be studies that provide evidence in favor of the automaticity hypothesis whereas others do not do so.

Having a closer look on literature on different types of approach-avoidance tasks, it becomes obvious that the degree of automaticity (in terms of independency of evaluation intentions) might vary across different task settings.Krieglmeyer and Deutsch (2010) contrasted three types of tasks. First, they used the manikin paradigm, where participants were instructed to imagine being a manikin presented on the screen (De Houwer et al., 2001, Experiment 4; Krieglmeyer et al., 2010, 2011, 2013). This manikin has to be moved toward or away from valenced stimuli via button presses. Second, the regular joystick paradigm has been used where participants were instructed to push or pull the joystick in response to valenced stimuli. Third, a feedback-version of the regular joystick task has been introduced where pushing and pulling the joystick always resulted in a visual zooming effect so that the stimulus either appears to come closer of disappear after responding. A direct comparison of these three types of approach-avoidance tasks revealed that the affective compatibility effect occurred independently of the goal to evaluate the stimuli only when participants performed the manikin or the feedbackjoystick task, i.e., those versions of the approach-avoidance task that provided a clear approach- or avoidance-related visual feedback. Therefore, whether an action is interpreted as an approach or avoidance movement depends on the perceivable action consequences. As perceivable action effects seem to play a crucial role in automatic activations of approach-avoidance behavior Van Dantzig et al. (2008) proposed to conceive approach- and avoidance-related action tendencies as "flexible action plans that are represented in terms of their effects" (p. 1298).

# **Evaluative Coding Account**

So far we have seen that the existence of perceivable approach- or avoidance-related action effects seems to be a necessary precondition for automatic occurrence of affective stimulus–response compatibility effects (Van Dantzig et al., 2008; Krieglmeyer and Deutsch, 2010). However, the idea that action effects are an important determinant of the activation of motor responses is not new. Indeed, in the field of cognitive psychology anticipated action consequences are intensively discussed to be involved in the generation of any motor response (Prinz, 1997; Hommel et al., 2001; Kunde et al., 2007). Following the argumentation of Lavender and Hommel (2007), for this reason affective and non-affective stimulus–response compatibility phenomena share so many basic characteristics that one should seriously doubt that both emanate from different mechanisms. In fact, Lavender and Hommel suggest that both perceptual (for example, the Simon effect; Simon, 1990; Lu and Proctor, 1995) and affective S–R compatibility effects can be explained by a general framework accounting for perception–action interactions, namely the theory of event coding (TEC; Hommel et al., 2001). Deriving from ideomotor principle (Lotze, 1852; Harless, 1861; James, 1890) the TEC postulates that perceived stimuli and response features (including their perceptual action effects) were coded as structurally identical event codes in a common representational domain. Therefore, it is assumed that actions were represented in terms of their anticipated consequences. Furthermore, specific actions can be primed as a result of feature overlap. That is, if a stimulus and an action share specific features, responses would be faster.

Eder and Rothermund (2008) applied this logic to affective S–R compatibility effects. Thus, on the one hand, valence can be considered as one stimulus feature (among others) either being coded as *positive* or *negative*. On the other hand, in affective S–R compatibility paradigms participants were provided with specific action goals. Approach movements can be considered as responses *toward* the self/object whereas avoidance movements can be considered as responses *away from* the self/object. Hence, these response codes themselves carry a specific valence so that a *positive* stimulus code might trigger the goal to respond *toward* a reference point whereas a negative stimulus code might trigger the goal to respond *away from* a reference point. Indeed, Eder and Rothermund (2008) convincingly demonstrated that the affective S–R compatibility effect can be interpreted as a compatibility effect between stimulus valence and response label valence.

of approach-avoidance responses would occur independently of the goal to evaluate stimuli. Therefore, by using the term "automaticity" we refer to the feature of goal independency.

Responses to positive stimuli were faster when introduced with a positive response label (such as: Move the joystick *up* or *toward* the self/object!). On the contrary, responses to negative stimuli were faster when introduced with a negative response label (such as: Move the joystick *down* or *away from* the self/object!). Consequently, a simple feature overlap mechanism as postulated by the TEC can account for affective stimulus–response compatibility effects.

Accordingly, in terms of the TEC, valence is considered as only one stimulus feature among others (such as color and shape). Accepting this theory to entirely account for the observed affective compatibility phenomena reported in the literature, one must deduce that it is not the motivational orientation that mediates the link between evaluative processes and response activation but rather a simple feature comparison mechanism. On the basis of this argumentation, Lavender and Hommel (2007) suggest "exploring the possibility of explaining all compatibility phenomena within the same theoretical framework—and only construct separate models if this attempt turns out to fail" (p. 1293).

# **Contrasting Motivational Orientation vs. Evaluative Coding**

In the literature there are two different theoretical approaches that try to explain the mechanisms underlying affective stimulus–response compatibility phenomena. The motivational orientation account assumes that approach and avoidance motivational systems mediate the link between stimulus evaluation and response activation. Within this domain, several authors argued that these motivational systems are linked to specific muscle groups so that positive stimuli would prepare the organism to respond with approach-like motor movements (e.g., arm flexion) whereas avoidance-like motor movements (e.g., arm extension) were activated by negative stimuli (*specific muscle activation hypothesis*; Cacioppo et al., 1993; Chen and Bargh, 1999). However, other researchers convincingly argued that affective stimuli not necessarily activate specific muscle groups but rather those responses that are situationally interpreted as approach or avoidance for example due to their perceivable action effects (*distance regulation hypothesis*; Markman and Brendl, 2005; Seibt et al., 2008; Van Dantzig et al., 2008). Until now, evidence for the specific muscle activation hypothesis seems to be rather weak.

On the contrary, in recent years it has been argued that general theories of human motor control might be sufficient to explain non-affective as well as affective S-R compatibility phenomena (Lavender and Hommel, 2007; Eder and Rothermund, 2008). Therefore, it has been proposed an evaluative coding account of affective S-R compatibility effects (Eder and Rothermund, 2008) in which it is assumed that positive (negative) stimuli activate positively (negatively) connotated responses due to feature overlap. Hence, several researchers deny that motivational processes would mediate the link between evaluation and behavior.

But which of these two theoretical approaches is more suitable to account for affective stimulus–response compatibility phenomena? Until now, there are only a few published studies that tested these theoretical accounts against each other. Laham et al. (2014) recently conducted a meta-analysis across 68 studies that examined affective S–R compatibility effects. In a multiple regression analysis it has been tested whether response labels and/or motivational framing would be significant predictors of the effect sizes. Interestingly, the authors reported that response labels had a significant influence whereas motivational framing does not. Thus, it has been concluded that "to the extent that motivational framing disambiguates action meaning it does so via assigning affective labels to responses" (Laham et al., 2014, p. 16).

However, in adopting the manikin paradigm Krieglmeyer et al. (2010) tried to directly contrast the motivational orientation and the evaluative coding account in an empirical study. In a series of experiments, the manikin was presented either above or below a centrally presented valenced word. The task was to move the manikin up or down depending on the word valence (Experiment 1) or the lexical category (Experiment 2 and 3). On the one hand, the evaluative coding account predicted that positively labeled responses (i.e., moving the manikin *upward*) should be faster in response to positive words whereas negatively labeled responses (i.e., moving the manikin *downward*) should be faster in response to negative words. On the other hand, the motivational orientation account predicted that moving the manikin toward positive and away from negative words should be faster than moving it in the reverse direction. With such an experimental setup, Krieglmeyer et al. (2010) provided evidence in favor of the motivational orientation account: responses to positive words were faster when they decreased the distance between the manikin and the stimulus whereas responses to negative words were faster when they increased the distance between the manikin and the stimulus. Most importantly, at least in two of the three experiments, this effect was independent of evaluative compatibility between stimulus valence and response label valence. Therefore, this study provided evidence for parallel running of both mechanisms.

In sum, there are numerous studies that repeatedly revealed an influence of evaluative coding mechanisms on affective stimulus–response compatibility effects (Eder and Rothermund, 2008; Krieglmeyer and Deutsch, 2010; Laham et al., 2014). On the contrary, the results concerning the influence of motivational orientations on response activation seem to be rather mixed. On the one hand, in the meta-analysis by Laham et al. (2014) the influence of motivational orientations on the affective compatibility effect disappeared when evaluative coding processes have been controlled. On the other hand, Krieglmeyer et al. (2010) clearly provided evidence in favor of the motivational orientation mechanism that seems to operate independent of evaluative coding processes. However, one major limitation of the results of the meta-analysis is the fact that several important studies (e.g., the one by Krieglmeyer et al., 2010) were excluded on the basis of specific selection criteria. Therefore, it can be reasoned that depending on the task affordances both mechanisms might contribute to affective S–R compatibility effects.

# **Effector as Determinant**

Taking a closer look at studies on approach/avoidance behavior it becomes obvious that most of the studies reviewed so far exclusively focused on manual responses. The spectrum ranges from lever movements (Chen and Bargh, 1999; Eder and Rothermund, 2008) over object movements (Lavender and Hommel, 2007) to button presses/releases (Wentura et al., 2000; Seibt et al., 2008). Hence, approach and avoidance is operationalized as an arm movement toward or away from a reference point or an arm movement that leads to an approach- or avoidance-related action effect (as for example in the manikin paradigm). However, when analyzing whether affective compatibility phenomena would be mediated by specific muscle activations, distance regulation processes or rather evaluative coding mechanisms only focusing on one specific effector of approach-avoidance responses might lead to biased interpretations of results.

Thinking of a human being that is acting in an environment, of course, the notion that object valence and specific arm movements were linked in a hard-wired fashion (Cacioppo et al., 1993; Chen and Bargh, 1999) makes little sense given that decreasing the distance between the self and an object can be achieved via arm extension or flexion depending on whether the object is already held in hand or not. Therefore, whether an arm movement can be interpreted as approach or avoidance must be extremely context dependent in order to efficiently act in a changing environment. On the one hand this might have biased interpretations toward the distance regulation hypothesis as compared to the specific muscle activation hypothesis. On the other hand, this context dependency paired with the inevitable flexibility of arm movements might also promote evaluative coding mechanisms. Since the type of approach-avoidance task seems to be an important factor (cf. Krieglmeyer and Deutsch, 2010) it is also conceivable that the effector of approach-avoidance responses plays a crucial role in the debate on whether motivational orientation accounts or the evaluative coding approach would be an appropriate theoretical framework to account for affective S–R compatibility effects.

# Whole-body Movements as Approach/Avoidance Responses

The studies reviewed so far have one important aspect in common: in all of them approach and avoidance have been conceptualized in terms of changes in physical distance. However, it can be argued that unimanual arm movements may not be the ideal instantiations of responses initiated to increase or decrease the distance between a stimulus and the self because basic action consequences that naturally occur when an individual increases or decreases the physical proximity between the self and a stimulus (e.g., changes in visual angle) fail to appear. Hence, Stins et al. (2011) introduced whole-body movements as an alternative measure of approach/avoidance responses (see also Koch et al., 2009). In their study participants had to step in a forward (approach) or backward direction (avoidance) in response to the valence of facial expressions presented on a computer screen. The authors observed faster response initiation in stimulus–response compatible, compared to incompatible, conditions. Similarly, Stins and Beek (2011) replicated this compatibility effect for whole-body movements using pictures of emotional scenes as stimuli.

Moreover, a recent study by Ly et al. (2014) provides evidence that even whole-body approach/avoidance responses were automatically triggered by valenced stimuli. The authors presented gems on the left or right side of the screen which were preceded by a task-irrelevant facial expression. Participants were instructed to step sideways either toward (approach) or away from the gem (avoidance). Ly et al. showed that whole-body approach (avoidance) movements could be initiated faster when preceded by a positive (negative) facial expression. In sum these results seem to further support Stins et al. (2011) who reasoned that whole-body movements might be a much "more direct measure of approach/avoidance behavior" (p. 604) with high ecological validity because with this alternative effector quite similar affective stimulus–response compatibility effects could be observed.

How can these affective stimulus–response compatibility effects for whole-body movements be explained? Typically researchers who investigated whole-body approach/avoidance responses (Stins and Beek, 2011; Stins et al., 2011; Ly et al., 2014) explain these compatibility effects in terms of the motivational orientation account. Due to this account affective stimuli should activate compatible motivational circuits so that any kind of appropriate behavior is primed. Thus, positive (negative) stimuli automatically activate the approach (avoidance) motivational system triggering approach-related (avoidance-related) movements. Considering a step forward (i.e., toward a stimulus) as an approach movement and a step backward (i.e., away from a stimulus) as avoidance, the motivational orientation account seems to provide an appropriate theoretical framework to explain whole-body affective S–R compatibility effects.

But is the specific muscle activation hypothesis or rather the distance regulation hypothesis an appropriate theoretical model to specify the motivational orientation account? We argue that the processing of affective stimuli must be flexibly linked to the activation of whole-body movements. First, from an evolutionary perspective, it makes little sense that a negative stimulus, such as a venomous snake, should always activate a tendency to step backward, because it clearly depends on the position of the snake whether a step backward or forward would be the optimal response to ensure survival. Second, the results reported by Ly et al. (2014) clearly demonstrated that it is not a specific movement that is activated by valenced stimuli. As already mentioned in this study a centrally presented emotional face appeared as a prime on a screen in front of the participant, following which, whole-body approach/avoidance movements were to be made in response to non-valent targets appearing on the left or right side of the screen. These approach/avoidance movements were implemented as steps to the left or right instead of steps in the forward or backward direction. The authors observed that whether a step to the left or to the right was triggered by positive or negative stimuli depended on the location of the target. Therefore, situational factors must have influenced whether a movement is interpreted as approach or avoidance.

However, whole-body affective S–R compatibility phenomena can also be explained in terms of the TEC (Hommel et al., 2001). According to the evaluative coding account responses should be speeded if stimuli and responses assigned to them share certain features. Taking a closer look at studies of whole-body approach/avoidance movements (Stins and Beek, 2011; Stins et al., 2011; Ly et al., 2014) it becomes obvious that in all of these studies movement direction and the affective connotation of response labels assigned to these movements were confounded: approach movements were always introduced as responses *toward* the stimulus whereas avoidance movements were always introduced as responses *away from* the stimulus. Therefore, the compatibility effects reported in the literature can easily be explained as a consequence of feature overlap (cf. Eder and Rothermund, 2008). Furthermore, empirical evidence for a flexible link between affective stimuli and whole-body movements again stems from Ly et al. (2014). In this study the emotional prime stimulus appeared in front of the participants following which sideways steps toward or away from the target had to be executed. Again, sideways steps *toward* the target stimulus could be initiated faster when preceded by a positive facial expression whereas sideways steps *away from* the target could be initiated faster when preceded by a negative facial expression. However, in one aspect such an effect is contradictory with the predictions derived from the distance regulation hypothesis because a step to the left as well as a step to the right after a centrally presented emotional stimulus always physically increases the distance between the self and the position where the prime stimulus appeared. Therefore, it is conceivable that it was the framing of responses as *toward* or *away* that produced the observed affective S-R compatibility effects for whole-body movements.

Although the motivational orientation account seems to be favored (Stins and Beek, 2011; Stins et al., 2011; Ly et al., 2014), as far as we know, until now there has been no study that directly tested whether whole-body affective S–R compatibility effects can be explained in terms of the evaluative coding or motivational orientation account. One might argue that the mechanisms underlying whole-body movements might be similar to the mechanisms underlying arm movements, as both effectors target chances in physical proximity between the self and a stimulus. But, until now there is no study that tested these two accounts against each other.

# Facial Muscle Contractions as Approach/Avoidance Responses

The literature we have reviewed so far defined approach and avoidance in terms of regulation of physical distance. However, approach and avoidance orientation might also be defined in terms of regulation of the social distance. Therefore, in addition to arm or whole-body movements, there are several other forms of recordable behavioral response channels that can be viewed as an indication of approach or avoidance; for example, horizontal versus vertical head movements (Wells and Petty, 1980; Förster and Strack, 1996), reflexes (Lang et al., 1990) or facial muscle contractions (Neumann et al., 2005). Given that arm movements (as well as whole-body movements, too?) seem to be extremely context dependent (Markman and Brendl, 2005; Eder and Rothermund, 2008; Seibt et al., 2008) it can be supposed that neither arm nor whole-body movements are the ideal operationalization of approach and avoidance orientations. Instead, the most critical test for the valence–approach/avoidance tendency link would therefore be the analysis of a response channel that unambiguously reflects approach versus avoidance orientation. According to Neumann et al. (2003) "facial expressions meet this criterion *. . .* [because] it does not depend on the point of reference that a smile reflects an approach and a frown an avoidance orientation" (p. 376). Moreover, facial muscles are controlled by both cortical and sub-cortical pathways (Rinn, 1984). This suggests that both

goals and evaluative processes might have an impact on facial muscles.

Reviewing the literature of facial muscle contractions it becomes obvious that there has recently been developed an experimental paradigm that is comparable to manual or whole-body approach/avoidance paradigms. This paradigm is based on voluntary facial muscle contractions recorded via electromyogram (EMG). In prototypical studies participants were required to voluntarily contract specific muscles of the face in response to significant stimuli (Dimberg et al., 2002; Neumann et al., 2005, 2014a; Kunde et al., 2011; Otte et al., 2011). Analyses mainly focus on mean response latencies with which specific muscles can be contracted whereas response latency is defined as the time from stimulus onset until response onset (cf. Chiew and Braver, 2010) or until the signal reaches a specific proportion of individual maximum muscle activity (cf. Neumann et al., 2014a).

In adopting this facial response paradigm several researchers provided further evidence for the assumption that positive stimuli would trigger approach whereas negative stimuli would trigger avoidance. In a study by Dimberg et al. (2002) participants were instructed to voluntarily contract either their *zygomaticus major* (i.e., muscle that raises the corners of the mouth producing a smile) or their *corrugator supercilii* (i.e., muscle that pulls the eyebrows together producing a frown) in response to positive and negative facial expressions (Experiment 1) or pictures (Experiment 3). The authors reported that contractions of the zygomaticus muscle could be initiated more quickly in response to positive stimuli and contractions of the corrugator muscle could be initiated more quickly in response to negative stimuli. This result was later replicated by Neumann et al. (2005) using positive and negative words as stimuli. Thus, research on facial muscle contractions revealed an affective stimulus–response compatibility effect comparable with research on manual or whole-body responses.

However, in contrast to manual S–R compatibility phenomena, the link between stimulus valence and facial approach/avoidance behavior seems to be much more automatic because several researchers repeatedly observed an affective S–R compatibility effect when participants performed a valence-irrelevant task (Neumann et al., 2005; Otte et al., 2011). In Experiment 2 of the Neumann et al. (2005) study participants were simply instructed to contract either the zygomaticus or the corrugator muscle whenever a word appeared on the screen. The task did not require them to evaluate these stimuli. Similarly, in the Otte et al. (2011) study voluntary zygomaticus and corrugator muscle contractions were to be made in response to the gender of stimulus persons showing different emotional expressions. In both studies, zygomaticus (corrugator) responses were faster in response to positive (negative) stimuli although valence was clearly task-irrelevant.

Further evidence for an automatic activation account stems from studies recording spontaneous muscle activity when participants perceived valenced stimuli. For example, Dimberg et al. (2000) demonstrated an increased activity over zygomaticus major when participants were unconsciously exposed to positive facial expressions whereas activity of the corrugator supercilii increased when participants were unconsciously exposed to negative facial expressions. Moreover, simply viewing positive pictures leads to increased activity of the zygomaticus major whereas viewing negative pictures increases the activity of the corrugator supercilii (Lang and Bradley, 2007). Consequently, valenced stimuli seem to automatically trigger facial approach/avoidance responses even in the absence of an intention to evaluate.

How can these affective S–R compatibility effects for facial responses be explained? Recent research suggests that affective stimulus–response compatibility phenomena can be explained within the TEC (Lavender and Hommel, 2007; Eder and Rothermund, 2008). First evidence for the assumption that feature overlap mechanisms even influence the activation of facial muscles comes from a study by Kunde et al. (2011). In this study voluntary zygomaticus versus corrugator contractions were to be made to indicate the color of dots appearing on the screen. Immediately after responding a facial expression was presented in the center of the screen that served as an action effect. In one experimental block the facial expression was always compatible with the response shown by the participant (i.e., a smiling face after zygomaticus response or frowning face after corrugator response) whereas in the other block the facial expression was always incompatible (i.e., a smiling face after corrugator response or frowning face after zygomaticus response). Therefore, responses which had noticeable features (e.g., feeling the corners of the mouth rising up) always resulted in a perceivable action effect. In terms of the TEC it is assumed that overlap between certain stimulus and/or response features leads to faster responses. Therefore, one would expect that facial responses should be faster when anticipating a compatible, compared to an incompatible, action effect to occur. Indeed, Kunde et al. (2011) demonstrated faster responses in response–effect compatible, compared to response–effect incompatible, conditions. Consequently, anticipated response consequences seem to be integrated in the activation of facial muscles.

Whether the TEC can also entirely account for facial affective stimulus–response compatibility effects has recently been investigated by Neumann et al. (2014a). In this study the experimental logic introduced by Eder and Rothermund (2008) has been adapted to facial responses. Thus, responses were introduced using positive versus negative response labels. Participants were instructed to classify stimulus words due to their valence by either showing the "sun"-response (i.e., positive response label) or the "rain"-response (i.e., negative response label). Half of the participants were required to respond with joystick movements away or toward themselves (i.e., manual responses) whereas the other half were required to respond with voluntary contractions of the zygomaticus or corrugator muscle (i.e., facial responses). For manual responses Neumann et al. (2014a) replicated the results reported by Eder and Rothermund (2008): there was a pronounced compatibility effect between stimulus valence and response label valence indicating faster "sun"-responses to positive words and faster "rain"-responses to negative words irrespective of the arm movement that was required. This result further confirms the assumption that feature overlap mechanisms contribute to affective stimulus–response compatibility effects as observed for arm movements. However, for facial responses the compatibility effect between stimulus valence and response label valence was much weaker compared to manual responses. In fact, Neumann et al. (2014a) reported having found a pronounced stimulus

valence–muscle compatibility effect indicating faster responses to positive words with the zygomaticus muscle and faster responses to negative words with the corrugator muscle. This compatibility effect was slightly reduced when response label valence mismatched the respective muscle (i.e., zygomaticus = "rain" response; corrugator = "sun"-response). Therefore, feature overlap does influence the activation of facial responses (cf. Kunde et al., 2011), too, but the results reported by Neumann et al. (2014a) strongly suggest that affective stimulus–response compatibility effects as observed for facial muscle movements cannot entirely be explained via a simple feature overlap mechanism as postulated by the TEC.

Can we conclude that the observed affective stimulus–response compatibility effects for facial muscle movements are mediated by motivational orientations? One alternative mechanism that could potentially account for the observed response pattern is automatically activated mimicking behavior. It is well known that the perception of an emotional expression automatically leads to imitation processes in the observer. Thus, a smiling face automatically triggers activation of the zygomaticus muscle whereas a frowning face triggers corrugator activity (Dimberg et al., 2000; Likowski et al., 2008, 2011). Such an account especially applies for studies using facial expressions as stimuli (Dimberg et al., 2000, 2002, Experiment 1; Otte et al., 2011). However, this objection has to deal with the vast body of literature demonstrating exactly the same response pattern using pictures (Dimberg et al., 2002, Experiment 3; Lang and Bradley, 2007) or even words as stimuli (Neumann et al., 2005, 2014a). Furthermore, Neumann et al. (2014b) recently provided evidence for the assumption that mimicking behavior might also be an evaluative process rather than an imitation mechanism. Amongst other muscles the authors recorded spontaneous muscle activity over levator labii (i.e., muscle that wrinkles the nose producing an expression showing disgust) in response to facial expressions. Interestingly, contraction of the levator labii in a stimulus person did not automatically trigger activation of the levator labii in the observer which could have been regarded as an indicator of motor-mimicry. Instead the perception of a facial expression of disgust leads to increased activity over the corrugator muscle which can rather be seen as a fast evaluative response to facial displays of others. Therefore, we think that imitation mechanisms do not provide a sufficient explanation for automatic responses to shortly presented emotional displays.

If motivational orientations mediate the link between stimulus evaluation and facial muscle contractions is the specific muscle activation or rather the distance regulation hypothesis an appropriate specification? In other words, are certain facial muscles inflexibly linked to motivational approach and avoidance circuits? Empirical evidence for this assumption stems from studies on cerebral asymmetry (Davidson et al., 1990; Allen et al., 2001; Coan et al., 2001). According to Davidson (1984, 1987, 1992) motivational orientations involve asymmetrical activations of the anterior cortical regions in the way that the left anterior region subserves the approach motivational systems whereas the right anterior region subserves the avoidance motivational system. Interestingly, the degree of cerebral asymmetry seems to be associated with the activation of specific facial muscle groups. In a study by Davidson et al. (1990) participants watched positive and negative film clips while recording EEG activity and videotaping their facial expressions. The authors observed greater left compared to right frontal activation during those time frames where subjects showed facial expressions of happiness whereas greater right as compared to left frontal activation occurred when subjects showed facial expressions of disgust. This link between motivational orientations and facial muscle contractions seems to be bidirectional. A causal influence of motivational orientations on the activation of facial muscle groups has been shown by Allen et al. (2001). They used biofeedback training to increase the relative left or right-sided activation of the frontal cortex. With this method they successfully manipulated the degree of frontal asymmetry. Afterward, facial muscle activation has been recorded with EMG while participants watched different film clips. The authors observed that increasing relative right-sided activation with biofeedback decreased activation of the zygomaticus major muscle whereas increasing relative left-sided activation decreased activation of the corrugator supercilii muscle. In addition to that it has been shown that manipulations of facial expressions also influence cortical activity in the anterior regions. In a study by Coan et al. (2001) the directed facial action task was used in which subjects were instructed to voluntarily adopt certain facial expressions while EEG activity was recorded. It could be demonstrated that adopting negative facial expressions resulted in relatively less left frontal activation than adopting positive facial expressions. All these findings are hardly explainable by a flexible link between motivational orientations and facial smuscle activation. In fact, we conclude that this link is rather hardwired so that the specific muscle activation hypothesis seems to provide an appropriate theoretical framework to explain affective S–R compatibility effects as observed for facial muscle responses.

# **Integration of Findings**

The literature reported above examined different types of effectors of approach/avoidance responses. On the one hand, we reviewed unimanual arm as well as whole-body movements, which are effectors that reflect approach/avoidance in terms of physical distance regulation. We have seen that evaluative coding mechanisms play a crucial role in the activation of manual approachavoidance responses. This–although not yet tested–might even be the case for whole-body approach-avoidance responses. These results might be seen as support for the notion that affective and non-affective S–R compatibility effects can be integrated into the same theoretical framework (cf. Lavender and Hommel, 2007). However, there are several exceptions were the TEC was not able to entirely account for the observed affective S–R compatibility effects. Instead, for example Krieglmeyer et al. (2010) provided clear evidence that both mechanisms contribute to affective S–R compatibility effects when focusing on the manikin version of the approach-avoidance task.

On the other hand, approach/avoidance can also be conceived as a means to regulate the social distance. In doing so, facial muscles seem to be an appropriate effector of approach/avoidance in the way that they unambiguously reflect specific motivational orientations. Focusing on the literature of facial approach/avoidance responses it becomes even more obvious that both motivational orientation as well as evaluative coding mediate the link between evaluative processes and response activation (cf. Neumann et al., 2014a). Thus, again, evaluative coding is an important mechanism contributing to the well-known affective S–R compatibility phenomena. However, independent of evaluative coding, motivational orientations seem to affect especially those responses that unambiguously reflect approach and avoidance orientations.

In sum, several findings contradict the postulate that one theoretical framework is sufficient to explain all affective as well as non-affective stimulus–response compatibility phenomena (cf. Lavender and Hommel, 2007). Instead, as reasoned by Krieglmeyer et al. (2010), p. 612), "[both mechanisms—the motivational orientation and the evaluative coding mechanism—seem] to operate independently of and in parallel to [each other]." The portion of the variance of the affective S–R compatibility effect that each mechanism might explain seems to vary across different types of approach-avoidance tasks and different effector of approach-avoidance responses.

# **Conclusion**

Taken together, we have seen that in the literature there are two opposite accounts for affective stimulus–response compatibility phenomena. The motivational orientation account states that valenced stimuli automatically activate corresponding motivational circuits: positive stimuli trigger approach motivation and negative stimuli trigger avoidance motivation (Davidson et al., 1990; Gray, 1990; Lang et al., 1990; Strack and Deutsch, 2004). These motivational orientations in turn trigger corresponding behavioral tendencies resulting in a close link between evaluative processes and approach/avoidance response tendencies. On the one hand, the specific muscle activation hypothesis predicts that motivational orientations inflexibly activate specific muscle contractions. On the other hand, the distance regulation hypothesis predicts that motivational orientations flexibly trigger those responses that situationally result in increased or decreased proximity between the self and an object. The motivational orientation account has recently been challenged by authors arguing that the TEC might also explain affective S–R compatibility effects (Lavender and Hommel, 2007; Eder and Rothermund, 2008). The TEC postulates that response episodes are stored in an event file including codes for each perceivable stimulus and response feature. Any of these features is sufficient to prime responses which share characteristics with that feature. Due to the fact that non-affective and (manual) affective S–R compatibility phenomena share basic characteristics it has been suggested to explain all S–R compatibility effects within the same theoretical framework—namely the TEC (Lavender and Hommel, 2007). The present paper explored to what extent this recommendation is applicable. Indeed, there is substantial evidence suggesting that the TEC is an economical model to explain affective S–R compatibility effects when examining manual and presumably whole-body responses, as well. But studies that adopted a different experimental paradigm as compared to the classical joystick task (e.g., the so-called *manikin task*) provided clear evidence in favor of the motivational-orientation account (Krieglmeyer et al., 2010, 2011, 2013). Moreover, focusing on the literature of approach/avoidance effectors used to regulate the social distance (i.e., facial muscle contractions), it becomes even more obvious that the TEC is not an appropriate theoretical framework to entirely account for the observed congruency effects. Instead of assuming the TEC to entirely account for the observed affective S–R compatibility phenomena, as did Lavender and Hommel (2007), one should rather assume that both mechanisms—the motivational orientation and the evaluative coding mechanism—can operate independently of each other and even in parallel (see also Krieglmeyer et al., 2010).

Furthermore, the effector of approach/avoidance responses seems to be an important determinant. However, until now there has been a lack of studies directly comparing different effectors of approach/avoidance. Similarly, Eder et al. (2013) stated that the "lack of cross-talk within and between different levels of behavioral analysis [within the research domain of approachand avoidance-motivated behavior] limits scientific insight into more general principles of approach and avoidance motivations, thereby contributing to fragmentation in the field" (p. 228). We

# **References**


think that future research should explicitly differentiate and compare the effectors of approach/avoidance responses in order to counteract this fragmentation. On the one hand, motivational orientation accounts provide a proper theoretical framework to explain the often observed link between evaluative processes and action tendencies. However, the observed affective S–R compatibility effects differ as a function of the effector by which responses were stimulated. Hence, from a theoretical viewpoint it would be an important step forward to refine motivational accounts by including the effector as an important factor in order to gain a deeper insight into the link between evaluative processes and emotional responses.

# **Acknowledgments**

The research summarized in this review was supported by the Research Group "Emotion and Behavior" (FOR 605; Ne721/2-3) sponsored by the German Research Foundation (DFG).


**Conflict of Interest Statement:** The Associate Editor, Fritz Strack, declares that despite being affiliated with the same institution as the author, Ljubica Lozo, the review process was handled objectively and no conflict of interest exists. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Kozlik, Neumann and Lozo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **Facial mimicry in its social setting**

*Beate Seibt 1, 2\*, Andreas Mühlberger 3, 4, Katja U. Likowski 3 † and Peter Weyers <sup>3</sup>*

<sup>1</sup> Department of Psychology, University of Oslo, Oslo, Norway, <sup>2</sup> Centro de Investigação e Intervenção Social, ISCTE - Instituto Universitário de Lisboa, Lisboa, Portugal, <sup>3</sup> Department of Psychology, University of Würzburg, Würzburg, Germany, <sup>4</sup> Department of Psychology, University of Regensburg, Regensburg, Germany

*Edited by:*

Jack Van Honk, Utrecht University, Netherlands

#### *Reviewed by:*

Peter A. Bos, Utrecht University, Netherlands Sebastian Korb, International School for Advanced Studies, Italy

#### *\*Correspondence:*

Beate Seibt, Department of Psychology, University of Oslo, Postboks 1094 Blindern, 0317 Oslo, Norway beate.seibt@psykologi.uio.no

#### *†Present Address:*

Katja U. Likowski, Deutsche Gesellschaft für Verbraucherstudien mbH, Berlin, Germany

#### *Specialty section:*

This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology

*Received:* 02 October 2014 *Accepted:* 20 July 2015 *Published:* 11 August 2015

#### *Citation:*

Seibt B, Mühlberger A, Likowski KU and Weyers P (2015) Facial mimicry in its social setting. Front. Psychol. 6:1122. doi: 10.3389/fpsyg.2015.01122 In interpersonal encounters, individuals often exhibit changes in their own facial expressions in response to emotional expressions of another person. Such changes are often called facial mimicry. While this tendency first appeared to be an automatic tendency of the perceiver to show the same emotional expression as the sender, evidence is now accumulating that situation, person, and relationship jointly determine whether and for which emotions such congruent facial behavior is shown. We review the evidence regarding the moderating influence of such factors on facial mimicry with a focus on understanding the meaning of facial responses to emotional expressions in a particular constellation. From this, we derive recommendations for a research agenda with a stronger focus on the most common forms of encounters, actual interactions with known others, and on assessing potential mediators of facial mimicry. We conclude that facial mimicry is modulated by many factors: attention deployment and sensitivity, detection of valence, emotional feelings, and social motivations. We posit that these are the more proximal causes of changes in facial mimicry due to changes in its social setting.

**Keywords: mimicry, facial expression, EMG, cooperation, competition, mood**

# **Introduction**

We humans have complex social lives, and our sociality is deeply ingrained in the makeup of our brains. Certainly, being able to cooperate, to lead and follow, to negotiate and to care for each other has given us an advantage over other species. Yet, how exactly we manage to coordinate, understand others' states and intentions and signal our own, still needs clarification. Of course, emotion psychologists have long been fascinated with emotional expressions, and social psychologists have been studying the influence of social situations on social behavior for over a century. However, the two fields of interest have only recently been brought together. This allows us for the first time to delineate how facial muscular reactions to the most important social stimulus, the human face, depend on the social context an individual is in, including one's traits and states, or one's relationship with the other person. In this paper, we explore facial reactions to facial expressions in order to better understand human nonverbal communication and its coordination.

The social situation in which facial mimicry typically occurs is that of a conversation. Whereas facial mimicry also occurs in other social situations such as between teacher and student, or actor and audience, the current review focuses on situations where only two individuals are involved who may or may not talk to each other. Such interpersonal encounters vary according to their setting, the states and traits of each of the interaction partners and their relationship. The current review aims at identifying the key findings as they pertain to these sources of influence.

# **Theories of Behavioral and Facial Mimicry**

The term mimicry describes the often unconscious and unintentional imitation of behavior shown by an interaction partner, like posture, prosody, or facial expressions, the latter called facial mimicry. Specifically, facial mimicry means congruent facial muscular activations in response to an emotional facial expression. For example, individuals react with an activation of the *Zygomaticus major* (muscle which lifts the corners of the mouth, forming a smile, labeled Zygomaticus throughout this review) when they look at a happy face, or they react with an activation of the *Corrugator supercilii* (muscle which draws the eyebrows together, forming a frown, hereafter labeled Corrugator), when they look at a sad face. These reactions can be of very low intensity and are therefore usually registered electromyographically (EMG) by placing electrodes over the respective muscle regions (Fridlund and Cacioppo, 1986). Changes in muscular activity are typically reported as difference scores between the reaction score, i.e., the muscle activity while watching the expression, and baseline, e.g., the muscle activity during the second before stimulus onset.

Measured this way, the muscular changes begin within the first 500 ms after stimulus onset and are typically outside of conscious awareness (Dimberg and Thunberg, 1998). They can be observed even after subliminal exposure (Dimberg et al., 2000; Bailey and Henry, 2009), and it seems that they cannot be suppressed, even if one is instructed to do so (Dimberg et al., 2002; Korb et al., 2010). This suggests that facial mimicry occurs automatically as a reflexlike reaction. There is also evidence for biological preparedness, because already neonates show it (cf. Field et al., 1982; Meltzoff and Moore, 1989).

# **Mimicry as Pure Motor Copy**

According to Chartrand and Bargh (1999), mimicry, both facial and gestural, is based on the *perception-behavior link*. According to this theory, perception is linked to behavior because of a common, or shared, representational system for perception and action (see also Prinz, 1990). As a consequence, the probability of a certain behavior increases with the mere observation of that behavior shown by another person, an effect also referred to as the *chameleon-effect* (cf. Lakin et al., 2003). This theoretical approach is discussed by Hess and Fischer (2013) as the *matched motor hypothesis*. Chartrand et al. (2005) describe mimicry as a universal process, which takes place in all people who can perceive and imitate behaviors, and which always represents the same meaning and function.

This view of mimicry is closely related to its postulated function. It is widely agreed that facial and other mimicry promotes affiliation and plays an important role in initiating and maintaining positive social relationships (Hess and Fischer, 2013). There is evidence that someone who mimics is liked more than someone who does not mimic, even when the mimicking vs. not mimicking agents are avatars (Bailenson and Yee, 2005). Thus, mimicry has been referred to as *social glue* (Lakin et al., 2003), which binds individuals together.

Lakin and Chartrand (2003) assume that they observed mimicry of foot shaking and face rubbing because "activating the desire to affiliate temporarily strengthened the perceptionbehavior link. Specifically, the desire to affiliate may cause people to pay more attention to what occurs in their social environments (i.e., they perceive more)" (p. 338). Applied to facial mimicry, this means that any situation that increases or reduces the desire to affiliate should also increase or reduce facial mimicry.

# **Facial Expressions Carry Meaning**

Yet, in facial mimicry, as opposed to gestural mimicry, the facial expressions of the sender and the receiver carry specific intrinsic meaning. According to Hess and Fischer (2013), smiles, fearful and sad expressions are more affiliative than frowns and disgusted expressions. But even the same expression can mean different things. A mother's reciprocation of her child's sad expression carries more information than her reciprocation of the child's foot shaking. It may mean that the mother cares about the sadness of her child, understands and shares it, which implies a communal relationship. It may mean that the mother is sad about the child's sadness, which relates to her own feeling state. It may mean that the mother wants the child to stop being sad, an appeal, or confirm the veracity of the sad news that the child anticipated, a factual information. It may even encompass several of these aspects at the same time.

In order to understand the facial response given to a facial expression, it is therefore necessary to make assumptions about the perceiver's understanding of the situation and relationship, as well as about his or her state and intentions. This is necessarily somewhat speculative, and made more difficult by the fact that many facial mimicry studies do not assess such variables. In this review, we therefore combine a top-down approach of making reasonable assumptions about likely intentions, states and interpretations, with a bottom-up approach of searching for stable findings and plausible explanations of them. Both moderators and mediators can be informative with regard to the message conveyed by a facial expression.

As mentioned earlier, to structure this review, we focus on moderation of facial mimicry by aspects of the social situational context. Specifically, we first examine factors relating to the setting, then to personal characteristics of the perceiver and the sender (except for clinical psychopathological characteristics), and finally to the relationship between perceiver and sender.

# **The Setting**

Imagine you attend a party with loud music. It is likely that in such a situation, you focus more on your interaction partner, and tune up your facial expressions and your facial mimicry, in order to make yourself understood despite the difficult verbal communication. Now imagine you attend a business meeting, everybody wears business attire and the conversation is about money. In such a situation, you might be much more restrained in your facial communication. The reason could be that you do not want to be "read" too easily, or that you are uncertain about the appropriate facial display.

# **Social Interaction**

As different social settings require different facial signaling, social norms and scripts develop regarding the normal, correct or desirable facial behavior in a setting. However, in most facial mimicry studies, the setting is impoverished compared to the multitude of signals present in natural settings, with participants passively watching photos, computer-generated images or short video sequences on a computer screen, with electrodes attached to their face. A few studies have, however, investigated facial responses during interactions of two naive participants, as described next.

Hess and Bourgeois (2010), using EMG measurement of two participants concurrently, found that in such interactions, women smiled more than men, and both genders showed more Duchenne smiles than polite smiles. The latter lack an *Orbicularis oculi* activation (labeled Orbicularis), which raises the cheeks and forms crow's feet around the eyes (see, e.g., Messinger et al., 2001). Interestingly, even when participants talked about an anger episode, only smiles, but not frowns, were mimicked. Likewise, during naturalistic observations in shopping malls with direct response coding by an observer, about half of the smiles of experimenters were returned but hardly any frowns (Hinsz and Tomhave, 1991). A set of studies by Heerey and Crossley (2013) allows a comparison between a natural conversation in the lab (using facial coding with the Facial Action Coding System; FACS, Ekman and Friesen, 1978a) and a highly controlled setting involving computer-displayed "senders" (using EMG). In both studies, Duchenne smiles were reciprocated earlier than polite smiles, with muscle contractions even starting before the onset of an expected Duchenne smile (Study 2).

# **Cognitive Load**

Another difference between lab settings and natural settings is that in lab studies, care is taken that participants do not hear or see anything that is not part of the experimental setup. Yet, in personal encounters, there is always additional stimulation: usually individuals are engaged in conversation, which can be more or less demanding, depending on the topic and the goal of the conversation. There is also typically distracting background noise, visual and other stimulation. Finally, a person may be distracted by additional tasks which have to be solved, or her own thoughts. Thus, the question is whether facial mimicry still occurs when individuals have reduced processing capacity due to cognitive load. If facial mimicry is diminished by cognitive load, then we can conclude that some aspect of the secondary task interferes with the processes leading to facial mimicry.

Regarding visual distraction, the task to indicate the color of the presented faces reduced facial mimicry (Cannon et al., 2009). A task to judge the genuineness of emotion expressions eliminated facial mimicry, and showed instead Corrugator activity as a function of task difficulty (Hess et al., 1998). Yet, when the task also involved a valence judgment of the expression, facial mimicry remained intact. A more recent study asking participants to judge the authenticity of smiles found that facial mimicry was still present and that it predicted ratings of authenticity (Korb et al., 2014). Regarding auditory distraction, video sequences of Ronald Reagan's emotional expressions only evoked facial mimicry when played without sound (McHugo et al., 1985).

We conclude that mimicry of Duchenne smiles plays an important role in conversations, and that anger mimicry may be uncommon in these settings. Furthermore, focussing on another aspect of the situation than valence and emotion diminishes facial mimicry, suggesting that facial mimicry depends on emotional processing. Yet, more research in naturalistic settings is needed to understand how they influence facial communication.

# **The Perceiver**

In conversations, individuals are always both perceiver and sender. In most experiments on facial mimicry, however, only the facial expressions of the sender are varied, which allows a clear distinction between both roles. Specifically, most research on perceiver characteristics measured facial reactions to static photos of persons or to computer generated faces, facing the perceiver with direct gaze and displaying a clear emotional expression, as described in the FACS (Ekman and Friesen, 1978a). Recently, more studies use short video sequences of actors posing the development of an expression or morphs between a neutral start frame and the full expression; we refer to these stimuli as dynamic facial expressions.

Given the importance of personal characteristics in interpersonal behavior, one can expect that across situations and relationships, some individuals tend to mimic more than others, because of different personal characteristics like cultural background, gender, and personality traits or because of their current state. Accordingly, we review evidence for modulation of facial mimicry by personal characteristics and by states.

# **Personal Characteristics**

Considering the functionality of facial mimicry for bonding, rapport and interpersonal connection, one should expect that traits positively related to affiliative motivation enhance facial mimicry.

# **Empathy**

One personal characteristic important for social rapport is empathy, which can be divided into at least two sub-constructs: emotional empathy or empathic concern ("I feel what you feel") and cognitive empathy or perspective taking ("I understand what you feel"). Evidence for this basic distinction is provided by evolutionary (de Waal, 2008) as well as neuropsychological approaches (Shamay-Tsoory et al., 2009). Individuals high in emotional empathy or empathic concern should be more likely to show facial mimicry, because they feel with the other, and because they are motivated to show their concern (for a summary and discussion see Goldman and Sripada, 2005). Conversely, facial mimicry may enable cognitive empathy, by working as a feedback mechanism about the other person's emotional state (e.g., Neal and Chartrand, 2011).

In a first study regarding emotional empathy and facial mimicry, Sonnby-Borgström (2002) recorded Zygomaticus and Corrugator reactions to happy and angry faces in participants scoring high and low on the Questionnaire Measure of Emotional Empathy (QMEE; Mehrabian and Epstein, 1972). High compared to low scoring participants showed stronger congruent Zygomaticus and Corrugator reactions even at short stimulus presentation times (averaged for the 17, 25, 30, and 40 ms). Furthermore, correlations between facial reactions and self-reported feelings were significantly higher in the high scoring participants. In a further study, Sonnby-Borgström et al. (2003) used less presentation times and a backward masking technique and found evidence of facial mimicry at 56 ms for high empathic participants only (as determined by median split). Other presentation times (17 ms; 2350 ms) did not produce significant EMG differences.

Dimberg et al. (2011) also presented happy and angry faces to the 30 highest and the 30 lowest QMEE scorers, selected from a larger sample of students. While the low empathic participants' facial reactions did not differentiate between the happy and angry faces, the high empathic participants showed larger Corrugator reactions to angry compared to happy faces and larger Zygomaticus reactions to happy compared to angry faces. In addition, the high empathy group rated both expressions as more intense than the low empathy group. Measuring emotional empathy (QMEE) after exposure to the facial expressions, Harrison et al. (2010) found a greater differentiation of Corrugator responses between angry and sad expressions on one hand and happy expressions on the other hand for high empathic individuals.

Much less is known about the influence of cognitive empathy on facial mimicry. Likowski et al. (2011a), using the Reading the Mind in the Eyes Test (Baron-Cohen et al., 2001) found that the experimental context influences cognitive empathy– with individuals in a competitive context having less–, and that high cognitive empathy predicted specifically more happiness mimicry. Hermans et al. (2009) used extreme scorers on the Autism Spectrum Quotient (AQ, Baron-Cohen et al., 2001), which assesses, as one main component, difficulties with social interactions, presumably related to deficits in cognitive empathy. They found only for low scoring women (but not for female high scorers and men) a significant difference in Corrugator reactions to smiles (congruent Corrugator relaxation) vs. frowns (congruent Corrugator activation) and, descriptively, also a stronger Zygomaticus reaction to smiles in this group. However, the small sample (only six women in that group) precludes generalizations from this study. Sims et al. (2012) (see below) found that while low AQ-scorers showed more smile mimicry for positively conditioned faces than for negatively conditioned faces, high scorers' mimicry reactions were independent of conditioning.

In sum, the available evidence shows stronger congruent facial reactions to happy and angry faces in individuals high in emotional empathy, and suggests that they perceive emotional expressions as stronger than low empathic individuals. A tentative conclusion from the available evidence on cognitive empathy is that it may boost affiliative smile mimicry toward rewarding interaction partners. However, research with more specific measures of trait cognitive empathy is needed to corroborate these results.

#### **Attachment Style**

Attachment styles are classifications of a person's relationship with attachment figures (Bowlby, 1969). Here, of particular interest is whether attachment needs are expressed (secure and anxious attachment) or concealed (avoidant attachment). These styles may impact affiliation behavior more generally, with avoidant adults concealing their negative reactions to negative affiliation signals. To test this, Sonnby-Borgström and Jönsson (2004) had participants watch happy and angry facial expressions with different presentation times. With a presentation time of 56 ms, representing an automatic level of processing, avoidant participants did not show any Zygomaticus responses, but compared to non-avoidant participants a tendency toward stronger Corrugator responses to angry faces. With a presentation time of 2350 ms, representing a cognitively controlled level of processing, avoidant participants showed no Corrugator response, but increased Zygomaticus activity, i.e., a smiling response, to angry faces while non-avoidant participants reacted with a congruent Corrugator activation. This suggests that avoidant individuals show enhanced anger mimicry when they are not aware of the stimulus face, but they tend to conceal this mimicry, and instead display a smile under conscious exposure conditions.

# **Social Anxiety**

Are individuals with fear of public speaking more sensitive to signs of social disapproval, such as an angry face? Four studies, measuring fear of public speaking with the Public Report of Confidence as a Speaker questionnaire (PRCS; Paul, 1966), investigated this issue.

Dimberg and Christmanson (1991) selected participants according to extreme scores on the PRCS. While the low fear group exhibited Zygomaticus activation to happy faces and Corrugator activation to angry faces, i.e., congruent responses, the high fear group's facial responses were not affected by the quality of the stimuli. However, in the first phase of this study, participants were presented with neutral stimuli as well as faces, which might have primed the anxious population to assess the images more cognitively and less affectively. This may therefore have led participants with extreme social fear to emotionally disengage.

Dimberg (1997) exposed women above and below the PRCS median to angry and happy facial expressions. As before, low compared to high fear women showed larger Zygomaticus responses to happy faces. However in this study, high compared to low fear women showed larger Corrugator responses to angry faces. Corrugator responses in Dimberg and Thunberg's (2007) study, which used 1 s instead of 8 s exposure in the other studies on this question, differentiated better between angry and happy photos in high-fear women than in low fear women. This was mainly due to a clear Corrugator deactivation to happy photos in the high-fear group. In this study, high fear women also showed larger Zygomaticus responses to happy faces than low fear women.

Vrana and Gross (2004) selected their participants from a pool of introductory Psychology students according to their PRCS scores. Participants for the high fear group were chosen from the top 10% scorers and participants for the low fear group from the students scoring one standard deviation around the mean. Low compared to high fear participants overall showed larger Zygomaticus activation, i.e., smiled more, especially to happy and neutral expressions. Corrugator reactions showed only a descriptive trend toward stronger activation in response to angry faces in high compared to low fear participants. Interestingly, in this study both groups showed activation of both muscles compared to baseline for happy, angry, and neutral expressions, possibly indicating amusement or cognitive effort.

Thus, three studies, all with 8 s exposure and EMG assessment, show weaker Zygomaticus activation to smiles in the high fear compared to the low fear group, suggesting reduced affiliative behavior toward strangers due to high social anxiety. The one study with 1 s exposure to expressions, however, shows stronger Zygomaticus responses to smiles in the high fear group. Also, in three studies, high compared to low fear participants reacted with stronger Corrugator activation toward angry faces. A possible interpretation is that this reaction is an emotional one indicating fear elicited by social threat. However, the *Frontalis* (muscle which raises the eyebrows) as indicator of a fearful expression has not been measured in either of the studies to validate this hypothesis. A related interpretation is that socially anxious individuals are more sensitive to all emotional expressions, as shown with short exposure times; they, however, inhibit their smile to happy faces at longer exposure times due to fear of social contact (Dimberg and Thunberg, 2007).

#### **Gender**

While most facial mimicry studies used male and female participants, few publications report tests for gender effects. In a study specifically concerned with participants' gender as well as the gender of the facial stimuli, Dimberg and Lundquist (1990) presented angry and happy facial expressions for 8 s each. Women reacted stronger with the Zygomaticus to happy faces. No effects of the poser's sex could be detected. This finding fits results from natural settings of more smile mimicry in female than in male dyads (Hess and Bourgeois, 2010), and of less smile mimicry by men to men passing by than in any of the other three constellations (Hinsz and Tomhave, 1991). However, Vrana and Gross (2004) failed to replicate Dimberg and Lundquist and instead found no effects of participants' gender, but an effect of the poser's sex: angry women and happy men elicited more congruent reactions.

Soussignan et al. (2013) varied participants' gender and avatars' sex and gaze direction and measured specific muscle sites for anger, fear, happiness, and sadness. Analyses revealed main effects of participants' gender not influenced by eye gaze–women compared to men showed less anger mimicry (Corrugator), and more sadness mimicry (*Depressor Anguli Oris*; the muscle which pulls the lip corners downward). For fearful and happy faces, a complex interaction emerged: women mimicked fearful expressions with averted gaze (Frontalis), while men did not mimic these expressions at all. Conversely, happiness mimicry (Zygomaticus) was stronger in the direct gaze condition, but this was only true for men. Using a similar setup with less muscle sites (Corrugator and Zygomaticus) and emotions (anger and happiness), Schrammel et al. (2009) found that female participants showed stronger Zygomaticus activation to male rather than female happy faces independent of gaze, whereas the same was not true for male participants. A further interaction involving the Corrugator, genders of perceiver and sender, and facial expression was not further decomposed.

To conclude, there is no consistent pattern of gender effects on facial mimicry. Women may show more same-sex smile mimicry and may be more attuned to environmental (fear with averted gaze) and self-disclosure information (sadness). Yet given the exploratory nature of some of these analyses, a replication is important. In addition, it seems likely that social goals and motives, as well as the concrete relationship, moderate gender effects on facial mimicry. For example, whether gender is seen as a salient intergroup dimension, whether individuals are motivated to flirt or to protect their relationships, and whether they interact with known others or strangers is likely to influence gender effects.

# **Age**

Do younger and older adults differ in their facial mimicry? When presenting angry facial expressions to younger (age range 18– 26 years) and older (age range 65–83 years) adults, Corrugator reactions did not depend on age (Bailey et al., 2009). In a further experiment, angry and happy faces were presented subliminally and masked with neutral faces (Bailey and Henry, 2009). There were congruent Zygomaticus and Corrugator activities to happy and angry faces, respectively, with no effect of age group.

Hühnel et al. (2014) presented a larger set of dynamic facial expressions (happy, angry, sad, and disgusting), with the expressions presented by younger and older men and women, to younger (18–30 years) and older (62–85 years) women. They also measured the activity of more muscles (Corrugator. Orbicularis, Zygomaticus; *Levator labii*, the muscle which lifts and everts the upper lip, hereafter labeled Levator). While they found similar congruent facial muscular reactions to happy (Zygomaticus and Orbicularis activation), angry, and sad expressions (Corrugator activation) for both age groups, a difference emerged for disgust expressions: only the older age group showed consistent mimicry (Levator) in response to this expression. Expression recognition accuracy in the older group was worse for happy and sad expressions; thus, the different measures show dissociation for these two expressions. No sender x perceiver interactions for the facial reactions were reported by the authors.

Thus, overall the available evidence shows more similarities than differences in facial mimicry across the investigated age groups.

#### **Conclusions**

Self-reported emotional empathy enhances facial mimicry of angry and happy expressions. From the reviewed studies, however, it is unclear whether this latter effect is mediated by enhanced sensitivity to emotional signals, enhanced emotional responding or enhanced emotional expressivity. Also, more evidence is needed for the role of cognitive empathy in facial mimicry.

The lack of anger mimicry in avoidantly attached individuals at longer presentation times and the lack of mimicry in individuals high in social fear in a study by Dimberg and Christmanson (1991) can be the result of chronic emotionregulation strategies. Directing one's attention away from an emotional stimulus or re-appraising it are strategies to downregulate negative emotions, and thereby to disengage and detach (Gross, 2014). Avoidantly attached individuals seem to detach by suppressing the activation of the attachment system (Fraley and Shaver, 1997). The present findings suggest that this only happens at longer stimulus exposure. Similarly, at longer exposure times, socially anxious individuals show a negativity bias for facial stimuli, which may be the result of an avoidance orientation (Schwarz and Clore, 1996).

To understand how these effects play out in day-to-day interactions, settings with known others have to be studied as well. Furthermore, other traits influencing social behavior, such as agreeableness, extraversion or chronic power and affiliation motives should also be tested as moderators of facial mimicry. Finally, Person x Situation x Emotion expression experiments can test whether traits influence facial mimicry especially in specific trait-relevant situations with respect to specific expressions, which would strengthen the causal models from trait to facial behavior.

Regarding the demographic categories gender, age, and culture, more studies with sufficient test power are needed. The findings for gender so far fit an evolutionary perspective, according to which women are more vulnerable to environmental threats and should therefore pick up on danger cues more easily, and men are more ready to engage in ingroup and intergroup aggression, and therefore pick up more easily on direct anger expressions (Navarrete et al., 2010).

Investigating facial mimicry in different cultures and across cultures is practically important for cultural understanding and theoretically important, in that it can help distinguish culturally learned from innate propensities. Recent evidence suggests vast differences in dynamic facial expressions between East Asians and Westerners (Jack et al., 2012). Their finding that East Asian models of several emotions show specific early signs of emotional intensity with the eyes is in line with the finding that Japanese look more at the eyes regions and US Americans more at the mouth of emotional expressions (Yuki et al., 2007).

# **Current State of the Perceiver**

In addition to relatively stable personality factors, the perceiver's psychological and physiological state also moderates facial mimicry.

## **Fearful Mood State**

Participants in an experiment by Moody et al. (2007; Exp. 2) watched neutral or fear-inducing film clips and afterwards neutral, angry, and fearful expressions. In the fearful condition, participants showed fearful expressions to angry and fearful faces, as was indicated by heightened Frontalis activity already in the second half of the first second after stimulus onset. These responses may be explained by a fast and vigilant information processing style, because being in a fearful state indicates that one has to watch out for danger cues in the environment (e.g., Schwarz and Clore, 1996; Luu et al., 1998).

#### **Sad Mood State**

Likowski et al. (2011b) induced happy and sad mood states through film clips. Afterwards, participants viewed happy, angry, sad, and neutral expression, and facial muscular activity was measured over the Zygomaticus, Corrugator and Frontalis regions. EMG analyses for the second half of the first second showed stronger mimicry of happy, sad, and angry faces for the happy compared to the sad group. Furthermore, the sad group showed hardly any significant facial muscular reactions to the emotional faces, except for a Corrugator deactivation to the happy faces. On the other hand, after having watched the happy film clip, participants showed clear signs of congruent muscular reactions to happy and sad expressions, and also a tendency to mimic anger expressions.

To conclude, facial mimicry is determined by the quality of the negative mood state and not its valence: Participants in a fearful mood showed a fear response to angry faces, while participants in a sad mood did not show any significant reaction to this expression. The latter effect can be explained by self-focused attention (e.g., Wood et al., 1990; Sedikides, 1992; Green and Sedikides, 1999).

Interestingly, participants in the sad condition reacted to happy faces with a Corrugator deactivation, just as the participants who watched the happy film clip, and they rated the sad faces as more arousing compared to the participants in the happy condition. Thus, obviously these participants still paid some attention to their social environment; however, the lack of a Zygomaticus reaction to happy faces indicates that they did not have the capacity or the motivation to show signs of affiliation.

## **Manipulation of the Current State by Drug Application**

Hormones and neurotransmitters in the body and the brain are connected with mood, arousal, motive, and need states. Thus, to understand how psychological states influence facial mimicry, it seems promising to administer drugs in double-blind, placebocontrolled designs, as a means to modify the psychological state of the participant. Hermans et al. (2006) applied testosterone or placebo to healthy female participants on two separate occasions in a double-blind, placebo-controlled, mixed factorial crossover design, and 4 h after medication, Zygomaticus and Corrugator responses to dynamic happy and angry facial expressions were measured. Testosterone, compared to placebo, decreased the congruent responses to both expressions.

This result might indicate a trade-off of status and affiliation motives, with acutely rising testosterone levels evoking the status motive and suppressing the affiliation motive (see Eisenegger et al., 2011). However, it is somewhat surprising that testosterone did not increase anger mimicry, especially given that testosterone leads to increased vigilance (van Honk and Schutter, 2007) and increased heart rate to angry faces (van Honk et al., 2001). Given that anger mimicry in status competition is potentially costly, it may well be that testosterone prepares for confrontation, but that other factors determine whether the confrontation is actually sought. Manipulating the norepinephrine system had no impact on facial mimicry (Harrison et al., 2010). We suggest that oxytocin, which is assumed to play a critical role in social cognition and behavior (cf., Churchland and Winkielman, 2012; Kanat et al., 2014), enhances facial mimicry, e.g., by enhancing the recognition of facial expressions (Shahrestani et al., 2013).

## **Conclusions**

The perceiver's mood modifies facial reactions to emotional faces by changing the perception and interpretation of the social environment. A fearful reaction to angry expressions in a fearful state reflects the perceiver's internal state (see Moody et al., 2007), but it also carries a relationship meaning (I submit) and an appeal (do not hurt me). The reduced mimicry after testosterone application and in sad mood arguably have different causes. It is plausible that status motives inhibit affiliation motives, whereas a sad mood may lead to a temporary inability to engage in affiliation due to self-focused attention, not to a lack of motivation. Future studies should test mediation models for these states, and also expand the range of states examined to emotional states like anger and pride (cf. Dickens and DeSteno, 2014, for pride and behavioral mimicry). Of practical importance is furthermore the question whether and how effects of these states differ from those of chronic forms, such as neuroticism or anxiety disorders, depressive disorders, and chronically elevated testosterone levels.

# **The Sender**

Not only perceivers, but also senders have characteristics that influence perceivers' reactions to facial expressions. Their sociodemographic variables such as senders' gender and age have been discussed briefly under perceiver characteristics. Cultural background of the sender has been studied as a determinant of group membership and will be discussed there. The senders' traits and states will influence which emotional expressions they show with which frequency, clarity and intensity. Here, we focus on two expressive features which have been experimentally investigated: eye gaze and the dynamic of the expression.

# **Eye Gaze**

An important cue to interpreting facial expressions is gaze direction. It helps us understand who an emotional expression is directed at. Is the person afraid of me, angry at me, glad to see me (Adams and Kleck, 2003, 2005)? Thus, evaluations of expression *and* gaze direction determine the relevance of the expression (cf. Graham and LaBar, 2012).

Yet, only few studies so far have investigated whether facial mimicry is moderated by gaze. Rychlowska et al. (2012). (Exp. 3) presented photographic images of smiling people with direct and with averted gaze and found stronger Zygomaticus activation for direct gaze, which was also judged as more positive. In an experiment by Schrammel et al. (2009), avatars "walked" to the middle of the computer screen, turned to the participant or sideways, displayed a dynamically developing happy or angry expression or a neutral expression, and then left again, to the other side. Zygomaticus activity was stronger while watching happy expressions compared to angry and neutral expressions only when the smiling avatars faced the participants. Corrugator activity was higher while looking at angry and neutral compared to happy faces, and this again was more pronounced in the direct gaze condition. As described already above, the results by Soussignan et al. (2013) show higher order interactions between emotional expression, gaze direction and perceiver's gender.

# **Dynamic Expressions**

In real-life encounters, facial expressions are often ambiguous, sometimes a mix of several emotions, often very slight and always dynamic, moving from neutral or from another emotion to the current emotional or neutral display (cf. Mühlberger et al., 2011). Yet much of the research on facial mimicry used photographic images of rather idealized emotional expressions. How valid are these findings for predicting facial mimicry in an interactive setting? To start studying this question, researchers have compared responses to still photographs of prototypical emotions with responses to dynamic video sequences or morphs, starting from a neutral expression.

Rymarczyk et al. (2011) compared muscular responses to static and dynamic (neutral to emotional) happy and angry expressions of the same actors within participants. Happy dynamic expressions produced faster and stronger mimicry than static ones. Results were less clear for angry faces: Corrugator responses were small, and the only difference was a faster Corrugator activation for dynamic angry faces.

Using FACS coding of responses to dynamic and static expressions, Sato and Yoshikawa (2007) found evidence of anger and happiness mimicry only for the dynamic versions. Sato et al. (2008) found enhanced facial EMG to happy and angry dynamic expressions, compared to the static ones, on the Zygomaticus and Corrugator, respectively. Yet they did not find differential Corrugator deactivation in response to dynamic and static smiles. In another study with a similar setup, the Corrugator showed a greater deactivation—and the Zygomaticus a greater activation—to dynamic compared to static happy expressions, yet no differences for the anger expressions were observed (Weyers et al., 2006).

In sum, dynamic, self-directed expressions generate the largest response, especially to smiles. In social encounters, emotional expressions always unfold. Compared to still images, this dynamic draws attention to the change occurring, and it is also a further cue, in combination with direct gaze, that the smile is directed at the participant. For anger expressions, the evidence is less clear, with some studies finding evidence of more anger mimicry for dynamic than for still expressions, and others not. Importantly, the available studies, while suggesting that working with dynamic stimuli increases test power, do not invalidate findings from studies with static stimuli, as static and dynamic stimuli did not produce qualitatively different effects. Apart from increasing test power, dynamic stimuli can also involve the disappearance of an expression (cf. Mühlberger et al., 2011) or the change from one expression to another. Such dynamics are frequent in interactions, yet little is known about the conditions for their mimicry.

# **The Relationship**

The relationship between the interaction partners can be described in many ways. One of the fundamental distinctions concerns whether there is a pre-existing relationship or whether strangers interact. Pre-existing relationships can be characterized according to their predominant relational model (Fiske, 2004) or relational orientation (Clark et al., 1998) whereas for strangers, important dimensions are warmth and competence (Fiske and Fiske, 2007).

# **Familiarity**

Despite the obvious importance of interactions in existing relationships, we know of only two empirical publications measuring facial mimicry with long-standing relationship partners. In one study, a friend or family member vs. stranger observed the sender's disgust and pride responses to the tasks she performed (Fischer et al., 2012). A FACS analysis of the videotaped expressions revealed no disgust mimicry, and smile mimicry (here, as part of the pride display) only among intimates (friend or family). In the other, photos of the romantic partners of participants were displayed on the computer alongside photos of strangers, and EMG measures to angry expressions were taken (Häfner and Ijzerman, 2011, Study 1). Results showed increased Zygomaticus responses to the anger expression of romantic partners toward whom participants had a communal orientation. This can be interpreted as a soothing smile to regulate the partner's anger, and shows the importance of relationship variables to understand facial mimicry in existing relations. In sum, among intimates, smiles in response to smiles and to other emotional expressions seem to regulate the relationship. Mimicking negative emotions may be uncommon among intimates and in social settings. How is facial mimicry between strangers influenced by their relationship, in particular their attitudes, goals, and group membership? Having a positive attitude toward another person means assessing them as warm, friendly, good-natured and sincere. The socialcognitive content model (e.g., Fiske and Fiske, 2007) maintains that this warmth dimension of social judgments essentially answers the question: friend or foe? A person judged as warm is judged to have good intentions and goals at least compatible with one's own. According to the model, this is the case for ingroup members and close allies. Thus, attitudes, goal compatibility and group membership are naturally confounded dimensions of relationships. Nevertheless, outgroups can be seen as positive or neutral, as is typically the case between men and women, or between adults and children, and having temporarily incompatible goals in a chess game does not preclude a generally friendly relationship. It is therefore informative to manipulate these factors separately to understand how they influence facial mimicry. Next, we will review evidence regarding attitudes without a salient group membership.

# **Attitudes**

# **Explicit Attitude Manipulations**

Considerable evidence supports the view that positive attitudes automatically elicit approach behavior toward objects and people, while negative attitudes automatically induce avoidance behavior (e.g., Chen and Bargh, 1999; Neumann and Strack, 2000; Neumann et al., 2004; Seibt et al., 2008). If mimicry is a means to affiliate, and thus related to approach behavior, then a positive attitude toward a person should lead to an approach orientation and hence—enhanced—mimicry, while a negative attitude should lead to an avoidance orientation and hence reduced mimicry. To test these assumptions, we manipulated attitudes experimentally (Likowski et al., 2008) and expected positive attitudes toward a person to cause stronger mimicry of facial expressions posed by that person, compared to neutral and negative attitudes; for negative attitudes, we expected no congruent or even incongruent reactions.

First, computer generated female and male neutral faces were introduced to female participants as avatars with different characters designed for computer games. Next, three character descriptions, by using adjectives, were paired to avatars with three different hair colors, and participants were asked to memorize the avatar characters with their specific traits for a later recalltask. The positive characters were described as kind, nice, likeable, and self-confident, the neutral characters as reserved, serious, calm, and neat, and the negative characters as malicious, aggressive, egoistic, and deceitful. After a recall task, showing that participants had processed the information, they watched the different characters again, however, now with happy, neutral, and sad expressions, and Zygomaticus and Corrugator activities were recorded.

Zygomaticus activation was stronger to happy faces of positive characters than to those of neutral and negative characters, with no difference in the reactions to the latter two. Additional testing for changes against baseline showed significant congruent Zygomaticus activation in response to happy faces of positive and neutral, but not of negative characters. Furthermore, while participants reacted with a congruent Corrugator activation to sad faces of positive characters, they reacted with an incongruent Corrugator deactivation to sad faces of negative characters.

Thus, facial mimicry reactions were altered after only one pairing of the avatars with the characterizing adjectives. From a functional perspective, the results make perfect sense. While a positive attitude toward a person leads to approach and affiliative tendencies and thus mimicry of happy and sad expressions, there is hardly any reason to approach and thus mimic somebody toward whom one holds a negative attitude, unless one follows a certain goal. The incongruent Corrugator deactivation to sad faces of negative characters may indicate schadenfreude, but future studies should provide more direct evidence for this interpretation. Based on the results for the negative characters, it is worthwhile to examine whether facial muscular reactions to emotional expressions can be used as a reliable and valid implicit measure in attitude research (cf., Vanman et al., 2004, for reactions to neutral expressions) and whether they can be changed by disconfirming information (cf. Gregg et al., 2006).

# **Implicit Attitude Manipulations**

Attitudes are evaluations, and can be acquired through conditioning. Is it therefore possible to replicate the results of an explicit attitude manipulation with a conditioning procedure? To find out, Sims et al. (2012) conditioned four different neutral faces with different amounts of reward: participants won in 90 or 60% of the trials or they lost in 90 or 60% of the trials in the presence of the respective face Afterwards Zygomaticus and Corrugator reactions to faces of the same persons, now, however, with dynamic happy and angry expressions were measured. Indeed, Zygomaticus reactions to the happy faces were a positive linear function of the reward value conditioned to the respective neutral faces; in contrast, reward value had no significant effect on the Corrugator response to these expressions. For angry expressions, Zygomaticus response was unexpectedly highest in the highest loss condition, with no differences between the other three conditions. If this unexpected finding replicates, it suggests that extremely negative valence also activates the Zygomaticus, either directly or indirectly by activating muscles in its vicinity (see Sims et al., 2012).

Interestingly, even a pairing of a person's neutral face with an emotional expression of that same person changes the reactions to that person's neutral face. Aguado et al. (2013) presented neutral static expressions and immediately thereafter a happy or angry static expression shown by the same persons. They found that participants who reacted with a differential pattern, i.e., higher Corrugator activation to angry than happy faces and higher Zygomaticus activation to happy than angry faces, showed this differential pattern already to the neutral faces of the respective persons in the second half of the experiment. Thus, the affective valence of a person changes according to his or her typical facial expression toward the perceiver. In sum, attitudes formed through associative conditioning moderate facial mimicry.

#### **Interdependence**

In order to reach our goals, we often depend on other people. We usually cooperate in work teams, compete with other individuals, teams or companies, we depend on others' fairness, and we work in hierarchies, with some individuals having more status and power than us and others less. How do these factors influence facial mimicry?

#### **Cooperation and Competition**

First, we examine three experiments manipulating cooperation and competition with female participants. In one experiment (Likowski et al., 2011a), participants were told that they would later play a game of dice with an avatar. In the game, both players would throw dice alternately. Then, participants either read that both players would win if the sum of their final scores exceeded a certain value (cooperation), that the one with the highest score would win (competition), or that they had to reach a certain score for winning, independent of the avatar's score (neutral condition). Furthermore, participants would only see their own results; instead of the avatar's results they would see the avatar's facial expression in response to her result. Thus, the avatars' facial expressions now had a specific meaning in the situation. Next, participants played an example round.

For the EMG measurement, participants just watched happy, neutral, sad, and angry expressions, which were described as potential reactions of the avatars to their result. Finally, all stimuli were presented again with the instruction to recall the game situation and to indicate the amount of joy, sadness, and anger evoked by the faces. Then, we measured cognitive empathy (Decety and Jackson, 2004) with the "Reading the Mind in the Eyes Test" (Baron-Cohen et al., 2001; German adaption: Bölte, 2005). To evaluate the goals participants pursued, they were asked to remember the game situation and rate the importance of several aspects like *to appear likable,to have a harmonious and smooth interaction, or to understand the other person's feelings and thoughts*.

In two further studies, we subliminally primed participants with interdependence-related terms. Specifically, four primes were presented 20 times each parafoveally for 90 ms and immediately masked. Participants' task was to indicate where the "flash" had appeared. Cooperation primes were *cooperate*, *partner*, *together*, and *confederate*; competition primes were *compete* (*win* in Weyers et al., 2009), *rival*, *opponent*, and *competition*; and neutral primes were *neutral*, *background*, *street*, and *blackboard* (Weyers et al., 2009; Seibt et al., 2013 without cooperation condition). The EMG procedure was the same in all three studies, but no angry expressions were shown in Weyers et al. (2009). In the end, manipulation checks confirmed successful manipulations and awareness checks confirmed subliminality of the primes.

The results of these studies are summarized in **Table 1**. In discussing them, we will focus on results that replicated across studies. Because only Seibt et al. (2013), but not Likowski et al. (2011a), found differential interaction effects on the Orbicularis, we will not describe these results here.

For neutral priming, we found congruent Zygomaticus and Corrugator reactions to happy and sad faces, respectively; thus participants reacted to affiliative facial expressions in a congruent

**TABLE 1 | Overview of effects of interdependence manipulation on facial responses to emotional facial expressions.**


"Act" means activation of the muscle relative to baseline, "Rel" means relaxation or deactivation relative to baseline, and "0" means no significant change relative to baseline. Parentheses indicate that the test against 0 is not significant but the difference from the other conditions is not significant either.

way (cf. Bourgeois and Hess, 2008). By contrast, participants in the neutral conditions did not show a congruent Corrugator contraction to angry faces.

We did not find enhanced congruent facial reactions to the affiliative expressions in the cooperation compared to the neutral conditions. The results for the game (Seibt et al., 2013) showed that indeed, participants primed with competition behaved more competitively than those primed with cooperation or neutral words, but the latter two groups did not differ from each other. According to van Vugt et al. (2007) women are more cooperative than men, so our female participants presumably had a cooperative stance in the control condition as well. A study including male participants could shed light on this hypothesis.

After the explicit cooperation manipulation, we observed an incongruent Zygomaticus activation to sad expressions. This activation could be fully explained by the goal to have a smooth and harmonious interaction, i.e., it can be interpreted as encouragement. Furthermore, subliminal priming for cooperation led to Zygomaticus and Orbicularis activity increases toward angry faces. These effects can be seen as evidence for a soothing smile toward a cooperation partner in order to prevent the cooperation from failing. Thus, both Zygomaticus reactions can be due to context-specific motivations.

The results of all three studies show a complete lack of Zygomaticus activation to happy faces in the competition conditions. One reason for not mimicking a happy face is a rejection of the opponent's affiliative offer (see Hess and Fischer, 2014). Another one is that a competitor's happiness signals goal progress, which has negative implications for oneself. Yet, we also found Corrugator relaxation to happy expressions in competitive contexts, which is a sign of a positive valence. Mediational analyses revealed that the lack of a congruent Zygomaticus reaction to happy faces in the competitive game condition could be explained by a decrease in state cognitive empathy: The lower the current cognitive empathy, the lower the congruent Zygomaticus response to happy faces. This suggests reduced interest in others due to reduced affiliation motivation.

Our findings for the competition conditions replicated findings by Lanzetta and Englis (1989) that competition leads to incongruent facial reactions: Specifically, participants with a competitive mind-set showed Corrugator relaxation rather than contraction to sad and angry faces. Corrugator reactions are inversely and linearly related to valence (Lang et al., 1993; Larsen et al., 2003). Thus, the competition groups presumably evaluated the sad and angry faces positively because a competitor's sad or angry face indicates an advantage for oneself. It is rather unlikely that the participants considered the anger to be a sign of aggression directed toward them, because in that case a relaxation of the Corrugator muscle, i.e., a positive affective reaction, would not make sense. Support for this interpretation comes from the significant mediations of these Corrugator relaxations to sad and angry expressions by joy in the competitive game condition.

In sum, the results show a modulation of facial mimicry by cooperation and competition. However, with one experiment (Likowski et al., 2011a) we created a specific situation, namely a game of luck in which one does not have control over the outcome. Thus, further studies should investigate a broader range of situations, for example situations in which participants have to make strategic decisions. Furthermore, it would be interesting to vary the amount of rewards that participants are promised for successfully cooperating or competing. This might modulate particularly affective reactions to the emotional expressions.

# **Fairness**

A game of dice as used by Likowski et al. (2011a) is fair because the a-priori likelihood of winning is equal for both sides. Now imagine a situation in which you have to give a certain amount, and your interaction partner is free to split the amount she disposes off evenly or to keep everything. If the latter happens several times you will certainly disagree with your partner's behavior and think of it as unfair.

Hofman et al. (2012) examined the effect of such a fairness manipulation on facial mimicry. Participants were first shown morphs of developing happy and angry facial expressions. In the second part, participants received 1600 points for the upcoming game and were told that the remaining points would be exchanged for money after the experiment. They were then instructed to offer in each of the following trials 25 points to one of two neutrally looking partners who would then have 50 points in total to distribute freely. The "confederates," however, were pre-programmed to either behave fairly (in 75% of the trials the sum was split evenly), or unfairly (in 75% of the trials the opponent kept everything). After the feedback of the trial's result, the chosen partner showed either a happy or an angry expression, identical to those used at the beginning. In the last part, all morphs from the introductory part were presented again.

During the first part of the experiment, participants showed congruent reactions to the happy and angry expressions, i.e., Zygomaticus activation to happy and Corrugator activation to angry expressions. During the second part, facial mimicry in response to happy expressions of unfair partners was reduced in comparison to the responses in the first part. And in the third part, participants showed increased anger mimicry to formerly unfair proposers and decreased anger mimicry to formerly fair proposers, while happy mimicry did not differ from the first part. Thus, the facial reactions which were mimicking responses at the beginning,*changed* according to the participants' learning experience, based on the partners' behavior during the game. The authors assume that the reduced happy mimicry after unfair offers, i.e., after a violation of social norms, is a sign of resentment, thereby not providing reinforcement of the preceding behavior.

# **Power Relationship**

Imagine an interaction between two individuals of different status or social power. What will happen, for example, in case the person high in power looks angrily at the person low in power? Based on power theories as well as studies concerned with emotional perception and responding, Carr et al. (2014) assumed that one's power in a relationship should shape one's facial reactions to the other's emotional expressions. Participants were primed neutrally or with high or low power by a writing task and afterwards watched the faces of target persons whose status was manipulated in power by ascribing a high or low power profession to them. The targets showed happy and angry expressions, and Zygomaticus as well Corrugator reactions were recorded. Low power participants showed Zygomaticus activation (i.e., a smiling response) to all expressions, independent of the targets' power. Their Corrugator reactions to angry and happy faces differentiated stronger for high compared to low power targets. High power participants showed Zygomaticus activation to happy faces of low, but not high power targets. And they reacted with stronger Zygomaticus activation to angry faces of high power persons. These results show that the power (status) relationship modulates facial mimicry. However, its interpretation is not always clear. For instance, why do low power individuals activate the Corrugator to angry faces of high power individuals? Future studies should measure additional muscles, like the Frontalis for fear, and disambiguate the meaning of perceived and emitted facial expressions.

# **Group Membership**

An important part of who we are concerns our group memberships. Members of closely knit groups imitate each other and converge on group norms for clothing, hair style, accent, and non-verbal behavior. It is therefore reasonable to assume that group members show more facial mimicry among each other than toward outgroup members. Furthermore, when group identity is salient (Brewer and Gardner, 1996), group members also tend to feel group emotions following group-based appraisals (Mackie et al., 2000; Smith et al., 2007). This can be a further reason for picking up each other's emotional expressions. Finally, because group membership is important for us, being excluded from groups should motivate us to show affiliative facial behavior to get included again.

#### **Ingroup vs. Outgroup**

Hess (2001) reported that negative racial attitudes toward members of an ethnic out-group covaried with the facial reactions to pictures of facial expressions of these out-group members: French Canadians did not mimic the happy and sad facial expressions displayed by Japanese actors, and the more negative their racial attitudes toward the members of the other ethnic group were, the more they showed incongruent facial reactions to these expressions. Specifically, they smiled at the Japanese actors' sad facial expressions and frowned at their happy ones. Participants in another study watched video sequences of emotional displays of two politicians (without sound) and negative attitudes toward the better known politician (Ronald Reagan) predicted less congruent facial reactions toward his happy expressions (McHugo et al., 1991). Yet, in a prior study, political attitudes did not modulate facial mimicry to Ronald Reagan's videotaped facial expressions (McHugo et al., 1985).

Bourgeois and Hess (2008) investigated facial reactions toward happy and angry displays by two politicians, and toward happy, sad and angry displays by alleged basketball players or nonplayers from an ethnic ingroup or outgroup of the participants. Happy displays were mimicked in all conditions, yet sad displays were only mimicked for faces presented as basketball-players by basketball-players (and as non-players by non-players) and angry displays were only mimicked for a politician by supporters of this politician. The context of a political debate provided a meaning of the display as directed toward the political enemy, not toward the self. Because anger mimicry in more ambiguous contexts can escalate a conflict, it is not surprising that it is avoided in such contexts. Given the lack of a smiling response to happy displays in competitive contexts (see above), it is surprising that the smiles of the competing politician were mimicked in this study. Sadness mimicry, conversely, may become more selective the more social a situation gets, because in social settings, mimicking sadness can become costly by inviting emotional sharing.

A study on French and Chinese participants' (living in France) estimates of the duration of stimulus display of angry and neutral ethnic ingroup and outgroup members sought to find evidence for differential mimicry with an indirect method (Mondillon et al., 2007). In particular, the prediction was that French participants would overestimate the duration of angry ingroup members' displays because they would tend to imitate these displays. This should lead to higher arousal, which in turn would be the proximal cause for the bias. Results confirmed these predictions. Chinese participants, however, did not show a differential estimation. For them, French and Chinese expressions may have been equally relevant, because they lived in France, leading to equal imitation of both groups. The task in this study was a non-social one, which might explain that anger mimicry presumably occurred.

van der Schalk et al. (2011) showed female psychology students angry, happy, and fearful displays of male models, allegedly also studying psychology (ingroup) or studying economics (outgroup). In a second study, they showed Dutch participants of both genders dynamic facial expressions of Dutch and other nationals of unspecified gender. Replicating Bourgeois and Hess (2008), no effect of the group manipulations was found for the mimicry of happiness displays. Conversely, participants showed more facial mimicry in response to ingroup anger and fear than to the corresponding outgroup displays, as measured by EMG in Study 1 and FACS in Study 2. The finding for sadness fits with the Bourgeois and Hess findings, yet they found no anger mimicry for the basketball ingroup. A difference between these two studies is that Bourgeois and Hess used male models (photos) and participants while van der Schalk et al. used male models and female participants in Study 1 and dynamic expressions in Study 2.

Studying teenagers' and adults' reactions to same-age and different-age video-morphings, Ardizzi et al. (2014) found enhanced ingroup mimicry for teenagers, but not for adults. Specifically, the study found enhanced Corrugator reactions in teenagers to teenagers than to adults, while adults' reactions did not differentiate between target groups. From the graphs, it becomes apparent that this difference is carried by teenargers' stronger Corrugator responses to teenagers' vs. adults' sad, fearful, and angry expressions. Both age groups, however, showed similar congruent Zygomaticus reactions to happy faces, independent of the sender's age. Contrary to these results, Hühnel et al. (2014, see above) did not observe an ingroup vs outgroup interaction effect.

# **Social Exclusion**

Social exclusion is a powerful social stressor leading to a wide range of cognitive and behavioral changes intended to regulate one's social relationship because of a fundamental motivation to belong with others or groups (Baumeister and Leary, 1995). Thus, one should expect that affiliative motivation increases after social exclusion, thereby promoting facial mimicry at least to affiliative expressions, and this has indeed been shown by Kawamoto et al. (2014). They used a ball-tossing game (Cyberball) to manipulate social exclusion and found stronger facial mimicry to happy faces after social exclusion compared to social inclusion, as indicated by larger Zygomaticus responses.

## **Conclusions**

The described experiments indicate that group membership is a powerful moderator of the facial reactions to emotional faces. Being a member of a specific group leads to affiliative signs, i.e., smile mimicry, and also to mimicry of sad expressions of members of one's own group, the latter indicating empathy and possibly support. Regarding age groups, only teenagers, but not adults, showed ingroup effects in facial mimicry. These effects could be either due to attitudes, or to shared and non-shared group identity (cf., Schubert and Häfner, 2003). Finally, being excluded from a group increases smile mimicry, possibly indicating increased affiliative tendencies.

# **General Conclusions**

This review examines what is known about facial mimicry in social encounters. We found that many factors that are important in social encounters moderate facial mimicry. We also discovered that this moderation is not just a matter of more or less mimicry, but that the intensity and type of facial response shown to facial displays also depend on the facial expression and the gaze direction. This result fits the observation that facial expressions carry intrinsic meaning, such that imitating a smile does not mean the same as imitating an anger expression. This makes it difficult to discover general rules for when individuals mimic what. Investigating any possible combination of setting, perceiver, relationship, interaction dynamic, expression, muscle site, and gaze direction is impractical, possibly akin to trying to predict the exact verbal reply to a particular statement somebody makes.

On the other hand, this review did show the value of isolating important modulating factors. By trying to understand how these factors influence facial mimicry, we can hope to get closer to the proximal causes and functions of facial responses to facial expressions, and thereby to a predictive model. Different approaches have been used to study the processes underlying facial mimicry. One approach is to test if emotional reactions to the stimuli covary with facial responses or mediate them. Using this approach, Likowski et al. (2011a) discovered that a positive facial response to a negative expression of a competitor was mediated by joy. That same paper reports mediation of other responses by situational goals, and of others by cognitive empathy.

Not only self-reports, but also other responses such as eventrelated potentials can help understand facial mimicry. Achaibou et al. (2008) found that facial mimicry covaried with early event related potentials in the EEG indicative of perceptual processing, which suggests that perception and attention modulate facial mimicry. Regarding not primarily the causes, but rather the functions of facial mimicry, several researchers have blocked facial mimicry to show that it can help in expression recognition (e.g., Oberman et al., 2007; Stel and van Knippenberg, 2008; Maringer et al., 2011; Rychlowska et al., 2014).

While these studies suggest the involvement of perceptual, attentional, emotional, and motivational processes in shaping facial responses to facial expressions, only one paper tested potential mediators of facial responses to facial displays (Likowski et al., 2011a) so the interpretation has to remain speculative. Given that many of the studies reviewed included angry and happy expressions of the sender, we will next discuss likely processes behind facial responses to these expressions.

# **Responses to Happy Expressions**

Happy expressions typically evoke Zygomaticus and Orbicularis contractions and Corrugator relaxations. These responses are rather robust to moderating influences. Even in a sad mood, with no other facial responses, Likowski et al. (2011b) observed a Corrugator deactivation to happy expressions, and this was also the only congruent facial reaction shown to competitors (Likowski et al., 2011a; Seibt et al., 2013). We suggest that this robustness is due to several processes jointly determining happiness mimicry.

First, genuine smiles act as social rewards (Heerey and Crossley, 2013), and therefore evoke a positive response and a tendency to return the reward (Sims et al., 2012). Second, smiles clearly signal a desire to get along and thereby form a solid basis for friendly relationships (Hess and Fischer, 2013; Rychlowska et al., 2015). Likewise, smile mimicry signals to the mimicker that the sender is being authentic (Korb et al., 2014), thus reinforcing affiliative motivation. Third, Corrugator relaxation and Zygomaticus contraction are not specific to happiness and can indicate any positive emotion or affect (Lang et al., 1993; Larsen et al., 2003), as well as other types of smiles, such as dominance or affiliation smiles (Niedenthal et al., 2010). And fourth, returning a smile is usually not a costly signal—no promise is made by returning a smile (with a few exceptions), such that a strong habit can develop to automatically return a smile in most circumstances.

Thus, the next interesting questions to address are not so much whether smiles are responded to with any of these muscles, but more how we can recognize which meaning the smile has (see also Niedenthal et al., 2010), how smiling behavior differs between strangers and friends, between and within groups, and which muscle indicates which aspect of the response.

# **Responses to Angry Expressions**

More puzzling are the responses to angry expressions. If the reason we show mimicry is for the social goal to affiliate with others, mimicking their angry expression does not make a lot of sense. Anger carries the meaning: "You are responsible for my negative outcome" and thereby does not exactly invite affiliation and bonding. Rather, it has been characterized as an aggressive expression (Krieglmeyer and Deutsch, 2013), which may be strategically employed to enforce norm compliance (Hofman et al., 2012). Why, then, did so many studies find anger mimicry? We suggest several explanations.

First, what looks like anger mimicry need not actually be an anger expression at all. Various studies test Corrugator to angry vs. happy expressions, thus effects can also be carried by the Corrugator deactivation to smiles. Furthermore, a contracted Corrugator can also be a sign of global negative affect (Larsen et al., 2003), disapproval (Cannon et al., 2011), incoherence (Topolinski et al., 2009), surprise (Topolinski and Strack, 2015), doubt (Sanna et al., 2002), or mental effort (Stepper and Strack, 1993; Hess et al., 1998; Strack and Neumann, 2000; Koriat and Nussinson, 2009). This goes back to (Darwin, 1955 [1872]) who characterized the frown as a reaction to an obstacle (p. 220). Thus, anger expressions can be "frowned upon" because they are surprising, impolite, and unmotivated.

Second, the less social a situation, the more individuals may allow themselves to engage in mimicry as a way to understand an expression. That is, anger mimicry may be much less common in real encounters than in lab situations (Hess and Bourgeois, 2010). Thus, it may well be that the more "serious" the anger expression of the sender is, and the more real the response, the less likely the anger mimicry. For example, communal partners smiled to angry expressions of their romantic partners, but not of strangers (Häfner and Ijzerman, 2011), and high power individuals did not show pure anger mimicry to anger expressions of other high power individuals, because they also showed Zygomaticus activation (Carr et al., 2014). This latter finding resonates with research finding a preference for complementarity in dominant and submissive postures, rather than imitation (Tiedens and Fragale, 2003). Third, anger mimicry can make sense when the anger is felt as a group emotion toward a common opponent (van der Schalk et al., 2011). And fourth, anger mimicry may also serve to deter aggression, which may explain that men are more likely to show it (Soussignan et al., 2013).

Given these various possibilities, it is important to measure these potential processes in order to understand the meaning of a particular finding.

## **Methodology**

Facial mimicry research always involves facial stimuli with varying expressions and a measure of facial responses. Variations in these aspects across studies complicate the comparison of results. **Table 2** shows methodological differences among the studies reviewed.

#### **Stimulus Material**

Differences across studies can, in part, be due to differences in the stimuli presented. For instance, fluency of processing is a source of positive affect, and can activate the Zygomaticus (Topolinski et al., 2009). Strong expressions, high visual contrast, familiarity and ingroup-status can increase fluency. Liking and attractiveness of faces also increase positive affect. The dynamic unfolding of expressions increases their salience and thereby guides attention. Furthermore, responses differ between, but also within expressions (e.g., polite vs. genuine smiles).

For these reasons, varied stimuli within the same study decrease statistical power (Westfall et al., 2014). To control for such variance, many studies used computer generated stimuli. Such stimuli can be introduced as avatars for concrete persons, allowing to investigate responses to known others in a highly controlled study. However, while responses to avatars and to photos are comparable (Spencer-Smith et al., 2001; Moser et al., 2007; Mühlberger et al., 2009), computer-generated stimuli, when falling in a specific "gap" in approaching realism ("Uncanny valley"), may engender negative reactions (de Borst and de Gelder, 2015).

In addition, human interaction entails different motivations and dynamics than passively looking at photos or avatars. For example, the presence of others enhances smiling expressions (Fridlund, 1991; Hess et al., 1995). But are reactions to stimuli on a screen and to interaction partners qualitatively the same, only more or less intense, or are they fundamentally different? Among the few articles using both kinds of situations, Heerey and Crossley (2013) found parallel effects of different types of smiles on the onset of mimicry reactions for FACS and EMG analyses of interaction and non-interaction situations, respectively. This is tentative evidence for qualitatively similar responses, but more direct evidence is needed. Thus, researchers should seek convergent evidence from well-controlled and from naturalistic settings.

#### **Measurement and Analysis**

Having electrodes in the face is not very natural; thus, studies concerned with ecological validity employ FACS coding instead of EMG. This difference in methods, however, may also directly influence results. While only visible changes can be picked up by FACS, EMG can also pick up muscle activation that is invisible to the eye. The confound between natural situation and assessment method poses a problem: if EMG, but not FACS, reveals anger mimicry, it might be that in interactions, people do not show anger mimicry, or it might be that anger mimicry usually remains below the visibility level.

Many other methodological choices are likely to influence results of facial mimicry studies. Here, we highlight three. Facial mimicry studies vary considerably in the time period of measurement: some look at the time course of the activity changes, typically during the first second after stimulus onset, while others present the mean difference from baseline for a whole stimulus presentation period (e.g., 6 s). An important next step will be to study the time course of facial mimicry modulations in more detail. A second issue concerns the treatment of EMG data. This review examined whether social situations influence the mimicry of different emotions to different degrees. However, several studies tested angry against happy expressions, or standardized the difference scores before analysis, making it impossible to gauge the net-effect per emotion. The third issue concerns replicability of effects. Different method choices in different labs and sometimes considerable numbers of individual tests per study suggest that replication in this field is important and challenging.


**TABLE 2 | Overview of studies referenced.**

(Continued)


**TABLE 2 | Continued**


etal.,2004);MMI,FacialExpressionDatabase(Panticetal.,2005);ATR,FacialExpressionDatabase(DB99,ATRPromotions,Kyoto,Japan).

 hSites for EMG electrodes; Action units for FACS coding: Zyg, Zygomaticus major; Corr, Corrugator; Orb, Orbicularis Oculi; Dep, Depressor anguli oris, Lev, Levator labii; Front, Frontalis; KS07, Coding according to Kring and Sloan (2007); KJ79, response scoring according toKraut and Johnston (1979).

## **How Do These Processes Develop?**

A facial response to a facial expression, like any other response, can be based on a reflective process, or it can be impulsive, based on learned associations (Strack and Deutsch, 2004). Anger and happiness expressions are assumed to be unconditioned stimuli, and should therefore evoke unconditioned affective reactions (Tomkins, 1962; Ohman and Dimberg, 1978). In addition, other expressions are assumed to become conditioned stimuli, such as a competitor's sadness signaling victory (Lanzetta and Englis, 1989). Finally, operant conditioning can reinforce any response as long as it has positive consequences, which can explain a smiling reaction to one's partner's anger (Häfner and Ijzerman, 2011).

Unconditioned responses can be distinguished from learned responses on the basis of developmental studies with children of different age groups, which is a promising avenue for further research. To determine whether a response is given impulsively and thus the result of conditioning processes, or reflectively, researchers can vary the exposure time to the emotional expression (Sonnby-Borgström, 2002; Sonnby-Borgström et al., 2003), the response period observed (early vs. late reactions, see Moody et al., 2007), or concurrent cognitive load (Cannon et al., 2009). Further, combining EMG with fMRI measurement can help discover the pathways of facial mimicry (Likowski et al., 2012). Given the rapid progress in fMRI technology, this seems a particularly promising avenue for future research (Heller et al., 2014). If one assumes that there is a fast route to "mirror" facial expressions, the question at what stage of information processing this route is modulated becomes inevitable. Moody et al. (2007) propose that such effects can originate at perception, with heightened sensitivity to emotion-relevant expressions, at information processing, with amplification, biased interpretation and evaluation of relevant input or at response preparation, with pre-activation of relevant facial responses.

#### **The Social Encounter**

As soon as individuals actually interact, they learn about each other (see Hofman et al., 2012; Aguado et al., 2013), which can, in turn, rectify pre-existing assumptions or change individuals' mood states. Factors such as the topic of the conversation, the facial mimicry of the sender, or the clarity, dynamic and type of facial signals will influence the emotional tone of the conversation

# **References**


and the cognitive and emotional states of the interaction partners. This, in turn, impacts the facial mimicry of the perceiver.

Furthermore, do interaction partners reciprocate the general amount of mimicry? Given that the brain's common currency is reward and punishment, it is likely that individuals distribute facial rewards and punishments just like other rewards and punishments in a tit-for-tat fashion (Axelrod and Hamilton, 1981), mimicking those who mimic them and stopping to mimic when the other is not mimicking them. Finally, relationship variables like power, attitudes, interdependence and fairness can be established, reinforced or mitigated through facial expressive and mimicry behavior. Studying facial mimicry in social encounters can help answer these questions. In addition, with refined software, it should be possible in the near future to test these propositions by manipulating the reciprocation of facial expressions via computer generated dynamic expressions shown by avatars or androids and triggered in real time by the participant's facial actions (see Bartlett and Whitehill, 2011; Littlewort et al., 2011; Hofree et al., 2014).

# **Summary**

Facial mimicry is embedded in the overall context. Congruent facial reactions are but one possible response to facial expressions. Another possibility is an incongruent response whose valence is opposite to that shown by the expresser. The selection of the reaction is determined by context-specific learning history. The collected evidence suggests that congruent facial expressions are by far not the reflex-like response they were once thought, and that facial reactions are not only quantitatively but also qualitatively modulated by contextual factors. To better understand facial mimicry, we have to study it in its social setting. Additionally, we have to design studies that shed light on functions and processes.

# **Acknowledgments**

The research by the authors as described in this article was supported by the German Research Foundation (DFG WE2930/2-1, DFG WE2930/2-2; DFG Research Group "Emotion and Behavior" FOR605). The publication was funded by the German Research Foundation (DFG FOR605). We thank Nina Matthiesen for assistance with preparing the tables.


*Psychol. Sci.* 16, 814–819. doi: 10.1111/j.1467-9280.2005. 01619.x


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Seibt, Mühlberger, Likowski and Weyers. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

**REVIEW ARTICLE** published: 01 December 2014 doi: 10.3389/fpsyg.2014.01351

# Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

# *Antje B. M. Gerdes1\*†, Matthias J. Wieser 2 † and Georg W. Alpers1,3*

<sup>1</sup> Clinical and Biological Psychology, Department of Psychology, School of Social Sciences, University of Mannheim, Mannheim, Germany

<sup>2</sup> Department of Psychology, University of Würzburg, Würzburg, Germany

<sup>3</sup> Otto-Selz Institute, University of Mannheim, Mannheim, Germany

#### *Edited by:*

Luiz Pessoa, University of Maryland, USA

#### *Reviewed by:*

Sarah Jessen, Max Planck Institute for Human Cognitive and Brain Sciences, Germany Mathias Weymar, University of Greifswald, Germany

#### *\*Correspondence:*

Antje B. M. Gerdes, Clinical and Biological Psychology, Department of Psychology, School of Social Sciences, University of Mannheim, L13,17 D-68131 Mannheim, Germany e-mail: gerdes@uni-mannheim.de

†Antje B. M. Gerdes and Matthias J. Wieser have contributed equally to this work.

In everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice's emotional tone will jointly create the emotional experience.This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broader contexts including human, animal, and environmental cues. In this article, we review current research on audiovisual emotion processing beyond face-voice stimuli to develop a broader perspective on multimodal interactions in emotion processing.We argue that current concepts of multimodality should be extended in considering an ecologically valid variety of stimuli in audiovisual emotion processing.Therefore, we provide an overview of studies in which emotional sounds and interactions with complex pictures of scenes were investigated. In addition to behavioral studies, we focus on neuroimaging, electroand peripher-physiological findings. Furthermore, we integrate these findings and identify similarities or differences. We conclude with suggestions for future research.

**Keywords: multimodal emotion processing, emotional pictures, emotional sounds, audiovisual interactions, emotional scene stimuli, auditory stimuli**

#### **INTRODUCTION**

In daily life, a wide variety of emotional cues from the environment reaches our senses. Typically, multiple sensory channels, for example vision and audition are integrated to provide a complete assessment of the emotional qualities of a situation or an object. For example, when someone is confronted with a dog, the evaluation of its potential dangerousness or friendliness will be more effective if visual (e.g., big vs. small dog; tail wagging or not) and auditory information (growling vs. friendly barking) can be integrated. While some of the information carried in either one of the channels may be redundant, the channels may also interact; i.e., a fierce bark may boost visual attention to the dog's bared teeth.

Despite the obvious relevance of multimodal perception in everyday life, emotion research has typically only investigated unimodal cues – with an apparent emphasis on visual stimuli. To cope with (a) limited processing capacities within a sensory modality and (b) the need to detect information which is relevant for survival, emotionally relevant cues have been suggested to modulate attention and selectively enhance perception (Vuilleumier, 2005; Pourtois et al., 2012).

Indeed, for the *visual domain* it has been shown that emotional cues – especially with threatening, but also with appetitive content – are preferentially processed in very early sensory areas (Schupp et al., 2003b; Pourtois et al., 2005; Gerdes et al., 2010). Emotional pictures influence perceptual processing and attract enhanced attention (Öhman and Wiens, 2003; Alpers and Pauli, 2006; Alpers and Gerdes, 2007; Gerdes et al., 2008, 2009; Stienen et al., 2011; Pourtois et al., 2012; Gerdes and Alpers, 2014).

Furthermore, distinct and intensive behavioral responses, physiological reactions, and brain activations are robustly evoked by emotional pictures (e.g., Lang et al., 1998; Neumann et al., 2005; Alpers et al., 2011; Eisenbarth et al., 2011; Plichta et al., 2012). According to Lang (1995) and Lang et al. (1998), the emotional response system is founded on an appetitive and defensive motivational system. Emotional states reflect these basic motivational systems and can be described in terms of affective valence and arousal. For example, a number of physiological measures are shown to covary with the valence or arousal of emotional cues: electromyographic (EMG) activity, heart rate responses and the startle reflex were shown to be sensitive to valence, whereas skin conductance and slow cortical responses are more sensitive to arousal (for more elaborative reviews on the processing of emotional pictures see, e.g., Bradley and Lang, 2000b; Brosch et al., 2010; Sabatinelli et al., 2011). Generally, enhanced processing gains of emotional cues may help individuals to quickly initiate adequate approach or avoidance behavior and therefore increases the chance of survival or well-being (Lang et al., 1997).

On the neural level the amygdala has long been identified as a key structure of emotional detection both in humans and animals (for reviews and meta-analysis see LeDoux, 2000; Phan et al., 2002; Costafreda et al., 2008; Armony, 2013). Relevant for the present context, via the thalamus the amygdala receives input not only from the visual modality but from all senses (Nishijo et al., 1988; Amaral et al., 1992; Amaral, 2003). The amygdala is instrumental in the relevance detection for biologically relevant

cues and has been documented to operate independently from the sensory modality which conveyed the information (Armony and LeDoux, 2000; Sander et al., 2003; Zald, 2003; Öhman, 2005; Stekelenburg and Vroomen, 2007; Scharpf et al., 2010; Armony, 2013). There is empirical evidence that the amygdala processes, e.g., emotional visual (Royet et al., 2000; Phan et al., 2002) and auditory cues (Fecteau et al., 2007; Klinge et al., 2010), as well as olfactory (Gottfried et al., 2002), and gustatory cues (O'Doherty et al., 2001).

Despite the evidence that visual and auditory emotion processing recruits similar brain structures, research on emotional auditory information and on multimodal cues is relatively scant. Until recently, this research field has mainly examined multimodal integration in social communication, i.e., face–voice stimuli (for a recent review see Klasen et al., 2012).

# **FACE–VOICE INTERACTION**

From studies using combined face-voice stimuli, we know that audiovisual integration canfacilitate and improve perception, even beyond the emotion effects within each separate channel. Emotion recognition is improved in response to multimodal compared to unimodal face-voice stimuli (Vroomen et al., 2001; Kreifelts et al., 2007; Paulmann and Pell, 2011). Furthermore, the identification of an emotional facial expression is facilitated when the face is accompanied by an emotional congruent voice and the evaluation of emotional faces is biased toward the valence of a simultaneously presented voice (de Gelder and Vroomen, 2000; de Gelder and Bertelson, 2003; Focker et al., 2011; Rigoulot and Pell, 2012). Such interactions appear to be independent of attentional allocation, i.e., even when participants are instructed to pay attention to only one sensory modality, emotional information of a concurrent but non-attended sensory channel influences the processing of the attended modality (Collignon et al., 2008). Likewise, if emotional faces and voices tap the same emotional valence (emotional congruency), they were processed faster than emotionally incongruent stimulus pairs or unimodal stimuli even when the attentional focus was explicitly directed to the faces or to the voices (Focker et al., 2011). Furthermore, this cross-modal influence was independent of a demanding additional task which had to be performed in parallel (Vroomen et al., 2001).

On the neuronal level, face–voice integration can occur at early perceptual stages of stimulus processing (for more specific information see the review of Klasen et al., 2012). Furthermore, specific brain areas such as superior and middle temporal structures and the fusiform gyrus, as well as parts of the emotion processing network including the thalamus, amygdala, and insula are consistently involved in emotionalface–voice integration (see alsoKlasen et al., 2012). Taken together, the integration of emotional faces and voices is an important part of social interaction and the prioritized processing and early integration of emotional face–voice pairs is an essential feature of social cognition (de Gelder and Vroomen, 2000).

#### **BEYOND EMOTIONAL "FACE–VOICE" PROCESSING**

In this review, we argue that a broader variety of stimuli should be considered in audiovisual emotion processing. While stimuli which are directly linked to human communication may represent an important subset of cues, a meaningful extension to existing concepts of multimodality should be carried forward by considering other domains as well. In this review we focus on visual and auditory cues across a wide range of semantic categories.

We start out with a short overview of studies mainly from the visual domain which focus on differences between emotional human communication and scene processing. This section demonstrates that the processing of communication vs. scene stimuli recruits different brain structures and elicit distinct electroand peripher-physiological responses. Thus, we argue that a distinction between those kinds of stimuli in multimodal emotion research is important and useful.

Because research on emotional sounds is much less frequent than research on emotional pictures we give a short overview of how emotional complex sounds can affect self-report ratings, physiological responses, and brain processes, and then summarize similarities and differences between emotional sound and picture processing.

Furthermore, we review studies which investigate how emotional information in one sensory modality can influence information processing of neutral cues in another sensory modality. Finally, we will summarize the existing studies that focus on interactions of the concurrent processing emotional visual and emotional auditory cues beyond faces and voices. We will conclude with a short summary and an outlook on research questions where the application of multimodal stimuli is particular interesting.

## **HUMAN COMMUNICATION VS. EMOTIONAL SCENE STIMULI**

Beyond face–voice integration, there are only few studies which focus on audiovisual interactions in emotion research. On the one hand, quite similar to face–voice interactions, some studies investigated multimodal integration in human communication with regard to bodily gestures and vocal expressions (Stekelenburg and Vroomen, 2007; Jessen and Kotz, 2011; Stienen et al., 2011; Jessen et al., 2012). On the other hand, there are several studies which examine influences of music on the processing of visual stimuli (e.g., Baumgartner et al., 2006; Logeswaran and Bhattacharya, 2009; Marin et al., 2012; Hanser and Mark, 2013; Arriaga et al., 2014). Because music is man-made, many theorists claim, that it is in essence another form of human communication. Therefore, music may be more similar to the communicative channels described above than to other naturally occurring sounds. In a similar vein, it has been argued that music has no obvious survival value (Juslin and Laukka, 2003).

Generally, it has been demonstrated that a fast and effective integration of stimuli across different modalities is necessary in several (survival-relevant) contexts. These contexts are certainly not limited to social situations. Examples for non-social situations of high biological significance may be a growling bear, swirling wasps, or an approaching thunderstorm. In these examples congruent visual and auditory information is transmitted (to the individual), whose prioritized processing may help the organism to survive. The first obvious advantage of multimodal information is that uncertainties in one sensory channel can be easily compensated and complemented by the other channel. In addition, even if the (emotional) information conveyed by ear and eye is obviously redundant, multisensory integration effects

can be clearly distinguished from redundancy effects within one modality. Again, the only empirical evidence supporting this claim stems from research on face–voice pairings. Redundant emotional information within the same modality leads to (post-) perceptual interferences shown by a lesser accuracy and longer response latencies within an emotional expression discrimination task. In contrast, congruent information of faces and voices was integrated early and pre-attentive with a clear perceptual benefit (Pourtois and Dhar, 2013). Thus, both senses supplement each other to create a distinct multisensory emotional percept (for a detailed discussion on stimulus redundancy; Pourtois and Dhar, 2013). Compared to information from face and voice, in many natural situations, not only concordant but also unrelated information during the same event can be conveyed by the different sensory channels (e.g., the sound of an emergency siren while watching children playing on a playground).

Empirical evidence for multifaceted differences between human communication and scene stimuli comes from the visual domain. On the peripher-physiological level, Alpers et al. (2011) showed that emotional scenes and faces were rated similarly, but the pattern of physiological responses measured by startle reflex, heart rate, and skin conductance was different. Startle responses to emotional scenes were modulated by valence with lowest amplitudesfor positive, intermediatefor neutral, and highest amplitudes for negative scenes, whereas the startle response was similarly enhanced in response to negative and positive faces. Furthermore, negative scene picture show a greater heart rate deceleration compared to neutral and positive scene; whereas negative and positive faces were followed by heart rate deceleration. These results indicate that scenes result in a valence based modulation, faces an arousal based modulation. In contrast, the skin conductance was arousal-modulated for the scene pictures with higher responses to the negative pictures and a valence-specific modulation for the faces with highest responses to positive faces. The facial EMG showed similar responses to both contents, but responses were slightly greater in response to scenes. Likewise comparing emotional faces and scenes, a recent study (Wangelin et al., 2012) showed that emotional scenes evoked stronger reactions in autonomic, central, and reflex measures in comparison to faces. In a meta-analysis (Sabatinelli et al.,2011), it was shown that emotional scenes elicited activation in occipital regions, the pulvinar, and the medial dorsal nucleus of the thalamus whereas the fusiform gyrus and the temporal gyrus were specifically activated in response to faces. Thus, measured emotion effects can strongly depend on the presented class of stimuli. Particularly, in emotion research one may argue that while face stimuli certainly convey emotional information, they do not necessarily elicit emotions in the observer (Ruys and Stape, 2008). Taken together, evidence from the visual domain clearly highlights the importance of a separate consideration of face and scene stimuli in emotion research. Analogous to that, for the auditory domain there is empirical evidence that human vocal and non-vocal sounds generally have different electrophysiological correlates and can elicit distinct responses in auditory regions (Meyer et al., 2005; Bruneau et al., 2013).

Thus, investigations with affective scene cues which have been consistently demonstrated to elicit emotional responses on behavioral and physiological levels (for emotional sounds see below) are needed to answer research questions about multi-modal emotion processing beyond face-voice stimuli.

#### **EMOTIONAL SOUND PROCESSING**

Compared to visual cues, sounds are still investigated only rarely. It is likely that this is due to the development of research traditions according to practical considerations rather than a reflection of the relative importance of auditory cues. Compared to pictures, sounds may be somewhat less amenable to experimental designs in the laboratory. However, sounds can clearly prompt strong emotional responses as has been shown in a large internetbased survey (Cox, 2008b). The development of the International Affective Picture System (IAPS; Lang et al., 2008) has been followed by a similar collection of sounds, the International Affective Digitized Sounds (IADS; Bradley and Lang, 2007) – a series of naturally occurring human, non-human, animal, and environmental sounds (e.g., bees buzzing; applause, explosions). Existing research on emotional sound processing (beyond voices) has been almost exclusively used this series as stimulus material (see below). In two experiments by Bradley and Lang (2000a), it was shown that valence and arousal ratings of these sounds were comparable to affective pictures from the IAPS. Furthermore, emotionally arousing sounds were also remembered better than neutral sounds in a free recall task. On a physiological level, emotionally arousing sounds elicit larger electrodermal activity which is generally known to be sensitive to the arousal of emotional stimuli (Bradley and Lang, 2000a). In comparison to pleasant sounds, the startle response to unpleasant sounds is enhanced and unpleasant sounds were accompanied by stronger corrugator activity and larger heart rate deceleration. This suggests that unpleasant sounds reliably activate the defensive motivational system (Bradley and Lang, 2000a). Another study showed that emotional sounds were accompanied by larger pupil dilatation which is an index of higher autonomic activity elicited by emotion (Partalaa and Surakka, 2003). Electrophysiological results suggest that aversive auditory cues (as, e.g., squeaking polystyrene) compared to neutral sounds were accompanied by a more pronounced early negativity and later positivity of event-related brain potentials as a measure of enhanced allocation of attention (Czigler et al., 2007) similar to what has been observed in emotional pictures (Schupp et al., 2003a). In contrast, unpleasant environmental sounds capture enhanced attention (shown by increased P3a amplitudes) but do not influence earlier components of perceptual processing (Thierry and Roberts, 2007). Similarly, two fMRI studies (Scharpf et al., 2010; Viinikainen et al., 2012) measured brain activation in response to emotional sounds from the IADS. Both studies showed that emotional sounds elicited strong activation in the amygdala compared to neutral sounds. Specifically, Viinikainen et al. (2012) showed that there was a quadratic U-shaped relationship between the sound valence and brain activation in the medial prefrontal cortex, auditory cortex, and amygdala with the weakest activation for neutral and increased activation for unpleasant and pleasant sounds.

Importantly, in an fMRI-study (Kumar et al., 2012) there was evidence that the amygdala encodes both the acousticfeatures of an auditory stimulus and the perceived unpleasantness. Specifically, acoustic features modulate effective connectivity from auditory cortex to the amygdala whereas valence modulates the effective connectivity from amygdala to the auditory cortex. Thus, control of acoustic features is of specific importance in research on emotional sounds.

A recent study from our research group investigated the processing of emotional sounds from the IADS within the auditory cortex (Plichta et al., 2011). Because fMRI scanner noise can interfere with auditory processing we used near-infrared spectroscopy (NIRS) which is a silent imaging method. In addition, the sound material was carefully controlled for several physical parameters such as loudness and spectral frequency. Unpleasant and pleasant sounds enhanced auditory cortex activation as compared to neutral sounds suggesting that the enhanced activation of sensory areas in response to complex emotional stimuli is apparently not restricted to the visual domain.

Further support for this observation comes from an MEG-Study investigating the influence of emotional content of complex sounds on auditory-cortex activity, both during anticipation and hearing of emotional and neutral sounds (Yokosawa et al., 2013). Indeed, during the hearing as well as during the anticipation period, unpleasant and pleasant sounds evoked stronger responses within the auditory cortex than neutral sounds.

In sum, there is now considerable evidence that complex highly arousing pleasant and unpleasant sounds are processed more intensively on a peripheral as well as on early sensory processing levels. Thus, using standardized emotional sounds (e.g., the IADS) can serve as a useful research tool to elicit emotions and investigate emotion processing.

# **EMOTIONAL SOUNDS AND PICTURE PROCESSING: SIMILARITIES AND DIFFERENCES**

Generally, emotional sound and picture processing is very comparable. The pattern of behavioral and physiological and electrophysiological reactions elicited by emotional sounds is comparable to emotional pictures (Bradley and Lang, 2000a; Schupp et al., 2003a; Czigler et al., 2007). However, there is some evidence that reactions to emotional sounds are weaker (Bradley and Lang, 2000a) and occur later (Thierry and Roberts, 2007). On the neuronal level, both emotional sounds and pictures gain privileged access to processing resources in the brain. Brain responses to visual, auditory, and olfactory stimuli were measured with PET showing for all three modalities, that all emotional stimuli activated the orbitofrontal cortex, the temporal pole, and the superior frontal gyrus (Royet et al., 2000). In addition, Scharpf et al. (2010) compared brain activation to sounds with responses to IAPS pictures. Independent of the sensory modality, the amygdala, the anterior insula, the STS, and the OFC showed increased activation during the processing of emotional as well as social stimuli. Also comparing brain activation to emotional pictures from the IAPS and to emotional sounds from the IADS, increased amygdala activity in response to both, emotional pictures and sounds were reported (Anders et al., 2008). Differentially, the left amygdala was sensitive to the valence of pictures and negative sounds whereas the right amygdala responded to the valence of positive pictures. A recent study directly aimed at investigating whether affective representations differ with sensory modality (Shinkareva et al., 2014).

Therefore, emotional picture and sound stimuli were presented in an event-related fMRI experiment. The results mainly provide evidence for a modality specific instead of a modality-general valence processing effect. Specifically, voxels were identified that were sensitive to the valence of pictures within the visual modality, as well as voxels that were sensitive to the valence of sounds within the auditory modality, but no voxels that were sensitive to valence across the two modalities.

To sum up, emotional pictures and sounds mainly elicit similar reactions on the level of self-report, behavioral, physiological, and neuronal – both types of stimuli strongly activate appetitive and defensive motivational circuits (Bradley and Lang, 2000a; Lang and Bradley, 2010). The reported processing differences (e.g., intensity of reaction, laterality effects, and timing) might be – at least partly – the result of methodological differences and different stimulus characteristics which are obvious between sounds and pictures (e.g., the dynamic nature of sounds). Thus, to account for such differences and to interpret potential processing differences, systematic and direct comparisons between emotional picture and sound processing with well controlled and (physically) comparable stimuli (e.g., conditioned stimuli) are urgently needed.

## **AUDIO–VISUAL INTERACTIONS**

#### **INTERACTION OF VISUAL AND AUDITORY PROCESSING**

Generally and beyond the emotional domain, it is well established that visual information can foster early stages of auditory processing and vice versa. For example, auditory speech perception can be strongly influenced by the viewing of visual speech stimuli on the perceptual (McGurk and MacDonald, 1976) as well as on the neuronal level (see, e.g., Kislyuk et al., 2008). Likewise, visual processing can be strongly altered by concurrent sounds even at the earliest stage of cortical processing (Bulkin and Groh, 2006; Shams and Kim, 2010). Based on these findings it seems plausible that such interaction may also occur when emotional information is conveyed by (at least) one of the sensory channels. Indeed, a small but growing number of studies suggest that (emotion) processing in the auditory system can be influenced by (non-related) emotional information coming from the visual modality and vice versa (see below).

#### **INTERACTION OF EMOTIONAL VISUAL AND NON-EMOTIONAL AUDITORY PROCESSING**

On a behavioral level, a recent study investigated the influence of emotional IAPS pictures on the classification of high and low pitch tones but did not find an effect of picture valence on the auditory classification (Ferrari et al., 2013).

On the physiological level, it is well-known that emotional visual stimuli can modulate the acoustic startle reflex elicited by loud, abrupt, and unexpected sounds: negative pictures enhance, positive pictures dampen the blink magnitude in response to the unexpected sound (e.g., Lang et al., 1990). Moreover, the electrocortical response to the acoustic startle probe (P3 component) was also found to be modulated by the arousal of the emotional pictures in the foreground with smaller amplitudes for high arousing pictures (Keil et al., 2007).

Regarding electrophysiological responses, the presentation of unpleasant pictures has a significant impact on event-related potentials of the EEG to strongly deviant tones. During the presentation of unpleasant pictures, high deviant tones elicited larger N1 and P2 responses than during the presentation of pleasant pictures which was interpreted as a sensitization to potentially significant deviant events (N1) and enhanced attention (P2) to regular external events (Sugimoto et al., 2007). Similarly, auditory novelty processing was enhanced by negative IAPS pictures. Participants had to judge (emotional) picture pairs as equal or different while ignoring task irrelevant sounds. During the presentation of negative IAPS-pictures, novel sounds compared to the standard tone provoked enhanced distraction effects shown on the behavioral level as well by the modulation of event-related potentials (enhanced early and late novelty P3; Dominguez-Borras et al., 2008a,b). In a similar vein, pleasant pictures were shown to modulate auditory information processing such that they significantly attenuated the Mis-Match-Negativity (MMN) in response to a change within an auditory stimulus stream. Thus, pleasant pictures can be seen as a kind of safety signals and probably reduce the need for auditory change detection (Surakka et al., 1998).

Further support for crossmodal influences of emotion is provided by a MEG study showing that unpleasant pictures diminish auditory sensory gating in response to repeated neutral tones as an index of neuronal habituation (Yamashita et al., 2005). Presenting neutral tones subsequent to emotional pictures, another study showed that neutral tones prompted larger ERP amplitudes (N1 and N2) when emotional relative to neutral pictures were presented before, indicating enhanced enhance attention and orienting toward neutral tones encoded in the context of emotional scenes (Tartar et al., 2012). All studies reported above indicate that emotional visual information can enhance auditory processing. However, the question arises whether increasing demands of an auditory task might interfere with the processing of emotion in the visual domain. This may be due to competition for limited processing resources, a process which has been documented for competing emotional pictures within the same modality (Schupp et al., 2007). However, the processing of emotional IAPS pictures was not modulated by an additional auditory detecting task with increasing complexity (Schupp et al., 2008). Thus, emotion processing in the visual domain was not affected by task demands in the auditory modality. This finding is in line with the multiple resource theory which assumes that each sensory modality has separate pools of (attentional) resources (Wickens, 2002).

Taken together, emotional cues in the visual domain are able to enhance concurrent as well as subsequent auditory processing even at very early processing stages with no or low costs for emotion processing.

# **INTERACTION OF EMOTIONAL AUDITORY AND NON-EMOTIONAL VISUAL PROCESSING**

To the best of our knowledge, evidence that emotion cues from the auditory modality can also influence non-emotional (basic) visual information processing is nearly missing. One study investigated the influence of emotional sounds on visual attention in a spatial cueing paradigm (Harrison and Davies, 2013). Here, non-speech environmental sounds from the IADS were presented spatially matched to the locations of subsequent visual targets. Indeed, results show for right-sided targets that neutral and

positive sounds elicited faster responses to valid trials (where the sound and the visual target were presented on the same side) compared to invalid trials. In contrast, after negative sounds, the reaction time to valid trials was slower suggesting faster attentional disengagement from negative sounds.

Another study used spoken emotional and neutral words that were followed by a visually presented neutral target word (Zeelenberg and Bocanegra, 2010). It was found that identification of a masked visual word was improved by preceding spoken emotional words as compared to neutral ones. These findings can be interpreted as first evidence that affective sounds may influence at least subsequent visual (word) processing. However, much more studies are needed in which the effects of emotional sounds on concurrent and subsequent visual processing are investigated.

# **INTERACTION OF EMOTIONAL AUDITORY AND EMOTIONAL VISUAL PROCESSING**

We assume that audio–visual interactions are possible in both directions. As reviewed above, emotional visual as well as auditory information are preferentially and intensively processed. Thus, one can expect that audio–visual interactions of emotional stimuli occur even stronger if emotional information is conveyed by *both* modalities. Regarding that question, self-report data of an internet-based survey suggest that sound stimuli were significantly perceived as more horrible when they were accompanied by pictures that show associated information (e.g., the sound of a crying baby combined with a picture of a crying baby) compared to pictures with unassociated pictures (Cox, 2008a). Using affective IADS sounds; Scherer and Larsen (2011) found significant cross-modal priming effects for negative sound primes on emotional visual word targets. Experimentally, self-report (valence and arousal) and physiological variables were measured in response to unimodal and bimodal presented emotional sounds and pictures in a within subjects design. Unpleasant and pleasant stimuli had similar effects on self-report, heart rate, heart rate variability, and skin conductance with no effect of stimulus modality. Contrary to expectations, bimodal presentation with congruent visual and auditory stimuli did not enhance the effects (Brouwer et al., 2013).

In a similar vein, we recently conducted an EEG-study in which unpleasant, pleasant, and neutral IAPS pictures were preceded by unpleasant, pleasant, and neutral IADS sounds. Ratings and electrophysiological data suggest that (emotional) sounds clearly influence emotional picture processing (Gerdes et al., 2013). We could demonstrate that audiovisual pairs with pleasant sounds and pictures were rated as more pleasant than pleasant pictures only. In addition, valence congruent audiovisual combinations were rated as more emotionally as other incongruent combinations. Electrophysiological measures showed that ERP amplitudes (P and P2) were enhanced in response to all pictures which were accompanied by emotional sounds compared to pictures with neutral sounds. These findings can be interpreted as evidence that emotional sounds may unspecifically increase sensory sensitivity or selective attention (P1, P2) to all incoming visual stimuli. Most importantly, unpleasant pictures with pleasant sounds prompted larger ERP amplitudes (P1 and P2) compared to unpleasant pictures with unpleasant sounds. The reduced amplitudes in response to congruent sound-picture pairs suggest that the processing of

unpleasant pictures is facilitated (i.e., less processing resources are needed) when they were preceded by congruent unpleasant sounds.

Taken together, the above mentioned studies strongly suggest that emotion processing in one sensory modality can strongly affect emotion processing of another modality during very early stages of neuronal processing, as well as on the self-report level.

For a short overview of the here reviewed studies see **Table 1**.

#### **CONCLUSION AND FUTURE DIRECTIONS**

As the review summarized at the beginning, the majority of research on unimodal auditory emotion processing provides clear evidence that complex emotional auditory stimuli (mainly investigated with IADS) elicit similarly intensive emotional reactions on behavioral, physiological, and on neuronal levels as traditionally used complex visual emotional scenes. Furthermore, emotional cues from both modalities guide selective attention and receive enhanced processing. This preferential processing can even alter (emotional) information processing in other sensory channels. Specifically, there is evidence that the processing of complex emotional information in one sensory modality can strongly affect (emotion) processing of another modality during very early stages of neural processing, as well as self-reported emotions, and that these effects are bidirectional. As hitherto existing research mainly focused on stimuli of human communication such as faces and voices, the here reviewed work expands the concept of multimodality to a broader variety of human, animal, and environmental cues. However, as this review also indicates research in this field is in its infancy. Consequently, to carve out similarities and differences between the different classes of stimuli, disentangling emotional from social relevance (see, e.g., Bublatzky et al., 2014b), and the impact on audiovisual combinations are promising areas of future research.

From a methodological viewpoint, complex scene stimuli additionally offer the opportunity to separate effects of semantic (or contextual) and emotional (in)congruence in multimodality which is usually confounded in face–voice pairings. Similarly important is the dissociation of (task) difficulty from incongruence in multimodal emotion integration (Watson et al.,2013). Furthermore, effects of presentation order and timing of the stimuli should be investigated systematically, because such methodological differences seem partly responsible for inconsistent findings (see Jessen and Kotz, 2013). Also important, research on multimodality should strongly reveal effects simply caused by stimulus redundancy or by intensity amplification in contrast to inherent multimodality (Pourtois and Dhar, 2013). In order to investigate the (interactive) effects of sounds, more diverse stimulus sets would be highly welcome. Specifically, research on influences of emotional sounds on (subsequent) visual processing is pending. To account for physical differences of emotional picture and sounds, investigations with well controlled and (physically) comparable stimuli (e.g., with instructed fear or conditioning procedures) are urgently needed (see, e.g., Bröckelmann et al., 2011; Bublatzky et al., 2014a).

Regarding the impact of attention and automaticity, there has been a controversy for unimodal emotional cues whether they are processed outside of explicit attention or whether attentional disengagement can reduce neural responses and behavioral output (Pessoa,2005). For example some studies claimed that visual threat cues activate the amygdala independently from attentional allocation (Straube et al., 2006) or that attentional distraction actually resulted in reduced activation (Pessoa et al., 2002; Alpers et al., 2009). We are not aware of systematic investigations of multimodal emotional stimulation and variations in attention to one or multiple channels. Interestingly, multimodal presentations may provide a particularly fruitful avenue for this debate because it is possible to attend to one channel and ignore the other.

With regards to other electrophysiological indices of preferential processing and attention, the N2Pc or steady-state visual evoked potentials (ssVEPS) which are established in the visual domain (Wieser et al., 2012b; Weymar et al., 2013), would lend themselves for the examination of sound as well as for audio–visual cues and interactions. For example, it would be interesting to see how the N2pc as an index of visual–spatial attention to a salient stimulus in visual search paradigms is modulated by concurrent (emotional) sounds. Also, the influence of emotional sounds on sustained attentional processes (as measured by ssVEPS) would be an interesting research question. The use of different paradigms would help inform us about the different stages at which presumably emotional cues from different modalities interact. Recently, novel paradigms have been introduced to examine the behavioral output of preferentially processed emotional cues (see, e.g., Pittig et al., 2014). If integrated multimodal cues result in a more intensive emotional experience (and neural processing), this may also result in more pronounced behavioral consequences.

To make research more ecological valid and to evolve a broader and more complete concept of emotional multimodality, future research should not only concentrate on audiovisual emotion processing but should also incorporate cues from other sensory channels as, e.g., olfactory (Pause, 2012; Adolph et al., 2013), somatosensory (Francis et al., 1999; Gerdes et al., 2012; Wieser et al., 2012a, 2014), or gustatory signals (O'Doherty et al., 2001; Tonoike et al., 2013) which are also known to elicit emotional reactions and may interact with information processes of other modalities.

Another issue that is of great theoretical and practical importance is the consideration of different populations in the context of multimodal emotion processing. From a clinical perspective, the consideration of multimodal emotional processing is promising for the understanding of several mental disorders (e.g., anxiety disorders). Here, contextual (multimodal) information contributes to the acquisition and maintenance of the disorder (Craske et al., 2006). Accordingly, it has been argued elsewhere that for example in research on social anxiety disorder a crossmodal perspective may help to gain a more complete and ecological picture of cognitive biases and understand fundamental processes underlying biases in social anxiety (Peschard et al., 2014). Altogether, explicit knowledge on multimodal integration and interaction processes can improve the understanding of emotion processing (deficits) and consequently may help to optimize therapeutic approaches (see Taffou et al., 2012; Maurage and Campanella, 2013).

For the most part, the here reviewed interactions between emotional stimuli in the two senses can be explained on the background of the motivational priming theory (Lang, 1995).




**46**

 =

 =

 =

 =

 = According to that theory, emotion is considered to be organized around two motivational systems, one appetitive, and one defensive. These systems have evolved to mediate behavior that either promote or threaten physical survival (Lang et al., 1997).

Independent of the sensory modality, emotional information is thought to activate the appetitive or defensive motivational system. Consequently, the engaged motivational system modulates other (brain) processing operations which means that (perceptual) processing of other emotional information can be facilitated or inhibited. These modulatory effects are shown crossmodally, thus, there also seem to be independent of the stimulus modality (see, e.g., Bradley et al., 1990; Lang et al., 1998).

Taken together, the motivational priming theory is able to explain audiovisual *interactions* of emotional information. However, the motivational priming theory does not make any assumptions of how multisensory emotional inputs are combined and integrated. Actually, no specific model exists which accounts for the integration of multisensory emotional information. Generally, one can assume that multisensory integration of emotional information follows similar principles as multisensory integration of other types of complex information (see, e.g., Stein and Meredith, 1993; de Gelder and Bertelson, 2003; Ernst and Bülthoff, 2004; Spence, 2007). Within the scope of the motivational priming theory, motivated attention might influence the efficiency of this integration processes. However, the development and the systematic testing of a specific theoretical framework for multimodal emotion processing is definitely one of the next important future challenges.

# **ACKNOWLEDGMENTS**

This work was supported by the German Research Society (DFG; GE 1913/3-1-FOR 605). In addition, we thank Michael M. Plichta for his valuable feedback and discussions.

#### **REFERENCES**


in fear," in *The New Cognitive Neurosciences*, 2nd edn, ed. M. S. Gazzaniga (Cambridge, MA: The MIT Press), 1067–1079.


by magnetic P50 suppression. *Eur. Arch. Psychiatry Clin. Neurosci.* 255, 99–103. doi: 10.1007/s00406-004-0538-6


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 August 2014; accepted: 06 November 2014; published online: 01 December 2014.*

*Citation: Gerdes ABM, Wieser MJ and Alpers GW (2014) Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains. Front. Psychol. 5:1351. doi: 10.3389/fpsyg.2014.01351*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Gerdes, Wieser and Alpers. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Emotion regulation in heavy smokers: experiential, expressive and physiological consequences of cognitive reappraisal

*Lingdan Wu1,2\*, Markus H. Winkler1, Matthias J. Wieser1, Marta Andreatta1, Yonghui Li3 and Paul Pauli1\**

*<sup>1</sup> Department of Psychology (Biological Psychology, Clinical Psychology and Psychotherapy), University of Würzburg, Würzburg, Germany, <sup>2</sup> Department of Psychology, University of Konstanz, Konstanz, Germany, <sup>3</sup> Institute of Psychology, Chinese Academy of Sciences, Beijing, China*

#### *Edited by:*

*Marina A. Pavlova, Eberhard Karls University of Tübingen, Germany*

#### *Reviewed by:*

*Eric S. Allard, Cleveland State University, USA Kyung Mook Choi, University of Missouri-Kansas City, USA*

#### *\*Correspondence:*

*Lingdan Wu lingdan43@gmail.com; Paul Pauli pauli@psychologie.uni-wuerzburg.de*

#### *Specialty section:*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology*

*Received: 09 June 2015 Accepted: 25 September 2015 Published: 13 October 2015*

#### *Citation:*

*Wu L, Winkler MH, Wieser MJ, Andreatta M, Li Y and Pauli P (2015) Emotion regulation in heavy smokers: experiential, expressive and physiological consequences of cognitive reappraisal. Front. Psychol. 6:1555. doi: 10.3389/fpsyg.2015.01555* Emotion regulation dysfunctions are assumed to contribute to the development of tobacco addiction and relapses among smokers attempting to quit. To further examine this hypothesis, the present study compared heavy smokers with non-smokers (NS) in a reappraisal task. Specifically, we investigated whether non-deprived smokers (NDS) and deprived smokers (DS) differ from non-smokers in cognitive emotion regulation and whether there is an association between the outcome of emotion regulation and the cigarette craving. Sixty-five participants (23 non-smokers, 22 NDS, and 20 DS) were instructed to down-regulate emotions by reappraising negative or positive pictorial scenarios. Self-ratings of valence, arousal, and cigarette craving as well as facial electromyography and electroencephalograph activities were measured. Ratings, facial electromyography, and electroencephalograph data indicated that both NDS and DS performed comparably to nonsmokers in regulating emotional responses via reappraisal, irrespective of the valence of pictorial stimuli. Interestingly, changes in cigarette craving were positively associated with regulation of emotional arousal irrespective of emotional valence. These results suggest that heavy smokers are capable to regulate emotion via deliberate reappraisal and smokers' cigarette craving is associated with emotional arousal rather than emotional valence. This study provides preliminary support for the therapeutic use of reappraisal to replace maladaptive emotion-regulation strategies in nicotine addicts.

Keywords: nicotine addiction, smoking, emotion regulation, craving, reappraisal, facial electromyography, late positive potential

# INTRODUCTION

Nicotine addiction is the most prevalent type of drug addiction, and one of the leading causes of preventable diseases (Centers for Disease Control and Prevention, 2011a; World Health Organization, 2011, 2013). Globally, smoking is estimated to kill approximately 6 million people per year with an additional 600000 assumed to be dying from the effects of second-hand smoke (Mathers and Loncar, 2006; Oberg et al., 2011). This sum surpasses even the estimated amount of people killed by HIV/Aids, tuberculosis and malaria combined (World Health Organization, 2012). Smokers are aware of the deadly results of smoking and most of them have tried several times to quit smoking (Al-Yousaf and Karim, 2001; Winickoff et al., 2009; Centers for Disease Control and Prevention, 2011b). However, the majority of them relapse. The relapse rates were reported as high as 75–95% after successful intervention for smoking cessation within 6– 12 months (Garvey et al., 1992; Ferguson et al., 2005; Nakajima and Al'absi, 2012).

The social psychological/self-regulation failure view describes nicotine addiction as a cycle of spiraling dysregulation of the mesocorticolimbic dopamine (DA) system (Baumeister and Heatherton, 1996) that plays an important role in reward and motivation (Fibiger and Phillips, 1986). Initial regulation failure sets up impulsive smoking and adds additional negative emotions, until the large-scale breakdown in selfregulation, which results in compulsive smoking (Baumeister and Heatherton, 1996; Bechara, 2005). Supportively, neuroimaging studies implicated that nicotine addicts show abnormal brain functions in prefrontal cortex (PFC; e.g., dorsal medial PFC and both dorsal and ventral lateral PFC) and basal ganglia circuits (Bechara et al., 2001; Lubman et al., 2004; Galvan et al., 2011; Goldstein and Volkow, 2011; Sutherland et al., 2012). These brain regions were also consistently reported to be involved in cognitive emotion regulation (Ochsner et al., 2004; McRae et al., 2010; Mocaiber et al., 2011; Moratti et al., 2011). Overall, this line of evidence may point to emotion regulation deficits in nicotine addicts.

In the field of emotion regulation, cognitive reappraisal has received particular attention. Reappraisal refers to changing one's interpretation of a situation so as to alter emotion (Gross, 2002). Previous studies have shown that reappraisal is an efficient way to modify emotional responses, including emotional experience, expression, and psychophysiology (Gross, 1998, 2002; Ochsner and Gross, 2005; Gross and Thompson, 2007). Furthermore, compared to other regulation strategies (e.g., suppression, avoidance, drug use) cognitive reappraisal appears to be more effective and more beneficial to long-term physical health (Gross, 1998, 2002; John and Gross, 2004; Ehring et al., 2010).

Previous studies have investigated the relation between nicotine addiction and the use of emotion regulation strategies. The consistent findings are that early smoking initiation, enhanced smoking urges, and failures in smoking abstinence are associated with a more frequent use of maladaptive strategies (e.g., suppression); on the contrary, reduced craving to smoke, greater positive mood, and fewer depressive symptoms are associated with a more frequent use of reappraisal strategies (Fucito et al., 2010; Szasz et al., 2012). Mostly these studies relied on self-reports to investigate the use of emotion regulation strategies and emotional responses. Although self-reports are a valuable source of information about affective experience, emotional reactions are expressed on multiple levels (Lang, 1995). Smokers' emotional responses such as facial expressions and neuronal correlates as a result of emotion regulation have not been assessed yet.

Therefore, the present study combine multiple measures (e.g., self-reports, psychophysiological measures of facial expressions and neural reactions) to investigate emotion regulation via reappraisal in smokers. Based on previous studies (Bechara et al., 2001; Goldstein and Volkow, 2011; Sutherland et al., 2012), we hypothesized that compared to non-smokers (nonsmokers) smokers would show deficits in cognitive emotion regulation via reappraisal. In addition, we assessed the effects of smoking abstinence on cognitive emotion regulation, which has not been studied in previous studies. Some studies have shown that deprived smokers (DS) experience more negative emotions and higher cravings to smoke than non-deprived smokers (NDS), which may contribute to the high-rate of relapse (Cinciripini et al., 2006; Piper and Curtin, 2006). Therefore, we hypothesized that it would be more difficult for DS to regulate emotion as compared to NDS.

Further, most prior work focused on the regulation of negative emotions (Baker et al., 2004; Fucito et al., 2010; Szasz et al., 2012). Little has been known about regulation of positive emotions (with a few exceptions, e.g., Krompinger et al., 2008; Wu et al., 2012). It has been acknowledged that the overall balance of positive and negative emotions predicts subjective wellbeing (Fredrickson, 2001; Fredrickson et al., 2008). In addition, maladaptive positive emotions (e.g., larger appetitive reactions to smoking cues as compared to non-smokers) have been associated with nicotine addiction (Geier et al., 2000; Winkler et al., 2011). Therefore, the present study aimed to expand previous work by comparing smokers and non-smokers on general emotion regulation competency in the context of both positive and negative stimuli.

Lastly, considering that emotional responses have been widely described on two main dimensions, valence and arousal, it is important to examine how the impact of reappraisal on emotional valence and arousal, is related to cigarette craving in nicotine addicts. Previous studies have indicated that cigarette craving triggers cigarette smoking (Kober et al., 2010a) and cognitive emotion regulation involves neural dynamics parallel to craving regulation (i.e., prefrontal-striatal pathway; Kober et al., 2010b; Tabibnia et al., 2014). In line with this, previous studies have shown that more negative emotions are associated with more cigarette craving (Juliano and Brandon, 2002; Baker et al., 2004; Shiffman and Waters, 2004; Conklin and Perkins, 2005; Bradley et al., 2007; Battista et al., 2008; Nakajima and Al'absi, 2012) and individuals with mood disorders, such as depression and anxiety, are more likely to smoke than normal people (McCabe et al., 2004; Gonzalez et al., 2008; Fucito and Juliano, 2009; Morrell et al., 2010). Accordingly, one may hypothesize that regulating negative emotions might be associated with changes in cigarette craving. However, it is not clear yet whether altering emotional valence and arousal impacts cigarette craving similarly or differently.

To address the above issues, we compared deprived and NDS with non-smokers in general emotion regulation competency. We adopted the reappraisal paradigm in which prior to each emotional stimulus, participants are instructed to regulate emotional responses by reinterpreting the emotional stimulus, e.g., changing the perspective in order to feel less emotion (Hajcak and Nieuwenhuis, 2006; Gross and Thompson, 2007; Ochsner and Gross, 2008; Urry, 2009; Moser et al., 2010; Ray et al., 2010). We used pictorial stimuli from the international affective picture system (IAPS) that has been widely applied in previous studies to assess general emotion regulation competency (Ochsner et al., 2004; Hajcak et al., 2009; Moser et al., 2010). Since emotions are dispositions to action that involve multi-level responses (Lang, 1995), we collected self-ratings (Chae et al., 2008; Robinson et al., 2014), psychophysiological, i.e., facial electromyography (EMG; Geier et al., 2000; Winkler et al., 2011, and brain responses (late positive potential, LPP; Littel and Franken, 2011; Versace et al., 2011) to evaluate emotional changes as a function of cognitive reappraisal. Facial electromyographic (EMG) reactions of the corrugator supercilii and zygomaticus major muscle has been suggested as sensitive index of negative and positive emotions (Dimberg, 1990; Dimberg and Thunberg, 1998; Dimberg et al., 1998; Weyers et al., 2006; Mauss and Robinson, 2009; Wu et al., 2012). The LPP activity is a sensitive index of neural activity to emotional arousing stimuli (Hajcak and Nieuwenhuis, 2006; Hajcak et al., 2009; MacNamara et al., 2009). Therefore, we used EMG activity and LPP activity as well as self-ratings as outcome measures for successful or unsuccessful regulation.

# MATERIALS AND METHODS

# Participants

Twenty-five non-smokers (12 females) and 50 heavy smokers (25 females), aged between 18 and 53 years, were recruited through Internet advertisements and posters. Participants were pre-screened via phone or email and performed a breath-test in an initial assessment with a portable Smokerlyzer-R carbon monoxide (CO) monitor. Persons who smoked an average of at least 10 cigarettes per day during the last 12 months and CO > = 10 ppm were considered as heavy smokers, while persons who had smoked fewer than two cigarettes in their lifetime and CO < = 5 ppm were recruited as non-smokers. Participants who met the criteria for heavy smokers were randomly assigned to one of two groups: the non-deprived smoking (NDS) group (individuals were asked to smoke as normal and to consume one cigarette immediately before they come to the laboratory), and the deprived smoking group (DS; individuals were required to abstain from smoking overnight for about 12 h prior to their lab appointment; Mucha et al., 1999; Stippekohl et al., 2010). All participants had a high school diploma or equivalent, were not taking any prescription drugs, and were fluent German speakers. Extra exclusion criteria included: (1) having a personal history of drug addiction excluding nicotine dependence; (2) having current psychiatric or neurological disorders; (3) currently taking any smoking cessation medications and/or participating in smoking cessation programs. Most participants were students from the University of Würzburg receiving either money (6 euro/h) or course credit. DS were compensated with extra 10 euro for their efforts to abstain from smoking. The study, including all procedures and the consent form, was approved by the ethical committee of the Universities of Wuerzburg and was carried out in accordance with the ethical standards of the fifth revision of the Declaration of Helsinki.

# Materials

In total 125 pictures (25 neutral scenes, 50 positive scenes, and 50 negative scenes) from the IAPS, (Bradley and Lang, 1994; Lang et al., 2005) were used1 . The three picture categories differed significantly from each other with regard to IAPS normative valence ratings (negative pictures: *M* = 2.82, *SD* = 1.64; neutral pictures: *M* = 5.05, *SD* = 1.21; positive pictures: *M* = 7.28, *SD* = 0.48); positive pictures did not differ from negative pictures on arousal ratings (negative pictures: *M* = 5.71, *SD* = 2.16; neutral pictures: *M* = 2.91, *SD* = 1.93; positive pictures: *M* = 5.71, *SD* = 2.28). The mean difference for valence ratings (or arousal ratings) between positive and neutral pictures was the same as the mean difference valence ratings (or arousal ratings) between negative and neutral pictures. Each picture was displayed at a resolution of 600 pixels × 800 pixels on a computer screen at a viewing distance of 60 cm using Presentation software (Neurobehavioral Systems, Albany, CA, USA).

Auditory instructions ('maintain' and 'decrease') were recorded in advance. The auditory instructions were presented binaurally via speakers with a sound intensity of 68 dB. All of the neutral pictures were preceded by the 'maintain' instruction (i.e., to simply attend to the pictures, allowing themselves to experience whatever feelings happened during picture-viewing) forming a baseline condition. Half of the emotional pictures (i.e., positive and negative pictures) were preceded by the 'decrease' instruction (i.e., to reappraise the emotional pictures in order to feel neutral by imagining that the depicted negative or positive scenario would become more positive or more negative, respectively, over time). The other half was preceded by the 'maintain' instruction.

Self-Assessment Manikins (SAM; Bradley and Lang, 1994) were used to measure emotional experiences as indexed by self-reported valence and arousal. The SAM is a non-verbal instrument. It consists of five graphic figures representing ninelevel ratings for both valence (1 = highly positive, 5 = neutral, 9 = highly negative) and arousal (1 = low arousal, 9 = high arousal). To measure cigarette craving during the experiment, a similar instrument with five bar graphs instead of five graphic figures, developed by Stippekohl et al. (2010), was used to represent nine-level ratings for craving to smoke (1 = low craving, 9 = high craving).

A portable Smokerlyzer-R CO monitor (Bedfont Scientific Ltd, Kent, UK) was used to verify the smoking status of the participants. Questionnaires were used to measure the degree of smoking dependence, depressive and anxiety levels via the Fagerström Test for Nicotine Dependence (FTND; Heatherton et al., 1991), the German version of the State Trait Anxiety

<sup>1</sup>Neutral pictures: 2102 2393 2575 2580 2593 5530 5740 7002 7004 7010 7056 7090 7130 7140 7150 7175 7211 7217 7491 7500 7550 7595 7700 7705 7950;

Negative pictures: 1050 1201 1302 1930 2120 2130 2141 2205 2399 2661 2683 2688 2691 2700 2710 2716 2750 2810 3168 3220 3301 6020 6190 6212 6250 6312 6313 6570 6571 6830 6831 8230 9042 9050 9250 9400 9421 9425 9470 9490 9520 9584 9600 9611 9635 9800 9901 9911 9920 9921;

Positive pictures: 1463 1710 1811 2080 2150 2160 2340 2345 2352 2550 2655 4572 4608 4623 4660 5270 5300 5450 5460 5480 5600 5623 5626 5628 5629 5660 5700 5910 7501 7502 8030 8034 8040 8080 8090 8116 8117 8161 8170 8180 8190 8200 8210 8300 8370 8400 8490 8496 8502 8500.

Inventory (STAI; Laux et al., 1981), and the German version of the Beck Depression Inventory (BDI; Hautzinger et al., 1995).

# Procedure and Apparatus

All experimental sessions were conducted in the afternoon between 12:30 and 7:00 pm. After reading the instructions for the experiment and signing the informed consent, participants completed a CO test and filled in the questionnaires.

Participants were then seated in a comfortable chair in a sound attenuated and dimly lit room. The electroencephalograph (EEG) electrodes and facial EMG electrodes were attached to the scalp and face, respectively. Three initial practice trials were given to explain the procedure. An example of reappraisal was given prior to practice trials, showing participants how to reappraise an emotional picture in order to feel neutral. Participants were asked to speak out aloud how they reappraise pleasant and unpleasant pictures during the initial practice trials. Feedbacks were given till participants completely understood the reappraisal strategy.

Next, the experimental session started, consisting of 125 trials with 25 trials for each of the five experimental conditions (i.e., maintain-neutral, maintain-positive, maintain-negative, decrease-positive, and decrease-negative). The trials were pseudorandomized so that no more than three trials from the same condition were presented successively. Each trial began with a white fixation cross presented on a black screen for a period ranging randomly from 4 to 5 s. The fixation cross turned blue, 1 s before the onset of the auditory instructions (i.e., 'maintain' or 'decrease') that lasted for about 1 s. Following the instruction, there was a 1 s delay and then the corresponding picture was presented for 6 s. At the offset of each picture, the rating scales appeared on the screen and participants rated how they felt during picture presentation. There were breaks after every 25 trials. The whole experimental session lasted about 40 min.

# Psychophysiological Data Recording

Continuous EMG and EEG were recorded at 1000 Hz by using a V-Amp 16 amplifier (Brain Products Inc., Gilching, Germany). Facial EMG activity was measured over the corrugator and zygomaticus muscle regions according to guidelines provided by Fridlund and Cacioppo (1986). The EEG was recorded using an EasyCap (EasyCap, Hersching, Germany) from 10 positions including FCz, Cz, CPz, Pz, C1, C2, CP1, CP2, and the left and right mastoids. Vertical EOG was recorded from electrodes placed 1 cm above and below the right eye, and horizontal EOG was recorded with two electrodes 1 cm from the outer epicanthus of each eye. FCz was used as ground. Reference was placed at Cz during data recording and replaced by the mean of mastoids during off-line data analysis. Impedance was kept below 10 k at all sites.

# Data Reduction

Off-line analyses of the EMG and EEG activity were conducted with Brain Vision Analyzer Software (Version 2.0, Brain Products Inc., Gilching, Germany). On average 4.51% of the trials were rejected due to Íartifacts, which left an average of 23.87 trials per subject and per condition. Ten participants were excluded from data reduction and further analysis because of technical errors that resulted in a lack of markers in the raw EEG data. As a result, a total of 23 non-smokers (11 males), 22 NDS (10 males) and 20 DS (10 males) were included in data analyses.

Electromyography data were re-referenced to obtain bipolar recordings. The raw signal was filtered with a band-pass filter from 30 to 500 Hz and a 50 Hz notch filter. Subsequently, the data were rectified, smoothed using a 125 ms moving average filter, segmented into trials, and baseline corrected for each trial. Trials with EMG activity above 8 μV or below −8 μV during the baseline (mean EMG activity over 1000 ms preceding picture onset) and above 30 μV or below −30 μV during picture presentation were excluded. EMG activity was scored as the average activity in the time window 300–6000 ms (Dimberg et al., 2000). Before statistical analysis, EMG activity was measured as the difference between the mean activity during the 6 s picture period and the 1 s baseline.

Electroencephalograph data were band-pass filtered between 0.01 and 20 Hz and then segmented into trials (−100–6000 ms with respect to picture onset). Subsequently, the data were corrected for ocular artifacts using the method developed by Gratton et al. (1983). An automated procedure was used to reject remaining artifacts according to the following criteria: a voltage step of more than 50 μV between two sample points, a voltage difference of more than 300 μV within a trial, and a maximum voltage difference of less than 0.50 μV within 100 ms intervals. EEG recordings were then re-referenced to the numeric mean of the mastoids, and baseline (−100–0 ms) corrected. Based on previous research indicating that the LPP is typically starting approximately 300–400ms after stimulus onset (Hajcak and Nieuwenhuis, 2006; Hajcak et al., 2009) and maximal at around 1700 ms at posterior and parietal sites (Schupp et al., 2000; Keil et al., 2002), the LPP was scored as the average activity in the time window 300–6000 ms at CPz, CP1, and CP2. For each participant, self-ratings, EMG and EEG data were averaged across trials per each condition.

# Statistical Analyses

One way analyses of variance (ANOVAs) were conducted to test for differences between non-smokers, NDS, and DS in demographics, degree of smoking dependence, and depression and anxiety levels. To analyze the effect of emotion regulation via reappraisal, difference scores were calculated by subtracting data scores of the baseline condition (i.e., maintain-neutral) from that of the other conditions (i.e., maintain-positive, maintain-negative, decrease-positive, and decrease-negative). These difference scores were then submitted to a repeated measures ANOVA with picture valence (positive, negative) and reappraisal (decrease, maintain) as within-subject factors, and group (NS, NDS, DS) as the between-subjects factor. Dependent variables included self-reported valence, arousal, and craving, corrugator and zygomaticus activity, and the LPP activity. *Post hoc t*-tests were conducted to further examine significant effects. To investigate whether the decreases in emotional feelings are associated with decreases in smokers' cigarette craving, correlations between changes in cigarette craving and emotional valence and arousal were analyzed on the basis of difference scores calculated by subtracting rating scores under the conditions with 'decrease' instructions (decreasepositive, decrease-negative) from corresponding conditions with 'maintain' instructions (maintain-positive, maintain-negative). The difference scores were then submitted to Pearson correlation analysis.

For all analyses the alpha-level was set at.05 (two-tailed). The Greenhouse-Geisser correction was applied when the assumption of sphericity was violated. The uncorrected degrees of freedom and effect sizes (η<sup>2</sup> p) are reported.

# RESULTS

# Sample Characteristics

The sample characteristics are depicted in **Table 1**. The one way ANOVAs revealed that non-smokers, DS and NDS did not differ in age, sex ratio, BDI score, STAI-trait and STAI-states scores (*p*s > 0.19). As expected, the three groups differed in CO levels [*F*(2,64) <sup>=</sup> 88.30, *<sup>p</sup>* <sup>&</sup>lt; 0.01, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.62] with non-smokers having lower CO levels than NDS [*t*(43) = −11.06, *p* < 0.01] and DS [*t*(41) = −5.20, *p* < 0.01]. Importantly, DS had lower CO levels than NDS [*t*(40) = −8.06, *p* < 0.01], confirming a successful deprivation manipulation. NDS did not differ from DS with regard to the age when they initiated smoking and the number of years they had smoked (*p*s > 0.06), though they had higher FTND scores [*F*(1,41) = 6.09, *p* < 0.05] and reported more consumption of cigarettes per day [*F*(1,41) = 4.23, *p* < 0.05] than DS.

# Effect of Reappraisal on Emotional Experience

The mean changes in self-reported valence and arousal as a function of reappraisal condition among NS, NDS, and DS are shown in **Figure 1**.

#### Self-reported Valence

The ANOVA revealed main effects of reappraisal [*F*(1,62) <sup>=</sup> 48.55, *<sup>p</sup>* <sup>&</sup>lt; 0.01, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.44] and picture valence [*F*(1,62) = 238.99, *p* < 0.01, η<sup>2</sup> <sup>p</sup> = 0.79], and an interaction effect of picture valence by reappraisal [*F*(1,62) = 92.80, *p* < 0.01, η<sup>2</sup> <sup>p</sup> = 0.60]. Indicating successful regulation of negative and positive emotions via reappraisal, Paired *t*-tests showed that participants reported less negative emotion under the decrease-negative condition compared to the maintain-negative condition [*t*(64) = 5.46, *p* < 0.01], and similarly, less positive emotion under the decreasepositive condition compared to maintain-positive condition [*t*(64) = 11.09, *p* < 0.01]. However, neither the main effect of group nor related interaction effects reached statistical significance (*p* > 0.22). This suggests that all participants successfully down-regulated emotional valence via reappraisal.

## Self-reported Arousal

The ANOVA revealed a significant main effect of picture valence [*F*(1,62) = 70.40, *p* < 0.01, η<sup>2</sup> <sup>p</sup> = 0.53] and an interaction effect of picture valence by reappraisal [*F*(1,62) = 18.27, *p* < 0.01, η2 <sup>p</sup> = 0.23]. Paired *t*-tests showed that the participants reported less arousal under the decrease-negative condition compared to the maintain-negative condition [*t*(64) = 3.24, *p* < 0.01], but larger arousal under the decrease-positive condition compared to the maintain-positive condition [*t*(64) = 2.13, *p* < 0.05]. Neither the main effect of group nor interaction effects reached statistical significance (*p* > 0.22), indicating that the three groups of participants did not differ in the regulation of emotional arousal via reappraisal.

# Effect of Reappraisal on Psychophysiological Responses Corrugator Activity

The ANOVA revealed a significant main effect of picture valence [*F*(1,62) <sup>=</sup> 37.56, *<sup>p</sup>* <sup>&</sup>lt; 0.01, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.38] and a significant interaction of picture valence by reappraisal [*F*(1,62) = 21.04, *p* < 0.01, η2 <sup>p</sup> = 0.25]. Paired *t*-tests showed that the corrugator activity was smaller under the decrease-negative condition compared to the maintain-negative condition [*t*(64) = 2.00, *p* < 0.05], indicating less negative facial expression as a result of reappraisal. Similarly, the corrugator activity was greater under the decreasepositive condition compared to the maintain-positive condition [*t*(64) = 3.74, *p* < 0.01], suggesting a successful downregulation of positive emotion (see **Figure 2**). Neither the main effect of group nor related interactions reached statistical significance (*p*s > 0.31), indicating that non-smokers, NDS and DS did not differ in the reappraisal outcome on corrugator activity.



smokers (NDS), and deprived smokers (DS). Depicted are difference scores (specific emotion condition minus baseline condition; see Materials and Methods). The more positive difference scores represent more negative (top) and more arousing (bottom) self-reported emotion under specific emotion condition comparing to baseline condition. Error bars represent standard error of the mean (SEM). ∗*p* < 0.05, ∗∗*p* < 0.01.

FIGURE 2 | Mean changes in facial electromyography (EMG) activity as a function of reappraisal among NS, NDS, and DS. Depicted are difference scores (specific emotion condition minus baseline condition; see Materials and Methods) in corrugator activity (top) and zygomaticus activity (bottom). The more positive difference scores in corrugator activity (top) represent more negative facial expressions; in contrast, the more positive difference scores in zygomaticus activity (bottom) represent more positive facial expressions under specific emotion condition comparing to baseline condition. Error bars represent standard error of the mean (SEM). ∗*p* < 0.05, ∗∗*p* < 0.01.

# Zygomaticus Activity

The ANOVA revealed a significant main effect of picture valence [*F*(1,62) = 25.03, *p* < 0.01, η<sup>2</sup> <sup>p</sup> = 0.29], a main effect of reappraisal [*F*(1,62) <sup>=</sup> 18.94, *<sup>p</sup>* <sup>&</sup>lt; 0.01, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.23], and a significant interaction of picture valence by reappraisal [*F*(1,62) = 15.11, *p* < 0.01, η<sup>2</sup> <sup>p</sup> = 0.20]. Paired *t*-tests showed that zygomaticus activity was smaller under the decrease-positive condition compared to the maintain-positive condition [*t*(64) = 4.49, *p* < 0.01], indicating less positive facial expressions as a result of reappraisal (see **Figure 2**). The zygomaticus activity under the maintain-negative condition did not differ from the decreasenegative condition [*t*(64) = 0.35, *p* = 0.73]. Again, neither the main effect of group nor other related interaction effects reached statistical significance (*p*s > 0.16), suggesting similar patterns of emotion regulation among NDS, DS and non-smokers.

#### LPP Activity

The ANOVA revealed that none of the main or interaction effects reached statistical significance (*p*s > 0.22). However, for explorative purposes, we conducted a paired *t*-tests revealing that the LPP was smaller under the decrease-negative condition compared to the maintain-negative condition [*t*(64) = 2.02, *p* < 0.05], indicating an effect of emotion regulation on the LPP in the expected direction (see **Figures 3** and **4**). However, the difference in LPP activity between the maintain-positive condition and the decrease-positive condition was not significant [*t*(64) = 0.23, *p* = 0.82], suggesting that positive emotion regulation was not reflected in LPP responses.

# Correlation between Changes in Emotions and Changes in Smokers' Cigarette Craving

Correlation analysis showed that the changes in smokers' cigarette craving were exclusively correlated with the modulation of self-reported arousal irrespective of the valence of the pictorial stimuli [negative stimuli (*N* = 42; *r* = 0.48, *p* < 0.01), positive stimuli (*N* = 42; *r* = 0.37, *p* < 0.05)]. None of the other correlations reached statistical significance. These correlations reflect that an increase in arousal was associated with an increase in craving (see **Figure 5**).

# DISCUSSION

This study aimed to investigate whether general emotion regulation competence via reappraisal is deteriorated in nicotine addicts. The present study found that all participants were capable of regulating positive and negative emotions following reappraisal instructions in the context of moderately evocative pictures, suggesting that smokers, including NDS and DS, have no deficit in general emotion regulation via deliberate reappraisal.

According to theoretical models of nicotine addiction (e.g., self-medication model and self-regulation failure model), people encounter repeated emotion regulation failures are prone to develop nicotine addiction because they expect that smoking could help them regulate emotions (Baumeister and Heatherton, 1996; Khantzian, 1997; Yucel et al., 2007). Neuroimaging studies have demonstrated that nicotine addicts are associated with abnormal PFC functions that are involved in cognitive emotion regulation (Lubman et al., 2004; Ochsner et al., 2004; McRae et al., 2010; Mocaiber et al., 2011; Moratti et al., 2011; Sutherland et al., 2012). It was assumed that heavy smokers would show general emotion regulation deficits in a reappraisal task. However, our results failed to support this hypothesis. The examined smokers were capable to regulate emotions via deliberate reappraisal, neither in case of positive emotions nor in case of negative emotions. Yet, some characteristics of the emotion regulation task and the experimental stimuli used may explain the

FIGURE 4 | Mean changes in LPP activity. Depicted are difference scores (specific condition minus neutral baseline condition; see Materials and Methods) in LPP activity as a function of reappraisal among NS, NDS, and DS. The more positive difference scores represent larger LPP activity under specific emotion condition comparing to baseline condition. Error bars represent standard error of the mean (SEM). ∗ *p* < 0.05.

inconsistent findings. First, we specifically examined reappraisal as a cognitive emotion regulation strategy and instructed the participants to regulate emotions using this strategy. Second, we used pictorial stimuli with moderate emotion to investigate the general ability of smokers to regulate emotions. This is different from real life situations in which individuals may often experience more intense and arousing stimuli (e.g., smoking related stimuli) and have to decide by themselves when and how to regulate their emotions. Therefore, the present study indicates that heavy smokers may have no deficit in general emotion regulation via reappraisal, which may not exclude emotion regulation dysfunctions in real life situation. Without deliberate instructions, it might be possible that smokers select maladaptive emotion regulation strategies which may result in a failure in regulating emotions (Ehring et al., 2010). To extend this conclusion, future studies are needed to investigate how smokers differ from non-smokers in the spontaneous selection of emotion regulation strategies in a real life situation when presented with more arousing or addiction relevant stimuli.

An additional aim of this study was to investigate the effect of smoking deprivation on general emotion regulation. Prior work showed that DS performed less well than NDS on a variety of cognitive tasks such as attention, memory, and affective processing (Cinciripini et al., 2006; Piper and Curtin, 2006). Thus, we assumed that smoking deprivation may worsen a hypothesized deficit in cognitive emotion regulation. However, our results demonstrated that DS performed as well as NDS when they were instructed to regulate emotions via reappraisal. This suggests that overnight abstinence from smoking does not affect deliberate regulation of emotion in smokers.

The present study expands previous studies by investigating cognitive emotion regulation in terms of both positive and negative stimuli among smokers. It has been noted that regulations of both positive and negative emotions contribute to human well-being and prevent people from substance abuse (Fredrickson, 2001; Fredrickson et al., 2008). However, most emotion regulation research focused on altering negative emotions (Ochsner et al., 2002, 2004; Ochsner and Gross, 2007; McRae et al., 2010; Mocaiber et al., 2011; Parvaz et al., 2012), with a few exceptions that have investigated regulation of positive emotions (Delgado et al., 2008; Giuliani et al., 2008; Krompinger et al., 2008; Wu et al., 2012). Overall, there is a lack of information on the regulation of positive emotions in nicotine addicts. The current findings showed that reappraisal is an efficient way for smokers and non-smokers to regulate both positive and negative emotions, with the outcomes of positive emotion regulation were somewhat different from the ones of negative emotion regulation. Specifically, both smokers and non-smokers successfully reduced negative emotions as indexed by self-ratings of unpleasantness, experienced arousal, corrugator activity, and LPPs. With respect to positive emotions, participants successfully decreased selfreported pleasantness and zygomatic activity, but increased selfreported arousal and failed to change LPPs. These results suggest that changes of emotional valence and arousal as a function of reappraisal are congruent in the context of negative picture stimuli but incongruent in the context of positive picture stimuli. In line with this, previous studies have been demonstrated that more negative stimuli were consistently rated as more arousing, whereas the more positive stimuli were associated with either higher arousal ratings or lower arousal ratings (Lang et al., 2005). Therefore, it should be cautious for future studies to differentiate valence and arousal when addressing regulation of positive emotions.

This study is the first to address the correlation between the effects of reappraisal on emotional valence, arousal and craving in smokers. Previous studies indicated emotion regulation and craving regulation activate common brain regions (Koob and Volkow, 2010). Accordingly, it was assumed that emotions and cravings would be altered simultaneously by reappraisal.

# REFERENCES


The present study showed that smokers' cigarette craving is positively correlated with emotional arousal with regard to both the negative and the positive stimuli. This expands our understanding of an association between emotions and craving, i.e., arousing stimuli or scenarios may trigger cigarette craving in smokers irrespective of their valence (Velicer et al., 1990; Shiffman et al., 2012). Therefore, it might be plausible to conclude that cigarette craving is linked to emotional arousal rather than emotional valence.

Finally, there are some limitations of this study. First, emotional events in real-life situations could be more intensive than the pictorial stimuli used in the present study. The present study showed that smokers might have an intact ability to regulate emotions via reappraisal, although this does not exclude an inability to select and apply adaptive strategies to regulate emotions in real-life situations, and, in particular, to regulate the motivational responses to smoking related stimuli. Second, the focus of this study was constrained on smokers who do not have a personal history of drug addiction excluding nicotine dependence and do not have current psychiatric or neurological disorders. Those smokers performed as well as non-smokers in the emotion regulation task. To expand this conclusion, future studies are needed to investigate emotion regulation in smokers with comorbid psychiatric disorder.

In sum, the current study illustrates that heavy smokers are able to regulate emotion via deliberate reappraisal, irrespective of the valence of the emotional stimuli. Moreover, we found no indication that over-night deprivation from smoking does affect the performance in the deliberate reappraisal task. From these results, we suppose that heavy smokers do not have a cognitive impairment in general emotion regulation via deliberate reappraisal, although this does not exclude their inability to select and apply appraisal strategies to regulate emotions in real-life situations.

# ACKNOWLEDGMENT

This work was supported by the German Research Foundation [Research Group "Emotion and behavior", FOR 605, PA 566/9-1, PA 566/9-2, and GRK 1253/1 scholarship to LW].


relevant distracters on vigilance. *Psychol. Addict. Behav.* 28, 752–760. doi: 10.1037/a0035122


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Wu, Winkler, Wieser, Andreatta, Li and Pauli. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# How absent negativity relates to affect and motivation: an integrative relief model

# *Roland Deutsch1\*, Kevin J. M. Smith1, Robert Kordts-Freudinger <sup>2</sup> and Regina Reichardt <sup>3</sup>*

<sup>1</sup> Department of Psychology, Technische Universität Dresden, Dresden, Germany

<sup>2</sup> Department of Psychology, University of Paderborn, Paderborn, Germany

<sup>3</sup> Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany

#### *Edited by:*

Paul Pauli, Julius-Maximilian's University, Germany

*Reviewed by:* Arielle Ryan Baskin-Sommers, Yale University, USA Anne Krendl, Indiana University, USA

#### *\*Correspondence:*

Roland Deutsch, Department of Psychology, Technische Universität Dresden, Zellescher Weg 17, 01069 Dresden, Germany e-mail: roland.deutsch@tu-dresden.de The present paper concerns the motivational underpinnings and behavioral correlates of the prevention or stopping of negative stimulation – a situation referred to as relief. Relief is of great theoretical and applied interest. Theoretically, it is tied to theories linking affect, emotion, and motivational systems. Importantly, these theories make different predictions regarding the association between relief and motivational systems. Moreover, relief is a prototypical antecedent of counterfactual emotions, which involve specific cognitive processes compared to factual or mere anticipatory emotions. Practically, relief may be an important motivator of addictive and phobic behaviors, self destructive behaviors, and social influence. In the present paper, we will first provide a review of conflicting conceptualizations of relief. We will then present an integrative relief model (IRMO) that aims at resolving existing theoretical conflicts. We then review evidence relevant to distinctive predictions regarding the moderating role of various procedural features of relief situations. We conclude that our integrated model results in a better understanding of existing evidence on the affective and motivational underpinnings of relief, but that further evidence is needed to come to a more comprehensive evaluation of the viability of IRMO.

#### **Keywords: relief, avoidance, motivation, reward, appraisal**

A fundamental feature differentiating various emotions is whether they refer to present or absent events. For example, the presence of positive events typically triggers happiness, whereas the absence of a desired positive state triggers anger (Carver and Harmon-Jones, 2009). Likewise, the expected presence of negative stimulation (NStim) of all sorts such as pain, social rejection, or failure at work, typically triggers fear, while the prevention or offset of NStim triggers relief (Lohr et al., 2007; Riebe et al., 2012). Generally, many theories of emotion suggest links between emotions, affective valence, and motivational orientations of approach and avoidance, but not so in a consistent manner. Very simplified, one class of theories, which we label *valence theories*, suggests that all positive emotions are associated with approach motivation, whereas all negative emotions are associated with avoidance motivation (e.g., Gray, 1987). A second class of theories, which we label *goal theories*, assumes the key feature along which emotion and motivational orientation are matched to be the goal that is pursued by an actor (e.g., Carver, 2001), whereas motivation and affective valence are seen as orthogonal.

Interestingly, these two classes of theories make markedly diverging predictions when it comes to absence-based emotions such as anger and relief (for discussions, see Carver, 2004, 2009; Carver and Harmon-Jones, 2009). Therefore, studying the affective and motivational underpinnings of absence-based emotions not only improves the understanding of these particular emotions. It also helps evaluating theoretical notions about the relation between emotions and broad motivational systems of approach and avoidance. Carver and Harmon-Jones (2009) recently reviewed existing evidence on the affective valence and

motivational orientation associated with anger. Although the relation between anger and motivational orientation seems to be moderated by various factors such as the goal of approach movements (e.g., Krieglmeyer and Deutsch, 2013; Bossuyt et al., 2014), the evidence generally favors the notion that anger is of negative affective valence but derives from an approach motivational orientation. This is in line with core assumptions of goal theories of emotion–motivation interactions (e.g., Carver and Scheier, 1990; Higgins, 2001; Harmon-Jones et al., 2003).

The aim of the present article is to provide an integrative review on the affective and motivational underpinnings of relief, an emotion triggered by the absence of expected or experienced NStim (Lazarus, 1991; Roseman and Evdokas, 2004; Lohr et al., 2007; Leknes et al., 2011; Riebe et al., 2012; Gerber et al., 2014). We are certainly not the first reviewing research on relief (Lohr et al., 2007; Riebe et al., 2012; Bastian et al., 2014; Gerber et al., 2014; Navratilova and Porreca, 2014). Yet, these reviews are focused on theories, paradigms, and findings within a particular range, such as relief from the stopping of pain (Bastian et al., 2014; Gerber et al., 2014; Navratilova and Porreca, 2014), relief from the termination of fear (Lohr et al., 2007; Riebe et al., 2012), long-term decrease in fear responding (Riebe et al., 2012), or neuronal underpinnings of relief (Gerber et al., 2014; Navratilova and Porreca, 2014). Importantly, these reviews were not focused on tackling the questions of affective valence and motivational orientation, and also provide limited cross cutting perspectives. The present review seeks to overcome both limitations. In what follows, we will first briefly explain the importance of relief, and provide a conceptual clarification of relief. We then review diverging theoretical perspectives

on the motivational orientation associated with relief and present a first step toward an integrative relief model (IRMO) that aims at combining parts of earlier, more focused theories of relief. We claim that integrating parts of these theoretical fields will help reconciling contradictory conceptualizations of and empirical results on relief. As a derivation, we identify two parameters of relief that can be expected to go hand in hand with differences in valence and motivational orientation. Finally we review evidence on the parameters and discuss the evidence in relation to theoretical notions on relief and motivational orientation.

## **THEORETICAL AND PRACTICAL IMPORTANCE OF RELIEF**

As suggested above, reviewing affective and motivational underpinnings of relief is important because it helps evaluating diverging theories of emotion–motivation interactions. Gaining a better understanding of relief is also of great importance because relief contributes to a number of phenomena of great practical importance. There is a growing literature on the mechanisms of relief from acute and chronic pain in general (e.g., Leknes et al., 2008, 2013; Bastian et al., 2014), and the role of relief in maintaining self-infliction of harm (e.g., Favazza, 1998; Franklin et al., 2013a). Relief is hypothesized to be a major force in phobia and avoidance behavior (Mowrer, 1960; Lohr et al., 2007). Moreover, although craving positive end-states plays a major role in addiction (e.g., Robinson and Berridge, 1993), there is also a contribution of relief from negative affect (e.g., Baker et al., 2004; Ostafin and Brooks, 2011). Also, relief may promote social influence (Dolinski and Nawrat, 1998; Dolinski et al., 2002) but at the same time may prevent creativity (Baas et al., 2011). Answering the question of whether relief is of positive vs. negative valence, as well as whether it goes along with an approach vs. avoidance motivation will contribute to a better understanding and perhaps ultimately control of these phenomena.

# **CONCEPTUAL CLARIFICATION**

Emotions are complex constructs involving facets such as subjective experiences, physiological response-patterns, cognitions, and behavioral tendencies that are typically triggered by a class of stimuli. As with other emotions (cf., Ortony and Turner, 1990; Prinz, 2004), formal definitions of which specific manifestations of the above facets constitutes relief slightly differ depending on the author (e.g., Ortony et al., 1988; Lazarus, 1991; Carver, 2001; Roseman and Evdokas, 2004; Lohr et al., 2007; Riebe et al., 2012; Gerber et al., 2014). However, one facet is shared by almost all researchers: relief derives from situations in which an expected or previously experienced NStim is reduced or absent. For example, Roseman and Evdokas (2004, p. 4) characterize relief as a consequence of "...appraising an event as consistent with an aversive pain-minimizing motive..." Lazarus (1991, p. 122) suggests the cause of relief to be "...a distressing, goal-incongruent condition that has changed for the better or gone away." Leknes et al. (2011, p. 1) characterize relief as "...reward induced through omission or reduction of an aversive event..." Moreover, research in the tradition of appraisal theories has tried to uncover conditions under which people label an affective state as relief. Such studies revealed that appraising negativity as absent was reliably associated with subjective relief (Roseman et al., 1990; Roseman, 1991, 1996), while results for other appraisal dimensions were less clearcut. In an attempt to capture the essence of existing definitions, the present review will use the term relief to refer to the emotion that is triggered by the absence of expected or previously experienced NStim. Moreover, we will refer to situations in which expected NStim does not occur, or in which experienced NStim stops or is reduced, as relief situations.

# **THEORETICAL PERSPECTIVES ON RELIEF**

Clearly, relief is part of many emotion theories. As can be derived from **Table 1**, these theories widely agree on the valence of relief, which is identified as positive. There is also agreement that relief presupposes a prior negative situation: "For it is only when an animal anticipates... punishment (fears) that it can be affected by the omission of punishment ('relief')" (Gray and McNaughton, 2000, p. 50). But there is some inconsistency in how the omission of experienced vs. expected NStim relates to relief. While the above quote implies that the omission of expected punishment is considered relief, Lohr et al. (2007; cf. Gerber et al., 2014) suggested differentiating between situations where a negative state stops (labeled relief) and situations where the non-occurrence of a potential negative state is experienced (labeled respite), whereas other theorists identify the latter case as relief too. For example, Riebe et al. (2012, p. 164) associate fear relief with recognizing "...the absence and/or disappearance of a threat..." In such situations, the fear associated with threat stops, but the dreaded event was never experienced. Moreover, there seems to be some agreement that relief results from the reduction of the psychological impact of a negative situation. But there is disagreement on whether relief presupposes a certain and complete omission of negativity. Some theories allow for something that Leknes et al. (2013) termed relative relief, where negativity must not necessarily be fully averted but only reduced (cf. Lazarus, 1991; Carver, 2001; Fujiwara et al., 2009; Leknes et al., 2013). Other theories, however, at least implicitly, associate relief with the full and certain omission of negativity (e.g., Roseman and Evdokas, 2004). We believe that all these facets of relief are important and should be considered in an integrated way. In what follows, we will focus on how various theories associate relief with motivational orientations.

#### **RELIEF, APPROACH, AND AVOIDANCE**

As can be derived from **Table 1**, different theories make diverging assumptions about the motivational orientation underlying relief. In line with earlier analyses (e.g., Higgins, 1996, 1997; Carver, 2001, 2009), we recognize two broad clusters of theories relating relief and motivational orientation. *Valence theories* assume affective valence to be the key feature along which emotion and motivational orientation are matched (e.g., Schneirla, 1959; Gray, 1971; Lang et al., 1990, 1992; Neumann et al., 2003; Strack and Deutsch, 2004). For example, Lang et al. (1992, p. 44) suggested "...that pleasant states are driven by the appetitive system and unpleasant states by the aversive motivation system..." Consequently, to the degree that relief can be considered to be of positive valence, relief is assumed to be an emotion of the approach system. Similarly, Gray's (1987) reinforcement sensitivity theory


**Table 1| Emotion theories and their assumptions regarding the association between relief and valence, as well as approach and avoidance motivation.**

(RST) states that the valence of stimuli determines whether appetitive [behavioral approach system (BAS)] or aversive motivation [fight–flight system (FFS); behavioral inhibition system (BIS)] dominates behavior. More specifically, the BAS is supposed to be distinctively activated by primary and secondary reward stimuli, including relief, resulting in the formula "hope = relief " (Gray, 1987, p. 248). In essence, valence theories suggest that positive emotions are driven by the approach system, and that therefore relief is an emotion of the approach system.

*Goal theories* assume the key feature along which emotion and motivational orientation are matched to be the type of goal that is pursued by an actor (e.g., Higgins, 1996, 1997; Carver and Scheier, 1998; Carver, 2001). Whereas valence theories assume that all positive affects (e.g., elation and enthusiasm) are associated with approach motivation and negative affects (e.g., fear and distress) with avoidance motivation, goal theories assume that valence is orthogonal to approach/avoidance. Rather, valence is hypothesized to be strongly dependent on the success of the goal pursuit (Higgins et al., 1997; Carver and Scheier, 2011). Accordingly, positive as well as negative affect can result both from approach and avoidance motivation. If an avoidance goal is pursued, doing poorly is predicted to result in anxiety and fear, whereas doing well will result in relief and calmness (Carver, 2001). Therefore, goal theories suggest relief to be a positive affect that derives from avoidance processes. Importantly, some goal theories explicitly suggest that relief derives from avoidance motivation but at the same time deactivates avoidance motivation (e.g., Roseman, 2013). Other goal-theories are less clear about whether relief activates or deactivates avoidance motivation. Carver's (2001) theory suggests that emotions provide feedback on the success of goal pursuit, with relief signaling that avoidance processes are progressing well. This suggests that relief might occur even when the avoidance goal is not yet fulfilled. From this perspective, assuming relief to deactivate avoidance processes would be dysfunctional. At the same time, the theory suggests that relief is "...part of the process... of regrouping, restoring one's access to energy supplies... preparatory to turning to some new activity" (Carver, 2001, p. 351), which may imply abandoning avoidance goals.

A third theory ascribes a dual motivational nature to relief. Specifically, the revised version of Gray's RST (Gray and McNaughton, 2000) maintains the notion that relief situations activate the BAS. However, the theory also suggests that in relief situations "...both the behavioral inhibition and the BAS will be activated concurrently, with some patterns of behavior being produced by the one system and some by the other" (Gray and McNaughton, 2000, p. 55). One reason for this prediction is that stimuli associated with (successful) avoidance behavior"...can, and often will, predict that some other (usually many other) responses will produce, or fail to avoid, the aversive stimulus" (Gray and McNaughton, 2000, p. 55).

A final theory has been immensely influential on relief researchers (e.g., Leknes et al., 2008; Andreatta et al., 2013), but makes only conditional predictions regarding the motivational orientation of relief: opponent process theory (OPT; Solomon, 1980). OPT suggests that a psychophysical process *A* typically triggers a process *B* that counteracts the effect of the original process

*A*. Moreover, "The *B* process (the opponent process) is postulated to be (a) of sluggish latency, (b) inertial, or slow to build to its asymptote, and (c) slow to decay..." (Solomon, 1980, p. 699). In the case of pain (or other intense aversive stimulation), the *B* process is predicted to be of positive valence and "... individuals feel an emotional state which entails opponent, namely appetitive properties" (Andreatta et al., 2013, p. 1). OPT therefore conceptualizes relief as the persisting *B* process after NStim (Leknes et al., 2008). OPT's predictions for affective valence are straightforward: "Because the *b* process is an opponent process, its affective or hedonic quality must be opposite to that of the *a* process" (Solomon, 1980, p. 699). But what is the opposite motivational orientation of an *A* process representing unspecific negative affect (Russell, 2003) or specific negative emotions such as fear, anger, or sadness, from which one might feel relieved if they stopped? To answer this question, one must obviously know the motivational orientation associated with the *A* process. As explained in the previous paragraphs, however, this is still a question of considerable debate. For example, from the perspective of goal theories (Carver and Harmon-Jones, 2009), anger belongs to the approach system, so that the opposite motivational orientation would be avoidance. From the perspective of valence theories (Lang et al., 1992), anger belongs to the avoidance system, so that the opposite motivational orientation would be approach. Clearly, OPT makes easy predictions as long as affect is concerned. Moreover, fear or pain as *A* processes go along with avoidance motivation in all considered theories, and hence OPT predicts relief to be approach-oriented in these cases. But predictions regarding other emotions or hedonic states, such as hunger, require additional theoretical assumptions regarding the relations between these constructs.

#### **TOWARD AN INTEGRATIVE MODEL OF RELIEF**

Given the heterogeneity of theoretical assumptions on the concept of relief and its affective and motivational bases, we suggest that integrating these diverging views is a pressing goal. In what follows, we describe a first version of an IRMO. While construing IRMO, we draw on four classes of existing emotion theories relevant to relief: theories of fear and learning (Lang et al., 1990; Gray and McNaughton, 2000; Lohr et al., 2007) provide a taxonomy of different relief situations and cues that feed into relief. We included assumptions of regulatory theories of emotion (Carver and Scheier, 1990, 2002; Carver, 2001) regarding the dynamic nature of relief and the feedback function of positive affect during active relief. We draw on mechanisms of OPT (Solomon, 1980; Leknes et al., 2008; Andreatta et al., 2010) to explain the occurrence of positive affect as a consequence of the absence of expected or experienced NStim. Finally, we incorporate basic notions of appraisal theories (Lazarus, 1991; Roseman and Evdokas, 2004; Reisenzein, 2009), highlighting the importance of certainty- and motive-congruence appraisals. Also, IRMO specifically combines assumptions of goal theories of motivation and affect (Higgins, 1996; Carver, 2001) with assumptions of OPT (Solomon, 1980). As suggested by goal theories, IRMO assumes relative independence of the generation of positive vs. negative affect on one hand, and the instigation of approach vs. avoidance motivation on the other hand. Applying OPT, IRMO assumes that both

affect and motivation come with their sets of specific *A* and *B* processes, with both following the principles outlined in OPT. Hence, the parts of which IRMO is made are not new, but we consider their combination an innovative step forward toward a better understanding of psychological processes related to absent negativity.

#### **DYNAMIC NATURE**

In line with regulatory theories of emotions (e.g., Higgins, 1996; Carver, 2001), we suggest that relief is best understood as part of a dynamic process instead of a static, one-shot phenomenon. First, relief is a dynamic phenomenon because it presupposes a change from expected or experienced NStim toward their reduction or full absence. Second, relief is a dynamic phenomenon because the shift toward full absence often may evolve over a longer action sequence during which the intensity of positive affect signals the effectiveness of avoidance behavior (Carver and Scheier, 1998; Lawrence et al., 2002). IRMO therefore suggests a tri-phasic sequence of relief episodes, which is an idealized abstraction of a continuous progression from unconditional NStim to unconditional absence of NStim (see **Figure 1**). IRMO suggests that organisms monitor two probabilities throughout these phases, and each phase is characterized by a specific combination of these two probabilities. The subjective probabilities will correlate with objective probabilities but are subject to biases associated with probability estimation (e.g., Gilovich et al., 2002). The first is the probability of NStim in the current situation without any salient features related to safety, expressed as *p*(NStim|situation). The situation may include cues that correlate with the occurrence

of NStim (CS+aversive; e.g., the smell of a dentist's office), or the actual exposition to aversive stimuli (USaversive; e.g., the drill touching the dental pulp). In the latter case, *p*(NStim|situation) necessarily equals 1. The second is the probability of NStim if cues related to safety become salient in the situation. Such cues are safety signals (CS−aversive; e.g., the sound of an airconditioning system that just sprang into action), or avoidance behaviors (Rrelief ; e.g., running away from a fire). We express this as *<sup>p</sup>*(NStim|(CS−aversive <sup>∪</sup> Rrelief))1. The availability of Rrelief can be signaled by discriminative relief stimuli (SD relief ; e.g., the sight of a box of aspirin signals that taking aspirin will stop pain), and therefore perceiving S<sup>D</sup> relief will result in a decrease in *p*(NStim|Rrelief).

The *negative stimulation phase* (NStim phase) is characterized by the experience or expectation of NStim, with *p*(NStim|situation) = 1 in the case of experience, and *p*(NStim|situation) > fear threshold in the case of expectation. *Fear threshold* is the probability at which an individual starts experiencing fear in the face of a threat. It may vary as a function of type of the potential NStim and as an individual difference factor. In the NStim phase, no Rrelief are available, going along with appraisals of low controllability, although organisms will likely start searching for available Rrelief . Also, no CS−aversive or S<sup>D</sup> relief are present that might result in a decrease in the subjective probability of NStim.

The *engaged relief phase* is characterized by the shift from a subjectively high absolute probability of NStim toward a lower

<sup>1</sup>We use the logical disjunction to express that one of the two or both might be present.

absolute probability assessment. Processing CS−aversive, SD relief , or engaging in Rrelief is responsible for the subjective change in probability. In the engaged relief phase, the original threat is still present, such that there is a high probability of NStim if nothing is done and no safety cues are present. There is, however, a lower *p*(NStim|(CS−aversive ∪ Rrelief)), that is a lower probability of NStim if avoidance behavior occurs and/or safety signals are present.

The final *disengaged phase* is characterized by the mental disengagement from earlier punishment, threat, safety cues, and avoidance behavior. In this phase, *p*(NStim|situation), and *p*(NStim|(CS−aversive ∪ Rrelief)) are appraised as equally low and below the fear threshold. Therefore, this phase is characterized by unconditional safety.

#### **PROCESS ASSUMPTIONS**

#### *Negative stimulation phase*

The integrative relief model proposes a cascade of processes mediating the shift from NStim to disengaged relief (see **Figure 2**). Starting point is the appraisal of a situation or a concrete stimulus (USaversive or CS+aversive) in that situation, resulting in the generation of a subjective *p*(NStim|situation) > fear threshold. Such negative appraisals go along with the activation of negative affect and avoidance motivation. Both are considered as *A* processes as conceptualized in OPT, and hence they are assumed to trigger opponent *B* processes, characterized by a slower temporal dynamic and lower intensity than the *A* processes (Solomon, 1980; Leknes et al., 2008). In response to an initial negative appraisal and resulting negative affect, a search for potential coping opportunities will set in. This includes scanning the environment for CS−aversive or S<sup>D</sup> relief and scanning memory for avoidance schemata (Rrelief ) that fit the current situation. As long as this search is without success, the organism remains in the NStim phase.

#### *Engaged relief phase*

If a safety signal is detected, *p*(NStim|CS−aversive) will be estimated and compared to *p*(NStim|situation). The intensity of the affectively negative *A* process as well as the motivationally avoidant *A* process will be reduced to the degree that the safety signal is appraised as reducing the probability of NStim2. Based on the principles of OPT, the reduction in the negative affective and motivationally avoidant *A* processes will result in a temporary relative strengthening of the positive affective and approach-oriented *B* processes. In other words, recognizing an increase in safety signaled by environmental stimuli goes along with a positive affective signal and a shift away from avoidance and a relative strengthening of approach motivation generated on the basis of opponent processes.

<sup>2</sup>In **Figure 2**, this will result in a negative difference in probabilities and thus an inhibitory influence on negative affect.

If an SD relief is detected, an additional regulation loop is expected to set in (Carver and Scheier, 1990; Carver, 2001; Lawrence et al., 2002) that serves the specific affordances of active avoidance. An SD relief goes along with high controllability appraisals, which are expressed by the difference in the probability of NStim with avoidance behavior and without avoidance behavior [i.e., *p*(NStim|Rrelief) – *p*(NStim|situation)]. Perceiving controllability will result in a decrease in the negative affective *A* processes and hence a temporary upswing in positive affect based on opponent processes. It will further result in an activation of the avoidance behavior that was the basis of the controllability appraisal3. Importantly, IRMO does not predict that perceiving SD relief , appraisals of controllability, or avoidance behavior directly decrease avoidance motivation. This prediction is based on the notion that avoidance motivation is the energizing part of avoidance behavior. As such, a successful organism will maintain high avoidance motivation as long as avoidance behavior is necessary to generate relative safety.

The actual *p*(NStim|situation) and *p*(NStim|Rrelief) are monitored, and the originally expected probabilities are adjusted based on the observed ones. If the adjustment results in an increase in expected controllability (i.e., the probability difference becomes more negative), the inhibition of negative affect will further increase, resulting in a temporary increase in positive affect based on OPT. Moreover, the avoidance behavior will be further activated. If the observed controllability is worse than the expected one, the inhibition of negative affect will be reduced, going along with a decrease in the activation of the avoidance behavior. If the Rrelief increased the *p*(NStim) compared to doing nothing, negative affect will increase, going along with an inhibition of the avoidance behavior. This provides a feedback loop driven by the tracked success of ongoing avoidance behavior (Carver and Scheier, 1990; Carver, 2001; Lawrence et al., 2002), potentially resulting in a situation with high controllability appraisals, phases of dominant positive affect (if controllability improves), and high avoidance motivation maintained by the continuously perceived threat of *p*(NStim|situation).

In some cases, instrumental behaviors may generate outcomes that signal the absence of threat for a distinct period of time (Berger and Brush, 1975; Berger and Starzec, 1988). That is, behavioral outcomes may function as CS−aversive, and IRMO predicts such signals to reduce avoidance motivation and consequently avoidance behavior as long as they are present (see broken line in **Figure 2**). For two reasons, such self-generated CS−aversive can even be expected to have more intense effects than external CS−aversive (cf. Cándido et al., 2004). First, because they are part of the instrumental action, they may receive more attention than stimuli that are only passively observed, resulting in better learning (e.g., Hommel, 2010). Second, probability estimates of self-generated CS−aversive may be more optimistic than passively acquired ones. Contingency assessments are biased toward overestimating control (Langer, 1975), especially under conditions of

acting (Langer and Roth, 1975; Blanco et al., 2011). For the same reasons, active relief in general may be more positive than passive relief even when no safety period is signalled (cf. Eder and Dignath, 2014).

#### *Disengaged relief phase*

According to IRMO, a shift toward disengaged relief goes along with reductions in *p*(NStim|situation), indicating that the situational threat is eliminated. Situational threat can be appraised in multiple ways. One way is to briefly stop avoidance behavior and explore the results. For example, a person who has been taking a pain reliever may briefly stop doing so to see whether the pain is still there. Another way would be to check whether an obvious cause of the threat is gone (e.g., whether the dangerous stray dog has been captured). The principles outlined in IRMO (see **Figure 2**) imply that if *p*(NStim|situation) is reduced, the activation of negative affect and avoidance motivation *A* processes is reduced, and approaches zero if situational threat falls below the fear threshold (cf., Pekrun et al., 2002; Roseman, 2013). As a consequence, avoidance behavior will lose momentum and *B* processes (positive affect and approach motivation) will dominate for a while. As has been theorized by Carver(2009, p. 133), relief "...represents a signal that the person does not have to attend to the threat any longer and attention broadens to consider other available possibilities for goal pursuit..." This, according to Carver (2009), only applies when the threat is eliminated. In that sense, a shift toward new, attractive goals can be expected primarily for the disengaged relief phase.

#### **APPRAISALS**

What is the relationship between the processes specified in the IRMO and appraisals proposed by appraisal theories of emotions (e.g., Roseman, 1984; Ortony et al., 1988; Lazarus, 1991)? We argue that some of these processes can be conceptualized as appraisals. In the case of active relief, we argue that the expected reduction in the probability of NStim through avoidance behavior can be conceptualized as an appraisal of controllability (cf. Mowrer, 1960). Moreover, assessing the absolute probability of NStim as low, or perceiving a reduction in the probability of NStim can be seen as an appraisal of motive congruency – i.e., with the motive to avoid or end punishment (Roseman, 2013). More generally, all probability assessments of IRMO can be thought of as appraisals of certainty of the respective events (i.e., absence or presence of NStim). These appraisals determine the strength of the activating or inhibiting effect on the aversive *A* process.

#### **CONSEQUENCES FOR THE RELATION TO AFFECT AND MOTIVATION**

There are numerous empirical consequences that follow from these process considerations. IRMO was designed to incorporate as many known relief phenomena with as few process assumptions as possible. So it comes as no surprise that it indeed covers many of these phenomena. But many of these consequences are not specific to the question whether relief is associated with approach vs. avoidance motivation and positive vs. negative affect, which is why we will refrain from discussing them here. In addition, the goal of IRMO is to provide an integrative perspective on the affective and motivational underpinnings of relief, which we found to be conceptualized quite differently in various theories. In what follows,

<sup>3</sup>Although the probability difference is negative in the case of controllability, the negative correlation attached to the path from controllability assessment to behavior in **Figure 2** results in an increase in activation.

#### **Table 2 | Forms of relief in IRMO.**


we will describe how the principles outlined in IRMO might help to reconcile these perspectives. Particularly, we will discuss procedurally different forms of relief (prevention vs. stopping-relief; active vs. passive relief; see **Table 2**) and demonstrate that IRMO predicts them to relate to approach and avoidance motivation to varying degrees. Although the crossing of the two features (active vs. passive; prevention vs. stopping) suggests four types of relief, we will discuss empirical consequences in the sense of main effects of the two features.

The dynamic perspective of IRMO suggests that affective and motivational underpinnings of relief coarsely vary depending on the relief phase. In the NStim phase, negative affect and avoidance motivation prevail, while in the disengaged relief phase, positive affect and approach motivation prevail. In the engaged phase, the affective tone is positive but motivational orientation differs depending on whether relief is active (avoidance) or passive (approach). Importantly, the phases are characterized by different combinations of subjective probabilities, corresponding to certainty appraisals of predictions regarding NStim or its absence. As a consequence, the certainty of these predictions can be considered moderator variables that affect the intensity and quality of the processes outlined in our model.

#### **CERTAINTY**

The integrative relief model predicts the certainty of NStim to positively correlate with negative affect and avoidance motivation (cf. Ortony et al., 1988), and, in line with learning and appraisal theories, also to positively correlate with positive affect and approach motivation if NStim is prevented or stopped (cf. Ortony et al., 1988; Gray and McNaughton, 2000; Reisenzein, 2009). IRMO further suggests that the certainty with which NStim can be avoided positively correlates with positive affect (cf. Roseman, 1984; Ortony et al., 1988), whereas it's relation to motivation depends on whether relief is active (no effect) or passive (decrease in avoidance). Depending on the certainty of expected nonpunishment compared to expected punishment, IRMO predicts fear to be reduced in intensity and opponent processes of fear to dominate. Hence, the certainty of absence of NStim determines the relative strength of positive vs. negative representations and approach vs. avoidance motivation at a given point in time. This

is at odds with a view of relief as a purely positive emotion, but, as will be seen later, backed up by evidence.

#### **ACTIVE vs. PASSIVE**

A first focal feature that differentiates theories, research, and findings in the realm of relief is whether the relief is caused by behavior of the subject or whether the relief occurs independently from the subject's behavior. Based on an earlier analysis by Zvolensky et al. (2000), Lohr et al. (2007) provide a taxonomy for the realm of anxiety disorders that we deem to also be highly useful outside the clinical context. They suggest differentiating between offset control of aversive stimulation on one hand, and offset prediction on the other hand. Offset prediction is a prototypical example of passive relief, as in Pavlovian conditioning, where CS−aversive elicit relief from fear of NStim (Gray, 1971; Cole and Miller, 1999; Genud-Gabai et al., 2013). Offset control of NStim is a prototypical example of active relief, where overt behaviors of a subject cause the prevention or the stopping of a negative event.

#### *Engaged relief phase*

The integrative relief model assumes active vs. passive relief to correspond to differences in underlying processes for engaged relief. Active relief presupposes engaging in behaviors that cause aversive stimulation to stop or to be prevented, whereas passive relief does not. According to the assumptions made in IRMO, affect and motivation respond in a manner that distinctively supports passive or active relief. If a CS−aversive is processed, the expectancy of NStim drops and negative affect and avoidance motivation *A* processes are reduced accordingly. As a consequence, a temporary increase in positive affect and approach motivation will result. Active avoidance behavior will become less likely. Instead, the safety phase signaled by the CS−aversive goes along with a higher probability of approach behaviors to be triggered by environmental cues.

For active relief, reducing avoidance motivation as a consequence of processing an SD relief would be dysfunctional. S<sup>D</sup> relief signal the opportunity to actively reduce the probability of NStim based on a comparison of the probability of NStim under the condition of action vs. inaction. If this comparison results in an appraisal of controllability, negative affect will be reduced, temporarily resulting in an overshoot of positive affect *B* processes. Avoidance motivation, however, is not reduced. Instead, the appropriate avoidance behavior is activated, energized by avoidance motivation, and its success is monitored. If the behavior reduces the probability of NStim as expected, affect and activation of the behavior remain the same. If the reduction in the probability of NStim is greater than expected, negative affect will further be reduced and another temporary positive affect *B* process will emerge and the avoidance behavior is further activated. If the reduction in the probability of NStim is smaller than expected, activation of negative affect will increase and the avoidance behavior will be inhibited (Carver, 2001, p. 20; Lawrence et al., 2002). If active relief involves the generation of CS−aversive that signal the absence of threat for a period of time, active and passive relief will be indistinguishable during the safety period predicted by the behavior-generated CS−aversive. Taken together, this suggests that in the engaged phase, active relief goes hand in hand

with temporary positive affect and avoidance motivation, whereas passive relief goes hand in hand with temporary positive affect and a shift toward approach motivation. As argued above, the effects on valence may be more pronounced for active than for passive relief. A special case are signaled safety periods in active avoidance, which are predicted to resemble passive relief but may exert stronger effects due to heightened attention and illusions of control (see process assumptions; Hommel, 2010; Blanco et al., 2011).

#### *Disengaged relief phase*

Once *p*(NStim|situation) falls below the fear threshold, the shift toward the disengaged phase has occurred. The process differences in the engaged relief phase extend their effects on affect and motivation to the disengaged phase. Generally, if the situational threat is eliminated at the beginning of disengagement, this goes along with a reduction in the activation of avoidance motivation, which would then allow *B* processes of approach motivation to dominate for a while. Passive relief in the engaged phase, however, already goes hand in hand with a reduction of negative affect and avoidance motivation, such that *B* processes already dominate during engagement. Therefore, at disengagement, passive relief will generate a weaker overshoot of *B* processes than active relief. In the latter case, avoidance motivation was in full activation during engagement, and consequently *B* processes too. If avoidance sets off at disengagement, *B* processes are still active at a high level.

#### **PREVENTING vs. STOPPING**

Theories, research procedures, and observations greatly differ with respect to whether the absence of negativity comes in the form of preventing or in the form of stopping NStim. IRMO adopts parts of Lohr et al.'s (2007) theoretical reasoning in assuming that prevention relief (i.e., an expected NStim does not materialize) and stopping relief (i.e., NStim is experienced but then ends) are associated with different processes. This is also in line with Gray and McNaughton (2000, p. 52) who concluded: "... we need to distinguish carefully between the primary events (Pun+, Rew−), on the one hand, and CSs for those primary events, on the other, since they can have quite opposite eliciting properties and functional requirements". As we will see, IRMO generates diverging predictions for affect and motivation as a function of prevention vs. stopping.

#### *Negative stimulation phase*

In the NStim phase (see **Figure 1**), prevention relief implies that the person has generated an expectation of NStim, be it based on the context or the presence of a CS+aversive. This requires anticipation processes based on learned associations, as well as appraisal processes that infer motive incongruence, and varying certainty depending on the predictive validity of the CS+aversive. Also, NStim is still appraised as uncontrollable through own behavior, although a search for such control-opportunities might set in. As a result, fear of NStim will be experienced (cf. Riebe et al., 2012), and, in line with the assumptions of OPT (Solomon, 1980; Leknes et al., 2008), processes opposing fear will set in. In the case of stopping relief, the experience (instead of the anticipation) of NStim (USaversive in **Figure 2**) represents the foundation of the NStim

phase. Depending on the type of NStim (e.g., food deprivation, noise, tissue damage), different sensations and emotions (e.g., hunger, frustration, pain) will result. Appraisals include motive incongruence, high certainty and low controllability of NStim. Also, opponent processes specific to the quality of the NStim will set in. In the NStim phase, prevention relief therefore differs from stopping relief in that it presupposes anticipation processes, and involves mainly fear and varying certainty, whereas stopping relief involves no anticipation, diverse negative sensations and emotions, and high certainty appraisals. However, in this phase, both forms of relief go along with negative affect and avoidance motivation.

#### *Engaged relief phase*

In the case of prevention relief, CS−aversive or S<sup>D</sup> relief signal that a previously expected NStim will not occur. The comparison between past (*p*(NStim|situation)) and present (*p*(NStim|(CS−aversive ∪ Rrelief)) expectancies of NStim will yield a reduction in fear that is proportional to the drop in expectancies. That is, prevention relief corresponds to a reduction or stopping of fear of USaversive (cf. Riebe et al., 2012). Stopping relief, however, is based on the actual experience of a USaversive, which then stops. For this to occur, no expectations and no previous experience with the stimulus or the general situation are necessary – the NStim may simply end (cf. Leknes et al., 2008). The comparison process between past and present experience of USaversive will yield a drop in negative affect proportional to the drop in USaversive.

There are three important predictions derived from the process-differences outlined above. First, prevention relief comes, on average, with a greater degree of uncertainty than stopping relief. This follows from the notion that in the case of stopping relief, the desired end-state (i.e., the reduction or stopping of NStim) is experienced and thus factual, whereas in the case of prevention relief, the desired end-state is only detected based on counterfactual reasoning (i.e., the observation that the expected NStim does not materialize). Detecting the validity of a CS−aversive presupposes a highly accurate representation of the typical timing of USaversive. For example, in order to be certain that a local anesthetic at the dentist's office really prevents pain, knowledge about when exactly pain can be expected during the treatment is necessary. If such knowledge is absent or imprecise, a residual fear that the CS−aversive did not work may prevail until the treatment is over. Detecting the offset of a USaversive, on the other hand, is clearly perceivable. For example, if a dentist applies a local anesthetic to stop a toothache, this will result in a drop in pain that in itself is 100% certain. As a consequence, stopping relief as compared to prevention relief will be associated with stronger inhibition of negative affect and stronger corresponding positive affect resulting from *B* processes. Likewise, a stronger decrease in avoidance motivation and a stronger corresponding increase in approach motivation can be expected for stopping compared to prevention relief as long as it is not active (see previous section).

Second, the differences in certainty and perceivability of prevention vs. stopping relief can also be expected to correspond to a difference in the speed of change in subjective probabilities of NStim. More specifically, the offset of NStim will correspond to a sudden decrease in the subjective probability of NStim, whereas the uncertainty that goes along with the ambiguity of the negation of an expectation in prevention relief will result in a more gradual change in the subjective probability of NStim. OPT mechanisms suggest that sudden decreases in negative *A* processes result in a stronger dominance of positive *B* processes than gradual decreases, and data strongly support this conclusion (Leknes et al., 2008). Thus, stopping relief can be expected to result in a more sudden offset of negative affective *A* processes and therefore a stronger overshoot in positive affective *B* processes than prevention relief. For the same reason, a stronger overshoot of approach motivation in the case of stopping instead of prevention relief can be expected, but only if it was not active (see previous section).

Third, elicited counter-regulatory processes as described in OPT can be expected to differ profoundly for prevention vs. stopping relief. In the case of stopping relief, the person has endured *A* processes related to factual NStim (e.g., pain, shame) for some time. Consequently, *B* processes specific to this NStim (e.g., activation of the endogenous opioid system in the case of pain; Leknes et al., 2008, p. 800), and the quality of these processes feeds into the experience of relief after the reduction or stopping of NStim. In the case of prevention relief, the feared NStim (e.g., pain) is not immediately experienced prior to a CS−aversive or prior to avoidance behavior. Instead, it is rather the fear of the NStim (e.g.,fear of pain) that constitutes the *A* process, and therefore opponent processes to fear act as the *B* process that uniformly shapes prevention relief. In other words, this reasoning results in the prediction that stopping relief will be psychologically and physiologically quite diverse depending on the nature of the NStim (and hence the nature of resulting *B* processes), whereas prevention relief will be uniformly present as relief from fear, independent of the NStim that is feared.

#### **INDIVIDUAL DIFFERENCES**

The process assumptions of IRMO provide a map for potential precursors of individual differences in relief. Such individual differences may reside in cognitive, affective, and motivational variables. For example, factors biasing probability judgments (Gilovich et al., 2002) can be predicted to affect relief. In the NStim phase, a bias toward increased expectancies of NStim will later increase relief. In the engaged relief phase, a bias toward increased expectancies of NStim will decrease relief unless the prevention or stopping of NStim is rendered fully certain by situational or internal factors (e.g., destruction of a threatening object; full behavioral control over threat). In IRMO, relief is assumed to result from an interplay of negative, avoidance related *A* processes and positive, approach related *B* processes. Given that *B* processes are predicted to be partially determined by the intensity of *A* processes, variability in trait negative affectivity as well as avoidance motivation can be expected to positively correlate with relief. At the same time, variability in trait positive affect and approach motivation can also be expected to correlate with relief, given that they influence the arousability of *B* processes in the realm of prevented or stopped negativity.

Moreover, stronger avoidance motivation might lower an individual's fear threshold, resulting in more frequent and intense fear, and accordingly, in more frequent and intense prevention relief due to the operation of opponent processes. Alternatively, stronger avoidance motivation might manifest itself in higher perceived probabilities of NStim. A final possibility is that stronger avoidance motivation is associated with a more efficient search for safety signals or appropriate avoidance behaviors in the NStim phase (cf. Shah and Higgins, 2001). People may also differ in their perceptual sensitivity toward specific NStim. A higher sensitivity can be expected to result in higher levels of relief.

# **EVIDENCE**

In what follows, we will review evidence that is informative on the validity of some predictions derived from IRMO. As we will see, however, the existing evidence rarely is based on experimental comparisons of the critical procedural features of relief that, according to our framework, will correspond with different processes and hence different motivational orientations. As such, we are almost exclusively dependent on cross-experimental comparisons. Another issue rendering existing evidence ambiguous is the lack of agreed-upon inductions of relief and measurements of motivation and affect in relief situations. For the present review, we included studies pursuing one of two research strategies. The first is a situation-based strategy that experimentally creates relief situations (see **Table 3** for an overview). Experimental paradigms to induce the offset of experienced or the prevention of expected NStim include various learning protocols, such as differential aversive conditioning, where a reinforced CS+ signals NStim, whereas a non-reinforced CS− signals the absence of NStim. Some theorists therefore refer to this situation as conditioned relief (Gray and McNaughton, 2000, p. 55).

Besides situational inductions based on real hedonic experiences in the experimental setting (e.g., pain, noise), some studies rely on the mere imagination of relief situations (e.g., Idson et al., 2000), thereby modeling general imagination-based techniques of emotion induction (Lench et al., 2011). Often, studies using this first strategy do not include additional measures of subjective experiences, but instead investigate other correlates of relief. For example, Leknes et al. (2013) were interested in how the context in which pain relief was experienced affects the hedonic quality (positive vs. negative) of relief. To achieve this, participants experienced the offset of a heat stimulus in various contexts, and subjective ratings of hedonic pleasantness as well as various biological measures were sampled. However, no direct ratings of subjective relief were taken. Instead, the presence of relief was assumed based on the strong situational induction (see **Figure 3A**). Similar assumptions are necessary in animal studies, where the presence of relief is mainly inferred from the situational conditions (e.g., Navratilova et al., 2012). The second research strategy relies on subjective measures of relief in neutral, experimental, or imagined relief situations (e.g., Ellsworth and Smith, 1988; Leknes et al., 2008), and may then use subjective ratings as predictors of other variables of interest (see **Figure 3B**).

We included studies using various operationalizations of affective valence and approach vs. avoidance motivation4. Measures of

<sup>4</sup>Following other approaches (Carver and Harmon-Jones, 2009), we refrain from further distinguishing facets of approach-avoidance motivation, such as moving toward vs. away or appetitive vs. aversive.

**Table 3 | Paradigms implemented in the investigation of situationally defined relief (non-occurrence of expected or cessation of actual negative stimulation or event).**


Aversive CS<sup>+</sup> refers to a stimulus which predicts the occurrence of a negative event/stimulation.

<sup>a</sup>Cessation of experienced NStim (stopping) vs. prevention of NStim (prevention); <sup>b</sup>relief independent of subject's behavior (passive) vs. relief caused by subject's behavior (active); <sup>c</sup>level of perceived certainty of the non-occurrence or cessation of NStim; <sup>d</sup>depends on the specifics of the conditioning procedure; <sup>e</sup>depends on the specifics of the behavior and on the time of measurement (i.e., during avoidance or after successful avoidance).

approach/avoidance motivation include the tendency to physically move toward a relief situation or away from it (see Krieglmeyer et al., 2013; Phaf et al., 2014), and the modulatory effects of relief situations on appetitive behavior (e.g., eating) or aversive behavior (e.g., fleeing) as in Pavlovian-instrumental transfer (PIT; e.g., Holmes et al., 2010). A similar variety of measures exists for stimulus valence. For example, the valence of relief situations has been assessed via self-report or its reinforcing effect on instrumental behavior.

Unfortunately, specificity for and sensitivity to valence vs. approach/avoidance motivation still remain unclear for some widely used measures. One example is eye-blink- and postauricular-startle modulation (Lang et al., 1990, 1992), which by some authors is classified as a measure of affective valence (e.g., Franklin et al., 2013b) and by other authors as a measure of motivational orientation (e.g., Lang et al., 1990; Peterson and Harmon-Jones, 2012) or the rewarding nature of a stimulus (e.g., Andreatta et al.,2013). Likewise, some brain structures [e.g., amygdala (AMY) and nucleus accumbens (NAcc)] are often interpreted as reflecting a specific affective valence (e.g., AMY and negative affect), or motivational functions (e.g., NAcc and reward processes). Yet some recent evidence casts doubt on a simple relation between activation of these structures and valence or motivation. For example, NAcc activity is often interpreted as a reward response, but there is some debate whether activation or deactivation reflects reward (e.g., Carlezon and Thomas, 2009). Furthermore, recent evidence suggests that NAcc activity generally codes motivational relevance or intensity (e.g., Jensen et al., 2003, 2007; Levita et al., 2009). Likewise, recent evidence suggests that different neuron-populations in the AMY code reward, punishment, and non-punishment (e.g., Genud-Gabai et al., 2013; Sangha et al., 2013). To deal with this issue, we will review results deriving from these measures separately in each of the following sections.

#### **CERTAINTY**

As outlined above, IRMO suggests that different levels of certainty relate differently to the affect and motivation associated with relief. Studies that were inspired by appraisal theories give an ambiguous picture of the association between the experience of relief and certainty, and are uninformative on the question of affective and motivational underpinnings. By and large, these studies suggest that relief sometimes goes along with appraisals of high certainty (Roseman, 1984, 2013; Frijda et al., 1989; Reisenzein and Spielhofer, 1994), sometimes with appraisals of low certainty (Roseman et al., 1990; Tong, 2015), or is unrelated to certainty (Ellsworth and Smith, 1988; Roseman, 1991). These diverging results may be due to the fact that often rather broad measures of certainty were taken, and that the affective vs. motivational facets of relief were not separated in the measures of subjective relief.

#### *Affective valence and startle modulation*

In an experimental study drawing on aversive Pavlovian conditioning, Andreatta et al. (2013) examined the subjectively rated affective valence of stimuli associated with threat (forward CS+) and stimuli associated with the situation-caused stopping of negativity (backward CS+) by using subjective ratings of valence as well as eye-blink startle modulation as dependent variables. Importantly, the stopping relief stimuli were either perfectly predicted by forward CS+, or occurred after painful stimulation that was not signaled by preceding stimuli. While the stopping relief could be predicted even before the onset of pain in the first condition, no such anticipation of relief was possible in the second condition. Results indicate that the predictable stopping relief stimulus was subjectively positive. Also, this stimulus decreased startle reactivity below baseline, whereas the unpredictable stopping relief stimulus was subjectively negative and increased startle reactivity. A study by Leknes et al. (2011) provides further evidence on the role of certainty for the affective valence of relief based on personality variables. More specifically, they observed a positive correlation between trait pessimism (a trait proxy for certainty of being punished) and (a) the anticipatory fear of being punished, and (b) the subjectively reported pleasantness of relief after passive pain relief. These observations support the theoretical notion that certainty of being punished increases anticipatory negative affect, and that the intensity of anticipatory negative affect positively influences the intensity of relief.

#### **PREVENTION vs. STOPPING**

For the engaged relief phase, the following expectations can be derived from IRMO: (A) stopping compared to prevention relief goes along with stronger approach relative to avoidance motivation; (B) stopping compared to prevention relief goes along with more positive relative to negative affect. Typical experimental inductions of prevention relief are presenting CS− in aversive Pavlovian conditioning, or training participants in instrumental avoidance behavior. Typical experimental inductions of stopping relief involve applying a pain stimulus and then removing it, either by or without participants' behavioral intervention. Dependent variables are either measured immediately after removal of the NStim or in response to stimuli that were systematically paired with the experience of stopping relief. Unfortunately, we did not find any empirical studies informative on potential differences in certainty appraisals as a function of prevention vs. stopping relief. We therefore have to focus on measures of affective valence, motivational orientation, and measures probably tapping into both valence and motivational orientation.

#### *Affective valence*

A relatively large literature on prevention relief concerns the valence of CS−aversive (Rescorla, 1969; Gray, 1971; Savastano et al., 1999). Many studies of this kind suggest that CS−aversive are evaluated more favorably than CS+aversive (e.g., Baas et al., 2002). However, studies comparing CS−aversive to a neutral control condition rather indicate that CS−aversive just became less aversive, but not more positive than control (Lipp et al., 2003; Mallan and Lipp, 2007). This is also supported by the results of an animal study by Fernando et al. (2013), who found safety stimuli to be just as rewarding as control stimuli, while at the same time being less rewarding than appetitive stimuli. Human studies using hypothetical relief from monetary punishment point into a similar direction (Idson et al., 2000). A study by Andreatta et al. (2012), however, suggests that CS−aversive were positive in the sense that they were rated above the scale midpoint, but at a similar level as control stimuli that were never presented during learning. As such, the above-midpoint rating may not reflect a learning-based increase in positivity in CS−aversive, but rather context-effects during test. With stopping relief, there is more evidence in favor of positive valence. For example, Zanna et al. (1970) observed that words associated with the stopping of electro shocks were evaluated more positively than baseline. Leknes et al. (2013) observed that a reduction in pain was evaluated as positive above a well-defined neutral anchor of visual analogue scales. A similar result was observed for stimuli associated with the offset of pain in one study (Andreatta et al., 2013). Yet, there is also contradictory evidence. Two studies yielded more negative evaluations compared to a pre-conditioning baseline for stimuli associated with stopping-relief (Andreatta et al., 2010, 2012), and another study observed a decrease in positive as well as negative self-reported affect after pain offset (Bresin and Gordon, 2013).

#### *Motivational orientation*

One method to study how prevention relief affects approach/ avoidance motivation draws on PIT (Rescorla and Solomon, 1967). For example, Rescorla and Lolordo (1965) trained dogs to differentiate between danger- and safety-stimuli in the context of receiving electric shocks. Also, the dogs trained behavior instrumental to avoid shock. In a test phase, danger and safety stimuli were presented during the instrumental avoidance

behavior. As a result, danger stimuli increased whereas safety stimuli decreased instrumental avoidance behavior. Apparently, the safety stimuli had acquired the potency to suppress fear and/or avoidance motivation. Many other studies support the notion that safety signals reduce instrumental avoidance behavior (e.g., Arcediano et al., 1996), and may increase appetitive instrumental behavior (Ray and Stein, 1959). Other evidence, however, suggests that this latter effect is presumably weak and highly moderated (e.g., Hoffman and Fleshler, 1964; Hammond, 1966). Evidence drawing on different methods to assess the motivational nature of prevention relief adds to this ambiguity. While some studies bolster the notion that safety stimuli boost appetitive motivation (Bromage and Scavio, 1978), other studies again suggest that this is highly moderated (DeVito and Fowler, 1994). Additionally, there is evidence that safety stimuli induce approach motivation in the sense that animals develop a preference for the place of their occurrence (Rogan et al., 2005) and are faster to run toward prevention relief signals (Haraway et al., 1984).

Observations regarding the motivational properties of stopping relief are often based on a backward conditioning paradigm. In such studies, stimuli are presented together with the offset of NStim. Experimentally, such stopping relief stimuli gain the power to inhibit avoidance behavior (Moscovitch and LoLordo, 1968; Grelle and James, 1981; Cole and Miller, 1999), and are also approached by animals (Tanimoto et al., 2004; Yarali et al., 2008). Stopping relief might also facilitate appetitive behavior. Drawing on animal research subjects,Amsel and Maltzman (1950) observed increased drinking behavior after stopping relief compared to baseline. Similar observations were made in other studies (Davis et al., 1976; Walasek et al., 1995), but another study found stopping relief stimuli to be no more appetitive than neutral control stimuli (Krank, 1985).

#### *Startle modulation*

Mirroring the observations with subjective valence measures, numerous studies indicate that stimuli associated with prevention relief are less aversive than threat stimuli, but still more aversive than baseline (e.g., Hamm et al., 1993; Falls and Davis, 1995; Lipp et al., 2003; Josselyn et al., 2005; Jovanovic et al., 2006; Mallan and Lipp, 2007; Weike et al., 2008). For stopping relief, available evidence provides a different picture. For example, Franklin et al. (2013a) observed that post-auricular startle was enhanced and eyeblink startle was reduced after pain off-set compared to baseline. Given that post-auricular startle may be interpreted as an indicator of approach motivation or positive valence (Benning et al., 2004; Gable and Harmon-Jones, 2009), this suggests that stopping relief is more than a mere reduction in aversive motivation or negative valence, but instead shifts the affective-motivational tone toward positive or approach. Similar findings were obtained in a number of other studies (Leknes et al., 2008; Andreatta et al., 2010, 2012, 2013; Franklin et al., 2010, 2013b).

#### *fMRI studies*

Leknes et al. (2011) investigated brain-activation patterns associated with prevention relief and reward. Functional magnetic resonance imaging (fMRI) analyses demonstrated some brain regions (ventromedial prefrontal cortex, rostral anterior cingulate) to jointly respond to prevention relief and imagined reward, whereas other regions were either specific to relief (e.g., right anterior insula, NAcc) or to imagined reward (e.g., posterior cingulate). This suggests that prevention relief shares some processes with reward but also has some distinct features. A similar conclusion can be drawn from a study by Genud-Gabai et al. (2013), who observed that neuron populations in the AMY not only respond to fear stimuli, but likewise to safety stimuli in a prevention setting. Sangha et al. (2013) compared neural responses to stimuli signaling shock, safety, or reward (sucrose). They observed that about 18% of the recorded neurons responded to both threat and prevention relief signals. There were two other neuron populations in the basal AMY, one responding selectively to prevention relief signals, another one selectively firing to prevention relief and to reward signals. This pattern may be interpreted as further evidence for the ambivalent nature of the reactions triggered by prevention relief signals, overlapping with reactions to threat stimuli (aversive component), and reward stimuli (appetitive component), and including a component idiosyncratic to prevention relief. A recent study drawing on experimentally inflicted pain and pain relief in animals suggests that stopping relief corresponds with dopamine release in the NAcc (Navratilova et al., 2012), which is often interpreted as implying a rewarding nature. Corroborating this finding, Andreatta et al. (2012) observed increased activity in the ventral striatum in human participants in response to CS associated with pain offset (i.e., stopping relief) compared to control stimuli.

#### **ACTIVE vs. PASSIVE**

The following expectations can be derived from IRMO for the engaged relief phase: (A) Active relief goes along with high avoidance motivation, passive relief results in reduced avoidance motivation and possibly increased approach motivation; (B) Outcomes of active relief that signal a safety period induce processes similar to passive relief; (C) Active and passive relief are of positive affective valence through opponent processes, with active relief possibly being more positive due to enhanced attention and controllability appraisals; (D) Actively generated CS−aversive inhibit negative affect and avoidance motivation more strongly than passively learned CS−aversive. Based on OPT, this will result in more positive affect and approach motivation. Affective and motivational effects of passive relief may be measured during the experience of the offset of NStim, or in response to stimuli associated with prevented or eliminated NStim. Active relief, on the other hand, can be studied in operant settings where participants learn to avoid negativity through engaging in specific behaviors. A simpler variant excludes learning processes, such as in active pain relief where participants can simply stop the application of pain, for example by voluntarily removing their hand from ice water. As with prevention vs. stopping, often cross-study comparisons are necessary to evaluate possible differences between the two types of relief.

#### *Affective valence*

Numerous studies show that passive prevention relief is less negative than threat, albeit still negative when compared to baseline (e.g., Mallan and Lipp, 2007). Passive stopping relief, however, sometimes turned out to be more positive than a neutral baseline (Andreatta et al., 2013) although the evidence is mixed (Andreatta et al., 2010; Bresin and Gordon, 2013). Evidence suggests SD relief as well as self-produced safety signals to be of positive valence, or at least to decrease negative affect. Murray and Strandberg (1965) observed that self-produced safety signals had reinforcing properties in rats, andKinsman and Bixenstine (1968) demonstrated such reinforcing effects to be stronger for self-produced safety signals than for SD relief . This difference is in line with the predictions of IRMO.A study by Eder and Dignath (2014), which compared stimuli associated with passive and active (prevention) relief, observed that active and passive relief result in different levels of positivity. Colors which signaled the successful outcome of active avoidance behavior were rated as positive on both an explicit and an implicit measure of valence, whereas colors which signaled passive relief were rated as positive on an explicit, but not an implicit measure of valence. Accordingly, this study indicates that the outcome of active prevention relief might be more consistently positive than CS−aversive. Similar results were obtained by Niznikiewicz and Delgado (2011), who observed greater positivity along with higher emotional intensity for active relief compared to passive relief on an explicit self-report measure.

#### *Motivational orientation*

The integrative relief model assumes that, during the engaged relief phase, CS−aversive will inhibit avoidance motivation, whereas SD relief will not. The inhibiting effect of passive relief stimuli is attested by the inhibiting effect of passive and self-generated CS−aversive on instrumental avoidance behavior (e.g., Rescorla and Lolordo, 1965; Berger and Starzec, 1988; Arcediano et al., 1996). However, IRMO suggests actively produced safety signals to have stronger inhibiting effects than passive relief stimuli. Relevant evidence for this prediction comes from a study by Cándido et al. (2004), who compared the effects of stimuli signaling successful avoidance and pf passive relief stimuli on the intensity of an independent, secondary fear response. Results suggest that actively produced safety signals suppressed fear more intensely than passive relief stimuli.

The integrative relief model also predicts active relief to go along with high avoidance motivation, whereas passive relief is predicted to reduce avoidance motivation. Supporting evidence for the first part of the prediction comes from studies by Weiss and Schindler (1989) and Weiss et al. (1996). In one study (Weiss et al., 1996), rats in one condition learned that a first discriminative stimulus (e.g., a clicker) signaled that they could gain food by pressing a bar, whereas a second discriminative stimulus (e.g., a tone) signaled that they could postpone an aversive shock by pressing on the bar. Importantly, in a test phase, both discriminative stimuli were presented simultaneously. Results indicate that the compound of an active relief and an active joy (i.e., reward attainment) stimulus resulted in decreased bar pressing compared to a compound of two joy or two relief stimuli. Thus, while the motivational power of two stimuli with identical incentives (joy/joy; relief/relief) added up, this was not the case for mixed joy/relief stimuli. This suggests that the possibility to avoid shock by bar pressing and the possibility of gaining a reward through bar pressing were associated

with the activity of different motive systems, providing indirect support for a lack of inhibition of avoidance motivation through active relief. Similar results were reported by Weiss and Schindler (1989).

Frijda et al. (1989) observed that participants who were asked to recall a situation in which they had experienced relief rated the situation as high in self-agency, an appraisal dimension we associate with active relief, and as high on the motivation to approach.While this may imply that active relief will activate an approach motivational orientation, it should be emphasized that the methodology of the study does not allow for a certain statement about the phase in the relief process that participants' appraisal of approach motivation refers to, nor about whether it refers to a safety period established by active avoidance or not.

#### *Startle modulation*

Regarding the modulation of eye-blink startle, evidence suggests passive prevention relief to still have aversive qualities, albeit being less aversive than NStim (e.g., Mallan and Lipp, 2007), while passive stopping relief was positively associated with approach or positive valence. A study which investigated active stopping relief also observed decreased eye-blink startle reactivity relative to baseline, indicating a strong reduction in negative affect or avoidance motivation by successful active stopping relief (Franklin et al., 2010). Although IRMO does not predict a reduction in avoidance motivation through active relief, these results are still compatible with IRMO under the assumption that the period after pain cessation was experienced as a safety period by participants. These self-produced safety periods or safety signals are predicted to reduce negative affect and avoidance motivation.

#### *fMRI studies*

A study by Levita et al. (2012) examined activity in the AMY and the NAcc in response to situations in which participants could actively avoid NStim or NStim did not occur when participants remained passive. Results indicate that active avoidance caused an increase in (primarily right) NAcc activity, whereas passive avoidance caused a decrease. Moreover, state anxiety predicted NAcc activation and deactivation. A similar pattern was observed for (primarily right) AMY activation, which was increased in active but decreased in passive avoidance (for a comparable finding, see Delgado et al., 2009). Similarly, Schlund et al. (2010) observed AMY activation to be increased during active avoidance compared to a neutral control, and the intensity of avoidance behavior to be positively correlated with AMY activity. Relatedly, Kohls et al. (2013) observed increased NAcc activation when participants prepared to avoid negative outcomes compared to a control condition and compared to the situation when the negative outcome was finally avoided. Typically, NAcc activity is interpreted as a reward response, but recent evidence is more compatible with the notion that NAcc activity generally codes motivational relevance or intensity (e.g., Jensen et al., 2003, 2007; Levita et al., 2009). This also better fits the dissociation between active and passive avoidance observed in Levita et al. (2012). From this perspective, the most conservative interpretation of these studies would imply stronger motivation in the case of active vs. passive relief.

Although it does not bear on the comparison between active and passive relief, a study by Kim et al. (2006) is informative regarding a distinction in IRMO between two different steps in the process of active relief during the engaged relief phase, namely appraising that an avoidance behavior can be performed in response to the expectancy of NStim, and the appraisal that the executed avoidance behavior reduced the expectancy of NStim. According to IRMO, the latter should be associated with more positive affect than the former, since the expectancy of NStim is further reduced as a result of successful avoidance. Kim et al. (2006) used a choice task in which participants could increase their chances of attaining a reward (a monetary gain) or avoiding a negative outcome (a monetary loss) by making the correct choice between two stimuli. They found that successfully avoiding a negative outcome in the choice task increased neural activity in the medial orbitofrontal cortex, a region associated with encoding the reward value of stimuli (Kim et al., 2006), just like actively attaining a reward, indicating that successfully avoiding a negative outcome, just like successfully attaining a positive outcome, is rewarding. However, a different picture emerged for brain activity at the time of choice, i.e., when avoiding NStim (or attaining a reward) was possible, but no feedback about the success of the avoidance (or reward attainment) was yet received. Here, activity in regions found to correlate with the expectation of a future rewarding outcome (the medial and lateral orbitofrontal cortex) decreased over time for trials in which a negative outcome could be avoided, while activity in regions associated with expectations of future aversive outcomes (the right dorsolateral prefrontal cortex and the anterior cingulate cortex) increased over time. The reverse pattern was found for trials in which a positive outcome could be achieved. These results indicate negative affectivity during an avoidance process prior to feedback about the outcome of avoidance behavior, and positive affectivity after the receipt of feedback indicating successful avoidance.

#### **INDIVIDUAL DIFFERENCES**

Some studies provide evidence on the relation between relief and individual difference variables. Based on OPT mechanisms, IRMO suggests an association of relief with both trait avoidance and trait approach. In line with this assumption, one study observed the experience of relief to be associated with an individual's chronic level of avoidance motivation. Specifically, Higgins et al. (1997) observed that avoidance motivation increased the impact of NStim on the frequency and intensity of quiescence-related emotional experiences (which include relief). Similar results were obtained for chronic avoidance motivation and failure on a task (Idson et al., 2000) or outgroup members (Shah et al., 2004) as NStim. Additional studies investigating chronic as well as situationally induced avoidance motivation observed similar results (e.g., Higgins and Tykocinski, 1992; Shah, 2003; Falomir-Pichastor et al., 2008;Yi and Baumgartner, 2008; Adams et al., 2011; Falomir-Pichastor et al., 2011). However, there are also a few studies that did not show this pattern or even a contradictory mapping of emotional tone and motivational orientation (Faddegon et al., 2008; Yi and Baumgartner, 2009; Winterheld and Simpson, 2011; McKay-Nesbitt et al., 2013).

Other studies also provide evidence on the role of approach related dispositions. In two studies, Carver (2009) measured BAS and BIS strength and assessed their impact on felt relief by using a scenario technique. Results revealed that the intensity of relief was positively correlated with both the strength of the BIS, as well as with one subscale of the BAS, namely reward responsiveness, but that the association between the BIS and relief was stronger than the association between reward responsiveness and relief, thereby supporting the dual nature of relief as predicted by the OPT assumptions of IRMO. Another study drawing on a conditioned inhibition paradigm even suggests prevention relief to be solely related to BAS reward responsiveness and no other components of the BIS/BAS questionnaire (Migo et al., 2006). However, some studies have failed to find associations between the experience of relief or other quiescence-related emotions and chronic approach or avoidance motivation (Leone et al., 2005; Yen et al., 2011). Leknes et al. (2011) measured individual differences in the subjective pleasantness of appetitive reward scenarios. There was a substantial positive correlation between the pleasantness of actual pain relief and the pleasantness of appetitive reward, further corroborating a link to approach related dispositions. Leknes et al.'s (2011) study is also informative as to the role of trait variables related to probability estimates and to the fear threshold. As would be expected, trait pessimism was positively correlated with relief and with acute dread. Interestingly, pessimism and dread did not correlate with appetitive reward.

## **IMPLICATIONS FOR THEORY AND RESEARCH**

We started out by describing how theories of relief converge and differ regarding the affective valence, and motivational orientation associated with relief (cf. Carver, 2009). Some theories suggest relief to be part of the BAS (e.g., Gray, 1987; Gray and McNaughton, 2000), whereas other theories conceptualize relief as a positive emotion of avoidance processes (e.g., Carver and Scheier, 1990, 2002; Carver, 2001). We proposed an integrative model, IRMO, that combines process assumptions and conceptual distinctions from a number of existing theories. Based on IRMO, we derived a number of predictions regarding the moderating nature of features related to the relief situation (certainty, active vs. passive, prevention vs. stopping) on the affective and motivational nature of relief. Unfortunately, systematic empirical research on moderators is rather scarce. Nevertheless, a preliminary evaluation of the validity of some of IRMO's predictions is possible, mostly based on cross-study comparisons though.

#### **PREVENTION vs. STOPPING**

The integrative relief model predicts stopping relief to trigger more positive affect and approach motivation than prevention relief. The rationale for this prediction was that stopping relief has a strong experiential component of the factual offset of NStim, whereas prevention relief does not. This may also correspond with the fact that stopping always entails a proof of being factual, whereas prevention relief is anticipatory and may still come with some degree of uncertainty. In line with this reasoning, reviewed evidence on the valence of CS−aversive as a proxy for prevention relief suggests that while such stimuli might be rated as being more positive than CS+aversive (Baas et al., 2002), CS−aversive seem to show little positivity and approach motivation compared to neutral control conditions, and fMRI evidence points to the possibility that CS−aversive activate representations of punishment and reward at the same time. With stopping relief, however, the reviewed evidence implies different regularities. Drawing on startle modulation as a dependent variable, evidence suggests that eye-blink startle reactivity decreases below baseline during stopping relief or in the presence of stimuli associated with stopping relief. Moreover, post-auricular startle – a marker of positive affect or approach motivation – as well as activation in reward associated brain regions increased during stopping relief. This evidence, although relying on crossexperimental comparisons, is supportive of the predictions derived from IRMO.

Evidence drawing on the modulation of instrumental behavior is less clear, however. There clearly is evidence that passive prevention relief decreases instrumental avoidance behavior, and, perhaps to a weaker degree, increases instrumental approach behavior. There is also clear evidence that stopping relief inhibits avoidance behavior and facilitates approach behavior. Both main effects are compatible with IRMO, as the mechanisms predict both prevention and stopping relief to reduce avoidance motivation and hence, by means of OPT, increase approach motivation. The more distinctive prediction (i.e., stronger shift toward approachfor stopping), however, presupposes a within-experiment manipulation of prevention vs. stopping relief, which, according to our search, is still missing.

#### **ACTIVE vs. PASSIVE**

The integrative relief model predicts that during the engaged relief phase, both active and passive relief will be associated with an increase in positive affect, with a potentially stronger effect for active relief. Active and passive forms of relief are expected to differ, however, with regard to their association with approach and avoidance motivation. Whereas passive relief is predicted to go hand in hand with a shift toward approach motivation, this is not expected to be the case for active relief. Rather, active relief is assumed to involve the activation of avoidance behavior, and a strengthening of this behavior if avoidance successfully reduces the expectancy of NStim. The exception to these predictions, however, are signaled safety periods produced as a result of active avoidance, which are assumed to exert the same effects as passive relief – i.e., increase in positive affect and approach motivation – albeit to a stronger degree due to processes such as illusions of control which favor actively produced over passively endured outcomes. The results reviewed in this article support these predictions for the most part. While there is evidence for the positivity, or at least decreased negativity, of both passive and active relief, successful active relief was indeed found to be more consistently positive than passive relief (Eder and Dignath, 2014). Cross-experimental comparisons appear to further support this point, as stimuli associated with successful active relief were found to reinforce the acquisition of a novel response (Murray and Strandberg, 1965; Kinsman and Bixenstine, 1968), whereas stimuli associated with passive relief were not (Fernando et al., 2013).

Regarding the effect of active and passive relief on approach and avoidance motivation, passive relief was indeed found to be associated with a decrease in avoidance motivation (Rescorla and Lolordo, 1965), whereas ongoing active relief was not (e.g., Weiss et al., 1996). Moreover, successful avoidance strengthens the avoidance behavior that led to the avoidance or escape from NStim (Dinsmoor, 2001). During the presence of stimuli associated with a self-produced safety period, however, avoidance motivation was reduced to a stronger degree than during the presence of stimuli associated with passive relief (Cándido et al., 2004), as predicted by IRMO. While it comes with some ambiguity, one plausible interpretation of fMRI studies (Delgado et al., 2009; Levita et al., 2012; Kohls et al., 2013) is that active relief goes along with greater avoidance motivation than passive relief.

# **IMPLICATIONS FOR VALENCE- vs. GOAL-THEORIES**

The observed patterns of results are relevant for evaluating the viability of valence theories (e.g., Gray, 1987; Gray and McNaughton, 2000) and goal theories (e.g., Carver and Scheier, 1990, 2002; Carver, 2001) of emotion for explaining the affective and motivational underpinnings of relief. These classes of theories uniformly associate relief with positive affect, but valence theories associate relief with approach motivation, whereas goal theories associate relief with avoidance motivation (Carver, 2009). As our review demonstrated, neither prediction fully matched available evidence. Some studies indicate that relief stimuli are of negative valence and avoidance motivation albeit less so than fear signals (e.g., Hamm et al., 1993; Falls and Davis, 1995; Lipp et al., 2003; Josselyn et al., 2005; Jovanovic et al., 2006; Mallan and Lipp, 2007; Weike et al., 2008), whereas other studies suggest that relief and associated stimuli are more positive than baseline or control stimuli and associated with approach motivation (e.g., Dinsmoor and Sears, 1973; Andreatta et al., 2010, 2013; Franklin et al., 2013a; Eder and Dignath, 2014). fMRI based studies demonstrate on the one hand some overlap of relief and reward (Kim et al., 2006; Leknes et al., 2011; Genud-Gabai et al., 2013; Sangha et al., 2013). At the same time, relief goes along with brain-activation that is specific for relief and independent from reward (Leknes et al., 2011; Genud-Gabai et al., 2013; Sangha et al., 2013), and some neuron populations respond to both threat and relief at the same time (Kim et al., 2006; Genud-Gabai et al., 2013; Sangha et al., 2013). Studies drawing on personality measures corroborate a heterogeneous nature of relief when it comes to motivational orientations, suggesting that it contains both approach and avoidance components (Migo et al., 2006; Carver, 2009). Clearly, relief is neither only positive, nor only negative. And it does not unambiguously match with approach or avoidance. This suggests that goal- and valence-theories might profit from extensions so that they can accommodate the more differentiated empirical patterns observed so far and – optimally – generate novel predictions. IRMO can be interpreted as such an attempt.

# **RELATION TO FEAR, FRUSTRATION, AND HOPE**

Relief is not the only emotion showing complex relationships with affect and motivational orientation. A similar picture emerges for emotions resulting from frustration situations such as anger or sadness. Frustration situations are situations in which an expected or experienced positive stimulation is reduced or absent (cf. Dollard et al., 1939; Berkowitz, 1989; Papini and Dudley, 1997). As such, frustration situations are the mirror image of relief situations. Clearly, emotions resulting from frustration situations such as anger and sadness have negative valence. Whether frustration situations are associated with an approach or avoidance motivation, however, is less clear. On the one hand, empirical evidence indicates a relation between anger and approach motivation (Carver and Harmon-Jones, 2009). On the other hand, frustration situations have been shown to trigger avoidance behavior in animal studies (Papini and Dudley, 1997). As with relief, various moderators may determine which motivational orientation is triggered by frustration situations. In particular, appraised control or coping potential determines the nature of emotional responses to frustration situations (Wortman and Brehm, 1975; Roseman, 2001; Smith and Kirby, 2001) and may thus also determine whether an approach or avoidance motivation is activated. For instance, research investigating hemispherical lateralization as an indicator of motivational orientation has shown that coping potential moderates the extent to which anger-inducing situations elicit an approach motivation (Harmon-Jones et al., 2003). Furthermore, appraisals of agency influence the extent to which anger arises (Roseman, 1991) and may thus influence the extent to which approach motivation is triggered. However, empirical evidence on the moderating influence of appraisals on motivational orientation is scarce. If one were to apply IRMO to frustration, one could conceptualize the appraisal of high coping potential as a situation where the probability of attaining a positive outcome is higher when performing a behavior than when not performing the behavior, in analogy to active relief.

While frustration, anger and sadness are mirror images of relief, fear is one potential precursor of relief. IRMO conceptualizes prevention relief as a reduction in fear, more specifically a reduction in the probability of experiencing NStim conditional on the availability of safety signals or avoidance responses. Prevention relief therefore goes hand in hand with a reduction in fear. Note, however, that IRMO suggests that the avoidance motivation triggered by fear will decrease through passive relief. But IRMO also suggests the avoidance motivation to continue even after reduction in fear for active relief during the engaged relief phase. As already hypothesized in several theories (e.g., Ortony et al., 1988; Reisenzein, 2009), we predict that the intensity of relief will be related to the intensity of antecedent fear. IRMO offers several reasons why this might be the case. First of all, more intense fear will result in a stronger *A* process, which will result in a stronger *B* process, and consequently larger residual activity of the *B* process in the disengaged relief phase. Moreover, more intense fear, conceptualized as a higher perceived probability of NStim, will make relative relief more likely. This is expected to be the case because even if safety signals or the availability of avoidance responses are associated with a probability of NStim that is still high in absolute terms, the difference between this probability and the probability of NStim in the absence of safety signals or avoidance responses might be rather large in the case of intense fear. Even in the case of stopping relief, which is driven by actual instead of by expected NStim, the possibility that pain might return could trigger anxiety, which will motivate the search for relief cues or adequate avoidance responses during the engaged relief phase. IRMO predicts a complete absence of fear and anxiety only in the disengaged relief phase.

Finally, hope is an emotion that might be considered in relation to relief. Based on Roseman (1984, 2013) one might consider that the degree of certainty of non-punishment corresponds to emotions ranging from fear (very uncertain nonpunishment/somewhat uncertain punishment) over hope (intermediate certainty of non-punishment/punishment) to relief (certain non-punishment/no chance of punishment). From Roseman's perspective, then, not all facets of relief as described in IRMO would actually be labeled relief. More specifically, this label would only apply to the disengaged relief phase, where conditional and unconditional threat of NStim is low. On one hand, this may be seen as an issue of labeling. One might simply decide to label the inner responses during engaged relief as hope. At the same time, this perspective would be incompatible with theory and research suggesting the existence of relative relief (Leknes et al., 2013), and the feedback-function of relief during active avoidance (Carver and Scheier, 1990).

# **OPEN QUESTIONS AND AVENUES FOR FUTURE RESEARCH** *Systematic research on certainty*

The integrative relief model suggests that various certainty appraisals play an important role for the occurrence of relief. Coarsely, the certainty of NStim in the present context, as well as the conditional probability of NStim when safety cues or avoidance behavior are present, determine the three phases of relief. More specifically, IRMO predicts very specific relations between levels and changes of certainty appraisals, affect, and motivational orientation. To our knowledge, few studies have systematically manipulated or measured certainty, and we are not aware of studies that measure or manipulate all types of certainty appraisals that IRMO deems relevant for relief. At the same time, experimental manipulations of all underlying probabilities seem easily achievable and highly desirable at the same time.

#### *Systematic research on types of relief*

Although the present review provided some evidence for the importance of active vs. passive, and stopping vs. preventing relief, most of the conclusions were drawn from cross-study comparisons. While such comparisons are informative to some degree, they still suffer from serious threats to validity because of confounding factors. For example, stopping vs. prevention is often confounded with certainty, since it is usually quite apparent that NStim has ended, whereas NStim that has not occurred might still occur, rendering stopping relief more certain than prevention relief in a lot of cases. A potential way to solve this problem is to implement stimuli which signal that a NStim will end soon as stopping relief stimuli. This could be accomplished in a within subjects design in which subjects are presented with a NStim of a certain length in every trial, unless the trial is preceded by a safety signal (i.e., the prevention relief stimulus). Moreover, on some trials the NStim will end earlier than usual, namely a short time after the presentation of another stimulus (i.e., the stopping relief stimulus). After participants have learned the meaning of these

two stimuli, their valences and effects on approach and avoidance motivation can be measured by using the stimuli as target stimuli in appropriate measures (e.g., an affective priming task, a Manikin task).

Moreover, studies on the affective and motivational consequences of relief do not always include (a) baseline or control conditions, and (b) independent measuresfor positive affect/approach motivation and negative affect/avoidance motivation. If these measurements are missing, it is hard to evaluate whether relief goes along with decreased negativity/avoidance motivation or increased positivity/approach motivation. For sure, some models of affect and motivation assume strict negative correlations between positive/approach on the one hand and negative/avoidance on the other hand. It would still be informative to have the opportunity to test the strength of this assumption in all experiments.

# *Diverse negativity* = *diverse relief?*

The core of all definitions of relief is that something negative is prevented, stopped, or reduced. But negativity can come in many forms, which may result in differences in relief that follows these different sources of negativity. More specifically, based on the OPT

components of IRMO, the nature of the negative *A* process determines the nature of the positive *B* process, and the interplay of the two partially determines the character of a relief episode. What are potential differences in NStim? First, evidence and theory suggests that negative affect may result from approach processes, which occurs when goal-pursuit is blocked briefly (resulting in frustration and anger, e.g., Carver and Harmon-Jones, 2009) or prolonged (resulting in sadness or depression, e.g., Higgins, 1987; Carver, 2004; Roseman, 2013). Consequently, relief, as defined here, may include the prevention or stopping of frustration. As a consequence, the underlying *A* process would be negative and of approach motivation, whereas the resulting *B* process would be positive and of avoidance motivation. Experimentally inducing relief fromfrustration or anger seems possible, and testing its motivational nature would help further evaluating the viability of OPT assumptions in the realm of relief. Second, there are many specific and qualitatively different NStims, such as heat, bad smell, cold, or social rejection etc., each potentially associated with diverging *A* and *B* processes. For example, Leknes et al. (2008, p. 800) theorize "A putative neurobiological mechanism for the opponent process of pain is the endogenous opioid system". Would opioid

**Table 4 | Exemplary measures of valence and of approach and avoidance motivation implemented as dependent variables.**


release be the appropriate *B* process for hunger? Whereas pain may indeed trigger opioid reactions, hunger would go hand in hand with glycogenolysis or gluconeogenesis. From this perspective, the *B* processes associated with pain and hunger might partially differ, and hence different phenomenologies of relief from pain vs. relief from hunger might result. From this perspective, systematically studying differences of relief from different NStims would be a worthwhile endeavor.

#### *Other measures of motivation*

As documented in our review, studies on relief have drawn on an impressive number of research methods to assess affective valence and motivational orientation (see **Table 4**). However, one approach to measuring motivational orientations is surprisingly missing: cortical asymmetries as assessed by EEG recordings. Such asymmetries – both assessed in resting state as well as in response to emotion-relevant stimuli – proved to be a helpful piece of the puzzle of the motivational orientation underlying anger and sadness (Carver and Harmon-Jones, 2009). Assessments of cortical asymmetries come with the advantage of high temporal resolution and indirect measurement. Based on the experiences of research on sadness and anger, applying such measures to relief would be highly desirable.

#### *Trajectory of avoidance goals*

An interesting avenue for future research concerns the disengaged relief phase. On one hand, some theories of relief (e.g., Pekrun et al., 2002; Roseman, 2013) and general motivation (Förster et al., 2007) suggest that experiencing relief goes hand in hand with a deactivation of avoidance motivation or avoidance goals, and this view is also compatible with Carver's (2009) notion of motivational reorienting after a threat is eliminated. On the other hand, recent evidence suggests that goals may remain accessible in memory after goal fulfillment until they are replaced with alternative goals (Walser et al., 2012, 2014). If the latter perspective would also apply to relief, this would suggest that the disengaged relief phase is characterized by accessible avoidance goals that are only weakly shielded against competing goals. This leads to the interesting prediction of disengaged relief resulting in an increased propensity to re-engage in earlier avoidance goals as long as they have not been replaced by other goals primed by the organism or the environment.

#### **SUMMARY**

The present paper reviewed existing theory and evidence on the affective and motivational underpinnings of relief. The evidence suggests that relief is a heterogeneous phenomenon in that it can come with positive affect, negative affect, and ambivalent affect. Moreover, evidence suggests that relief may go along with dominant approach and dominant avoidance motivation. As such, the evidence is by and large incompatible with two broad classes of emotional theories that characterize relief as of positive valence, with valence theories linking relief uniformly with approach, and goal theories linking relief uniformly with avoidance. We also presented an IRMO that aims at integrating existing process assumptions regarding relief. It was designed to cover a large number of known effects regarding

relief. With respect to affect and motivation, IRMO pointed at variants of relief, that are characterized by active vs. passive avoidance as well as stopping vs. preventing NStim. IRMO suggests that these variants will determine the affective tone as well as the motivational nature of relief. As such, IRMO may help to understand existing variability in empirical evidence on affective and motivational underpinnings of relief. The reviewed evidence provides first support for the viablity of the process assumptions outlined in IRMO. At the same time, this evidence often fails to experimentally manipulate the theoretically important variables. Instead, our conclusions were typically based on cross-experiment comparisons. Therefore, conducting direct tests of the moderator predictions generated by IRMO is a desirable goal for future research. This may lead to some confirmations and possibly some disconfirmations of predictions generated by IRMO. In any case, we believe that such an edeavor will better our understanding of relief.

#### **ACKNOWLEDGMENTS**

RD, KJMS, Department of Psychology, Technische Universität Dresden, Germany; RK-F, Department of Psychology, University of Paderborn; RR, Department of Education and Psychology, Freie Universität Berlin. The preparation of the present paper was supported by Grant DE 1150/2-2 of the Deutsche Forschungsgemeinschaft (DFG) to RD. We thank Marta Andreatta and Paul Pauli for providing helpful comments on an earlier version of this manuscript.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 August 2014; accepted: 30 January 2015; published online: 10 March 2015. Citation: Deutsch R, Smith KJM, Kordts-Freudinger R and Reichardt R (2015) How absent negativity relates to affect and motivation: an integrative relief model. Front. Psychol. 6:152. doi: 10.3389/fpsyg.2015.00152*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology.*

*Copyright © 2015 Deutsch, Smith, Kordts-Freudinger and Reichardt. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

**REVIEW ARTICLE** published: 30 January 2015 doi: 10.3389/fpsyg.2015.00026

# The impact of perception and presence on emotional reactions: a review of research in virtual reality

# *Julia Diemer 1, Georg W. Alpers <sup>2</sup> , Henrik M. Peperkorn3 , Youssef Shiban1 and Andreas Mühlberger 1\**

<sup>1</sup> Clinical Psychology and Psychotherapy, Department of Psychology, University of Regensburg, Regensburg, Germany

<sup>2</sup> Chair of Clinical and Biological Psychology and Psychotherapy, Mannheim School of Social Sciences, University of Mannheim, Mannheim, Germany

<sup>3</sup> Department of Psychology I, University of Würzburg, Würzburg, Germany

#### *Edited by:*

Fritz Strack, University of Würzburg, Germany

#### *Reviewed by:*

Desmond Jay Oathes, Stanford University, USA Rosa M. Baños, University of Valencia, Spain

#### *\*Correspondence:*

Andreas Mühlberger, Clinical Psychology and Psychotherapy, Department of Psychology, University of Regensburg, Universitätsstrasse 31, 95053 Regensburg, Germany e-mail: andreas.muehlberger@ psychologie.uni-regensburg.de

Virtual reality (VR) has made its way into mainstream psychological research in the last two decades. This technology, with its unique ability to simulate complex, real situations and contexts, offers researchers unprecedented opportunities to investigate human behavior in well controlled designs in the laboratory. One important application ofVR is the investigation of pathological processes in mental disorders, especially anxiety disorders. Research on the processes underlying threat perception, fear, and exposure therapy has shed light on more general aspects of the relation between perception and emotion. Being by its nature virtual, i.e., simulation of reality, VR strongly relies on the adequate selection of specific perceptual cues to activate emotions. Emotional experiences in turn are related to presence, another important concept in VR, which describes the user's sense of being in a VR environment. This paper summarizes current research into perception of fear cues, emotion, and presence, aiming at the identification of the most relevant aspects of emotional experience in VR and their mutual relations. A special focus lies on a series of recent experiments designed to test the relative contribution of perception and conceptual information on fear in VR. This strand of research capitalizes on the dissociation between perception (bottom–up input) and conceptual information (top-down input) that is possible inVR. Further, we review the factors that have so far been recognized to influence presence, with emotions (e.g., fear) being the most relevant in the context of clinical psychology. Recent research has highlighted the mutual influence of presence and fear in VR, but has also traced the limits of our current understanding of this relationship. In this paper, the crucial role of perception on eliciting emotional reactions is highlighted, and the role of arousal as a basic dimension of emotional experience is discussed. An interoceptive attribution model of presence is suggested as a first step toward an integrative framework for emotion research in VR. Gaps in the current literature and future directions are outlined.

**Keywords: virtual reality, perception, fear, anxiety, emotion, presence**

# **INTRODUCTION**

In virtual reality (VR), researchers can simulate intricate real-life situations and contexts to investigate complex human behaviors in highly controlled designs in a laboratory setting. These characteristics of VR have proven especially attractive for the investigation of pathological processes in mental disorders, and this technology has steadily gained momentum since the 1990s (Rothbaum, 2009). The main application of VR scenarios in this field is research into the processes underlying anxiety disorders and their treatment. Here, VR has become established as a medium for investigating threat perception, fear, and exposure treatment (Mühlberger et al., 2007; Rothbaum, 2009; Opris et al., 2012; Glotzbach-Schoon et al., 2013; Shiban et al., 2013; Diemer et al., 2014).

For research into emotional experiences and emotional behavior, such as fear, anxiety, and exposure effects, it is vital that VR can actually induce emotional reactions. By its very nature, VR as a medium is "unreal" and relies on perceptual stimulation (including perceptual feedback of one's own actions) – in

particular, visual cues, sounds, and sometimes touch and smell – to trigger emotional reactions. Historically, the first VR scenarios applied in the field of mental disorders used powerful visual stimuli to provoke emotional responses, in particular, height (Hodges et al., 1995). Soon, more complex multimodal presentations of visual, acoustic, and vestibular stimuli were developed, for example, to simulate airplane travel (e.g., Mühlberger et al., 2001, 2003, 2006). Still, as it is the very nature of VR the emotional cues relied on perceptional simulations. However, more recent studies have highlighted the need to consider not only bottom-up processes of perception, but also top-down effects when it comes to understanding how VR can be emotionally engaging – e.g., a background narrative to a VR scenario may enhance emotional experience (Bouchard et al., 2008; Gorini et al., 2011; Mühlberger et al., 2012; Peperkorn and Mühlberger, 2013). What is interesting about this perspective is that VR, as a perceptual medium (e.g., all experiences may be interpreted as not-evidence based), enables researchers to dissociate perceptual, i.e., bottom–up input, and higher-level, i.e., top–down processes based on information, and

to manipulate them independently to study their effects separately and in combination.

Another VR phenomenon linked to emotional experience is presence. Presence is a dimensional construct and describes the extent to which a user feels present in a VR environment (Slater and Wilbur, 1997; Schubert et al., 2001; Botella et al., 2009). Theories of presence can be divided into descriptive and structural models. Descriptive models focus on delineating the components of presence, like the model embedded in the Igroup Presence Questionnaire (Schubert et al., 2001). Via factor analysis, these authors identified three dimensions of presence: spatial presence, involvement, and realness (Schubert et al., 2001). On the other hand, structural models aim at an understanding of how the experience of presence is generated in the mind. These models focus on cognitive processes and generally suppose that directing attention to the VR environment (e.g., Witmer and Singer, 1998) and creating a mental representation of this environment (Sheridan, 1999) are necessary processes that enable us to experience presence (Sheridan, 1999; Schuemie et al., 2001). The most recent structural model of presence, proposed by Seth et al. (2012), goes beyond earlier theories. Their perspective is not limited to VR, but instead, Seth et al. (2012, p. 12) point out that presence is an everyday phenomenon, "a basic property of normal conscious experience". Seth et al. (2012) argue that extremes of disturbed presence (with regard to normal reality) can be observed, for example, in schizophrenia and depersonalization disorder. The basic precept of Seth et al.'s (2012) *interoceptive predictive coding model* is that presence rests on continuous prediction of emotional (interoceptive) states. For example, when expecting the encounter with an anxiety-related stimulus, the prediction would be fear, together with the changes the organism usually undergoes during fear. When encountering the feared stimulus, the organism compares the actual interoceptive state (fear and its symptoms) with the predicted state. According to Seth et al. (2012), there will practically always be a certain degree of mismatch. Seth et al. (2012) postulate that presence is the result of successful suppression of this mismatch between the predicted and the actual interoceptive state – i.e., the prediction prevails over the mismatch signals. The idea that suppression of information that is incompatible with the VR experience is vital for presence is not new (Schuemie et al., 2001). For example, Sheridan (1999) posits in his estimation theory that presence is the result of a continuously updated interior model of the environment, stressing the necessity for suspension of disbelief. However, Sheridan (1999) is concerned with the prediction of environmental, i.e., external events. What is unique to Seth et al. (2012) is their emphasis on the prediction of interoceptive states (rather than external events), which affords a crucial role to emotional experience.

The aim of this paper is twofold. First, we provide a review of current research into the relationship between perception and information on emotional experience in VR environments. Since exposure therapy has so far been the most common application of VR technology in clinical psychology, our focus lies on VR concerned with fear and anxiety in both healthy and clinical populations. We present a series of our own experiments that were designed to examine the significance of perception vs. conceptual

information and presence for the experience of anxiety, and fear in VR environments. Second, an integration of the literature regarding immersion, presence, and emotional experience in VR is still outstanding. Different VR systems, diverging operationalizations of presence, and study samples ranging from healthy controls to patients with anxiety disorders make it difficult to draw firm conclusions. Based on a review of presence research, we suggest a new interoceptive attribution model of presence as a step toward an integrative framework for emotion research in VR.

#### **EFFECTS OF PERCEPTION VS. INFORMATION ON FEAR**

The most influential theoretical conceptualization of dysfunctional fear to date is offered by the *emotional processing theory* by Foa and Kozak (1986; McNally, 2007). According to this theory, dysfunctional fear can be viewed as a memory network comprising information about the feared stimulus (e.g., its characteristics), the fear response (i.e., behavioral plans concerning escape and avoidance), and propositions of meaning (e.g., association with danger or threat; Foa and Kozak, 1986). Importantly, this fear network can be partly or fully activated by input that matches part of the network. Fear, according to this theory, is an index of network activation and can be measured both subjectively and physiologically (Foa and Kozak, 1986).

Fear can be activated by at least two pathways: The perceptual (e.g., visual fear-related cues) and the conceptual (fear-related information) paths. Perceptual fear-related cues are assumed to rapidly evoke physiological and behavioral fear reactions, whereas fear-related information is expected to produce subjective fear reactions, but only a poorer physiological activation (Hofmann et al., 2008). Strack and Deutsch (2004) in their *reflective-impulsive model of social behavior* propose that impulsive, emotional reactions are fast, and governed by the laws of association (spreading activation), while reflective behavior is subject to more flexible, cognitive control. However, the impulsive and the reflective systems are supposed to interact, allowing conceptual information (input to the reflective system) to activate rapid emotional reactions (Strack and Deutsch, 2004). In practice the separation of the two paths is difficult to investigate as emotionally relevant situations typically comprise input to both paths simultaneously.

Virtual reality is a particularly suitable tool as it offers an opportunity to differentiate the two paths for eliciting emotion. In VR, cue propositions can be activated by presenting feared objects perceptually (e.g., visually), and, unrelated to the perceptual presentation, activating the meaning propositions by informing a person of the existence of a feared object, or situation outside the VR scenario they are immersed in. The laboratory setting of VR further allows the online assessment of different fear reactions (subjective, physiological, and behavioral) in a highly controlled setting.

#### **EMPIRICAL FINDINGS ON SPECIFIC PHOBIA**

In a series of studies we investigated the relative importance of perceptual fear-related cues and conceptual fear-related information on the activation of fear in different anxiety disorders. We assumed that fear reactions in specific phobia (animal type) are primarily caused by simple perceptual fear-related cues like a spider, whereas the impact of information on fear (i.e., knowing about the presence of a spider without seeing it) should be less pronounced. We directly and separately manipulated the two paths by using VR to present the visual cues on the one hand and the independent information about the existence of a real fear-evoking stimulus on the other hand.

In a first study with patients suffering from spider phobia (Peperkorn et al., 2014), we found that specific perceptual cues (in this case visual simulations of a spider) and conceptual information (verbal report that an unseen spider was present in front of the participant) presented separately activated the fear network, albeit via different routes. Specifically, perceptual cues vs. conceptual information led to different degrees of fear activation, with the perceptual route being significantly more fear provoking than the informational route, as was expected for spider phobia. Fear ratings (mean of five exposure trials) of this experiment are shown in **Figure 1**.

In a second study, we addressed the question whether these findings generalize to other types of phobias. While in spider phobia, fear is characteristically triggered by a stereotypical object (the animal), in other phobias – those of the situational subtype, e.g., claustrophobia – triggers are more context-related, involving more complex perceptual stimuli (American Psychiatric Association, 2013). Therefore, we used the same design in a sample of patients suffering from claustrophobia (Shiban et al., submitted). Similar to the spider phobia study (Peperkorn et al., 2014), we found for claustrophobia that the perceptual condition (seeing the inside of a virtual box with a closed door) initially activates stronger self-reported and physiological fear responses compared to the information condition where patients knew they sat in an actual, closed claustrophobic box (the fear-specific information), but saw an open door in the corresponding VR environment. It is important to note that although both studies used mainly visual cues as perceptual cues, in the spider phobia study the cues were

**FIGURE 1 | Fear in spider phobia during exposure to spider cues in virtual reality (cue), to a real (but unseen) spider (info), and to both.** Error bars represent SEM.

specific (a virtual spider), whereas in the claustrophobia study they were more complex and context-related (a claustrophobic box).

In summary, in these studies we demonstrated for the first time in an integrated multimodal experiment that perceptual cues and conceptual information can provoke fear reactions in specific phobia, with additive effects if combined. Interestingly, perceptual cues alone seem to induce more self-reported fear than information alone, regardless of the type of specific phobia (animal vs. situational subtype). This is in line with findings that fear enhances perceptual, but not mental processing (e.g., Borst and Kosslyn, 2010), implying that there is a closer link between perceptual input and the experience of fear, than between fear and the mental processing of (purely conceptual) information.

#### **EMPIRICAL FINDINGS FOR SOCIAL ANXIETY**

As social fears are generally thought to be more cognitive in nature than specific phobias (Clark, 2005; Schulz et al., 2008; Wieser et al., 2010), we expected – in contrast to the results from studies on specific phobia – that anticipating a speech would be more fear-provoking when conducted in front of an audience a participant is informed are there (even if not seeing the audience: information condition) than in front of a virtual audience (perceptual cues) when knowing that actually no one will listen to the talk. Therefore, in a third study we applied a modified version of the paradigm described above to a public speaking challenge (Shiban et al., 2014; Diemer et al., in preparation). In contrast to the studies of specific phobia, anticipatory anxiety was chosen to avoid a possible confound in the physiological variables due to arousal caused by speaking (Gramann and Schandry, 2009). Also, anticipatory anxiety has been shown to share important parts of the neural network of acute anxiety (Nitschke et al., 2006).

We hypothesized that a real observer outside VR (information condition) would evoke significantly stronger subjective and physiological fear reactions than a visual observer in VR (perceptual cue condition). Further, we expected that a combination of real and VR audience (combined condition) would result in the strongest subjective and physiological fear reactions. The experimental conditions are presented in **Figure 2**. Finally, we expected high socially anxious participants to show stronger fear reactions than low socially anxious participants. We randomly allocated 48 healthy participants to either the information condition, the cue condition, or the combined condition. (for details of physiological data acquisition, see Peperkorn et al., 2014). As expected, socially anxious participants reported significantly higher subjective fear, but there were no differences between conditions (see **Figure 3**). Physiological parameters [heart rate, skin conductance level (SCL)] decreased significantly over time. There was a trend SCL to differ between groups, with the highest SCL in the visual cue condition (*p* = 0.066), but there were no other effects of social anxiety or condition. With a mean Social Phobia Inventory (SPIN; Connor et al., 2000) score of 21.8 (median: 21, SD: 10.5), our sample was above the mean of healthy controls (*M* = 12.1, SD = 9.3), but markedly below the mean (*M* = 41.1, SD = 10.2) of patients with social phobia reported by Connor et al. (2000).

**FIGURE 2 | Manipulation of fear cues and fear-relevant information in public speaking.**

While these results are disappointing in sofar as we could not find the expected effect of the information condition, the paradigm has shown promise. There was a clear effect of social anxiety, with significantly higher subjective fear in socially anxious participants, and in contrast to the studies on specific phobia, no superiority of the cue condition was found (Shiban et al., 2014; Diemer et al., in preparation). Therefore, we believe that it would be worthwhile to apply this paradigm in a larger sample of patients with social anxiety disorder, and to assess acute fear during public speaking.

#### **SUMMARY**

In summary, the VR designs reported here confirmed the possibility of eliciting fear reactions via different routes (perceptual vs. conceptual). Patients with specific phobia seem to be particularly sensitive to perceptual cues. Interestingly, this finding was the same for spider phobia (animal type) and claustrophobia (situation type). For social anxiety, no differences in activation of thefear structure between the two paths were found. These observations

are in line with Foa and Kozak's (1986) prediction about differential sensitivities of different anxiety disorders to different media of exposure (*in vivo* cues vs. imagination). However, the interpretation of our results on social anxiety remains preliminary, as we did not assess patients or acute fear as in the studies of specific phobia. It seems worthwhile to continue this research with different kinds of specific phobias and more complex anxiety disorders like agoraphobia, panic disorder, and social anxiety disorder.

# **PRESENCE AND EMOTION IN VR**

The association of presence and emotional experience in VR exposure therapy is an issue of debate. Presence is commonly regarded as a necessary mediator that allows real emotions to be activated by a virtual environment (Parsons and Rizzo, 2008; Price et al., 2011). While this conception implies a causal role for presence, research has not yet been able to clarify the relationship between presence and emotional experience in VR.

Presence has been conceptualized, and consequently operationalized and manipulated, in very different ways. These ranges from a manipulation of presence by providing more or less sophisticatedVR technology to the diverse methods of assessing presence, either by subjective ratings taken online during the VR experience, or afterward via questionnaires. Presence questionnaires vary greatly with regard to the constructs they measure; however, what they have in common is that they ask participants for a subjective judgment regarding their experience of presence. With this in mind, we will use the definition of Slater and Wilbur (1997) and Slater (1999) and call any manipulation at the level of technology a manipulation of immersion, rather than presence. Presence is defined as a subjective phenomenon that results from experiences induced by immersiveVR technology (Slater and Wilbur, 1997; Slater, 1999; Schubert et al., 2001). To avoid confusion with aspects of immersion (technology), for the purpose of this paper, only subjective measures of the presence experience (ratings or questionnaires) are considered presence measures. We chose not to include physiological parameters as indicators of presence as physiology is directly linked to emotional arousal, so considering physiological responses as operationalizations of presence would inevitably bring a confound of presence and emotion. The following section on presence and emotion considers two approaches to presence. First, the effects of immersive VR technology on presence and emotion are considered. Then, we will take a closer look at correlative findings of presence and emotion.

# **THE ROLE OF IMMERSION**

# *Immersion and presence*

VR simulations can be more or less graphically enhanced, multimodally integrated, and interactive. More sophisticated technology is often thought to result in more presence. Already Botella et al. (1999) reported more emotional reactions to a simple, neutral VR scene when a high-quality head-mounted display (HMD) was used, compared to a medium-quality HMD. Typically, studies assessing different degrees of immersion find higher presence in more immersiveVR systems compared to less sophisticated setups. Such effects have been reported for VR scenarios presented via a Cave Automatic Virtual Environment (CAVE) vs. HMD (Krijn et al., 2004; Juan and Perez, 2009), for HMD vs. computer monitor (Gorini et al., 2011), video wall (a large stereoscopic projection screen) vs. computer monitor (Baños et al., 2004), for active vs. passive navigation in VR (Freeman et al., 2005), and for stereoscopy vs. monoscopy (Ijsselsteijn et al., 2001; Ling et al., 2012). Although some researchers have failed to find an effect of immersion on presence (e.g., Baños et al., 2008, for stereoscopy), in general, research indicates that more sophisticated simulations (higher immersion) result in increased presence, especially in virtual environments not designed to induce particular emotions (Baños et al., 2004).

#### *Immersion and emotion*

As for possible effects of immersion on emotions, the picture becomes more complicated. While some authors report an increase in emotional responses in more immersive compared to less immersive VR systems (Botella et al., 1999; Juan and Perez, 2009; Visch et al., 2010), others did not find effects of immersion on emotion (Freeman et al., 2005; Ling et al., 2012). In more

detail, it seems that immersion effects on emotion might depend on the nature of the emotions under study. Visch et al. (2010) suggest that the effect of immersive technology is mediated by arousal. This idea appears plausible, as especially fear and anxiety, both of which are strongly arousing emotions, have been found to be stronger in more immersive VR setups (Juan and Perez, 2009), while happiness and relaxation appear to be much less influenced by the technology used (Freeman et al., 2005; Baños et al., 2008). Of note, the positive emotions induced in the studies by Freeman et al. (2005) and Baños et al. (2008) were not only of different valence than fear, but also non-arousing in nature. In a study of spider phobia, we also found stronger subjective and behavioral (avoidance) fear reactions in a stereoscopic vs. monoscopic VR (Peperkorn et al., submitted). By contrast, Ling et al. (2012) did not find an effect of stereoscopy on emotional reactions including fear. However, Ling et al. (2012) investigated healthy participants during a speech task, so arousal levels (mean heart rate about 75 beats per minute) appear to have been comparatively low.

Another possibility to test the influence of immersion is the use of different perceptual modalities or multimodal perceptual cues. Thus, we compared tactile cues (touching a spider model) with visual cues (visual VR spiders presented in the HMD) in patients with spider phobia (Peperkorn and Mühlberger, 2013). As expected, the combination of visual and tactile cues led to the highest fear ratings. Tactile cues alone activated significantly stronger fear reactions than visual cues alone. Interestingly, presence was also higher in the multimodal (perceptual plus tactile cues) than the single modus conditions, a finding that confirms the association of immersion and presence. However, the different perceptual paths that we investigated are few out of many; for example, acoustic stimuli can be important in specific phobia, and can be easily implemented in VR (Taffou et al., 2012).

Taken together, there is considerable evidence that the level of immersion a VR system provides exerts an effect on the presence experienced by the user (Ijsselsteijn et al., 2001; Freeman et al., 2005). This effect seems to be particularly prominent in the absence of emotional manipulations, i.e., the effect does not seem to be mediated by emotion. The fact that immersion does not *per se* increase emotional experience, but that the emotionally enhancing effect of immersion might be limited to arousing emotions (see the discussion above), supports this conclusion. For example, Baños et al. (2004) independently manipulated immersion (HMD vs. computer monitor vs. video wall) and emotional content (sad vs. neutral) of a VR scenario. They found an interaction effect, with immersion affecting presence ratings in the emotionally neutral condition, but much less so in the emotional (sad) condition. There was also a main effect of emotion, with higher presence in the emotional than in the neutral condition (Baños et al., 2004). However, it is not clear from these data why there was no immersion effect on presence in the emotional condition; unfortunately, Baños et al. (2004) do not report the strength of the actual emotions experienced by their participants. As manipulations of immersion are not direct manipulations of presence, it is impossible to determine from these findings whether presence is causal for emotional experience. It has been argued that immersion causes arousal, which in turn increases presence and emotion ratings (Visch et al., 2010). We will come back to the issue of arousal in the following section on correlative findings.

#### **PRESENCE AND EMOTION**

The association of presence and emotion has been mainly investigated by means of correlations between these two measures. Correlations between presence and emotional experience in VR have been consistently reported, especially in the literature on VR exposure therapy (Robillard et al., 2003; Price and Anderson, 2007; Riva et al., 2007; Bouchard et al., 2008; Alsina-Jurnet et al., 2011; Price et al., 2011), although some researchers have reported no relation between presence and the extent of experienced fear (Krijn et al., 2004). A common conclusion in this type of research is that in VR exposure therapy, presence and fear appear mutually dependent (Robillard et al., 2003; Price and Anderson, 2007). In a recent study, we confirmed the positive association, but additionally found indications that the relationship between presence and fear might change dynamically during exposure to phobic stimuli (Peperkorn et al., submitted). Interestingly, a general effect of presence on treatment outcome could not be established (Krijn et al., 2004; Price and Anderson, 2007), although Price et al. (2011) found that scores on the presence subscale "involvement," but not other presence scales, predicted treatment outcome in a sample of patients with social phobia (*n* = 31) undergoing VR exposure therapy.

In the case of fear in non-patients, results are less clear. On the one hand, there are results paralleling findings from patient samples. For example, Alsina-Jurnet et al. (2011) exposed a large sample (*n* = 210) of test-anxious students and non-anxious students (groups assigned according to questionnaire scores) to a VR environment that simulated a university exam, and a neutral VR. The authors reported no correlation between fear and presence in the neutral VR, and a considerably stronger correlation between presence and fear in the test anxious group (Alsina-Jurnet et al., 2011). We found similar results in a sample of spider fearful and control participants exposed to VR spiders, with significantly stronger presence in the fearful participants vs. controls, and a significant positive correlation between presence and fear in the fearful participants only (Peperkorn and Mühlberger, 2013). Whether this pattern of results is related to a floor effect and/or reduced variability in fear ratings in the healthy samples has not been investigated. On the other hand, research on emotions other than fear tends to produce mixed results. In an emotion induction paradigm in VR, Baños et al. (2004, 2008, 2012) tested the effects of different kinds of emotion on presence. They found correlations between presence and emotion in healthy controls for sadness (Baños et al., 2004), joy (Baños et al., 2008), and relaxation (Baños et al., 2008). Using non-immersive VR equipment, Baños et al. (2012) could not find significant correlations between emotion (joy, relaxation) and presence; however, they did observe relatively high presence ratings. By contrast, using a relaxation paradigm presented with different levels of immersion, Freeman et al. (2005) found only one significant correlation between the experience of happiness and a presence scale, which the authors interpreted as an artifact due to item overlap.

Interestingly, some authors have also tested the effects of emotions induced by information on presence. Gorini et al. (2011) had participants search for a blood container in a VR hospital, either with the information that this was urgently needed to save a child, or without this information. Bouchard et al. (2008) informed patients with snake phobia that there were snakes in aVR environment, while in fact, no snakes were shown. Both Bouchard et al. (2008) and Gorini et al. (2011)reported that this emotionally relevant background information enhanced presence, indicating a causal influence of emotions on presence. Other possible influences on presence could be personality or (spatial) intelligence (Alsina-Jurnet and Gutierrez-Maldonado, 2010). However, little is yet known about the influence of these, or other, traits on presence or emotion during a VR experience.

Taken together, results show that the stronger the feelings involved, either because of the nature of the emotion (e.g., fear vs. joy vs. relaxation), or because of the nature of the sample (patients with anxiety disorder vs. normal controls), the greater the likelihood of finding a significant correlation between presence and emotion. A possible explanation for this phenomenon could again be arousal. Already, Freeman et al. (2005) suggested that the correlation of presence and emotion might be limited to arousing stimuli. They proposed an arousal theory of presence, arguing that arousal leads to alertness, which in turn leads to higher presence ratings. According to Freeman et al. (2005, p. 2018), alertness increases a participant's readiness to respond to the stimuli that compose a given VR, as arousal represents a "call to action" – thus leading to a greater perceived physical and mental presence in VR. So far, this arousal theory has not been rigorously tested, although objective measures of arousal (i.e., physiological parameters) can be easily assessed during VR sessions (Mühlberger et al., 2007; Diemer et al., 2014). First evidence for a crucial role of arousal comes from the study by Gorini et al. (2011), who reported significantly higher heart rate in the group that experienced the hospital VR with a narrative that increased the relevance of the scenario. Unfortunately, Gorini et al. (2011) do not report correlations between heart rate and presence ratings.

# **DISCUSSION**

The findings reviewed here highlight important advances in the study of fear and anxiety in VR environments. The data on perceptual fear cues and conceptual information show that both are viable triggers of fear reactions (Bouchard et al., 2008; Gorini et al., 2011; Peperkorn et al., 2014; Shiban et al., in preparation). There is evidence that patients with specific phobia react more strongly to visual cues than to fear-specific information, a finding that lends preliminary support to dual-process theories like the impulsive–reflective model of social behavior (Strack and Deutsch, 2004). The possibility of activating fear separately by perceptual cue or information in VR opens up new research opportunities to investigate pathological processes specific to each route. This might be particularly relevant for cue-independent fears and anxiety, for example in obsessive–compulsive disorder, illness anxiety disorder, and generalized anxiety disorder.

As for presence, the literature shows the significance of immersion on presence. Specifically, greater immersion of a VR system increases presence, particularly in emotionally neutral VR scenarios, which indicates that the effect is not mediated by emotion. In fact, it seems that the "depth" of a VR experience in terms of presence and emotion is more strongly influenced by factors quite apart from the technological quality of the VR system. Certainly the effect of immersion – i.e., technological quality – on presence exists, but interestingly, it is strongest when no emotion is involved. As soon as a VR scenario engages emotions, presence is increased. Studies that manipulate emotion independently of the technological aspects and even the stimuli presented via VR (e.g., Gorini et al., 2011) demonstrate this effect quite convincingly. Further, correlations between (strong) emotions and presence have been consistently reported. The effects of immersion and emotion on presence are possibly explained by arousal (Freeman et al., 2005; Visch et al., 2010), but theories of emotion and presence in VR (Freeman et al., 2005; Seth et al., 2012) have so far been insufficiently tested. In the case of VR exposure therapy, neither general presence nor immersion seem to be related to treatment outcome (Mühlberger et al., 2005); rather, a certain degree of both appears a necessary requirement for VR exposure therapy, but increasing either does not *per se* enhance therapy effects (Krijn et al., 2004; Price and Anderson, 2007).

Before the findings reported here can be integrated into one model, more research is needed. While the data resumed so far indicate a crucial role for arousal, the position of arousal in an explanatory framework that comprises VR system factors, immersion, aspects of stimulation (e.g., perception vs. information), presence and emotion is not clear. First, we do not know how the effect of perception vs. information on emotion is produced. On the one hand, fear-related elements in VR are input cues to the fear network – as proposed in emotional processing theory (Foa and Kozak, 1986) – and might thus directly enhance emotional arousal. However, this theory does not explain why, in specific phobia, perceptual cues have a stronger effect on fear network activation than information alone. The reflective–impulsive model of social behavior (Strack and Deutsch, 2004) can explain different effects of perception vs. information on fear. On the other hand, however, emotionally relevant perceptual stimuli and information enhance a VR environment, making it more interesting, appealing to attention and ultimately, increasing, at least initially, arousal – irrespective of the emotional valence of the stimuli in question. Since arousal is a basic dimension of emotional experience, the effect of perception and information on emotion might be mediated by arousal. The role of arousal should be tested with emotions with different levels of arousal, using in particular physiological indicators of arousal. To broaden the range of emotions investigated, anger would be interesting as a highly arousing emotion other than fear that could also be activated in VR.

Concerning presence, the preliminary conclusion we would draw from the findings reviewed here is that the case for a crucial involvement of arousal in the experience of presence is compelling. However, the mechanism of this effect cannot be discerned yet. Freeman et al. (2005) propose that arousal increases presence by enhancing attention to a VR environment and the possibilities of action offered by this environment. A different explanation we suggest is an *interoceptive attribution model of presence* (see **Figure 4**). Since presence is a subjective experience, common

measures of presence explicitly call the participants to make a judgment of the degree of presence they feel in VR. Based on the results reviewed in this paper, we propose that participants make this judgment based mainly on two sources of information: (1) immersion and (2) the degree of arousal they feel. As for immersion, participants might base their presence judgment on the perceptual distance they experience from the real world setting, i.e., the less stimulation they receive from the real world, and the more stimulation from the VR scenario, the higher the level of presence they will indicate. Of course, this hypothesis needs further empirical confirmation. With regard to emotion, we believe that participants will give higher presence ratings if they feel emotionally affected. As arousal is a particularly strong indicator of emotional involvement, arousing emotions should lead to higher presence ratings, and correlate more closely, with presence ratings, than calm or serene emotional states – a picture that is in fact found in the literature. Interestingly, whether the experience of arousal *per se*, or the attribution of this arousal to the VR scenario is necessary for the experience of presence has not yet been investigated. Additionally, immersion itself is likely to increase arousal (Visch et al., 2010). In essence, the cognitive nature of presence – in that it is a subjective judgment – forms the core of our understanding of presence as it is usually conceptualized and assessed in its relation to immersion, stimulation, and emotion in VR research. We believe that our model is compatible with the predictive coding mechanisms put forward by Seth et al. (2012). In contrast to Seth et al.'s (2012) conception, our model focuses on the attribution process that gives rise to cognitive presence judgments. It is intended as a framework for research into emotional experience and presence in VR. Future studies should therefore differentiate

as precisely as possible between cognitive presence (presence as a subjective judgment), emotional presence (Seth et al., 2012), and on the other hand immersion (technological features of a given VR system), arousal (as a dimension of emotion), specific emotions (along both the arousal and the valence dimensions), and the population under study (patients vs. fearful participants vs. healthy controls). Further, to fully understand presence in VR and its unique characteristics, the investigation of presence in reality, e.g., during *in vivo* exposure as compared to VR exposure, appears vital (Seth et al., 2012). We can reasonably assume that, when making sense of a VR environment, people apply the same mechanisms to it as they do to everyday reality (Seth et al., 2012). A direct comparison of both worlds has, unfortunately, long been neglected.

#### **AUTHORS CONTRIBUTION**

Julia Diemer: data analysis, wrote the manuscript. Georg W. Alpers: study conception, contribution to data analysis, and the manuscript. Henrik M. Peperkorn: study conception, data acquisition, and analysis, contribution to the manuscript. Youssef Shiban: data acquisition and analysis, contribution to the manuscript. Andreas Mühlberger: study conception, data analysis, contribution to the manuscript. All authors have approved of the final version of the manuscript and its submission.

#### **ACKNOWLEDGMENT**

This research was funded by DFG grant MU2299-2/2 to Andreas Mühlberger and Georg W. Alpers.

## **REFERENCES**


**Conflict of Interest Statement:** The Guest Associate Editor Fritz Strack declares that, despite being affiliated to the same institution as author Henrik M. Peperkorn, the review process was handled objectively and no conflict of interest exists. Andreas Mühlberger is stakeholder of a commercial company that develops virtual environment research systems. Julia Diemer, Georg W. Alpers, Henrik M. Peperkorn and Youssef Shiban have no conflicts of interest.

*Received: 29 July 2014; accepted: 07 January 2015; published online: 30 January 2015. Citation: Diemer J, Alpers GW, Peperkorn HM, Shiban Y and Mühlberger A (2015) The impact of perception and presence on emotional reactions: a review of research in virtual reality. Front. Psychol. 6:26. doi: 10.3389/fpsyg.2015.00026*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology.*

*Copyright © 2015 Diemer, Alpers, Peperkorn, Shiban and Mühlberger. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Mutual influences of pain and emotional face processing

# *Matthias J. Wieser 1\* †, Antje B. M. Gerdes 2 †, Philipp Reicherts1 and Paul Pauli <sup>1</sup>*

<sup>1</sup> Department of Psychology, University of Würzburg, Würzburg, Germany

<sup>2</sup> Department of Psychology, University of Mannheim, Mannheim, Germany

#### *Edited by:*

Luiz Pessoa, University of Maryland, USA

#### *Reviewed by:*

Mathieu Roy, University of Colorado Boulder, USA Susanne Becker, Central Institute of Mental Health, Germany

#### *\*Correspondence:*

Matthias J. Wieser, Biological Psychology, Clinical Psychology, and Psychotherapy, Department of Psychology, University of Würzburg, Marcusstraße 9-11, D-97070 Würzburg, Germany e-mail: wieser@psychologie. uni-wuerzburg.de

†Matthias J. Wieser and Antje B. M. Gerdes have contributed equally to this work.

The perception of unpleasant stimuli enhances whereas the perception of pleasant stimuli decreases pain perception. In contrast, the effects of pain on the processing of emotional stimuli are much less known. Especially given the recent interest in facial expressions of pain as a special category of emotional stimuli, a main topic in this research line is the mutual influence of pain and facial expression processing. Therefore, in this minireview we selectively summarize research on the effects of emotional stimuli on pain, but more extensively turn to the opposite direction namely how pain influences concurrent processing of affective stimuli such as facial expressions. Based on the motivational priming theory one may hypothesize that the perception of pain enhances the processing of unpleasant stimuli and decreases the processing of pleasant stimuli. This review reveals that the literature is only partly consistent with this assumption: pain reduces the processing of pleasant pictures and happy facial expressions, but does not – or only partly – affect processing of unpleasant pictures. However, it was demonstrated that pain selectively enhances the processing of facial expressions if these are pain-related (i.e., facial expressions of pain). Extending a mere affective modulation theory, the latter results suggest pain-specific effects which may be explained by the perception-action model of empathy. Together, these results underscore the important mutual influence of pain and emotional face processing.

#### **Keywords: pain, emotion, facial expression, ERPs, perception-action**

# **INTRODUCTION**

Emotions possess immense power to alter pain perception. The influence of experimentally induced emotions on experimental pain has been investigated with various affective stimuli like affective pictures (e.g., Meagher et al., 2001; Rhudy and Meagher, 2001; Kenntner-Mabiala and Pauli, 2005; Kenntner-Mabiala et al., 2007, 2008), pain-related pictures (e.g., Godinho et al., 2012), odors (e.g., Villemure et al., 2003), and music (e.g., Roy et al., 2008, 2012). This research has demonstrated that negative emotions lead to increased pain perception while positive emotions result in decreased pain perception pain. Overall, it was found that emotions affect various measures of pain perception such as sensory and affective pain ratings, and neuronal correlates as measured with BOLD responses and EEG responses (for extensive reviews see Wiech and Tracey, 2009; Bushnell et al., 2013). Various brain mechanisms are implicated in the emotional modulation of pain (Roy et al., 2009; Wiech and Tracey, 2009). One key mechanism involves descending pain modulatory systems affecting the afferent transmission of spinal nociceptive signals to many brain regions (including the thalamus, amygdala, insula, and somatosensory cortex), which can lead to inhibition or excitation of afferent pain signals (Bushnell et al., 2013). Other mechanisms involve the integration of pain- and emotion-related signals in the anterior insula, also with regards to signals from the autonomic nervous system which are central to both pain and emotions (Craig, 2002). Overall, the multiplicity of mechanisms underlying the emotional modulation of pain is reflective of the strong interrelations between pain and emotion both on a

psychological and neuroanatomical level (Vogt, 2005; Roy et al., 2009).

A theoretical framework for the explanation of the emotional modulation of pain is the motivational priming hypothesis (Lang, 1995), which assumes that processing of unpleasant information is facilitated while processing of pleasant information is inhibited under aversive affect. Accordingly, unpleasant stimuli increase and pleasant stimuli decrease pain perception and physiological responses to pain. Interestingly, the majority of studies on emotion-pain interactions has unidirectionally examined the influence of emotional stimuli on pain perception, although bidirectional interactions are likely and plausible (Bushnell et al., 2013). This would also concur with predictions drawn from the motivational priming hypothesis that pain as an aversive state should facilitate the processing of unpleasant stimuli and inhibit the processing of pleasant stimuli.

As mentioned above and also common in emotion research, most of the studies investigating emotional modulation of pain used affective pictures (e.g., Meagher et al., 2001; Rhudy and Meagher, 2001; Kenntner-Mabiala and Pauli, 2005; Kenntner-Mabiala et al., 2007, 2008) or pain-related pictures (e.g., Godinho et al., 2012). Only recently, researchers started to investigate the effects of facially communicated pain stimuli (i.e., facial expressions of pain), their expression, perception, and possible effect on pain perception. While in general it is doubtful that facial expressions elicit strong emotional states in the observer (e.g., Bradley et al., 2001; Britton et al., 2006; Alpers et al., 2011) the social importance of non-verbal emotion communication makes

facial expressions (of pain) an interesting model for the interaction of facial signals of emotions and concurrent pain processing (Williams, 2002). However, the modulation of pain by this crucial feature in non-verbal emotion communication has been widely neglected so far. One of the few studies in this field demonstrated that emotional compared to neutral facial expressions increase pain perception accompanied by alterations of pain-related brain oscillations (Senkowski et al., 2011). Similarly, pain compared to neutral expressions were found to augment pain perception (Mailhot et al., 2012). However, the effect of pain on pain face processing was not quantified.

A possible theoretical explanation for the interaction of viewing others' facial expression of pain and the own sensation of pain is offered by the perception-action model (PAM) of empathy (Preston and de Waal, 2002). The PAM proposes that the capacity to feel the internal state of someone else activates the corresponding representations in an observer. Indeed, it was found that observing others'facial expression of pain also amplifies one's own facial and neural responses to pain, revealing a vicarious effect of facial pain expression (Vachon-Presseau et al., 2011, 2013; Mailhot et al., 2012). Additional support for the PAM derives from neuroimaging studies in which it was found that emotions observed in a target are mapped onto a self-reference framework supposed to serve the rapid understanding of others' feelings, goals, and intentions (e.g., Wicker et al., 2003; Jackson et al., 2006). Consequently, the PAM would predict selective pain-enhancement by watching pain faces of others compared to other negative facial expressions, whereas the motivational priming would assume a general enhancement of pain by negative facial expressions, but not necessarily selectivity of pain faces.

The aim of this mini-review is to selectively summarize research on the influence of visual affective stimuli on pain perception, but mainly on the opposite effect of pain on the processing of affective stimuli such as facial expressions. Given the growing interest in pain modulation by facial expressions of pain and the lack of studies which used other affective stimuli, we focus on studies on facial expressions. In addition we seek to extend the viewpoint of a mere affective modulation of pain with regards to the theory of vicarious pain and include a recent experiment from our lab which aimed at investigating the mutual effects of the perception of facial expressions of pain and pain perception. This review is far from exhaustive; it only summarizes the literature relevant for our work within the research group "Emotion and Behavior" at the University of Würzburg, Germany. We are fully aware that much more research is available on the topics of emotional modulation of pain and vicarious pain, and we direct the attention of the interested reader to the excellent reviews by Wiech and Tracey (2009), Lamm et al. (2011) and Bushnell et al. (2013).

# **PAIN-MODULATED PROCESSING OF AFFECTIVE PICTURES AND FACIAL EXPRESSIONS**

As mentioned above, the effect of pain on emotion processing has been investigated much less, although from a clinical perspective the high prevalence of mood disorders in chronic pain suggests effects in this direction (Bair et al., 2003; Campbell et al., 2003). One study found that when paired with pain, pleasant pictures were rated less pleasant and elicited attenuated

visual-evoked responses of the EEG (Godinho et al., 2008). However, no enhanced responses to negative stimuli were found. In an own study, evaluative facial responses congruent and incongruent to pictures of facial expressions were recorded during painful pressure stimulation (Gerdes et al., 2012). Normally, voluntary facial muscle reactions are facilitated (i.e., less errors and faster responses) in response to muscle-congruent facial expressions (i.e., facilitated reactions of Musculus Corrugator supercilii in response to negative facial expressions and facilitated reactions of Musculus zygomaticus major in response to positive facial expressions), which is interpreted as motor-compatibility and automatic evaluation of affective stimuli. In this study, pressure pain generally slowed compatible as well as incompatible muscle responses (Musculus zygomaticus and Musculus corrugator) and resulted in fewer erroneous incompatible (Musculus corrugator) responses to happy faces. However, pain did not affect muscle responses to angry faces and affective ratings. Together with the results by Godinho these findings point at the notion that pain particularly reduces responses to pleasant stimuli, but seems not necessarily to exacerbate processing of negative emotional stimuli. The latter observation may be partly explained by the pain-reducing effects of distraction which is caused by the ongoing pain and thus dampens the actual facilitatory effects of pain for unpleasant emotions.

In a further study, we then investigated the effect of tonic pressure pain on the electrocortical correlates of processing of facial expressions (Wieser et al., 2012). Here, fearful, happy, and neutral faces were presented while participants received tonic pressure stimulation. Face-evoked brain potentials revealed no affective but an attentional modulation by pain: early and late indices of attention allocation toward faces [P100 and late positive potentials (LPP) of the ERP] were diminished during tonic pain compared to the control condition. The latter finding is concurrent with earlier findings revealing the attentional interruptive function of pain (Eccleston and Crombez, 1999), which has been demonstrated for attentional processes (e.g., Seminowicz and Davis, 2006; Tiemann et al., 2010), visual object processing (Bingel et al., 2007), and early stages of memory formation (Forkmann et al., 2013).

We conclude that on the one hand there is some evidence that experimental pain alters perception and processing of positive affective stimuli (scenes and faces), although most effects were observed with regards to attentional mechanisms. On the other hand, little is known about how pain alters processing of facial displays of pain and vice versa. Given the match between observed and experienced pain, one may argue that selective enhancement and mutual influences have to be expected. Before we report an experiment in which these mutual influences were investigated, we will briefly summarize why facial expressions of pain may be special compared to other facial expressions.

# **PAIN-SPECIFIC MODULATIONS: FACIAL EXPRESSIONS OF PAIN, THEIR PERCEPTION, AND THEIR EFFECT ON PAIN**

The sensation of pain is accompanied by distinct albeit not uniform facial expressions (e.g., Prkachin, 1992; Prkachin and Craig, 1995; Kunz et al., 2009, 2011; Kunz and Lautenbacher, 2014). Pain expressions, on the one hand, may benefit the sender by observers' support and assistance in recovery, and on the other hand, inform observers about potential threat and danger (Williams, 2002). Thus, facial displays of pain serve important social functions and therefore are supposed to be of great importance for social interactions (Williams, 2002; Craig, 2009).

Compared to neutral facial expressions, facial expressions of pain receive prioritized processing and elicit enhanced initial orienting (Vervoort et al., 2013). Similar results were obtained in a dot-probe paradigm (Baum et al., 2013), whose results indicated both early attentional engagement and subsequent avoidance of facial expressions of pain. A recent study from our lab investigated whether facial expressions of pain are perceived differentially from other facial expressions and elicit distinct electro-cortical responses as measured by ERPs (Reicherts et al., 2012). To this end, participants watched painful, fearful, happy, neutral facial expressions (Simon et al., 2008) and were asked to rate these videos while EEG was recorded continuously. Videos of pain faces were rated as more intense and negative than other emotional (both positive and negative) expressions and concurrently elicited enhanced electrocortical responses (augmented late positive potentials, LPPs), which are supposed to index sustained motivated attention to salient stimuli (Schupp et al., 2000) (see **Figure 1**).

Corroborating our findings, Missana et al. (2014) reported enhanced LPPs of pain compared to angry faces. Additionally, two studies observed that pain faces were rated as more unpleasant and arousing than angry, happy, and neutralfaces, resulted in enhanced amplitudes of the ERPs and increased theta activity, and evoked greater corrugator response (González-Roldán et al., 2011, 2013). Taken together, these results emphasize that pain faces have a special salience different from other emotional expressions, although one has to bear in mind that in none of the aforementioned studies a comparison was made between all facial expressions (e.g., facial expressions of disgust) within one study.

Since emotional stimuli have a great impact on pain perception as summarized above, one may assume that a highly salient signal such as the facial expression of pain should also modulate pain perception. This assumption is further strengthened by the fact that the cortical regions involved in the decoding of pain (Bushnell et al., 1999) and emotional facial expressions (Adolphs et al., 2000), e.g., the somatosensory cortex, partly overlap.

The interaction of facial expressions of pain and perception of pain is rarely investigated. In one study, volunteers viewed videos showing different levels of pain expression before noxious electric shocks were delivered. Viewing stronger pain expressions generally increased pain unpleasantness ratings, the amplitude of the nociceptive flexion reflex, and corrugator responses to the noxious stimulation (Mailhot et al., 2012). In another study, the influence of pictures displaying noxious stimulation to the foot or hand of others compared to pain faces on pain perception was investigated (Vachon-Presseau et al., 2011). Again, enhanced pain perception was found when participants viewed others' pain, however, more robust facilitation of pain perception was found in response to images of noxious stimulation to the foot and hand compared to facial expression of pain (Vachon-Presseau et al., 2011). Roy et al. (2013) recently demonstrated that characteristic visual features of pain expressions are sufficient to induce this enhanced pain perception. Together, these studies confirm the augmentation of pain responses when observing pain in others, although no control for other (negative) facial expressions was used and hence it remains unclear whether this effect is specific to facial expressions of pain.

Besides the lack of information of the pain-specificity of these effects, little is known to date about the mutual effects of the perception of facial expressions of pain in others and the own pain sensations. Given the observations from functional neuroimaging that seeing others facial expression of pain leads to activations in pain-related areas in the brain of the observer, one may assume

strong interactions between seeing pain of others and feeling pain. A number of studies using methods such as functional magnetic resonance imaging and electrophysiological recordings have provided support for this view by showing increased activity in pain-related brain regions during perception of pain in others (Fan et al., 2011; Lamm et al., 2011) including facial expressions of pain (Vachon-Presseau et al., 2012).

In a recent study we aimed at investigating both the effects of facial expressions of pain on the actual perception of pain, and vice versa the influence of pain sensation on the affective ratings of facial expressions of pain (Reicherts et al., 2013). To this end, participants received painful thermal stimuli while passively watching dynamic facial expressions (joy, fear, pain, and a neutral expression). To compare the influence of complex visual with low-level stimulation, a central fixation cross was presented as control condition. Participants were asked to rate the intensity of the thermal stimuli and also to evaluate valence and arousal of the facial expressions. In addition, facial electromyography was recorded as an index of emotion and pain perception. Results show that faces in general compared to the low-level control condition decreased pain ratings, suggesting a general attention modulation of pain by complex (social) stimuli. In addition, the facial response to painful stimulation was found to correlate with pain intensity ratings. Most important, painful thermal stimuli increased the perceived arousal of simultaneously presented fear and especially pain expressions of others, and vice versa, pain expressions of others compared to all other facial expressions led to higher pain ratings. The independent effects of attention and facial expressions on pain ratings are depicted in **Figures 2A,B**, the selective enhancement of arousal ratings of pain faces by pain in **Figure 2C**.

These findings demonstrate that the relation between pain and emotion is bidirectional with pain faces showing selectively mutual influences. This study provides further experimental evidence that processing painful stimulation and the facial expressions of pain in others are highly interconnected. Extending previous findings it also shows pain-specific modulations of pain perception

such that highest pain ratings of painful thermal stimuli were obtained while participants watched faces of pain compared to other facial expressions. Importantly, this effect was also larger for pain compared to fear faces, suggesting that the facial expressions of pain enhance self-pain perception not only due to its negative valence but due to its pain relevance. In addition to the predictions from the motivational priming theory, these results support the notion that not only the valence of a facial expressions enhances pain perception, but that the expressed pain itself primes the sensorimotor system, which might drive a potentiating pro-algesic mechanism (Godinho et al., 2012). Additional evidence for this amplification of pain driven by the perception of others' pain comes from several studies showing that watching facial pain expressions results in augmented pain ratings (Vachon-Presseau et al., 2011, 2012, 2013; Mailhot et al., 2012).

Another potential mechanism of pain modification in addition to the affective priming hypotheses has been put forward as the PAM of empathy (Preston and de Waal, 2002). This model would postulate that the observation of others' pain activates a similar neural network implicated in the first-person experience of the very phenomenon (Jackson et al., 2005). Accordingly, the perceived pain expression of others is mapped on the observer's own neural representations and as such facilitates and primes own-pain perceptions. This shared representations account has been supported by neuroimaging studies (e.g., Jackson et al., 2006). However, it has to be noted that the brain responses to pain and to facial expressions of pain may not indicate shared representations of actual pain and observed pain, but a much more unspecific response to salient stimuli (Iannetti et al., 2013).

This mini-review featured recent work on the emotional modulation of pain perception by affective stimuli such as facial expressions, but more importantly on the reverse impact of pain on emotional face processing. The presented studies also further underscore the special relevance of facial expressions of pain. The functional significance of pain faces for human

**FIGURE 2 | Pain intensity ratings (VAS 0 = no pain – 100 = extreme pain) when watching facial expressions (A), and collapsed across facial expressions compared to trials with crosshairs (B).** Pain-specific enhancement of pain perception is shown in **(A)**, whereas **(B)** depicts the attentional effect of watching complex social stimuli such as faces compared to low-level visual stimulation (crosshairs) which results in reduced pain. Data

are taken and figure is adapted with permission from Reicherts et al. (2013). In **(C)**, the arousal ratings for facial expressions are given, when participants experienced painful and non-painful thermal heat. The figure has been reproduced with permission of the International Association for the Study of Pain® (IASP). The figure may not be reproduced for any other purpose without permission.

social interaction, however, is still under debate, therefore future work needs to clarify whether they elicit predominantly approach or avoidance behavior in the observer (Yamada and Decety, 2009; Hadjistavropoulos et al., 2011). This would probably be accomplished best by incorporating measures of behavioral consequences (Krieglmeyer et al., 2013). The most recent study in our research program aimed at combining methods investigating mutual influences of pain and emotion processing. These results revealed exciting insights on how pain and facial expressions of pain may interact. Clearly, future research along the aforementioned theories should clarify the specificity of pain enhancement due to pain faces but also further elucidate the common neural substrates of pain and facial emotion processing.

#### **ACKNOWLEDGMENTS**

The research summarized in this review was supported by the Research Group "Emotion and Behavior," FOR 605, Wi2714/3-2, and a research grant Pa 566/3-2, both sponsored by the German Research Foundation (DFG). The authors would like to thank Ramona Baur, Marion Höfle, and Yiting Liu for data collection and analysis, and all participants for taking part in the experiments.

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 29 July 2014; accepted: 24 September 2014; published online: 13 October 2014.*

*Citation: Wieser MJ, Gerdes ABM, Reicherts P and Pauli P (2014) Mutual influences of pain and emotional face processing. Front. Psychol. 5:1160. doi: 10.3389/fpsyg.2014.01160*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Wieser, Gerdes, Reicherts and Pauli. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited andthatthe original publication inthis journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Corrugator activity confirms immediate negative affect in surprise

# **Sascha Topolinski <sup>1</sup>\* and Fritz Strack<sup>2</sup>**

<sup>1</sup> Department of Psychology, Social and Economic Cognition, University of Cologne, Cologne, Germany <sup>2</sup> University of Würzburg, Würzburg, Germany

#### **Edited by:**

Paul Pauli, Julius-Maximilians-University, Germany

#### **Reviewed by:**

Nicholas T. Van Dam, Nathan S Kline Institute for Psychiatric Research, USA

Rainer Reisenzein, University of Greifswald, Germany

#### **\*Correspondence:**

Sascha Topolinski, Department of Psychology, Social and Economic Cognition, University of Cologne, Richard-Strauß-Straße 2, 50931 Cologne, Germany e-mail: sascha.topolinski@ uni-koeln.de

The emotion of surprise entails a complex of immediate responses, such as cognitive interruption, attention allocation to, and more systematic processing of the surprising stimulus. All these processes serve the ultimate function to increase processing depth and thus cognitively master the surprising stimulus. The present account introduces phasic negative affect as the underlying mechanism responsible for this switch in operating mode. Surprising stimuli are schema-discrepant and thus entail cognitive disfluency, which elicits immediate negative affect. This affect in turn works like a phasic cognitive tuning switching the current processing mode from more automatic and heuristic to more systematic and reflective processing. Directly testing the initial elicitation of negative affect by surprising events, the present experiment presented high and low surprising neutral trivia statements to N = 28 participants while assessing their spontaneous facial expressions via facial electromyography. High compared to low surprising trivia elicited higher corrugator activity, indicative of negative affect and mental effort, while leaving zygomaticus (positive affect) and frontalis (cultural surprise expression) activity unaffected. Future research shall investigate the mediating role of negative affect in eliciting surpriserelated outcomes.

**Keywords: surprise, fluency, affect, EMG, expectancy**

# **IMMEDIATE NEGATIVE AFFECT IN SURPRISE**

Surprise is a distinct emotional response to events that are discrepant with the schema of a current situation (e.g., Smedslund, 1990; Meyer et al., 1991, 1997; Ekman, 2003). There is no agreement in the literature on the particular valence of surprise as an emotion. While some approaches label surprise as a positive emotion (e.g., Fontaine et al., 2007; Valenzuela et al., 2010), other argue that surprise has no valence at all (e.g., Russell, 1980; Reisenzein and Meyer, 2009; Reisenzein et al., 2012). However, there is growing indirect evidence that particularly the immediate phase of cognitive interruption during surprise triggers negative affect (Noordewier et al., submitted). In several independent lines of research in social psychology it has been shown that inconsistencies, disruption, and lack of structure are experienced as unpleasant (Elliot and Devine, 1994; Kay et al., 2009; Gawronski and Strack, 2012; Proulx et al., 2012; Rutjens et al., 2013). For instance, Mendes et al. (2007) found that targets who disconfirm a certain stereotype schema, such as a Latino with high socio-economic status, or an Asian with a southern accent, are liked less than stereotypeconsistent exemplars. More directly demonstrating a negative component in surprise, Noordewier and Breugelmans (2013) examined the spontaneous facial expressions of individuals who were surprised by unexpected turning of events in TV shows. Facial codings of these expressions showed negative overall expressions.

The evolutionary function of surprise is to facilitate cognitive mastering of unexpected events (Plutchik, 1980; Meyer et al., 1991, 1997; Forabosco, 1992; Attardo, 1997). This is realized by a whole *emotion syndrome* (Reisenzein, 2000) of attentional and cognitive mechanisms that all serve an enhanced and more thorough processing of the surprising stimulus: interruption of ongoing mental operations (Meyer et al., 1991; Reisenzein, 2000), behavioral freezing (Scherer et al., 2004), recruitment of executive capacity (Näätänen, 1990), attention-allocation to the surprising event (e.g., Darwin, 1872/1999; Ekman et al., 2002), and increase in effortful processing (e.g., Meyer et al., 1991, 1997; Scherer, 2001; Horstman, 2006). As a consequence, surprising stimuli are more elaborated cognitively (cf., *sensemaking*, Pezzo, 2003; *cognitive mastering*, Attardo, 1997) and more likely remembered at a later time (e.g., Schützwohl, 1998) in comparison to nonsurprising stimuli. In a dual-system perspective, surprise is thus an effective functional switch between more automatic and routine processing to more effortful and reflective operations (Strack and Deutsch, 2004), functioning like a *cognitive tuning* from heuristic to systematic processing (Bless, 2001; Schwarz, 2002; Deutsch and Strack, 2008). The question is how this switch operates. Which powerful mechanism enables all these different yet functionally converging processes?

In this paper, we propose phasic negative affect as a mechanism. In the following, we argue that schema-discrepant events evoke cognitive disfluency which in turn triggers negative affect that tunes the cognitive system to more systematic processing. Thus, in contrast to earlier models that hold that schemadiscrepancy triggers surprise as an affectively neutral signal directly (e.g., Meyer et al., 1997), we introduce negative affect as a causal mediator between discrepancy and surprise. This procedural account of surprise lends insights from various cognitive and social psychological frameworks and deepens our understanding of the causal architecture of surprise.

#### **SURPRISE AS COGNITIVE DISFLUENCY**

A powerful psychological concept in explaining various phenomena is processing fluency, which is the content-independent speed and efficiency of ongoing mental operations (Reber et al., 2004), such as perceptual (e.g., Reber et al., 1998), or semantic processing (Whittlesea, 1993). High processing fluency elicits positive affect (Winkielman and Cacioppo, 2001), while low processing fluency elicits negative affect and more thorough processing (Alter et al., 2007). There are various ways to manipulate fluency experimentally, such as degrading and enhancing perceptual clarity (Reber et al., 1998), repeated exposure (Moreland and Topolinski, 2010), motor training (Topolinski and Strack, 2009c, 2010; Topolinski, 2010, 2012), or semantic priming (Whittlesea, 1993). Particularly semantic fluency is of particular interest for the case of surprise. Previous research has shown that the fluency of processing the meaning of a stimulus is facilitated when this stimulus occurs in a semantically predictive context (Whittlesea, 1993) or in semantic coherence with other stimuli (Topolinski and Strack, 2008, 2009a,b,d). For instance, a coherent word triple such like SALT DEEP FOAM (implying the common topic SEA) is processed faster than an incoherent word triple like BALL BOOK DREAM (implying no common topic; Topolinski, 2011).

How is surprise connected to fluency? According to the major theories on surprise, a surprising event is defined as a stimulus or stimulus change that is discrepant or inconsistent with the currently activated general schema of the situation (e.g., Smedslund, 1990; Meyer et al., 1991, 1997; Ekman, 2003). This may be the case because the stimulus was not expected or the expectancy of another stimulus or event had been evoked, like a sweet taste when one has expected a sour taste (Teigen and Keren, 2003). However, a conscious expectancy about an occurring event is not necessary for surprise to occur. Any event that is inconsistent with the current active schema will be surprising, even if no conscious expectancy was held. For instance, running into your house neighbor in a foreign airport is strongly surprising, although you have not held conscious expectancies in mind about the likelihoods of meeting certain people on the airport. In terms of semantic processing, surprising events can thus be conceptualized as events that are inconsistent with the currently activated semantic context. Thus, these events elicit semantic disfluency, which has been shown in previous research to evoke certain psychological consequences. In the following we argue that these consequences should also occur for surprising events.

#### **SURPRISE, DISFLUENCY, AND NEGATIVE AFFECT**

While relatively high processing fluency elicits positive affect, low fluency prompts immediate negative feelings (Hajcak and Foti, 2008; Topolinski et al., 2009; Hajcak, 2012). For instance, low compared to high-fluent stimuli are liked less, judged as being false, toxic, or less famous and funny (Topolinski and Reber, 2010a,b; Topolinski and Strack, 2010; Leder et al., 2013; Topolinski, 2014; Topolinski et al., 2014b). Such a phasic negative affect, independent from fluency, has been shown to function like negative mood in *cognitive tuning*, where negative compared to positive affective states inhibit automatic and heuristic processes and induce more systematic and effortful processing (e.g., Bless et al., 1996; Kuhl, 2000; Baumann and Kuhl, 2002; Ruder and Bless, 2003). For instance, phasic negative compared to positive affect, when induced randomly, changing from trial to trial and lasting only for around a second, decreases creative performance (Topolinski and Deutsch, 2012), or inhibits automatic semantic processing (Topolinski and Deutsch, 2013).

Crucially, also disfluency-triggered negative affect can function as a rapid cognitive tuner to more systematic processing. Alter et al. (2007) showed that experimentally induced disfluency facilitated systematic processing of syllogistic reasoning problems, a core facility of reflective processing (Strack and Deutsch, 2004).

Applying this to surprising events, we argue that surprising stimuli are cognitively disfluent (since they are schema-discrepant by definition) and thereby elicit immediate negative affect that then tunes cognitive processing from a more heuristic, impulsive mode to a more systematic, effortful mode in the further course of psychological functioning. It should be emphasized, however, that this is a subtle negative state that is not necessarily conscious (cf., Russell, 2003; Winkielman and Berridge, 2004; Topolinski and Deutsch, 2012) and not a strong emotional experience. Thus, we do not argue that this brief negative affect determines the eventual experiential evaluation once the surprising event is fully mastered. Of course, we can be positively surprised and feel joy in response to unanticipated positive outcomes, or can be negatively surprised by bad news. This eventual valence of the later surprise feeling is rather determined by the valence of the event itself (see also Noordewier and Breugelmans, 2013). Negative affect only pertains to the immediate state when encountering a schemadiscrepant event.

In sum, we argue that negative affect is the mechanism triggering a switch in operation mode from automatic or heuristic processing to more controlled and effortful processing, as well as all further attentional and motivational consequences such a switch from heuristic to analytical processing brings along, such as attention allocation and deeper processing (Meyer et al., 1991; Reisenzein, 2000).

The crucial first prediction of this account, however, is that surprising stimuli elicit negative affect. As already mentioned, this has been shown recently by Noordewier and Breugelmans (2013) for facial codings of TV show participants. However, these expressions were coded as whole-face responses and did not differentiate specific facial muscles indicative of more specific affective responses. Because of this shortcoming, the present study examined affective facial responses to surprising stimuli in a more controlled experimental set-up with a psychophysiological method, namely facial electromyography (EMG). This also allowed to disentangle independent changes in positive and negative affect (Winkielman and Cacioppo, 2001).

#### **AIM OF THE PRESENT RESEARCH**

The present study tested the initial stage of affective consequences of surprise as outlined above, the automatic elicitation of negative affect. Note that the present scope was not about the further cognitive consequences of surprise such as attentional allocation or deeper processing. The current notion of (dis)fluency only refers to the immediate online efficiency in encoding a stimulus, not the later increased mental effort that is elicited by surprise. As one operationalization of schema-discrepant, thus surprising, information, we chose trivia statements that had been pre-rated as being more or less surprising (Reisenzein, 2000). For the case of trivia, the surprise and thus disfluency does not stem from a situationally primed context or even expectancy, but from the degree to which certain trivia are (in)consistent with our chronically activated *general knowledge structures* (e.g., Bless et al., 1996). Thus, surprising trivia do not match semantic knowledge structure and thus exhibit *semantic disfluency*.

As a genuine indicator of spontaneous affect we assessed spontaneous facial muscle activity via facial EMG (Cacioppo et al., 1986; Dimberg et al., 2000; but see, for recent debate on the coherence between affect and facial expression, Hassin et al., 2013; Reisenzein et al., 2013). Specifically, three muscles were investigated that have been shown to be indicative of affective responses. The *M. zygomaticus major*, which raises the corner of the lips in smiling, is indicative of positive affect (Cacioppo et al., 1986; Scherer and Ellgring, 2007). The muscle has also been shown to be associated with gains in processing fluency (Harmon-Jones and Allen, 2001; Winkielman and Cacioppo, 2001), and with semantic fluency. Topolinski et al. (2009) presented word triples to participants who were told that these would be random words and were asked to merely read over these triples. In some of these triples, the words were not random but semantically coherent (e.g., DEEP FOAM SALT all related to the common concept of SEA), while in other triples the words were actually random, that is, semantically incoherent (e.g., DREAM BALL BOOK). It was found that participants showed higher zygomaticus activity for coherent than for random triples. This occurred because coherent compared to incoherent triples were encoded with a higher semantic fluency.

The *M. corrugator supercilii*, which furrows the brows, is indicative of negative affective states (e.g., Ekman, 1973; Cacioppo et al., 1986) and to difficulty in information processing, that is, disfluency (Cacioppo et al., 1985; Cohen et al., 1992; see, for inductions of subjective mental effort due to corrugator activation, Larsen et al., 1992; Stepper and Strack, 1993; Strack and Neumann, 2000). In Topolinski et al. (2009), higher corrugator activity was found during encoding of incoherent compared to coherent semantic information. This makes the corrugator the prime indicator of disfluency-triggered negative affect in the present argumentation.

Finally, the *M. frontalis medialis*, which raises the eye brows, surely is the prima facie indicator of surprise, since the iconic surprise face in cultural displays and actors involves raised eye brows (Scherer and Ellgring, 2007). And indeed, Topolinski et al. (2009) found increased activity of the frontalis for the encoding of incoherent (i.e., disfluent) semantic information compared to coherent information. However, several other studies that instantiated real surprising events in more ecologically valid ways, such as changing a whole room, found only a weak impact on frontalis activity (Reisenzein et al., 2006; Schützwohl and Reisenzein, 2012).

Following the current argumentation that surprise prompts negative affect, we predicted increased corrugator activity for highly compared to low surprising trivia statements (cf., Topolinski and Strack, 2009b). Furthermore, we predicted lower zygomaticus activity for highly compared to low surprising stimuli as disfluent surprising stimuli should elicit negative affect (cf., Winkielman and Cacioppo, 2001; Topolinski et al., 2009). Concerning the frontalis, it remained an empirical question whether frontalis would be susceptible to the present manipulation (see Topolinski et al., 2009 vs. Reisenzein et al., 2006). We did not implement any other measure (such as explicit affective ratings) to ensure that participants' facial responses reflected spontaneous affective processes running independent from a conscious evaluative mind-set or any demand effects.

# **MATERIALS AND METHODS**

#### **PARTICIPANTS**

Twenty-eight volunteers from various professional backgrounds from the city area of Würzburg participated for a financial compensation of e10 (21 female, 7 male, *M*age = 26, SDage = 5). They were recruited via a local mini-job online market. The volunteers were not screened for psychological or neurological disorders, and due to anonymity issues, we did not record other demographic variables such as ethnicity. Unfortunately, this sample size confines the power of the present tests, but it is common for such time-consuming and laborious methods (cf., Winkielman and Cacioppo, 2001).

#### **MATERIALS**

Trivia statements were developed using the pool of quiz items used in Reisenzein (2000). <sup>1</sup> These original items were compounds of questions with true and false answers. For instance, *The invention of matches is attributed to?*, *Johnny Walker* (correct answer), and *Robert Bosch* (false answer). For these items, norming ratings are available on how surprising the correct answer is to German samples. In these norming ratings, *N* = 60 participants had rated how surprising the given fact was to them. Using these ratings, 22 high-surprising (*M*surprising\_rating = 5.67, SD = 0.45) and 22 lowsurprising (*M*surprising\_rating = 1.80, SD = 0.66) facts were chosen, with a strong resulting difference in normative ratings on surprise in Reisenzein (2000), *t*(42) = 22.68, *p* < 0.001. Then, a trivia statement was created by combining the initial question with the correct answer, for instance *The invention of matches is attributed to Johnny Walker* (highly surprising), or *Women have a higher life expectancy than men* (low surprising). The stimuli (in German language) are available from the first author upon request.

#### **Manipulation check**

To check whether these materials still elicited similar surprise levels than in the rating by Reisenzein (2000), *N* = 12 individuals similar in age and education to the main sample and being

<sup>1</sup>We thank Rainer Reisenzein for kindly supplying the whole stimulus pool and norming ratings.

ignorant about the current research thrust received these 44 items and were asked to rate how surprising each item is (0 = *not at all surprising* to 10 = *very surprising*). As expected given the selection of extreme items, there was a very large difference between the items: (*M*high\_surprise\_items = 6.97, SD = 1.00 vs. *M*low\_surprise\_items = 2.10, SD = 0.90), *t*(11) = 15.17, *p* < 0.001, *d* = 5.23.

#### **PROCEDURE**

The study had ethical approval from the Deutsche Forschungsgemeinschaft (Str 264/25-1). After informed consent, participants were tested in single sessions. Participants were told that skin conductance was measured by electrodes to cover the actual facial muscle recording (see Dimberg et al., 2000). The trivia viewing task was part of a larger experimental battery involving other, unrelated tasks (studying non-sense words, Topolinski, 2012; watching neutral geometric shapes, Topolinski et al., submitted). In the trivia viewing task, participants were told that a recreation phase in between two other tasks, necessary for physiotechnological reasons, would follow in which participants should relax. For their entertainment, general trivia facts would be presented on the PC screen. They were not given any specific encoding instruction but were asked to stay relaxed and watch the events unfolding on the PC screen. Thus, participants simply read the trivia statements, and no response or any task was required. Then, in altogether 44 trials, 22 high surprising and 22 low surprising trivia items were presented in random order, re-randomized anew for each participant. In each trial, first a fixation cross appeared in the middle of the screen for 1000 ms followed by a blank screen for 1100 ms. Then the trivia item was presented in black font in the middle of screen for 6000 ms. Then, an inter-trial interval being a white screen followed with a length randomly varying between 5000 and 6000 ms. The whole task took around 10 min.

#### **EMG ASSESSMENT**

The electrical activity was measured over the *M. zygomaticus major*, the *M. corrugator supercilii*, and the *M. frontalis medialis* on the left side of the face using bipolar placements of 13/7 mm Ag/AgCl surface-electrodes (Fridlund and Cacioppo, 1986). The impedances of all electrodes were reduced to less than 10 kOhm. The EMG raw signal was measured with a V-Amp amplifier (Brain Products Inc.), digitized by a 24-bit analog-to-digital converter, and then stored on a computer with a sampling frequency of 500 Hz. These raw data were rectified offline and filtered with a 30 Hz low cutoff filter, a 500 Hz high cutoff filter, a 50 Hz notch filter, and a 125 ms moving average filter.

To control for baseline activity, EMG scores were calculated being the difference between the activity in the given trial and a pre-stimulus level, namely the mean activity during the last 1000 ms before stimulus onset. Trials with an EMG activity above 8 µV during the baseline period and above 30 µV during the stimuli presentation were excluded (<8%). Then, the EMG scores were aggregated over the 6000 ms of trivia item presentation, for high and for low surprising trivia, respectively.

#### **RESULTS**

The conditional means are shown in **Figure 1**. A 3 (muscle: zygomaticus, corrugator, frontalis; within) × 2 (trivia: high surprise,

low surprise; within) analysis of variance (ANOVA) over the electrodermal activity yielded a main effect of muscle, *F*(1,26) = 6.94, *p* = 0.004, η 2 <sup>p</sup> = 0.35, with generally higher activity of the corrugator compared to the other muscles overall, which is conceptually irrelevant. Furthermore, a main effect of trivia surfaced, *F*(1,27) = 6.04, *p* = 0.021, η 2 <sup>p</sup> = 0.19, with generally more muscle activity for surprising than for common trivia. Crucially, a marginal interaction between muscle and trivia was found, *F*(1,26) = 3.15, *p* = 0.060, η 2 <sup>p</sup> = 0.20.

According to our predictions of a differential impact of surprise on the single muscles, we ran single planned tests separately for each muscle. For the corrugator, there was higher activity during encoding of high surprising compared to low surprising trivia, *t*(27) = 2.61, *p* = 0.014, *d* = 0.48, 95% CI [0.06, 0.53]. There were no effects of surprise on zygomaticus and frontalis activity (both *t*s < 1.4, *p*s > 0.19), see also **Figure 1**. Corrugator activity was reliably above zero for both high and low surprising items (*t*s < 3.91, *p*s < 0.011). Zygomaticus activity was reliably below zero for less surprising items, *t*(27) = 2.61, *p* = 0.015.

#### **DISCUSSION**

Examining the initial affective responses to neutral surprising stimuli, we assessed spontaneous facial expressions during merely reading high and low surprising trivia statements. As we had argued, high compared to low trivia statements are inconsistent with the individuals' common knowledge structures (Bless et al., 1996) and are thus cognitively disfluent, eliciting negative affect (Alter et al., 2007; Topolinski et al., 2009; Noordewier and Breugelmans, 2013). We found increased corrugator activity for high surprising compared to low surprising trivia, while zygomaticus and frontalis were not significantly affected. Before discussing this result, it should be emphasized that the present rather small sample confined the present statistical power, and the present preliminary evidence should be interpreted with caution.

This finding corroborates our assertation that surprise entails an immediate negative affective state and provides more specific evidence on facial expressions than whole-face codings (see also Noordewier and Breugelmans, 2013). However, the present initial demonstration is only a starting point for experimentally manipulating and testing the whole procedural chain that we assume for surprise to be at work, and also to test the mediational role of negative affect. The current evidence also allows the reversed causal interpretation that negative affect is only a by-product of the whole attentional and cognitive syndrome of surprise of increased cognitive effort and cognitive mastering (Topolinski, 2014). Future studies should show that high compared to low surprising events do not only elicit negative affect, but that this negative affect in turn induces a more thorough processing of surprising stimuli and is thus correlated with further psychological consequences of surprise, such as attention allocation and more reflective processing (Plutchik, 1980; Schützwohl, 1998; Reisenzein, 2000). Furthermore, negative affect in the present setup might have stemmed not only from the initial dysfluency in reading the trivia, but from frustration in additional stimulus elaboration and memory-retrieval during failure of making sense particularly of surprising items (cf., Pezzo, 2003). However, note that the present time window of EMG measures was only the first 6 s after stimulus onset. Because (1) participants needed 1–2 s to read the trivia in the first place, (2) memory retrieval and factchecking itself requires another 1–2 s (Collins and Quillian, 1969), and (3) facial activity in response to higher mental processes requires itself time to unfold, we argue that such additional cognitive processes unlikely affected the current data.

A further limitation of the present evidence might be the use of the corrugator muscle as an indicator of negative affect, because the corrugator does not only mark negativity, but also mental effort (Cacioppo et al., 1985; Cohen et al., 1992). It could be argued that the presently found increased activity of the corrugator muscle is rather due to higher mental effort or increased processing depth in integrating the surprising compared to the less surprising trivia statements. However, note that the frontalis muscle was not affected by the present manipulation. In the literature examining the facial responses to mental effort more generally, whenever both corrugator and frontalis were assessed, the frontalis muscle was affected by mental effort to a comparable degree as the corrugator muscle (e.g., Van Boxtel and Jessurun, 1993; Waterink and Van Boxtel, 1994; Waersted and Westgaard, 1996; Bansevicius et al., 1997). Accordingly, the frontalis has been conceptualized as an independent indicator of mental effort (Van Boxtel and Ven, 1978; Fridlund and Cacioppo, 1986). In ergonomics, some authors even suggested frontalis being the more valid indicator of mental effort because it is, in contrast to the corrugator, not affected by valence (Zeier, 1979; de Waard, 1996; Piechulla et al., 2003). The impact of the present manipulation on corrugator but not on frontalis activity thus favors our interpretation that negative affect, but not mental effort *per se*, drove the present responses. However, convergent validity of this facial measure should be obtained in future studies using other measures of negative affect, such as evaluations or approach–avoidance movements (e.g., Topolinski et al., 2014c). Also, although the present item pool was already successfully used in earlier publications (e.g., Reisenzein, 2000), it is still possible that the present items elicited not only surprise, but also confusion or annoyance, which should be disentangled in future studies.<sup>2</sup>

The missing impact of surprise on frontalis activity, which is the prima facie muscle for iconic cultural displays of surprise (for a discussion, see Noordewier and Breugelmans, 2013), might be regarded as being at odds with earlier findings that frontalis is reliably affected by highly surprising events (e.g., Reisenzein et al., 2006). However, detectable frontalis activity in these earlier studies was either weak or occurred in only up to a third of the observed participants (Reisenzein, 2000; Reisenzein et al., 2006; Schützwohl and Reisenzein, 2012). Given that the trivia items we used were not validated in the present participant pool, it could well be possible that they did not elicit enough surprise to trigger frontalis activity (which is weak as it is). Future research is necessary to further ascertain the role of frontalis in the display of surprise.

The null-finding on zygomaticus activity was not predicted and deserves some discussion. In contrast to earlier findings that high compared to low fluency increases zygomaticus activity, thus induced positive affect independent from changes in negative affect (Winkielman and Cacioppo, 2001; Topolinski et al., 2009) we found no impact of surprise on zygomaticus activity. The first possible explanation of course is lacking power, since the present sample size was small (but similar to earlier studies in this domain, e.g., Winkielman and Cacioppo, 2001). However, the direction of the difference in zygomaticus activity was, descriptively, even opposite to what would be expected on fluency grounds: participants exhibited descriptively *higher* zygomaticus activity for highthan low-surprising items.

This can be explained by the relativity of fluency effects (Hansen et al., 2008; Hansen and Topolinski, 2011), as revealed by a closer comparison between the earlier EMG study on semantic fluency (Topolinski et al., 2009) and the present study. In Topolinski et al. (2009), participants received coherent and incoherent word triples and were told that these word triples were random. In addition to this information, one would usually not expect a regularity in word groups in general. Thus, coherent triples showed an unusual fluency gain, higher than one would expect when reading random word groups, eliciting a positive affect (as evidenced by increased zygomaticus activity, but also higher liking ratings, Topolinski and Strack, 2009a,d). In the present study, however, participants received high and low surprising trivia statements. Here, low surprising trivia, although of course having higher semantic fluency than highly surprising trivia, did not exhibit an usually high fluency *compared to what one would expect during reading trivia statements in general*. Thus, an unusual fluency gain and thus positive affect were less likely to occur in the present set-up. However, future research should more closely investigate the respective impact of semantic fluency on positive and negative facial responses under different fluency expectations. Finally, it is also possible that the present trivia statements were simply more complex than the word triples in Topolinski et al. (2009), imposing higher cognitive demand on participants and thereby inhibiting zygomaticus activity more generally.

Finally, a conceptual integration between the present account and earlier highly influential accounts on the mechanism of surprise should be made. While we completely concur with most aspects of the model provided by Meyer and Reisenzein (e.g., Meyer et al., 1991; Reisenzein, 2000), the difference between both

<sup>2</sup>We thank the reviewer for these thoughts.

accounts is that Meyer et al. (1991) assume that unexpectedness or schema-discrepancy triggers the neutral state of surprise along with all its components directly, while we assume that discrepancy impacts on the cognitive architecture via the link of brief negative affect. We add this mechanistic assumption because we do not know of any plausible perceptual, cognitive, or (other) affective route by which schema-discrepancy should directly causally change the information processing mode (as assumed by Meyer and Reisenzein). Given the previous independent evidence that disfluency triggers negative affect and modulates processing style (e.g., Winkielman and Cacioppo, 2001; Alter et al., 2007), negative affect is an obvious candidate to effectuate such a modulation; and the detection of brief negative affect in the present experiment is first supporting evidence for our claim. Furthermore, if unexpectedness directly triggers a change in processing mode without any further mediator (like in the model by Meyer and Reisenzein), the question is why immediate brief negative affect did occur in the present study as well as in Noordewier and Breugelmans (2013). Thus, we deem the assumption that schema-discrepancy modulates the cognitive operational mode by triggering a brief negative affect as the most parsimonious interpretation and integrative interpretation of the present and earlier findings.

Concluding, the present evidence showed immediate negative facial responses to neutral stimuli that were more or less surprising. Future research should map the mediating role of negative affect in eliciting further cognitive consequences of surprise.

## **ACKNOWLEDGMENT**

This research was funded by the Deutsche Forschungsgemeinschaft (DFG; Str 264/25-1).

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 June 2014; accepted: 26 January 2015; published online: 16 February 2015.*

*Citation: Topolinski S and Strack F (2015) Corrugator activity confirms immediate negative affect in surprise. Front. Psychol. 6:134. doi: 10.3389/fpsyg.2015.00134*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology.*

*Copyright* © *2015 Topolinski and Strack. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Avoidant decision making in social anxiety: the interaction of angry faces and emotional responses

# *Andre Pittig1,2 , Mirko Pawlikowski <sup>3</sup> , Michelle G. Craske2 and Georg W. Alpers1\**

<sup>1</sup> Chair of Clinical and Biological Psychology and Psychotherapy, Department of Psychology, School of Social Sciences, University of Mannheim, Mannheim, Germany

<sup>2</sup> Anxiety Disorders Research Center, University of California, Los Angeles, CA, USA

<sup>3</sup> General Psychology: Cognition, Department for Informatics and Applied Cognitive Science, University of Duisburg-Essen, Essen, Germany

#### *Edited by:*

Fritz Strack, University of Würzburg, Germany

#### *Reviewed by:*

Seung-Lark Lim, University of Missouri-Kansas City, USA Daniel S. Quintana, University of Oslo, Norway

#### *\*Correspondence:*

Georg W. Alpers, Chair of Clinical and Biological Psychology and Psychotherapy, Department of Psychology, School of Social Sciences, University of Mannheim, Box 103462, 68131 Mannheim, Germany e-mail: alpers@uni-mannheim.de

Recent research indicates that angry facial expressions are preferentially processed and may facilitate automatic avoidance response, especially in socially anxious individuals. However, few studies have examined whether this bias also expresses itself in more complex cognitive processes and behavior such as decision making.We recently introduced a variation of the Iowa Gambling Task which allowed us to document the influence of taskirrelevant emotional cues on rational decision making. The present study used a modified gambling task to investigate the impact of angry facial expressions on decision making in 38 individuals with a wide range of social anxiety. Participants were to find out which choices were (dis-) advantageous to maximize overall gain. To create a decision conflict between approach of reward and avoidance of fear-relevant angry faces, advantageous choices were associated with angry facial expressions, whereas disadvantageous choices were associated with happy facial expressions. Results indicated that higher social avoidance predicted less advantageous decisions in the beginning of the task, i.e., when contingencies were still uncertain. Interactions with specific skin conductance responses further clarified that this initial avoidance only occurred in combination with elevated responses before choosing an angry facial expressions. In addition, an interaction between high trait anxiety and elevated responses to early losses predicted faster learning of an advantageous strategy. These effects were independent of intelligence, general risky decision-making, self-reported state anxiety, and depression. Thus, socially avoidant individuals who respond emotionally to angry facial expressions are more likely to show avoidance of these faces under uncertainty. This novel laboratory paradigm may be an appropriate analog for central features of social anxiety.

**Keywords: decision making, behavioral avoidance, social anxiety, facial expressions, psychopathology**

# **INTRODUCTION**

Avoidance is the characteristic action tendency associated with anxiety (Hofmann et al., 2009). It is triggered by emotional responses toward specific fear-relevant stimuli and can protect these emotional responses from extinction (Lovibond et al., 2009). Adaptive behavior, however, requires the individual to obtain reward or positive consequences. Investigating avoidance by itself, therefore, only accounts for one side of a two-sided balance between avoidance and approach (Stein and Paulus, 2009). This broader view on avoidance takes into account that anxious individuals miss out on potential benefits and therefore suffer costs. In social anxiety, these two tendencies are often in conflict with each other, because socially anxious individuals are explicitly aware of lost benefits, even for their most feared situations (Kashdan et al., 2008). For example, socially anxious individuals are often afraid of job interviews, although they are aware of the potential benefits for their career. Thus, they are in a conflict of opposing choices; to approach these benefits or avoid the situation to reduce anxiety. Pathological avoidance is, therefore, indicated by a dysfunctional shift toward avoidant decisions

(Stein and Paulus, 2009). This shift results in the loss of benefits, which illustrates the impairments of patients with anxiety disorders. Studies investigating avoidance in the context of an approach–avoidance conflict in anxious individuals should, therefore, account for both fear-relevant as well as reward-related stimuli and consequences.

We recently combined these features in a novel experimental paradigm to investigate behavioral avoidance as a decision-making process (Pittig et al., 2014a). The paradigm was based on the Iowa gambling task (Bechara et al., 1994, 2000). In our modified gambling task, spider fearful participants continuously had to make decisions with the goal to maximize overall gains. Advantageous choices to obtain this goal were, however, associated with fearrelevant stimuli (i.e., pictures of spiders), such that avoidance of the fear-relevant stimuli resulted in the loss of long-term gains. In comparison to non-fearful participants, spider fearful participants consistently avoided the fear-relevant stimuli, despite the fact that these avoidant decisions resulted in overall cost in task performance. In addition, such avoidant decisions can result not only from specific fear-relevant stimuli, such as spiders, but also from novel fear conditioning experience, especially in anxious individuals (Pittig et al., 2014b). Thus, these laboratory experiments show that the presence of fear-relevant stimuli can trigger avoidant decisions in fearful individuals. As approach–avoidance conflicts are particularly relevant to social anxiety, we hypothesize that a similar bias may be observed in socially anxious individuals when confronted with stimuli specifically relevant to social interactions.

Facial expressions are one of the most important stimuli in social situations and determine the individual's social behaviors. Emotional facial expressions are processed preferentially in the visual system (e.g., Alpers and Gerdes, 2007; Bublatzky et al., 2014b) and result in specific behavioral responses (Eisenbarth et al., 2011; Gerdes et al., 2012; Neumann et al., 2014). Angry facial expression are biologically rooted signals of threat (Öhman, 1986) and are specifically fear-relevant in subclinical and clinical levels of social anxiety. In support, recent research has pointed to a preferential processing of angry faces in healthy individuals (Mogg and Bradley, 1999; Fox et al., 2000). This preferential processing is pronounced in socially anxious individuals (Gilboa-Schechtman et al., 1999; Mogg et al., 2004; Klumpp and Amir, 2009; Wieser et al., 2009), which is accompanied by elevated amygdala activity in patients with social anxiety disorder (SAD; Stein et al., 2002). In addition, facial expressions can influence approach or avoidance tendencies. In this regard, approach-related motor responses are faster for happy faces, whereas avoidance-related responses are facilitated by angry faces (Marsh et al., 2005; Seidel et al., 2010). The difference in approach toward happy and angry faces is also evident in whole-body movements (Stins et al., 2011). The effect of a stronger avoidance response to angry faces may be elevated in socially anxious individuals, although a similar tendency was found for happy faces (Heuer et al., 2007). Finally, recent findings suggest that the difference in approach or avoidance tendencies is most pronounced when comparing angry to happy faces (Horstmann, 2003; Marsh et al., 2005; Seidel et al., 2010). Thus, confrontation with angry facial stimuli (compared to happy facial stimuli) may trigger avoidant decisions in socially anxious individuals.

There is indeed first evidence that angry facial expressions may generally bias rational decisions in healthy individuals (Averbeck and Duchaine, 2009; Furl et al., 2012). In these studies, the participants' task was to find out whether selecting either a happy or an angry facial expression yielded more frequent reward. In general, decisions were biased toward selecting the happy face, even if prior evidence favored the angry facial expressions as advantageous choice (Averbeck and Duchaine, 2009). This effect of angry facial expressions on rational decisions may be more pronounced in socially anxious individuals. However, levels of social anxiety were not controlled in previous studies. To address this issue in order to evaluate whether repeated decisions are altered by subclinical levels of social anxiety and a tendency for social avoidance, the present study used a gambling task in which advantageous decisions were linked to angry facial expressions (similar to Pittig et al., 2014a,b).

In the field of decision making, several theories have stressed a general impact of emotional experience on decisions (Bechara et al., 1997; Loewenstein et al., 2001). These theories provide a powerful framework to investigate potential predictors of avoidant decisions in socially anxious individuals. For example, somatic marker theory (Damasio et al., 1991) suggests that emotional responses which are based on previous experience and activated during decision making can alter subsequent decisions. These responses are seen as embodied markers (or so called "gutfeelings") which are linked to specific choices. Recent research on somatic markers using the Iowa gambling task or related paradigms typically investigated skin conductance responses (SCRs) as correlates and predictors of decisions (Bechara et al., 1997; Suzuki et al., 2003; Lawrence et al., 2006; Starcke et al., 2009; Pittig et al., 2014b). Importantly, the processing of facial expressions, especially angry expressions, is also related to physiological responses (Johnsen et al., 1995; Stein et al., 2002; Springer et al., 2007; Anokhin and Golosheykin, 2010; for an overview see Alpers et al., 2011). Thus, if angry facial expressions are presented during the consideration of different options, emotional responses to these faces may bias subsequent decisions, especially in socially anxious individuals. Since emotional responses to fear-relevant stimuli habituate with repeated presentation (Bradley et al., 1993; Bublatzky et al., 2014a), a potential bias on decisions should also be most pronounced for initial presentations. However, only one study so far reported that elevated SCRs can generalize form fear conditioning and subsequently predict more pronounced avoidant decisions (Pittig et al., 2014b). To this respect, the present study used SCRs as indicators of emotional arousal during social challenges (Schulz et al., 2008) and especially during decision-making (see Bechara et al., 1997; Adolphs, 2002).

# **MATERIALS AND METHODS PARTICIPANTS**

Thirty-eight students at UCLA participated for partial course credit. Exclusion criteria were assessed through self-report screening before the assessment and included any serious medical conditions, substance abuse/dependence, current/history of bipolar disorder, psychosis, organic/traumatic brain damage, and current use of psychotropic medications or medications that may influence autonomic state. All participants provided written informed consent. All procedures were approved by the UCLA Internal Review Board. Demographic, questionnaire, and neuropsychological data of the sample are shown in **Table 1**.

# **PROCEDURES**

After informed consent was given, electrodes for the physiological measures were attached. Subsequently, participants completed a questionnaire battery including the self-rating form of the Liebowitz Social Anxiety Scale (LSAS-SR; Fresco et al., 2001), the trait version of the State-Trait Anxiety Inventory (STAI-T; Spielberger et al., 1983), and the Beck Depression Inventory (BDI-II; Beck et al., 1996). The LSAS is a 24-item questionnaire, commonly used to assess anxiety and avoidance in social interaction and performance situations. It has shown very good internal consistency, as well as good convergent and divergent validity (Heimberg et al., 1999). Participants rate each item twice, in terms of level of fear or anxiety (0 = "none"; 3 = "severe"; LSAS-Anxiety), and frequency of avoidance (0 ="never"; 3 ="usually";

**Table 1 | Demographical, clinical, and neuropsychological data of the sample.**


Means (and SD) for demographic, questionnaire, and neuropsychological data of the whole sample. N, number of participants; SPIN, social phobia inventory (Connor et al., 2000); LSAS, Liebowitz social anxiety scale (Fresco et al., 2001); STAI, state-trait anxiety inventory (Spielberger et al., 1983); BDI, beck depression inventory (Beck et al., 1996); GDT, game of dice-task (Brand et al., 2005); LPS-4, performance test system – subtest 4 (Horn, 1983).

LSAS-Avoidance). The social anxiety and avoidance scales served as main predictors in the present study.

Trait anxiety has been shown to influence decision making in the original Iowa gambling task. Although results were mixed (Miu et al.,2008;Werner et al.,2009), these studies demonstrate the need to control for general levels of unspecific trait anxiety. This was done with the 20-item STAI-T. The BDI is a self-report inventory which contains 21 items to measure the severity of depression. The BDI was used as a control measure, because depression symptoms may be associated with altered processing of reward (Mineka et al., 1998; Eshel and Roiser, 2010).

After a subsequent 5 min quiet sitting baseline, participants indicated their current state anxiety by completing the state version of the STAI.Afterward, they completed the gambling task,followed by the additional neuropsychological tasks. As the gambling task was always the first task to be completed, it was not influenced by the other tasks.

#### *Social anxiety gambling task*

The gambling task was used to measure the impact of presentation of angry facial expressions on decision making. It was modeled after the Iowa gambling task (Bechara et al., 1994, 2000), see the screenshot in **Figure 1**. The task comprised four decks of cards (A, B, C, and D). Card backs depicted pictures of two happy facial expressions (decks A and B) and two angry facial expressions (decks C and D; all pictures approximately 9.15◦ × 5.73◦ visual angle). The pictures were taken from the Karolinska Directed Emotional Faces (KDEF; Lundqvist et al., 1998), a well validated picture set with moderately expressive facial expressions (Adolph and Alpers, 2010). Due to copyright terms of the KDEF, placeholders are used in **Figure 1**. Referring to the KDEF database the following pictures were displayed on the specific decks: deck A = AF01HAS, deck B = AM01HAS, deck C = AF21ANS, deck D = AM10ANS. Participants had to select one card at a time from

one of the decks in a total of 100 trials. The mouse sensitive area for selecting a deck was reduced to a small square in the middle of each deck (approximately 0.57◦ visual angle), so that selecting a card required a fixation of the corresponding picture. After each trial, the mouse pointer moved back to the middle of the screen and had to be moved to the square again. Thus, each selection always required the participant to look at the corresponding deck and the depicted facial expression. After each selection a transparent gray shading of the decks was used to visualize this selection. This transparency ensured that the facial expressions were still visible, so that the selection could not be used as an avoidance strategy.

In order to compare physiological responses in each trial, there was another difference compared to the original Iowa gambling task. After each choice participants received feedback if they won or lost a specific amount of virtual money. In the original Iowa gambling task, gains and losses could occur simultaneously in one trial (Bechara et al., 1994, 2000). In contrast, in the present study participants could either win or lose on each trial to maintain a consistent single feedback after each trial.

Overall, decks A and B yielded large immediate gains, but occasionally even larger losses, resulting in long-term loss (−250 \$ per 10 selections), and are considered disadvantageous choices for the goal of maximizing overall gain. The disadvantageous decks always depicted the two happy facial expressions. In contrast, decks C and D yielded small immediate gains, but also small occasional losses. Therefore, they resulted in long-term gains (+250 \$ per 10 selections) and are considered advantageous choices. The advantageous decks always depicted the two angry facial expressions. Thus, to choose advantageously during the task, participants had to select the pictures of the angry facial expressions. As we focused on simulating the approach–avoidance conflict in individuals with elevated levels of social anxiety, we only investigated the link between advantageous choices and angry faces.

Participants were instructed to freely choose between the four decks with the goal to maximize their virtual monetary gains. In the beginning, they were not aware of the contingencies for gains and losses or the duration of the task. Hence, they had to use continuous feedback of gains and losses in order to learn which decks were advantageous or disadvantageous. Each participant started with a positive balance of \$2000 and could continue to play even if they lost the entire starting amount. Analogous to analyses of the Iowa gambling task (e.g., Bechara et al., 2000), the 100 trials of the gambling task were analyzed in five blocks of 20 trials each. The number of advantageous choices from decks C and D was used as the dependent variable. Higher scores indicate a higher overall outcome in the task due to choosing advantageous decks with angry facial expressions.

After completing the task, participants were asked to rate the pictures of each facial expression. All ratings were given on a 10-point Likert scale for valence/pleasantness (0 = "very unpleasant"; 9 = "very pleasant") and arousal (0 = "not at all aroused"; 9 = "extremely aroused").

#### *Neuropsychological control measures*

*Risky decision making: game of dice task.* The game of dice task (GDT; Brand et al., 2005) is a computerized dice task and was administered to control for general differences in risky decision making independent of fear-relevant stimuli. A virtual dice is thrown and participants are asked to maximize a fictitious starting capital by guessing the correct number thrown in 18 trials. Participants can guess a single number or two, three, or four numbers together. If the guess matches the thrown number, participants win a specific amount of virtual money. If not, they lose the same amount. Non-risky choices have a winning probability of 50% or higher and are linked to lower gains (i.e., a combination of three numbers with a 50% probability to win 200 € and a combination of four numbers with a 66.67% probability to win 100 €). Risky choices have a lower winning probability, but are linked to higher gains (i.e., a single number with 16.67% probability to win 1000 € or a combination of two numbers with a 33.33% probability to win 500 €). For analysis, a net score was calculated by subtracting the number of risky choices from the number of safe choices. Thus, a higher net score indicates more non-risky choices.

*Logical reasoning: performance test.* Potential differences in reasoning abilities were controlled using the Performance test system – Subtest 4 (LPS-4; Horn, 1983). This was done because such differences can influence decision making in the Iowa gambling task (Bechara et al., 1997). The LPS-4 is a non-verbal test used to estimate logical reasoning. Participants have 8 min to find a single error in a logical order of letters and numbers in a total of 40 rows. The number of errors correctly identified can be used to estimate the logical reasoning skills and intelligence of the participant.

#### **PHYSIOLOGICAL ASSESSMENT**

Electrodermal activity (EDA) was continuously recorded as measure of emotional responses during decision making using BIOPAC instrumentation (MP150 Data Acquisition System for Windows; BIOPAC Systems, Inc.). Data monitoring, acquisition, and analysis were conducted with AcqKnowledge software (Acq-Knowledge 4.1; BIOPAC Systems, Inc.). One disposable Ag/AgCl

electrode on the left clavicle served as ground electrode. EDA was recorded using BIOPAC skin conductance instrumentation with a constant voltage of 0.5 V (sampling rate = 62.50 Hz). Two disposable Ag/AgCl electrodes with electrodermal conducting gel were attached to the palmar surface of the middle phalanges of the second and third fingers of the non-dominant hand. Participants were instructed to avoid larger movement as to not bias the physiological responses. Data recording was monitored online and artifacts (e.g., movement, sneezing, etc.) were recorded by a research assistant who observed the assessment from an adjacent room. All sections with such events were removed from further analysis.

#### *Skin conductance responses*

Each choice during the gambling task was flagged by a digital marker. SCRs were analyzed for two intervals (see Bechara et al., 1997; Starcke et al., 2009): feedback SCRs were analyzed in the 5 s following each choice and anticipatory SCRs were analyzed 5 s before every choice. Six second intervals between two consecutive choices allowed sufficient time to score both types of SCRs without serious overlaps (Bechara et al., 1999; Starcke et al., 2009). The mean interval time between two consecutive choices in the present study was 10.71 s (SD = 0.93).

EDA data were filtered with a digital low-pass 2 Hz FIR filter. A 0.05 Hz FIR high-pass filter was used to obtain phasic SCRs. SCRs were calculated as the maximum increase in skin conductance during 5 s before and after each choice. A threshold of 0.02 Mirco-Siemens (μS) was used; all SCRs below this threshold were scored as zero (zero responses were included in the calculation of mean responses). For range correction, SCRs were divided by the largest SCR of each participant (SCR corrected = SCR raw/SCR maximum; Lykken and Venables, 1971) and the square root was taken to obtain normal distribution (Dawson et al., 2007). Skin conductance recordings could not be analyzed for two participants due to equipment failure.

The present analyses focused on SCRs during the first block of the gambling task in order to predict subsequent choices, because the impact of SCRs was expected to be strongest in initial trials. SCRs of the first block were analyzed in six different SCR categories (see Bechara et al., 1999; Starcke et al., 2009); two anticipatory SCR categories (before choosing an advantageous vs. disadvantageous deck) and four feedback SCR categories that were subdivided by deck (after choosing an advantageous vs. disadvantageous deck) and outcome (win vs. loss).

#### **STATISTICAL ANALYSES**

All questionnaire data were examined for outliers (defined as values >2.5 SD from the mean) and for normal distribution. Detected outliers (0.19% of all data) were replaced with the closest, non-outlier value (Winsor method; see Guttman, 1973). Hierarchical linear growth curve models (HLMs with random intercept and random slope) were built with gambling task scores as the dependent variable to model repeated decisions (level-1) nested in different individuals (level-2) using HLM 6.08 software (Raudenbush and Bryk, 2002; Raudenbush et al., 2004). HLM is particularly well-suited for the analysis of repeated data, because it does not require independence of observations for repeated observations and produces lower Type I error rates than standard GLM procedures (Raudenbush and Bryk, 2002).

Examination of the raw data and a non-significant result for quadratic change over blocks resulted in the use of linear level-1 components to model gambling task scores (i.e., changes across blocks were modeled in a linear way). Both intercept and slope for the unconditional linear model showed significant variability. Therefore, gambling task blocks (block 1–5) were entered as the repeated level-1 predictor. On level-2, the different clinical and neuropsychological variables were entered to test if these variables predicted gambling task scores for the first block (intercept) and linear change across blocks (slope). All variables were mean-centered before being entered into the model and before interaction terms were calculated to reduce multicollinearity. For interaction analyses, multiplication terms were entered together with their corresponding main effect variables. To investigate the impact of social anxiety and avoidance on decision making, the effects of self-reported social anxiety (LSAS-Anxiety) and social avoidance (LSAS-Avoidance) were tested on level-2. Further, effects of the additional control variables were tested in a similar way, including trait anxiety (STAI-T), state anxiety (STAI-State), depression (BDI), intelligence (LPS-4), and general risky decision making (GDT). Before building a combined model including multiple predictors, each predictor was separately tested on level-2 to ensure that a potential effect was not covered by the effect of another variable or poor statistical power. Afterward, significant predictors were combined in one model to test for incremental predictive effects. In addition, potential incremental effects of the different SCRs generated during the first block were investigated by entering SCRs of the six categories on level-2. To this end, interaction terms between anxiety or social avoidance and the different SCRs as well as main effects of the SCRs were entered into the model.

Tests of HLM assumptions did not yield serious violations. Assumptions were tested using the Kolmogorov–Smirnov test for normal distribution of level-1 and level-2 residuals, a χ<sup>2</sup> test for homogeneity of level-1 residual variance, and visual inspection of scatterplots. Reported *R*<sup>2</sup> effect sizes represent the proportion of variance explained by adding the level-2 variables; these effect sizes were calculated by subtracting the variance obtained with the level-2 predictors from the variance obtained without the level-2 predictor, and dividing by the latter (Raudenbush and Bryk, 2002). *R*<sup>2</sup> values were separately calculated for both HLM intercept and linear slope. In addition, the level-1-only model (without any level-2 predictors) served as a reference model to evaluate if the observed data were better explained by adding the level-2 variables to the model. Therefore, a χ<sup>2</sup> variance-covariance components based likelihood test was used (see Raudenbush and Bryk, 2002).

#### **RESULTS**

#### **GAMBLING TASK PERFORMANCE**

Means and SD of choices from the advantageous decks for the five blocks of the gambling task yielded sufficient variance for regression analyses, see **Figure 2** (block 1: *M* = 7.18, SD = 2.30; block 2: *M* = 8.40, SD = 3.03; block 3: *M* = 9.23, SD = 4.56; block 4: *M* = 9.60, SD = 4.24; block 5: *M* = 10.30, SD = 4.32). The level-1-only model including blocks(no level-2 predictors) yielded

significantly fewer advantageous choices with angry facial expressions in block 1, *B* = 7.42, SE = 0.35, *t*(36) = −7.30, *p* < 0.001, and a significant linear increase across blocks, *B* = 0.64, SE = 0.16, *t*(36) = 3.96, *p* < 0.001. So, all participants combined made fewer advantageous choices with angry facial expressions at the beginning and successively learned to make more advantageous choices. Learning was also evident in the assessment of individual contingency awareness1. However, at the end of the task all participants combined did not show a clear preference toward the advantageous deck, as indicated by **Figure 2**.

#### **EFFECT OF SOCIAL ANXIETY AND SOCIAL AVOIDANCE ON DECISION MAKING**

In the HLM with social avoidance as only level-2 predictor, higher social avoidance predictedfewer advantageous choicesfrom the decks with angry facial expressions in block 1 (intercept), *B* = −0.06, SE = 0.02, *t*(36) = −3.64, *p* = 0.001. In the HLM with social anxiety as only level-2 predictor, higher social anxiety yielded a similar effect, but with a smaller effect size, *B* = −0.05, SE = 0.02, *t*(36) = −2.31, *p* = 0.027. In both HLMs, no effect was found for change across blocks for social anxiety, *B* = 0.01, SE = 0.02, *t*(36) = 0.70, *p* = 0.489, or social avoidance, *B* = 0.01, SE = 0.01, *t*(36) = 0.88, *p* = 0.386.

#### **EFFECT OF CONTROL VARIABLES ON DECISION MAKING**

For all following results the specific control variable was entered as single level-2 predictor into an HLM with blocks entered as repeated level-1 factor. In the HLM with trait anxiety as only

<sup>1</sup>Participants were asked about their contingency knowledge after the task (see also Bechara et al., 1997; Maia and McClelland, 2004): (1) "Tell me all you know about what is going on in this game?", (2) "Did you find any difference between the decks?", (3) "Suppose you select 10 new card from deck A (B, C, D) will you on average win or lose money?", (4) "If you have to choose only one deck, which one will you choose in order to maximize your long-term gain?". Participants were judged as having explicitly learned the correct contingencies, if they indicated correct answers to question 3 and 4. Seventy four percent of the participants were judged to be explicitly aware of the contingencies. In addition, analyses that only include participants judged as aware of contingencies yielded the same pattern of results as analyses including the entire sample.

level-2 predictor, higher trait anxiety predicted fewer advantageous choices from the decks with angry facial expressions in block 1 (intercept), *B* = −0.07, SE = 0.03, *t*(36) = −2.84, *p* = 0.008, but had no effect on change across blocks, *B* = −0.01, SE = 0.02, *t*(36) = −0.62, *p* = 0.538. In addition, when depression scores were entered alone on level-2, higher depression scores also predicted fewer advantageous choices in block 1, *B* = −0.18, SE = 0.08, *t*(36) = −2.17, *p* = 0.036, but did not show a significant effect on change across blocks, *B* = 0.01, SE = 0.03, *t*(36) = 0.35, *p* = 0.732.

For all other HLMs with a single control variable as predictor, the effects on intercept and slope were not significant: (1) age: intercept: *B* = 0.03, SE = 0.21, *t*(36) = 0.13, *p* = 0.898, slope: *B* = −0.13, SE = 0.10, *t*(36) = −1.40, *p* = 0.171; (2) sex: intercept: *B* = −0.45, SE = 0.69, *t*(36) = −0.66, *p* = 0.517, slope: *B* = −0.21, SE = 0.39, *t*(36) = −0.54, *p* = 0.591; (3) state anxiety: intercept: *B* = −0.08, SE = 0.05, *t*(36) = −1.68, *p* = 0.101, slope: *B* < 0.01, SE = 0.02, *t*(36) = 0.07, *p* = 0.946; (4) GDT: intercept: *B* = 0.01, SE = 0.04, *t*(36) = 0.23, *p* = 0.819, Slope: *B* = 0.02, SE = 0.02, *t*(36) = 0.73, *p* = 0.468; (5) Estimated IQ (LPS-4): intercept: *B* = −0.02, SE = 0.03, *t*(36) = −0.57, *p* = 0.572, Slope: *B* < 0.01, SE = 0.02, *t*(36) = 0.50, *p* = 0.622.

Next level-2 predictors which yielded significant effects when entered as single level-2 predictor were combined in one model to test incremental predictive power. In the model including social avoidance and anxiety, trait anxiety, and depression scores, only social avoidance, *B* = −0.04, SE = 0.01, *t*(35) = −2.39, *p* = 0.023, and trait anxiety, *B* = −0.05, SE = 0.02, *t*(35) = −2.20, *p* = 0.034, incrementally predicted fewer advantageous choices from the decks with angry facial expressions in block 1. In the combined model, no significant effect was found for social anxiety, *B* = −0.03, SE = 0.03, *t*(35) = 1.10, *p* = 0.281, or depression, *B* = −0.08, SE = 0.12, *t*(34) = −0.70, *p* = 0.488. In addition, no variable predicted linear change across blocks in the combined model; social avoidance, *B* = 0.01, SE = 0.01, *t*(35) = 1.75, *p* = 0.089, social anxiety, *B* = 0.02, SE = 0.02, *t*(35) = 1.24, *p* = 0.225, trait anxiety, *B* = −0.02, SE = 0.02, *t*(35) = −1.26, *p* = 0.217. Thus, social anxiety and depression scores were droppedfrom the model. The inclusion of social avoidance and trait anxiety on level-2 resulted in significantly less model deviance of observed from estimated data compared to the level-1-only model, <sup>χ</sup><sup>2</sup> <sup>=</sup> 8.98, *<sup>p</sup>* <sup>=</sup> 0.003. In summary, after analysis of the additional variables, social avoidance and trait anxiety both yielded significant beta weights and were the only variables showing incremental predictive values in a model with more than just one predictor.

#### **EFFECTS OF TRAIT ANXIETY AND SOCIAL AVOIDANCE ON SUBSEQUENT BLOCKS**

To further investigate if the effects of self-reported social avoidance and trait anxiety were consistent throughout the task, the intercept analyses were repeated for block 2–5. For STAI-T, the effects were significant for all subsequent blocks; block 2, intercept: *B* = −0.07, SE = 0.02, *t*(35) = −3.28, *p* = 0.003; block 3, intercept: *B* = −0.09, SE = 0.03, *t*(35) = −2.99, *p* = 0.006; block 4, intercept: *B* = −0.11, SE = 0.04, *t*(35) = −2.54, *p* = 0.016; and block 5, intercept: *B* = −0.13, SE = 0.06, *t*(35) = −2.24, *p* = 0.032. In contrast, the effects for LSAS-Avoidance were

not significant beyond block 1; block 2, intercept: *B* = −0.02, SE = 0.02, *t*(35) = −1.11, *p* = 0.277; block 3, intercept: *B* = −0.01, SE = 0.03, *t*(35) = −0.17, *p* = 0.870; block 4, intercept: *B* = 0.01, SE = 0.04, *t*(35) = 0.32, *p* = 0.750; and block 5, intercept: *B* = 0.03, SE = 0.05, *t*(35) = 0.59, *p* = 0.560. Thus, selfreported trait anxiety consistently predicted fewer advantageous choices from the decks with angry facial expressions throughout the task, whereas social avoidance had an impact only in the first block.

#### **INTERACTIONS WITH EMOTIONAL RESPONSES DURING DECISION MAKING**

#### *Initial decisions: social avoidance and SCRs*

In order to evaluate the influence of emotional responses during decision making, the different SCR categories and their interaction with social avoidance and trait anxiety were analyzed in different HLMs (each included the main effect of the specific SCR category, social avoidance and trait anxiety, and the interaction terms). For intercept analyses, no significant main effects on intercept were found for any of the SCR categories of block 1, all *t*s < 1.90, all *p*s > 0.08. In addition, no interaction effects on intercept were found between SCRs in block 1 and trait anxiety, all *t*s < 1.70, all *p*s > 0.10. However, analyses yielded a significant interaction of social avoidance and anticipatory SCRs before choosing an advantageous deck with angry facial expressions, *B* = −0.43, SE = 0.16, *t*(32) = −2.62, *p* = 0.015. In order to illustrate the significant interaction, results were compared for different levels of the predictors (social avoidance and anticipatory SCRs before choosing an advantageous deck with angry facial expressions). Therefore, high and low scores for social avoidance and the anticipatory SCRs of the present sample were calculated by using mean scores for the upper and lower quartile. These scores for low and high levels were entered back into the regression and estimated results for high and low scores were plotted for visual interpretation of the interaction. **Figure 3** illustrates that in the first block, only high social avoidance in combination with higher anticipatory SCRs predicted less frequent choices of the advantageous decks with angry facial expressions. No further interactions between SCR categories and social avoidance were found, all *t*s < 0.65, all *p*s > 0.54.

#### *Learning of advantageous decisions: trait anxiety and SCRs*

Slope analyses were conducted with the same predictors as intercept analyses (main and interaction effect of social avoidance, trait anxiety, and the single SCR categories). No significant main effects on slope were found for any of the SCR categories, all *t*s < 1.00, all *p*s > 0.33. In addition, no interaction effects on slope were found for interactions between SCR categories and social avoidance, and most of the interaction effects between SCR categories and trait anxiety, all *t*s < 1.30, all *p*s > 0.21. However, the interaction between SCRs after loss feedbacks (for both advantageous and disadvantageous decks) and trait anxiety significantly predicted change across the task, *B* = 0.20, SE = 0.07, *t*(32) = 2.77, *p* = 0.010. **Figure 4** illustrates that high trait anxiety combined with lower SCRs after losses resulted in decreased learning. However, a steeper learning curve occurred in high trait anxious participants, if participants showed higher SCRs to losses in the initial block of the task.

**FIGURE 3 | Interaction illustration by HLM estimated number of choices from deck C** + **D in the first block of the social anxiety gambling task for high vs. low social avoidance with high vs. low skin conductance responses (SCRs) before choosing an advantageous deck with angry facial expressions.** Mean scores of the upper and lower quartile were used to illustrate the interaction effect. Higher scores indicate more frequent choices from the advantageous decks depicting angry facial expressions.

**FIGURE 4 | Interaction illustration by HLM estimated number of choices from deck C** + **D for high vs. low self-reported trait anxiety with high vs. low skin conductance responses (SCRs) after loss feedbacks in the first block of the social anxiety gambling task.** Mean scores of the upper and lower quartile were used to illustrate the interaction effect. Higher scores indicate more frequent choices from the advantageous decks depicting angry facial expressions. Positive slopes indicate increased learning of advantageous choices.

Finally, only the two significant interaction terms (social avoidance and anticipatory SCRs before choosing an advantageous deck for intercept; trait anxiety and SCRs after losses for slope) as well as the corresponding main effects were included into one model to test for incremental effects of the interaction terms. The inclusion yielded significantly less deviances of observed data from modeled data compared to the model with trait anxiety and social avoidance only, <sup>χ</sup><sup>2</sup> <sup>=</sup> 13.46, *<sup>p</sup>* <sup>=</sup> 0.001. Compared to the level-1-only model, inclusion of the level-2 predictors explained additional variance of initial decisions in block 1 (*R*<sup>2</sup> <sup>=</sup> 0.23) and change across blocks (*R*<sup>2</sup> <sup>=</sup> 0.10).

#### **PICTURE RATINGS**

After completion of the gambling task, both pictures with angry facial expressions (female angry face: *M* = 1.81; SD = 1.35; male angry face: *M* = 1.97; SD = 1.46) were rated as significantly less pleasant compared to the happy facial expressions (female happy face: *M* = 7.44; SD = 1.42; male happy face: *M* = 6.75; SD = 1.57), all *t*s > 12.16, all *p*s < 0.001. In addition, arousal ratings were significantly higher for the male angry face (*M* = 4.97; SD = 2.98) compared to both happy facial expressions (female happy face: *M* = 2.94; SD = 2.53; male happy face: *M* = 3.06; SD = 2.62), all *t*s > 3.30, all *p*s < 0.003, but not for the female angry facial expressions (female angry face: *M* = 3.69; SD = 2.92) compared to all happy facial expressions, all *t*s < 1.27, all *p*s > 0.21.

# **DISCUSSION**

The present study yielded evidence that task-irrelevant, but fear-relevant, angry faces exert a bias on choices in ambiguous situations, which is pronounced in individuals with elevated levels of social anxiety. This influence was observed in an experimental gambling task linking advantageous choices with angry facial expressions. The present findings were not explained by age, sex, state anxiety, logical reasoning, or risky decision making. Overall, all participants combined made fewer advantageous choices with angry facial expressions at the beginning. Furthermore, participants successively learned to make more advantageous choices, but did not exhibit a clear preference for the advantageous decks at the end.

These findings may suggest a more universal bias of angry facial expressions on decision making which is not limited to elevated levels of social anxiety and avoidance. Indeed, previous studies reported a similar initial bias of angry facial expressions on rational choices for healthy participants (Averbeck and Duchaine, 2009). However, in this previous study the bias was limited to early trials and participants integrated feedback information to change their decisions. In the present study, this correction to initial decisions was also evident in explicit contingency ratings and increasing advantageous choices across the task. However, across all participants no clear preference for the advantageous choices was found by the end of the task. The latter results are not in line with most findings of healthy participants completing the original Iowa Gambling Task (e.g., Bechara et al., 1997, 1999, 2000; Brand et al., 2007). There may be different explanations for this mismatch of contingency awareness and persisting behavioral preference toward happy facial expressions at the end of the task.

In line with the finding of a universal bias, angry facial expressions may have exerted a more sustained bias on decision making behavior than on cognitive evaluation (contingency awareness) in our study. This sustained bias would have led to fewer selections of angry facial expressions despite developing knowledge of an advantageous outcome of these decks (for examples of discrepant evaluation and behavior see Strack and Deutsch, 2014). Alternatively, it is not clear when precisely participants acquired awareness of contingencies, because explicit awareness was only assessed after completion of the task. Due to the limited experience with the advantageous decks at the beginning, it may be possible that participants became explicitly aware only during the last blocks of the task. Whereas this late awareness can be seen in explicit knowledge after the task, it would be too late to result in a significant preference of advantageous choices. To further clarify the mismatch of contingency awareness and a behavioral preference, future research may, therefore, incorporate online contingency and expectancy ratings throughout the task (e.g., seeMaia and McClelland, 2004).

#### **INITIAL CHOICES, SOCIAL AVOIDANCE, AND EMOTIONAL RESPONSES**

Across all participants, the decks with angry facial expressions were avoided initially. As hypothesized, these avoidant decisions were more pronounced in participants with a higher level of social avoidance. This effect of social avoidance was, however, limited to the beginning of the task when contingencies were still obscure and vanquished at the end of the task. These results are in line with recent findings of a bias on initial decisions in healthy participants (Averbeck and Duchaine, 2009). In this study, healthy participants also showed initial avoidance of angry compared to happy facial expressions, which diminished in later trials. In the present study, this initial avoidance of angry facial expressions was elevated in individuals with elevated levels of social anxiety. These present results further support recent studies that found similar avoidant decisions in individuals with fear of spiders (Pittig et al., 2014a) and in healthy participants after fear conditioning experience (Pittig et al., 2014b). Thus, avoidant decision making under uncertainty seems to be common across different types of fear and anxiety. Detailed understanding of the underlying mechanisms of development and reduction of avoidant decisions may inform behavioral treatments which target pathological avoidance behaviors.

For initial choices, a significant interaction was found between social avoidance and anticipatory SCRs before choosing an advantageous deck with angry facial expressions. Only high social avoidance in combination with higher anticipatory SCRs resulted in fewer choices of angry facial expressions. This interaction provides evidence that avoidance is not triggered by angry facial expressions for every individual with high levels of social avoidance, but rather depends on congruent elevated emotional responses. As social anxiety represents a multidimensional construct with multiple stimuli and cognitive processes as potential triggers for anxious responses (Hofmann et al., 2004; Bögels et al., 2010), it is likely that angry facial expressions were not fear-relevant stimuli for all participants with elevated levels of social anxiety. This may have resulted in lower emotional responses and subsequently in missing avoidant decisions for some participants.

As SCRs are sensitive to many cognitive and affective processes, it is difficult to interpret the specific interaction effects without doubt. Previous research suggested different explanations for anticipatory SCRs in the Iowa gambling task, for example, growing awareness of contingencies or reward and punishment expectancies (for a review see Dunn et al., 2006). Previous studies, however, did not find any specific anticipatory reactions in the initial block of the task (Bechara et al., 1997). In addition, the present results indicate that the interaction of social avoidance and emotional responses was specific for decks with angry facial expressions. Findings, thus, may favor the interpretation that the initial bias on decisions was related to the angry facial expressions. It seems that early responses to the angry facial expressions may have been the most prominent information for initial decisions. Whereas the facial expressions can be identified immediately, task-relevant

information or emotional labels related to gains and losses tend to develop at a later stage (Bechara et al., 1997; Brand et al., 2007). Thus,fear-relevant information may have had the strongest impact under uncertainty due to missing explicit knowledge. With little knowledge about outcomes, individuals with a high tendency to avoid social stimuli and situations may have followed their default avoidance strategy if they experienced higher emotional responses.

However, pronounced avoidance in highly avoidant individuals was limited to initial trials of the gambling task, when the contingencies were not yet detected. In later trials, no effect of elevated social avoidance was found above and beyond the general tendency to choose fewer cards from the decks with angry facial expressions (advantageous decks) across all participants. Thus, pronounced avoidance was limited to trials with uncertainty about gain-related consequences, in which it is likely that no strong conflict between avoidance and approach was evident. After further experience with the contingencies, initial avoidance may have been overcome in the presence of an opposing rewardrelated goal. In the present subclinical sample, socially avoidant individuals did not engulf long-term costs. This findings differs from our previous findings in a sample of spider fearful participants, where avoidant decisions were more consistent (Pittig et al., 2014a). Possibly, these differences may be explained by the specificity of the fear-relevant stimuli. Whereas spiders are specific fear-relevant stimuli for spider-fearful individuals, multiple stimuli, and cognitive processes may trigger anxious responses in social anxiety, as mentioned above (Hofmann et al., 2004; Bögels et al., 2010).

Current decision-making theories further highlight the importance of emotional and cognitive influences in decision making (Bechara et al., 1997; Loewenstein et al., 2001). Changes in both processes could have influenced the decrease in the effect of avoidance over time. First, participants were exposed to the same pictures in all 100 trials. This could have caused a habituation of emotional responses to the angry facial expressions. Thus, initial avoidance may have decreased due to reduced aversiveness of the angry facial expressions, which were subsequently not strong enough to bias decisions persistently. In addition, research on the Iowa gambling task showed that uncertainty is especially pronounced during the initial block, whereas explicit knowledge starts to develop in block 2 or 3 (Brand et al., 2007). With more experience, more explicit knowledge about the benefits of selecting the angry facial expressions was acquired. This acquisition of explicit knowledge may have mitigated the effect of the angry facial expressions and fostered the approach tendency.

# **LEARNING ACROSS THE TASK, TRAIT ANXIETY, AND EMOTIONAL RESPONSES TO LOSSES**

First results suggested a general negative effect of trait anxiety on advantageous decision making. This effect was further clarified by a significant interaction between trait anxiety and SCRs after early losses. High trait anxiety combined with low SCRs was associated with impaired learning of advantageous decisions. Conversely, elevated learning of advantageous choices occurred in high trait anxiety in combination with higher SCRs to losses during initial decisions.

These findings can be linked to previous studies on the impact of high trait anxiety and SCRs on decision making in the original Iowa gambling task. First, prior studies also reported a link between feedback SCRs and decision making (Suzuki et al., 2003; Lawrence et al., 2006) and showed that participants with higher SCRs to losses performed better in the Iowa gambling task (Starcke et al., 2009). Regarding elevated levels of trait anxiety, previous results were mixed.Whereas some findings suggest a negative impact of high trait anxiety on decision making (Miu et al., 2008), others contrarily suggest a beneficial effect (Werner et al., 2009). Here, the latter findings supported a positive correlation between high trait anxiety and physiological responses to feedback. Combined, these findings may suggest that the effect of trait anxiety in the present study could be more closely linked to outcome-related features of the task than to the facial expressions. This may indicate that stronger emotional responses can augment advantageous decision making, if they are related to goal-relevant features of the task.

The impact of initial reaction to losses was not observed in participants with low trait anxiety. This specificity of early losses in high trait anxiety may be linked to a higher loss aversion in highly anxious individuals (see Hartley and Phelps, 2012). In addition, the immediate impact of emotional responses on decision making may be lower in low trait anxious individuals, whereas the use of cognitive strategies may be more pronounced. Such cognitive strategies depend on explicit knowledge, which starts to develop at a later stage of the task (Brand et al., 2007). In contrast to the use of cognitive strategies, high trait anxious individuals may be more engaged in emotional reasoning. Here, the impact of responses to early losses could be mediated by a higher perceived intensity of negative outcomes in high trait anxious individuals (Maner and Schmidt, 2006) and the tendency to use fewer cues in reasoning tasks (Leon and Revelle, 1985). Thus, if highly trait anxious individuals show elevated responses to goal-relevant features, they may be more prone to immediately direct future decisions following these information.

#### **LIMITATIONS AND FUTURE RESEARCH**

There are some limitations to the present study which may be relevant for future research. First, we treated anxiety and depression as dimensions rather than nosological categories and participants were not selected for clinical levels of anxiety or depression. We expect that the effects observed here may be even be more pronounced in more severe degrees of anxiety an in patients with SAD. For example, individuals with clinical levels of social avoidance may show a more consistent pattern of avoidant decisions. However, a discussion of potential perspectives of social phobia warrants further examination in clinical populations. Future research may, therefore, replicate the present findings within a sample of patients with SAD. Similarly, the impact of clinical levels of depression may be tested. Results from our study in fear of spiders suggest that effects of fear-relevant stimuli on avoidant decisions may be similar in above-threshold individuals (Pittig et al., 2014a).

Second, whereas the present sample size provided enough power to detect effects of social avoidance and trait anxiety, it might not be large enough to detect small effects of the remaining variables. Future research may, for example, further investigate potential effects of gender, age, intelligence, or risky decision making in general. However, this was not the primary focus of the present study and controlling for these variables did not change the main results.

Third, exclusion criteria, especially the absence of medical or other psychiatric conditions were only assessed via self-report of the participants. No standardized clinical interviews were completed to more thoroughly rule out a history of psychiatric disorders. Although this might have resulted in a miss of specific or past psychiatric symptoms, the current sample mainly consisted of students. Nevertheless, future research may incorporate standardized interview to further examine the role of different or past psychiatric symptoms.

Fourth, awareness of reward contingencies was only assessed after completion of the task. This procedure was used to prevent a potential bias of repeated contingency ratings throughout the task on learning of an advantageous strategy. However, it remains unclear when participants developed awareness of contingencies and if they still continued to avoid the angry facial expressions afterward. In order to further clarify the relationship between explicit knowledge of reward contingencies and the effect of angry facial expressions, future research may incorporate online contingency and expectancy ratings throughout the task (e.g., see Maia and McClelland, 2004).

#### **CONCLUSION**

In summary, angry facial expressions trigger avoidant decisions in individuals with elevated levels of social anxiety, but only in those who initially experience strong emotional responses toward these stimuli. Emotional responses can also be beneficial and increase advantageous decision making, if they are goal-relevant. This highlights the opposing impact of emotions on decision making and calls for the need to account for both types of emotional responses when investigating avoidance behavior in anxiety.

# **ACKNOWLEDGMENTS**

This work was funded in part by a scholarship by the Deutscher Akademischer Austauschdienst (German Academic Exchange Service) and a scholarship by the Studienstiftung des Deutschen Volkes (German National Academic Foundation). Andre Pittig, Department of Psychology, School of Social Sciences, University of Mannheim, Germany, and Anxiety Disorders Research Center, University of California, Los Angeles, USA; Mirko Pawlikowski, General Psychology: Cognition, Department for Informatics and Applied Cognitive Science, University of Duisburg-Essen, Germany; Michelle G. Craske, Anxiety Disorders Research Center, University of California, Los Angeles, USA; Georg W. Alpers, School of Social Sciences, Department of Psychology, University of Mannheim, Germany.

# **REFERENCES**

Adolph, D., and Alpers, G. W. (2010). Valence and arousal: a comparison of two sets of emotional facial expressions. *Am. J. Psychol.* 123, 209–219. doi: 10.5406/amerjpsyc.123.2.0209

Adolphs, R. (2002). Recognizing emotion from facial expressions: psychological and neurological mechanisms. *Behav. Cogn. Neurosci. Rev.* 1, 21–62. doi: 10.1177/1534582302001001003


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 27 June 2014; accepted: 02 September 2014; published online: 29 September 2014.*

*Citation: Pittig A, Pawlikowski M, Craske MG and Alpers GW (2014) Avoidant decision making in social anxiety: the interaction of angry faces and emotional responses. Front. Psychol. 5:1050. doi: 10.3389/fpsyg.2014.01050*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Pittig, Pawlikowski, Craske and Alpers. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited andthatthe original publication inthis journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Being watched by others eliminates the effect of emotional arousal on inhibitory control

#### **Jiaxin Yu1,2 , Philip Tseng<sup>3</sup> , Neil G. Muggleton2,4,5 and Chi-Hung Juan<sup>2</sup>\***

1 Institute of Neuroscience, National Yang-Ming University, Taipei, Taiwan

2 Institute of Cognitive Neuroscience, National Central University, Jhongli, Taiwan

<sup>3</sup> Brain and Consciousness Research Center, Taipei Medical University – Shuang Ho Hospital, New Taipei City, Taiwan

4 Institute of Cognitive Neuroscience, University College London, London, UK

<sup>5</sup> Department of Psychology, Goldsmiths, University of London, London, UK

#### **Edited by:**

Paul Pauli, Julius-Maximilians-University, Germany

#### **Reviewed by:**

Mario F. Juruena, University of Sao Paulo, Brazil Eric S. Allard, Cleveland State University, USA

#### **\*Correspondence:**

Chi-Hung Juan, Institute of Cognitive Neuroscience, National Central University, No. 300, Jhongda Road, Jhongli City 320, Taiwan e-mail: chijuan@cc.ncu.edu.tw

The psychological effect of being watched by others has been proven a powerful tool in modulating social behaviors (e.g., charitable giving) and altering cognitive performance (e.g., visual search). Here we tested whether such awareness would affect one of the core elements of human cognition: emotional processing and impulse control. Using an emotion stop-signal paradigm, we found that viewing emotionally-arousing erotic images before attempting to inhibit a motor response impaired participants' inhibition ability, but such an impairing effect was completely eliminated when participants were led to believe that their facial expressions were monitored by a webcam. Furthermore, there was no post-error slowing in any of the conditions, thus these results cannot be explained by a deliberate speed-accuracy tradeoff or other types of conscious shift in strategy. Together, these findings demonstrate that the interaction between emotional arousal and impulse control can be dependent on one's state of self-consciousness. Furthermore, this study also highlights the effect that the mere presence of the experimenter may have on participants' cognitive performance, even if it's only a webcam.

**Keywords: cognitive control, conscientiousness, emotion regulation**

#### **INTRODUCTION**

The psychological effect of being watched by others has been proven a powerful tool in boosting honest or charitable behaviors (Bateson et al., 2006; Ekstrom, 2012) while reducing dishonest behaviors (Nettle et al., 2012). This watching effect increases self-awareness (Baltazar et al., 2014) and causes individual to consciously modify their behavior to increase compliance with social standards (Wedekind and Milinski, 2000; Milinski et al., 2002). However, besides these high-level socio-behavioral changes that are subject to participants' conscious decisions, it remains unclear whether the effect of watchful eyes can modulate some of the core elements of human cognition, such as cognitive control and emotion regulation. Several studies in social psychology have revealed that the Stroop interference effect can be significantly reduced if the experimenter looks frequently at the participants during the task (i.e., Huguet et al., 1999). This is known as the "social facilitation-inhibition" effect, where social context inhibits automatic and purpose-irrelevant stimulus (i.e., word meaning) and leads to performance enhancement (speed of color naming). In addition to Stroop interference, one recent study using visual search has also shown that people may alter their cognitive performance by searching slowly in order to achieve higher accuracy rates when they are being (or believe they are) watched by others (Miyazaki, 2013).

This type of facilitation effect could be potentially valuable if it can be applied to cognitive and emotional control since training programs to improve performance usually take a long period of time (e.g., Berkman et al., 2014), and the uses of drugs (Li et al., 2010) or brain stimulation techniques (Hsu et al., 2011) can raise ethical issues. As such, in this study we explored whether increased levels of conscientiousness, or the knowledge of being watched by others, may influence the effects that emotion has on cognitive control.

Studies investigating cognitive processes behind inhibitory control have shown that seeing emotional stimuli prior to performing a stop-signal task, which assesses the efficiency in inhibiting planned responses (Verbruggen and Logan, 2008), can significantly impair inhibitory control (Verbruggen and De Houwer, 2007) by prolonging participants' time to stop a preplanned response (stop-signal reaction time, SSRT). This is because the emotional stimuli elicit a response in participants (whether they like it or not) and therefore delay the onset of the actual inhibition processes. Recently, by using emotional stimuli of matched arousal and valence levels between female and male participants, Yu et al. (2012) found that erotic and painful stimuli impaired male participants' ability to inhibit motor responses, but not females. This finding suggests that, even when arousal and valence levels are equated between both genders, men in general suffer greater impairment in inhibitory control ability in the face of emotional stimuli or events. This was especially true when stimuli were erotic images (Yu et al., 2012). In light of these findings, here we adopted the same erotic pictures and stop-signal task in a naïve group of male participants to investigate whether the awareness of being watched by others would interact with the known effects of emotional stimuli on inhibitory control. Although we did not anticipate any effect of emotional arousal on females' inhibitory control in this experimental context, as a control condition, we also tested the same paradigm in a group of naïve female participants (Control Experiment 2).

The emotional stop-signal paradigm here was coupled with the manipulation of being watched by the experimenter. This manipulation was achieved by turning a webcam ON (being watched) or OFF (not being watched) in front of the participants, which should induce higher levels of conscientiousness in them (Javaras et al., 2012). An increase (i.e., worse performance) in the already-impaired SSRT would imply that being-watched has an additional negative impact over and above erotic pictures on men's inhibitory control, whereas a decrease in SSRT (i.e., better performance) would suggest improved inhibition; both of which would demonstrate a link between emotion and the effect of being-watched. Alternatively, a null effect would predict no additional changes to the emotion-driven impairment in SSRT. Hence, to compare the effect of being-watched in a more appropriate context, we also conducted a control experiment that examined the effect of being-watched in a classic stop-signal task (i.e., no erotic pictures).

Additionally, the control adjustment process was analyzed to evaluate the effects of camera and emotion. Post-error slowing was used as an index of strategy adjustment to assess whether individuals were prolonging their response time to make less errors (Li et al., 2008). If the awareness of being-watched affects participants' conscious decision boundary, we should observe increased post-error slowing RTs in all experimental conditions regardless of our manipulation of conscientiousness.

# **MATERIALS AND METHODS**

#### **PARTICIPANTS AND RECRUITMENT**

To make sure all participants understood the stop-signal task, they were required to pass a prescreen stop-signal task before being recruited. There were 72 go trials and 24 stop trials in the prescreen test, and all participants needed to meet two criteria (1) the go accuracy rate was higher than 95%, and (2) the stop-response rate was between 45 and 55%. Forty-two male participants passed the prescreen test and took part in this study, 26 of which (21.6 years, from 18 to 27) were randomly assigned to the main experiment, and 16 (22.4 years, from 18 to 29) to the control experiment. All experimental procedures were approved by the Institutional Review Board of National Taiwan University, Taipei, Taiwan.

#### **APPARATUS**

The visual stimuli were presented on a 23-inch LCD monitor. Stimulus presentation and data acquisition were done using Matlab with Psychtoolbox-3. To manipulate being watched or not, we placed one web camera (Microsoft LifeCam HD-3000) on top of the monitor.

#### **DESIGN AND PROCEDURES**

#### **Main Experiment**

Participants were randomly assigned to the ON or OFF group at the start of this experiment. Participants in these two groups were given the same instructions regarding the stop-signal task. An additional sentence was included in the ON group instruction: "during this experiment, your facial expressions will be recorded and will be analyzed after you finish the whole experiment<sup>1</sup> ," and the recording program would be opened in front of them momentarily to convince them that we were recording their facial expressions. All participants were told that the experimenter would be absent from the room until they finished the whole session.

For the stop-signal task, participants were to perform a speeded choice response task and were to withhold that response if a stop signal was presented. Participants' were required to press the left or right button of response box with their left or right index finger according to the direction of the arrow (go signal; **Figure 1**). In 75% of the trials, this made up the entire trial (go trials). In the remaining 25% of trials, a red dot (stop signal) appeared shortly after the go signal, prompting the participants to withhold their go response (**Figure 1**). Since the paradigm was set to adjust difficulty by altering the time of onset of the stop signal

<sup>1</sup>Participants were asked whether they thought the camera was recoding after they finished the experiment. All participants reported yes.

**FIGURE 1 | (A)** After an emotional stimulus, participants needed to do a speeded choice response task in which they were required to withhold response if a stop signal was presented. There were 144 go trials and 48 stop trials in each emotion condition. One erotic/neutral image was shown to participants shortly before each trial for 1 s. **(B)** Mean of stop-signal reaction times (SSRTs), with 95% confidence interval error bars. No difference was seen in SSRTs in the neutral condition (F = 0.137, p = 0.715) but a significant difference in SSRTs in the erotic condition (F = 8.540, p = 0.007) was seen between the two groups. \*p < 0.05, \*\*p < 0.01.

(stop-signal delay, SSD) with a staircase procedure, the obtained measures, SSD and go reaction time, as well as their difference (SSRT), provided an estimate of participants' ability to inhibit a prepotent response. All participants completed an emotional stop-signal task, including two erotic and two neutral blocks, on the same day in counterbalanced order. In the emotional stopsignal task, there were 144 go and 48 stop trials in each condition. One erotic or neutral image was shown to the participants shortly before each trial for 1 s (**Figure 1A**). For image stimuli, 48 erotic and 48 neutral images (picture size: 300 × 300 pixels) were retrieved from the International Affective Picture System (IAPS). It was ensured that the images differed in terms of their valence (erotic: mean = 7.2, standard error (SE) = 0.74; neutral: mean = 5.0, SE = 0.4) and arousal (erotic: mean = 6.5, SE = 0.89; neutral: mean = 2.73, SE = 0.67). More details, as well as all the images used in the present study, can be found in the study by Yu et al. (2012).

# **Control Experiment 1**

The purpose of the control experiment was twofold: (1) to assess the effects of being-watched on a classic stop-signal task (i.e., no erotic pictures), and (2) use a within-subject design to better control for any possible group differences that might be present in the main experiment. All participants completed a classic stop-signal task (without any emotional pictures, 144 go and 48 stop trials in each ON/OFF condition), including ON and OFF conditions in a group-counterbalanced order. All experimental procedures and materials were identical to the main experiment. Note that to avoid any possible gender interaction, the male experimenter in the main and control experiments was the same.

#### **Control Experiment 2**

Although our previous research has suggested that female participants are not affected by the emotion stop-signal manipulation, it was still necessary to check for any gender differences since we added a new webcam manipulation. In this control experiment, a female version of the task (female experimenter and female participants) was conducted for comparison with the males' results. Twelve female participants were randomly assigned to the camera ON or OFF group. All experimental procedures and materials were identical to the main experiment.

# **Data analysis**

Data analysis was conducted using Matlab (Mathworks) software for both experiments. The go RTs were filtered by removing incorrect trials. Trials with latencies more than two standard deviations from each participant's mean go RT for each emotional condition were also excluded. Each participant's critical SSD was calculated by averaging the stop signal durations of all stop trials. The SSRT was calculated using each participant's mean go RT and subtracting the critical SSD. Mixed model repeated measures analysis was carried out for correct go RT and SSRT under the two emotion conditions between the two groups in the main experiment. To ensure that there were no preexisting differences between participants that were randomly assigned to the watched/unwatched groups, an additional one-way ANOVA was applied to test differences in SSRTs in the neutral condition between these two groups of participants. Paired *t*-tests were used to compare both correct go RTs and SSRTs for ON and OFF conditions in the control experiment.

For post-error slowing, successful go trials were categorized into two types: go trials after a correct go trial (pG) and go trials after a stop-error trial (pSE). In the present study, the difference in RTs between pSE and pG served as an index of control adjustment, which might signify a deliberate and strategic slowing by the participants to increase the likelihood of successful stopping following an error.

# **RESULTS**

#### **MAIN EXPERIMENT**

For SSRTs, a main effect of emotion condition [*F*(1,24) = 15.434, *p* < 0.001], no effect of being watched [*F*(1,24) = 1.488, *p* = 0.23], and a significant interaction between these two factors [*F*(1,24) = 6.397, *p* = 0.02] was observed. Paired *t*-tests showed a significantly slower SSRT only in the unwatched groups when viewing erotic pictures, compared to the neutral pictures (*t* = 2.971, *p* = 0.01), with no difference in the watched group (*t* = −0.739, *p* = 0.47). No difference in SSRTs in the neutral condition (*F* = 0.137, *p* = 0.72] was seen, but a significant difference in SSRTs in the erotic condition (*F* = 8.540, *p* < 0.01) was observed between the two groups (**Figure 1B**). For Go RTs, there were no significant effects of emotion condition [*F*(1,24) = 0.005, *p* = 0.95], being watched [*F*(1,24) = 0.936, *p* = 0.34], or interaction of the two factors [*F*(1,24) = 2.106, *p* = 0.16]. For posterror slowing, there was no main effect of emotion condition [*F*(1,24) = 1.432, *p* = 0.24] or being watched [*F*(1,24) = 3.633, *p* = 0.07], and also no interaction [*F*(1,24) = 0.031, *p* = 0.86] was observed. All the results are summarized in **Table 1**.

#### **Table 1 | Summary of the main experiment results (mean** ± **standard error).**


#### **CONTROL EXPERIMENT 1**

Paired *t*-tests showed no difference between ON and OFF conditions for Go RTs (*t* = −0.379, *p* = 0.71), SSRTs (*t* = 0.153, *p* = 0.88), or post-error slowing (*t* = 0.036, *p* = 0.97). Therefore, the control experiment suggests that (1) there is no additional effect of being watched on the processes of motor inhibition, and (2) the marginally significant effect of post-error slowing in the being-watched condition from Experiment 1 is likely reflective of between-subject group differences, which were no longer present when a within-subject designed was used. All the results are summarized in **Table 2**.

#### **CONTROL EXPERIMENT 2**

For SSRTs, there was no main effect of emotion [*F*(1,10) = 0.284, *p* = 0.606], being watched [*F*(1,10) = 0.81, *p* = 0.389], and no interaction between the two [*F*(1,10) = 0.011, *p* = 0.917]. Therefore, consistent with previous findings (Yu et al., 2012), female participants were not affected by the emotion-arousal manipulation in the context of a stop-signal paradigm. Results are summarized in **Table 3**.

#### **DISCUSSION**

Studies have demonstrated the psychological effect of being watched by others as a powerful tool in changing social behavior. Our results showed that such awareness also alters individual's inhibitory control ability within an emotional context. From these results, there are several points to consider. First, in the condition where the observing camera is thought to be off, the present experiment replicates our previous findings (Yu et al., 2012), demonstrating that the presence of emotional stimuli such as erotic pictures can impair male participants' ability to inhibit motor responses. Second, when the camera is thought to be on, the awareness of being watched by the experimenter eliminated the impairing effect of erotic stimuli on SSRTs. This counteracting effect cannot be explained by an overall increase in



motivation that is often associated with a watchful experimenter (Miyazaki, 2013) because, if this was true, then we should have observed better SSRT in the camera ON condition of the control experiment, where erotic images were absent. Therefore, it is clear that the effect of being watched by others specifically acts on the (impairing) effect of emotion. Consequently, such monitoring eliminates the emotion-induced impairment of SSRT, but does not enhance SSRTs in general when emotion stimuli were absent.

Though acting specifically on the effect of emotion, it is important to note that the effect of being watched itself need not be an emotional one in nature. Support for this notion comes from a comparison between the effects of erotic pictures and the webcam observation. If the effect of being watched also elicits an emotional response within the participants, then the camera ON condition in the control experiment should also yield a change in SSRTs similar to the effect of erotic pictures, and perhaps even an additional and cumulative effect on SSRTs when erotic pictures and the camera observation were combined. One plausible alternative explanation for such rapid improvement in SSRT is the idea of conscientiousness (or vigilance), or the ability to maintain alertness and attentiveness over a period of time. Conscientiousness is thought to be related to effortful control (Caspi et al., 2005) and emotion regulation (Roberts et al., 2007). Although previous studies have only reported a negative correlation between conscientiousness and negative emotion recovery (i.e., Javaras et al., 2012), there is no solid evidence that conscientiousness would not down regulate high valence and high arousal emotion such as may be caused by the erotic stimuli in the male group. This attentional account would explain the dramatic improvement in inhibitory control, but would also imply a different mechanism here from the high-level, "being-watched" socio-behavioral literature reviewed above, and instead suggest that the present effect is located more on the implicit end of the conscientiousness spectrum. This idea is also supported by the lack of significant findings in our post-error slowing analysis, which suggests that participants were not, or could not, deliberately trying to slow down their response times in order to gain higher accuracy.

Participants in the being-watched group knew that their facial expressions would be assessed, which could lead them to suppress their facial expressions either consciously or unconsciously. Our results here are more supportive of the unconscious notion because, had our participants attempted to consciously suppress their expressions (which requires mental resource), such topdown control would in theory interfere with inhibitory control and consequently prolong the stopping processes (instead of decreasing SSRT in the emotion condition, and not affecting SSRTs in Control Experiment 1). In contrast, if the suppression

#### **Table 3 | Summary of the Control Experiment 2 results (mean** ± **standard error).**


is done unconsciously or automatically, based on the embodied approach of emotion, it could reduce participants' experience of emotion/arousal and resulting in a null effect of emotion (Niedenthal, 2007). It seems that the idea of being watched by others most likely unconsciously triggers such expression suppression and leads to the neutralization of emotion's impact on on-going cognitive processes. If this is true, the automatic nature of such a modulating effect possibly makes it a suitable method for improving inhibitory control in some clinical populations such as people who are substance-dependent and would benefit from enhancement of their cognitive control ability to battle cravings.

Nevertheless, irrespective of the origin of these differences, the present findings highlight the interaction between emotional arousal and cognitive inhibition processes depending on the participants' awareness of being watched by others. In laboratory settings, these findings also raise an important issue for the field to consider: that the presence of an experimenter or monitoring device may have a profound impact on the data we collect (e.g., Miyazaki, 2013). In addition, these results also provide a possible basis for clinical use by incorporating the conscientiousness factor into cognitive training programs and treatments for people who suffer impaired cognitive control. However, it should be noted that the present study mainly tested the watching effect in the context of positive valence emotions (i.e., erotic pictures in this study) in male participants. It remains unclear whether such an effect can also be applied in the context of negative pictures or different types of emotion (e.g., anger). Also, we did not estimate individual anxiety and stress levels for the camera ON/OFF conditions, which could be a possible mediator of the effects of emotion on cognitive processes. It would be fruitful for future studies to pinpoint the cognitive processes that would benefit from a highlevel of awareness of being watched, such as inhibitory control in men in this case, and take advantage of such interaction to facilitate cognitive functioning in different populations.

# **ACKNOWLEDGMENT**

This work was sponsored by the Ministry of Science and Technology, Taiwan (102-2420-H-008-001-MY3, 101-2628-H-008-001- MY4, 103-2420-H-008-023-MY3, 103-2811-H-008-008).

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 July 2014; accepted: 03 January 2015; published online: 20 January 2015. Citation: Yu J, Tseng P, Muggleton NG and Juan C-H (2015) Being watched by others eliminates the effect of emotional arousal on inhibitory control. Front. Psychol. 6:4. doi: 10.3389/fpsyg.2015.00004*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology.*

*Copyright* © *2015 Yu, Tseng, Muggleton and Juan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Musical activity and emotional competence – a twin study

# *Töres P. Theorell 1,2 \*, Anna-Karin Lennartsson1, Miriam A. Mosing1 and Fredrik Ullén1*

<sup>1</sup> Department of Neuroscience, Karolinska Institute, Stockholm, Sweden

<sup>2</sup> Stress Research Institute, Stockholm University, Stockholm, Sweden

#### *Edited by:*

Fritz Strack, University of Würzburg, Germany

#### *Reviewed by:*

Daniel S. Quintana, University of Oslo, Norway Hans J. Grabe, University of Greifswald, Germany

#### *\*Correspondence:*

Töres P. Theorell, Department of Neuroscience, Karolinska Institute, Retzius Väg 8, SE-171 77 Stockholm, Sweden e-mail: tores.theorell@ki.se

The hypothesis was tested that musical activities may contribute to the prevention of alexithymia. We tested whether musical creative achievement and musical practice are associated with lower alexithymia. 8000 Swedish twins aged 27–54 were studied. Alexithymia was assessed using the Toronto Alexithymia Scale-20. Musical achievement was rated on a 7-graded scale. Participants estimated number of hours of music practice during different ages throughout life. A total life estimation of number of accumulated hours was made. They were also asked about ensemble playing. In addition, twin modelling was used to explore the genetic architecture of the relation between musical practice and alexithymia. Alexithymia was negatively associated with (i) musical creative achievement, (ii) having played a musical instrument as compared to never having played, and – for the subsample of participants that had played an instrument – (iii) total hours of musical training (r = −0.12 in men and −0.10 in women). Ensemble playing added significant variance. Twin modelling showed that alexithymia had a moderate heritability of 36% and that the association with musical practice could be explained by shared genetic influences. Associations between musical training and alexithymia remained significant when controlling for education, depression, and intelligence. Musical achievement and musical practice are associated with lower levels of alexithymia in both men and women. Musical engagement thus appears to be associated with higher emotional competence, although effect sizes are small. The association between musical training and alexithymia appears to be entirely genetically mediated, suggesting genetic pleiotropy.

**Keywords: alexithymia, musicality, depression, emotional competence, twins**

# **INTRODUCTION**

It has been shown repeatedly that musical activity in children and adults is positively associated with performance in a broad range of non-musical cognitive tasks (Schellenberg, 2004, 2006; Schellenberg and Weiss, 2013). There is considerable interest within the cognitive sciences to explore the nature and biological underpinnings of these relations further. Music listening, performance, and composing also involve affective processing (Juslin and Sloboda, 2001), and a similarly important question relates to the potential of associations between musical activity and emotional skills (Wrangsjö and Körlin, 1995; Dissanayake, 2000; Baumgartner et al., 2006; Gabrielsson, 2011).

Emotional competence can be operationalised in different ways, but one key concept in the psychosomatic literature relating emotional ability to disease development is *alexithymia,* – i.e., a deficiency in the ability to process emotions. The alexithymia concept was introduced by Sifneos (Sifneos, 1996) and has been further developed by several authors (Nemiah, 1996). The most frequently used instrument for assessing alexithymia is the Toronto Alexithymia Scale (TAS) consisting of 20 questions TAS-20; Bagby et al., 2014). As expected, alexithymia has been shown to be negatively related to other measures of emotional competence (Baughman et al., 2013).

Several studies have investigated the biological basis of alexithymia. Neuroimaging studies have shown that alexithymia is related to structural (Borsci et al., 2009) and biochemical (14) differences in emotional systems of the brain, as well as to altered patterns of regional brain activity during the processing of emotional stimuli (Duan et al., 2010; Heinzel et al., 2010; Deng et al., 2013), mentalizing (Moriguchi et al., 2006) and imagery (Mantani et al., 2005). Based on a study of 8000 Danish twins Jorgensen et al. (2007) reported a heritability of alexithymia of 30–33% with an additional 12–20% of the variance due to shared environmental influences. These estimates have subsequently been confirmed by two other twin studies (Picardi et al., 2011; Baughman et al., 2013). Alexithymia has been linked to specific polymorphisms of the catechol *O*-methyltransferase (Ham et al., 2005) and serotonin transporter genes (Kano et al., 2012), suggesting that genetically based individual differences in monoamine systems of importance for emotional processing could be one factor underlying alexithymia.

Furthermore, alexithymia has been related to several health outcomes, such as hypertension (Jorgensen and Houston, 1986; Grabe et al., 2010), depression, and reduced likelihood of recovery from alcoholism and substance abuse (Picardi et al., 2011; Evren et al., 2012; de Haan et al., 2013). Further, it has been shown that subjects with early stage hypertension have a relatively poor ability to describe feelings (Theorell et al., 1984). In summary, the literature suggests that alexithymia is a clinically relevant construct. It is associated with differences in brain systems that are used for emotional processing, self-awareness and theory of mind. Individual differences in alexithymia appear to

largely depend on genetic factors and some additional shared environment.

The expression and perception of emotions are central elements of music listening and performance (Juslin and Sloboda, 2001). This relates to a more profound and complex question about the functional role of music in human beings and what role emotions may have in this (Madison, 2011). Some researchers have speculated that music may facilitate the experience of emotions and thus be of importance for the development of the ability to identify and differentiate emotions (Dissanayake, 2000; Laiho, 2004; Baumgartner et al., 2006; Gabrielsson, 2011). It therefore appears reasonable that musical engagement may be associated with emotional competence, i.e., lower alexithymia. However, no study to date has explored the relationship between musical activity and alexithymia. Such associations could, if their direction is from music practice to reduced alexithymia, potentially be of interest for intervention approaches. If musical engagement could facilitate emotional development, music education may potentially serve as an intervention strategy to reduce alexithymia, particularly in adolescence (Lichtenstein et al., 2006). However, clearly it may also be the case that alexithymic individuals are simply less likely to seek out musical engagement or that a third factor causes both, alexithymia and low musical engagement.

Here, we use a large sample of adult twins to explore – based on the considerations summarized above – whether alexithymia is negatively associated with active musical engagement operationalized as musical achievement as well as musical practice throughout life time. A reasonable assumption is furthermore that music practice with an ensemble is particularly strongly associated with social interaction. Since emotional and social competence are related to one another, ensemble playing is likely to contribute to the statistical variance in emotional competence. One possibility would be that the two (number of practice hours and experience of ensemble) have additive effects. Another possibility, would be a multiplicative effect, e.g., because emotionally competent individuals are attracted to ensemble playing or because ensemble practice stimulates emotional competence. Therefore we also examined whether playing/singing in an ensemble is associated with lower alexithymia score among those who have ever practiced – with formal tests of multiplicative interaction. Finally, we explored genetic and environmental influences on the relationship between alexithymia and music practice.

# **MATERIALS AND METHODS**

**PARTICIPANTS**

Data were collected as part of a web-survey sent out to a cohort of approximately 32,000 twins born between 1959 and 1985 – the STAGE cohort (Lichtenstein et al., 2002) – from the Swedish Twin Registry (STR), one of the largest registries of its kind (Formann and Piswanger, 1979; Lichtenstein et al., 2002; Magnusson et al., 2013). Zygosity of the twins in the STR has been determined with questions about intra-pair similarities and subsequently was confirmed in 27% of the twins using genotyping, showing that questionnaire based zygosity determination was correct for more than 98% of twin pairs. For further details on the STAGE cohort and zygosity determination in the STR (see Lichtenstein et al., 2002, 2006). The present study received approval from the

Regional Ethics Review Board in Stockholm (Dnr 2011/570-31/5, 2011/1425-31, 2012/1107/32). In total, 11,543 individuals participated in the web-survey. For the phenotypic analyses, to control for relatedness within the sample, we randomly selected one twin from each pair and all single twins (*n* = 8599). Of these, 5,881 individuals had valid TAS-20 scores comprising the sample for phenotypic analyses. For the twin analyses we included all individuals with a valid TAS-20 score and zygosity information (*n* = 8, 110), consisting of 1,755 complete twin pairs (851 monozygotic (MZ) and 845 dizygotic (DZ)) as well as 4,600 single twins without the co-twin participating. All participants were aged between 27 and 54 (mean 40.7, SD 7.7).

# **MEASURES**

#### *Music practice*

Participants were first asked to indicate whether they had ever played an instrument (including singing). Those who responded positively were questioned about their starting year of practicing and the typical weekly intensity of practicing during four ageintervals (age 0–5, 6–11, 12–17, and 18 till now). From these measures the lifetime total hours of playing was estimated. As expected, music training was positively skewed and kurtosed with many individuals having none or little training. Of the 5,788 participants who had played or sung, 5,777 participants (2,060 men and 3,717 women) reported hours of music practice. Among those who had ever played or sung, the skewness for number of music practice hours was less pronounced than in the total group (skewness 0.80 for men and 1.01 for women). In preliminary analyses, the data were log-transformed and univariate analyses were conducted on both transformed and untransformed data. However, given the large sample size and the fact that the results were very similar with and without log-transformation, untransformed data were used for the final analyses. The approximated total number of training hours in life up to the examination ranged from 0 to 18,400 in men and from 0 to 20,800 in women. More men (*n* = 1,327) than women (*n* = 995) reported that they had never played an instrument.

#### *Ensemble playing*

"Ensemble playing" was self-assessed by the participant in the larger questionnaire on musical background. Ensemble playing was treated as a dichotomous variable, i.e., participants who reported that they had practiced ensemble playing during any age period were assigned to the "ensemble" group and other subjects to the "no ensemble" group. Alexithymia levels were compared between the playing group and non-playing group.

#### *Musical achievement*

Musical achievement was assessed used the Music subscale of a Swedish adaptation of the Creative Achievement Questionnaire of Carson (Simonsson-Sarnecki et al., 2000). Musical creative achievement is self-rated using a 7-graded scale, ranging from zero (never) played or sung to seven (nationally or internationally awarded professional).

#### *Alexithymia*

A back-translated and psychometrically tested Swedish version of TAS-20 (Magnusson Hanson et al., 2014) was used, containing three subscales: the inability to handle emotions due to emotions being poorly recognised (difficulty recognizing), the inability to describe feelings (difficulty describing), and mismatch between coping behaviour and emotions (externally oriented thinking). Here, only the full scale score was used. The scores were normally distributed.

#### *Depressive symptoms*

Depressive symptoms were measured by means of a six item subscale of the Hopkins symptom checklist (SCL) depression scale. The items are graded from 0–4 giving a range of full scores (sum scores) of 0–24. It has been used in Swedish and Danish population studies (Magnusson Hanson et al., 2014). This was included in the analyses because we wanted to examine whether any putative associations with alexithymia would hold when controlling for depressive symptoms.

#### *Education*

Level of education was assessed by means of a 10-graded scale reflecting the level of formal school education according to Statistics Sweden. The lowest level was unfinished elementary compulsory school and the highest level an academic doctoral degree. The four lower levels corresponded to no more than high school education whereas the six upper levels corresponded to at least some exposure to college or university education.

#### *Wiener Matrizen-Test (WMT)*

To estimate general fluid intelligence the Wiener Matrizen-Test (WMT; Carson et al., 2005) was used. The WMT is a timed (maximal test taking time: 25 min.) matrix intelligence test similar to the standard progressive matrices (SPM) from Raven. It consists of 24 multiple choice items. The total score is the number of correctly answered items. The reliability of the WMT has been shown to be relatively high (Cronbach's alpha = 0.81) and the test correlates well with Raven's SPM (*r* = 0.92). The WMT score was normally distributed without any outliers.

#### **STATISTICAL METHODS**

#### *Group comparisons between practice and non-practice groups*

Means of alexithymia scores and estimated total hours of music practice were calculated separately in men and women. Alexithymia scores were compared, using ANCOVA with adjustment for age, in those who reported that they had never played or sung (1,015 men and 708 women) and those who reported that they were playing or singing now or had been doing so in the past (1,512 men and 2,646 women).

#### *Analyses within the practice group*

In these phenotypic analyses, only men and women who reported that they were playing or singing now or in the past have been included.


In order to facilitate interpretation, partial correlations (adjusting for age) analyses were computed not only between alexithymia scores and hours of music practice but also between these mainfactors and musical achievement, education level, depression scores, and WMT scores. Since 30 correlations were computed for each gender the *p-*value required for significance was set at 0.001 in these correlation analyses.

Since there was a significant correlation between education and alexithymia, separate means of alexithymia scores were also computed for subgroups (with and without college or university education). Similarly significant correlations were found between alexithymia and depressive symptoms. Therefore additional partial correlation analyses (adjusting for each depression and education) were conducted between alexithymia scores and music practice hours. All analyses were performed separately for men and women.

A MANCOVA was performed using ensemble/no ensemble playing and number of musical practice hours divided in tertiles as explanatory variables and alexithymia score as dependent variable with adjustment for age. This allowed us to test whether there was an interaction between ensemble playing and number of practice hours.

#### *Genetic analyses based on the full sample*

Due to the fact that identical (MZ) twins share all their genes and non-identical (DZ) twins share on average half of their segregating genes, twin studies allow for partitioning of the variance of a trait (and similarly the covariance between two traits) into additive genetic (A), shared environmental (C), and residual (also known as non-shared environmental – E) variation. Variance due to A results from the sum of allelic effects across genes. C results from environmental influences shared by siblings, e.g., home environment, common friends, or socioeconomic status; while E variance results from environmental influences that are not shared between siblings, such as illness or injury and this includes also measurement error. Structural equation modelling Mx (Neale et al., 2006) is employed to estimate the combination of A, C, and E influences that best explains the observed data. For further details on the twin modeling methods or the program used see (Neale and Maes, 2004). Here, a bivariate sex-limitation model was fitted for music practice and alexithymia corrected for age to explore genetic and environmental influences on the covariance between the two traits. The full sample – including subjects who had never practiced – was used for this analysis. The main rationale behind this was statistical power. The number of MZ twins was reduced when the analysis was confined to those who had experience of practice. In addition, since musical achievement and number of hours of music practice were strongly correlated we decided to select one of them – hours of practice – in these analyses.

# **RESULTS**

#### **RESULTS OF PHENOTYPIC ANALYSES**

Supporting our hypothesis, participants who reported that they had never played an instrument had a significantly higher age adjusted alexithymia mean score (47.5, SD = 9.94, *n* = 1,015 for men and 44.0, SD = 10.5, *n* = 708 for women) than those who reported having ever played (45.8, SD = 9.78, *n* = 1,512 for men and 41.5, SD = 10.2, *n* = 2,646 for women). Further, there were significant differences between the low and high education groups (*p* = 0.001) for both men and women. Also, a higher percentage of men compared to women never engaged in musical practice (40% vs. 21%).

Within the practice group, on average men had higher phenotypic alexithymia scores (*M* = 45.7, SD = 9.8) than women (*M* = 41.5, SD = 10.2).

**Table 1** shows the age-adjusted paired correlations between all study variables separately for men and women in the practice group. The patterns were very similar in men and women therefore in the text we only report the estimates for men (for estimates for both sexes see **Table 1**). *Number of hours of music practice* was significantly and negatively associated with alexithymia (*r* = −0.12) and positively correlated with musical achievement (*r* = 0.52) and education (*r* = 0.10).

*Alexithymia* was negatively correlated with musical achievement (*r* = −0.11) and education (*r* = −0.17) and positively correlated with depressive symptoms (*r* = 0.29). *Musical achievement* was correlated with education (*r* = 0.11) and with WMT (*r* = 0.15) but unrelated to depressive symptoms. *Depressive symptoms* were unrelated to education and WMT. As expected education was correlated with WMT (*r* = 0.30).

**Table 1 | Product moment correlations between the study variables in participants who have ever played or sung (men and women).**


\*Correlation is significant at the 0.001 level (2-tailed). n = 1488 –1510. Phenotypic correlations (n = 1488 –1510) with random exclusion of one member of each twin pair.


\*Correlation is significant at the 0.001 level (2-tailed). Phenotypic correlations (n = 2617 – 2639) with random exclusion of one member of each twin pair.

Since there was a significant correlation between education and alexithymia, separate age adjusted marginal means were also computed for the college/no college subgroups (see **Table 2**). Male participants without college or university education reported the highest TAS-20 scores. The remaining three groups had means falling in the following order: men with college education, women without college education, and women with college education. The differences between those without and with college education were highly significant both among men and women (*p* < 0.0001) according to ANCOVA analysis with age correction (with educational group as explanatory and alexithymia score as dependent variable). In addition, the TAS differences between men and women were highly significant in both educational groups (*p* < 0.0001).

In line with our second hypothesis, we found significantly lower alexithymia scores among those who played ensemble compared to the other musically active participants, both among men and among women. These differences were highly significant for both men and women (*p* < 0.0001). In addition, the TAS differences between men and women were highly significant both in the ensemble and the no ensemble group (*p* < 0.0001).

A MANCOVA using ensemble/no ensemble playing and number of musical practice hours divided in tertiles as explanatory variables, and alexithymia score as dependent variable with age as covariate, showed that both ensemble playing and number of practice hours contributed statistically significantly to a reduced alexithymia score (for men number of hours *p* < 0.0001 and ensemble *p* = 0.032 and for women number of hours *p* < 0.0001 and ensemble *p* = 0.007). There was no interaction between number of hours and ensemble experience in relation to alexithymia. This finding is illustrated in **Figure 1** (men) and **Figure 2** (women). The figure shows that for men the average difference in marginal means between men who have a low number of practice hours (lowest tertile) and no experience in ensemble playing and those who have a high number of practice hours (highest tertile) and experience of ensemble playing is 4.9, corresponding to approximately half a standard deviation. The corresponding difference for women is 4.0.

In a corresponding MANCOVA with age correction it was observed that musical achievement and number of hours of practice contribute statistically independently to alexithymia score

**Table 2 | Alexithymia marginal means (adjusted for age) in subgroups (no college/college and no ensemble playing/ensemble playing in men and women.**


Differences between no college/college and no ensemble/ensemble separately for men and women are all significant (ANCOVA p = 0.0001).

**FIGURE 1 | Age adjusted marginal Toronto Alexithymia Scale (TAS-20) means for ensemble (yes/no) and music practice tertiles among subjects (men) who have practiced playing an instrument.** Music practice tertiles: lower, range 40–800, median 360, Middle, range 840–3160, median 1600, Upper, range 3200–18400, median 5600.

**have practiced playing an instrument.** Music practice tertiles: lower, range 40–800, median 440, Middle, range 840–2360, median 1360, Upper, range 2400–20800, median 5040.

(for men number of hours *p* = 0.004 and musical achievement *p* = 0.033, and for women number of hours *p* < 0.0001 and musical achievement *p* = 0.002).

Both education and depressive symptoms were associated with alexithymia. Therefore we tested the effect of controlling for these variables on the correlations between number of music practice hours and alexithymia. The effects were very small (adjustment for education and age *r* = −0.10 in men and *r* = −0.08 in women and with adjustment for depressive symptoms and age *r* = −0.15 for men and *r* = −0.12). WMT did not show a significant association with alexithymia (**Table 1**) and adjusting for WMT score and age left the correlations between number of music practice hours and alexithymia unchanged (*r* = −0.12 for men and *r* = −0.09 for women). Since these results indicated that adjustment for education, depressive symptoms, and WMT score was of minor importance, the genetic analyses were performed without these adjustments.

#### **RESULTS OF TWIN ANALYSES**

Age and sex adjusted phenotypic correlation between hours of music practice and alexithymia was the same as the one reported above at r = −0.12. **Table 3** shows the twin correlations for hours of music practice and alexithymia. Since the twin correlations were slightly different for men and women, a sex-limitation model was fitted allowing the ACE-estimates to differ between the sexes. Multivariate model fitting results indicated that ACE influences on the TAS as well as the ACE-cross paths between musical practice and TAS could be equated between males and females without a significant deterioration of model fit. Further, all C-influences on the TAS as well as the E-cross path between the two variables could be removed without significant deterioration of the model fit, indicating that an AE model fits the TAS best with all the covariance between the traits being due to shared genetic influences (A). The best fitting model is shown in **Figure 3**. Heritability for the TAS was 36% (27–41%) without significant sex-differences.

# **DISCUSSION**

The results of the present study suggest that there is a small but significant association between alexithymia and creative achievement in music as well as musical practice. This relationship remained significant even when we controlled for depressive symptoms, education, and cognitive ability, factors that have been shown to be related to participation in activities like music practice (Ahlberg et al., 2004; Picardi et al., 2011). Further we found that ensemble playing was associated with lower alexithymia score compared to other musical engagement which is consistent with past research (Juslin and Sloboda, 2001). Since ensemble playing requires social interaction which requires emotional competence this association

**Table 3 |Twin correlations for monozygotic (MZ) and dizygotic (DZ) pairs (female, male, and opposite sex pairs separately) zygosity (bottom) for Toronto Alexithymia Scale and hours of practice corrected for age.**


MZ = Monozygotic; DZ = Dizygotic; DZOS = DZ opposite-sex; TAS-20 =Toronto Alexithymia scale.

could be due either to alexithymia individuals avoiding ensembles and vice versa or to an effect of ensemble playing on the development of emotional competence.

Further, in line with results from a previous similar Danish twin cohort study (Jorgensen et al., 2007; Baughman et al., 2013) we confirmed that about one third of the variance in TAS-20 scores is genetically determined. Our results differ, however, from those in the Danish study as we did not find any significant shared environment effect.

The relationship between musical practice and alexithymia could be explained by shared genes. This indicates that most of the association is due to genetic pleiotropy – the same genes making an individual more susceptible to develop alexithymia also predispose this individual to perform less musical practice and/or vice versa. It is important to note that our findings do not allow for any conclusions regarding causality. However, given that the relationship is due mainly to genetic pleiotropy it is rather unlikely that environmental manipulation of one of the traits would necessarily result in changes in the other trait. It is important to emphasize, however, that a rigorous examination of a causal role of musical practice in preventing alexithymia may require longitudinal controlled studies. The present study is an examination of the association of "volunteer musical practice" – which may be strongly influenced by family selection including genetic factors.

Music is known to induce emotions and it has been proposed that musical activities may contribute to the prevention of alexithymia (Juslin and Sloboda, 2001). There are many examples of

music deliberately constructed for the amplification of specific feelings in specific circumstances, for instance music for funerals and solemn or joyful celebration. Such emotional amplification could facilitate development of emotional skills – an emotion is partially "translated" by the music. In such a situation the social context has a decisive role in the emotional effect of the music. Visual impressions may amplify the neurobiological effect of the music and vice versa (Baumgartner et al., 2006). Konecni (2008) have challenged the idea that specific music experiences per se are related to specific emotions. They maintain that "being moved" and "esthetic awe" are the only established psychological reactions to music and that more specific emotional reactions are induced by a combination of musical experience with circumstances and previous experiences. Västfjäll (2001) have described methodology which facilitates the study of the complex interplay between music and emotional responses. In summary it is fair to say that musical experiences may contribute in a complex web of circumstances to specific or more general emotional states.

Ensemble playing is another example of a social musical context. In this study, ensemble practice added to the statistical variance in alexithymia. In future studies one could therefore focus more on the association between ensemble playing and alexithymia. In addition it would be valuable to examine the relationship between other cultural activities, in particular theatre, and alexithymia. A recent Australian quasi-experimental study with control groups showed beneficial effects of increased cultural activities in the school curriculum on emotional and social competence in the pupils (Vaughan et al., 2011). In the same vein, the effects of other kinds of structured emotion teaching should be examined.

#### **STRENGTHS AND LIMITATIONS**

The use of the self-administered TAS-20 builds upon the subjects' ability to adequately respond to the questions regarding ability to identify emotional states, to give words to them and to act accordingly. Such emotional insight (and ability to fill in the questionnaire adequately) may be gained mainly from people whom the subject interacts with. One may be told by others that one lacks emotional insight. Regardless of which ability the subject has in his/her way of describing this, capacity to deal with emotions may in itself arise in social interaction. It is therefore possible that a self-administered questionnaire of the type that TAS-20 may not capture all aspects of alexithymia. This could lead both to underestimation and overestimation of the true association.

As during recruitment it was clearly indicated that the present study was exploring musical engagement we expect to have a slight over-representation of music-interested individuals in the sample. Apart from that we are confident that our sample represents the Swedish population in the age group 27–54.

#### **CONCLUSION**

This study of a large Swedish twin cohort has shown that musical practice and ensemble playing are significantly albeit weakly associated with reduced alexithymia. The association between musical practice and alexithymia holds even after adjustment for age, education, depressive symptoms, and intelligence and is of the same magnitude in men and women. Accordingly to our twin modelling results, the association between musical practice and alexithymia is genetically determined.

#### **ACKNOWLEDGMENTS**

This research was supported financially by the Swedish Tercentenary Fund and by the Salén Foundation, which is gratefully acknowledged.

#### **REFERENCES**


environment in the pathogenesis of cardiovascular disease. *Scand. J.Work Environ. Health* 10, 373–380. doi: 10.5271/sjweh.2305


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 03 April 2014; paper pending published: 05 June 2014; accepted: 01 July 2014; published online: 16 July 2014.*

*Citation: Theorell TP, Lennartsson A-K, Mosing MA and Ullén F (2014) Musical activity and emotional competence – a twin study. Front. Psychol. 5:774. doi: 10.3389/fpsyg. 2014.00774*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Theorell, Lennartsson, Mosing and Ullén. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited andthatthe original publication inthis journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*