# EXTINCTION LEARNING FROM A MECHANISTIC AND SYSTEMS PERSPECTIVE

EDITED BY: Denise Manahan-Vaughan, Onur Gunturkun and Oliver T. Wolf PUBLISHED IN: Frontiers in Behavioral Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2016 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

*The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88919-908-2 DOI 10.3389/978-2-88919-908-2

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **EXTINCTION LEARNING FROM A MECHANISTIC AND SYSTEMS PERSPECTIVE**

Topic Editors:

**Denise Manahan-Vaughan,** Ruhr University Bochum, Germany **Onur Gunturkun,** Ruhr University Bochum, Germany **Oliver T. Wolf,** Ruhr University Bochum, Germany

The cover image reflects a side view of the brain's left hemisphere. It illustrates a schematic representation of the nerve fibers within the brain. The fibers displayed in this picture have been calculated using the specific technique in magnetic resonance imaging, known as diffusion tensor imaging (DTI). The contours that you see in the image represent the contours of the brain or the skull. The different colors in this figure encode the orientation of fiber tracts, i.e.: green: front-to-back, blue: top-to-down and red: right-to-left.

Extinction learning is a complex process that involves a large number of subcortical and cortical processes. This DTI-image demonstrates the richness of connectivities of the human brain of which some play a crucial role in extinction.

Image courtesy of Dr. Erhan Genc, Ruhr University Bochum, Faculty of Psychology, Department of Biopsychology.

**Citation:** Manahan-Vaughan, D., Gunturkun, O., Wolf, O. T., eds. (2016). Extinction Learning from a Mechanistic and Systems Perspective. Lausanne: Frontiers Media. doi: 10.3389/978-2-88919-908-2

# Table of Contents


#### **Neuronal correlates of extinction**


Jarid Goodman and Mark G. Packard

*31 The Role of the Medial Prefrontal Cortex in the Conditioning and Extinction of Fear* Thomas F. Giustino and Stephen Maren

#### **Neurotransmitter systems involved in extinction learning**


# **Extinction learning and memory reconsolidation**

*128 Extinction and Retrieval + Extinction of Conditioned Fear Differentially Activate Medial Prefrontal Cortex and Amygdala in Rats* Hongjoo J. Lee, Rebecca P. Haberman, Rheall F. Roquet and Marie-H. Monfils

*138 Assessing Fear Following Retrieval + Extinction Through Suppression of Baseline Reward Seeking vs. Freezing*

Jason Shumake and Marie H. Monfils

*147 Retrieval and Reconsolidation Accounts of Fear Extinction* Ravikumar Ponnusamy, Irina Zhuravka, Andrew M. Poulos, Justin Shobe, Michael Merjanian, Jeannie Huang, David Wolvek, Pia-Kelsey O'Neill and Michael S. Fanselow

### **Pharmacological and cognitive-behavioral modulation of fear and extinction**

*158 Increased perceived self-efficacy facilitates the extinction of fear in healthy participants*

Armin Zlomuzica, Friederike Preusser, Silvia Schneider and Jürgen Margraf *170 Gradual extinction reduces reinstatement*

Youssef Shiban, Jasmin Wittmann, Mara Weißinger and Andreas Mühlberger

*181 Morphine administration during low ovarian hormone stage results in transient over expression of fear memories in females* Emily M. Perez-Torres, Dinah L. Ramos-Ortolaza, Roberto Morales, Edwin Santini, Efrain J. Rios-Ruiz and Annelyn Torres-Reveron

#### **Stress and pain**

*188 Modulation of Fear Extinction by Stress, Stress Hormones and Estradiol: A Review*

Ursula Stockhorst and Martin I. Antov

*214 Differential Effects of Controllable Stress Exposure on Subsequent Extinction Learning in Adult Rats*

Osnat Hadad-Ophir, Noa Brande-Eilat and Gal Richter-Levin


#### **Contextual and additional factors**


# Editorial: Extinction Learning from a Mechanistic and Systems Perspective

#### Denise Manahan-Vaughan<sup>1</sup> \*, Oliver T. Wolf <sup>2</sup> and Onur Güntürkün<sup>3</sup>

<sup>1</sup> Department of Neurophysiology, Medical Faculty, Ruhr University Bochum, Bochum, Germany, <sup>2</sup> Department of Cognitive Psychology, Faculty of Psychology, Institute of Cognitive Neuroscience, Ruhr University Bochum, Bochum, Germany, <sup>3</sup> Department of Biopsychology, Faculty of Psychology, Institute of Cognitive Neuroscience, Ruhr University Bochum, Bochum, Germany

Keywords: extinction learning, fear conditioning, appetitive learning, predictive learning, Pavlovian conditioning, brain structure, neurotransmitter, renewal

**The Editorial on the Research Topic**

#### **Extinction Learning from a Mechanistic and Systems Perspective**

Throughout life, we learn to associate stimuli with their consequences. But some of the new information that we encounter forces us to abandon what we had previously acquired. This old information is then subject to a new learning process that is called extinction learning. This involves a large number of brain structures (Kattoor et al., 2013; Lissek et al., 2013, Lissek et al.; Merz et al., 2014). Extinction is an unusually complex learning process that can involve both Pavlovian (classical; Pavlov, 1927; Lattal and Lattal, 2012) and operant (instrument) conditioning (Skinner, 1938; Bouton et al., 2012). A further hallmark is its context-dependency (Bouton, 2004) that is likely to rely on a tight interaction between the hippocampus and other brain areas (e.g., André et al.; Icenhour et al., 2015). Thus, one of the aims of the present Research Topic was to incorporate studies that analyze the concert of neural structures that enable extinction learning.

Edited and reviewed by: Nuno Sousa,

#### University of Minho, Portugal \*Correspondence:

Denise Manahan-Vaughan denise.manahan-vaughan@rub.de

> Received: 26 April 2016 Accepted: 24 May 2016 Published: 13 June 2016

#### Citation:

Manahan-Vaughan D, Wolf OT and Güntürkün O (2016) Editorial: Extinction Learning from a Mechanistic and Systems Perspective. Front. Behav. Neurosci. 10:115. doi: 10.3389/fnbeh.2016.00115

The old memory trace may be partly, or not at all forgotten during extinction (Üngör and Lachnit, 2006). It tends to re-emerge after a passage of time (spontaneous recovery), when re-exposure to the context of original learning occurs (renewal), or unexpected exposure to the unconditioned stimulus takes place (reinstatement). Such invasive memories are key symptoms of anxiety or pain disorders. They especially occur in individuals with enhanced susceptibility (Mosig et al.; Glombiewski et al., 2015). Although pathological fear in anxiety disorders can be treated through extinction-based approaches, treatment is not always successful in the long-term, underscoring the need to understand the mechanisms underlying impaired extinction. Therefore, the second aim of the Research Topic was to include publications that are situated at the transition between basic and clinical neuroscience.

Given the relevance of extinction, it is astonishing how little we know about extinction learning, in terms of its neural fundaments and its development, especially when moving outside the realm of fear extinction in rodents. The third aim of the Research Topic was therefore to include papers on the uncharted territories of extinction learning that involve less-studied entities such as the immune system (Hadamitzky et al., 2016) or hormonal factors (Wolf et al., 2015; Maren and Holmes, 2016), less-studied species (Lengersdorf et al.) or novel paradigms (Wiescholleck et al., 2014).

One specific goal of this Research Topic was to offer a basis for trans-species comparisons, as reflected by the spectrum of animals described that range from snails, through mice, rats, and pigeons. Several of the studies also describe extinction learning in humans, including pharmacological approaches. A number of studies (André et al.; Lengersdorf et al.; André and Manahan-Vaughan; Andrianov et al.; de Oliveira et al.; Lissek et al.) addressed neurotransmitter systems that are known to be involved in other forms of learning (Morris, 2013; Seyedabadi et al., 2014; Bauer, 2015) and in synaptic plasticity that is believed to underlie learning (Harley, 2004; Lesch and Waider, 2012; Park et al., 2013; Hansen and Manahan-Vaughan, 2014; Hagena et al., 2015). Here, for example, antagonism of N-methyl-Daspartate receptors (NMDAR) prevented appetitive extinction in pigeons (Lengersdorf et al.), and GluN2B-containing NMDAR were found to play a key role in extinction of conditioned suppression of licking in rats (de Oliveira et al.). In an interesting corollary to the latter finding, Shumake and Monfils describe how conditioned suppression of licking is far more sensitive to extinction than freezing behavior, and along with Lee et al. investigated the impact of reactivating the original memory trace on extinction success. Examination of the role of dopamine receptors in appetitive learning in rats (André and Manahan-Vaughan) and predictive learning in humans (Lissek et al.), highlight differences that may relate to the species, or the extinction learning paradigm studied.

Studies with regard to the neural basis of extinction learning, and its associated brain structures, revealed a specific and experience-dependent role of microcircuitry within the basolateral amygdala (Sangha). In their review article, Giustino and Maren challenge the common assumption that the medial prefrontal cortex (mPFC) mediates the expression, whereas the infralimbic cortex (IL) mediates the suppression of fear responses, whereas Lee at al. offer experimental evidence that extinction learning and retrieval trigger differentiated responses in the mPFC and amygdala. Goodman and Packard differentiated between extinction learning of response and place learning, and provide evidence that the effectivity of the extinction learning strategy depends on the memory system (dorsolateral striatum vs. hippocampus) that encoded the original experience. In line with studies in rats (Gershman et al.), Shiban et al. observed that gradually reducing the frequency of aversive stimuli, in a Pavlovian fear conditioning paradigm in humans, is more effective in averting the return of fear than abrupt stimulus withdrawal, and Zlomuzica et al. demonstrate that improved self-efficacy also improves fear extinction. By contrast,

#### REFERENCES


Vervliet and Indekkeu show that low-cost avoidance behavior is resilient to extinction. Earlier studies indicate that extinction learning is reinforced by a context change (Bouton, 2004). Here, Sjouwerman et al. report that the timing of the context change is decisive with regard to the functional outcome with regard to both extinction and renewal. At the structural and/or molecular levels, several studies provided evidence for the direct involvement of the hippocampus in extinction learning (Lissek et al.; de Oliveira et al.; Wille et al.). Whereas, de Oliveira et al. provide evidence of the involvement of the dorsal hippocampus in conditioned suppression, Wille et al. describe how modulation of the expression of chromatin remodeling factors in the ventral hippocampus rescue impaired extinction of conditioned fear. Several studies examined hormonal control of extinction learning in fear, or stress-based, paradigms (Perez-Torres et al.; Hadad-Ophir et al.; Labrenz et al.): aspects that were also addressed in a review article by Stockhorst and Antov and a research perspective by Elsenbruch and Wolf.

What becomes apparent from these studies is the emergence of fine-tuning of our understanding as to which neural structures regulate extinction learning, what common denominators (and differences) exist between species, and how the regulation of extinction learning by neurotransmitter systems aligns with current knowledge as to the role of these systems in learning and memory. The papers compiled in this Research Topic offer new and valuable insights into the mechanisms and functional implementation of extinction learning at its different levels of complexity, and form the basis for new concepts and research ideas in this field.

## AUTHOR CONTRIBUTIONS

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

### ACKNOWLEDGMENTS

DM, OW, and OG are members of the research consortium FOR1581 "Extinction Learning: Neural mechanisms, behavioral manifestations, and clinical implications," that is funded by the German Research Foundation (DFG) and which supported this article.


fear extinction in a renewal paradigm. Hum. Brain Mapp. 36, 317–393. doi: 10.1002/hbm.22837


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Manahan-Vaughan, Wolf and Güntürkün. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Plasticity of Fear and Safety Neurons of the Amygdala in Response to Fear Extinction

Susan Sangha1, 2 \*

*<sup>1</sup> Department of Psychological Sciences, Purdue University, West Lafayette, IN, USA, <sup>2</sup> Ernest Gallo Clinic and Research Center, University of California, San Francisco, Emeryville, CA, USA*

Fear inhibition learning induces plasticity and remodeling of circuits within the amygdala. Most studies examine these changes in nondiscriminative fear conditioning paradigms. Using a discriminative fear, safety, and reward conditioning task, Sangha et al. (2013) have previously reported several neural microcircuits within the basal amygdala (BA) which discriminate among these cues, including a subpopulation of neurons responding selectively to a safety cue and not a fear cue. Here, the hypothesis that these "safety" neurons isolated during discriminative conditioning are biased to become fear cue responsive as a result of extinction, when fear behavior diminishes, was tested. Although 41% of "safety" neurons became fear cue responsive as a result of extinction, the data revealed that there was no bias for these neurons to become preferentially responsive during fear extinction compared to the other identified subgroups. In addition to the plasticity seen in the "safety" neurons, 44% of neurons unresponsive to either the fear cue or safety cue during discriminative conditioning became fear cue responsive during extinction. Together these emergent responses to the fear cue as a result of extinction support the hypothesis that new learning underlies extinction. In contrast, 47% of neurons responsive to the fear cue during discriminative conditioning became unresponsive to the fear cue during extinction. These findings are consistent with a suppression of neural responding mediated by inhibitory learning, or, potentially, by direct unlearning. Together, the data support extinction as an active process involving both gains and losses of responses to the fear cue and suggests the final output of the integrated BA circuit in influencing fear behavior is a balance of excitation and inhibition, and perhaps reversal of learning-induced changes.

Keywords: amygdala, fear, safety, extinction

# INTRODUCTION

Environmental cues signifying danger, safety, or reward availability can have a potent effect in emotion regulation. Accurately discriminating among these cues is important in initiating the proper emotional response in order to guide behavior. Maladaptive emotion regulation can lead to a wide-range of clinical problems, such as anxiety disorders and addiction. Since potentially rewarding and dangerous stimuli often occur simultaneously leading to opposing behaviors of approach or avoidance, respectively, reward- and fear-related circuits must interact in order to mediate these antagonistic behaviors. Approach and avoidance behaviors can also be modulated

#### Edited by:

*Denise Manahan-Vaughan, Ruhr University Bochum, Germany*

#### Reviewed by:

*Marie H. Monfils, University of Texas at Austin, USA Stephen Maren, Texas A&M University, USA Stefan Herlitze, Ruhr-University Bochum, Germany*

> \*Correspondence: *Susan Sangha sangha@purdue.edu*

Received: *27 August 2015* Accepted: *06 December 2015* Published: *24 December 2015*

#### Citation:

*Sangha S (2015) Plasticity of Fear and Safety Neurons of the Amygdala in Response to Fear Extinction. Front. Behav. Neurosci. 9:354. doi: 10.3389/fnbeh.2015.00354* by signals that inform the organism if the environment is safe or not. The inability to discriminate among danger, safety, and reward cues can lead to generalized fear responses that are enhanced in Post-traumatic Stress Disorder (PTSD) patients (Jovanovic et al., 2012).

Behavioral therapy for maladaptive fear often involves repeated exposures to the danger cue in the absence of an aversive outcome, a procedure known as extinction. Through repeated exposures, the subject feels an increasing sense of control over the situation and fear diminishes. Safety conditioning is another method of reducing fear. During safety conditioning, a safety cue in conjunction with a danger cue signifies no aversive outcome whereas the danger cue on its own does result in an aversive outcome. Thus, extinction and safety conditioning are related but distinct phenomena. Safety cues can even act as positive reinforcers, suggesting the mechanisms of safety learning may overlap with reward learning (Christianson et al., 2012; Sangha et al., 2013).

The amygdala has been consistently implicated in processing and regulating a myriad of emotional responses (for review see Janak and Tye, 2015). The basal amygdala (BA) in particular is important for discriminating among sensory stimuli that signal multiple outcomes of a similar valence (Málková et al., 1997; Corbit and Balleine, 2005; Balleine and Killcross, 2006), and it possesses neuronal populations selective for valence (Schoenbaum et al., 1999; Paton et al., 2006; Belova et al., 2007; Shabel and Janak, 2009; Sangha et al., 2013).

Evidence suggests that fear extinction learning induces plasticity and remodeling of inhibitory circuits and synapses within the amygdala (Heldt and Ressler, 2007; Lin et al., 2009; Sangha et al., 2012), as well as decreased synaptic efficacy in the medial prefrontal cortex-BA pathway (Cho et al., 2013). Within the BA, "extinction" neurons have been reported (Herry et al., 2008). These are neurons that are unresponsive to a fear cue before extinction but become responsive to the fear cue after extinction, when fear behavior is diminished. Diminished fear behavior is also seen during safety conditioning in response to a safety cue. Using a discriminative conditioning task that allows assessment of fear, safety and reward cue learning together, Sangha et al. (2013) demonstrated significant suppression of freezing behavior in response to a compound fear+safety cue compared to the high freezing seen in response to a fear cue. In addition, this study also reported several neural microcircuits within the BA that showed a discriminative response to these cues. In particular, 24% of recorded neurons were responsive to the compound fear+safety cue but unresponsive to the fear cue when presented alone suggesting these neurons are encoding safety. Similar to these "safety" neurons, the "extinction" neurons reported by Herry et al. (2008) were also unresponsive to the fear cue before extinction training. Since safety conditioning and extinction are related phenomena, neurons classified as "safety" neurons in Sangha et al. (2013) were here examined through extinction to see if they became "extinction" neurons, similar to the neurons reported by Herry et al. (2008).

To do this, firing rates of neurons classified as discriminative, nondiscriminative or unresponsive during discriminative conditioning (DC), based on their responses to the fear cue alone and the compound fear+safety cue, were examined in response to the fear cue during extinction training and recall as fear behavior decreased. The hypothesis tested is that there is a bias for the neurons that are safety cue responsive during DC to become responsive to the fear cue as fear extinction progresses.

# MATERIALS AND METHODS

# Subjects

Fourteen Long Evans male rats (Harlan) weighing 350–400 g at the beginning of experiments were single housed under a 12 h light/dark cycle (lights on 07:00) and handled for 1 week before commencing experiments. All procedures were performed during the light cycle and approved by the Gallo Center Institutional Animal Care and Use Committee in accordance with the National Institute of Health guidelines. Rats had ad libitum access to food and water up until the third reward learning session, at which point they were restricted to 22 g of food per day for the remainder of the experiment.

# Behavioral Apparatus

The experimental chambers, used in all experiments and obtained from MedAssociates, were Plexiglas boxes (32 cm length × 31 cm width × 35 cm height) encased in soundattenuating shells. A recessed port 3 cm above the floor and located in the center of one wall was used to deliver sucrose. Two lights (28 V, 100 mA) located 12 cm from the floor on the wall opposite the port provided constant illumination. A light (28 V, 100 mA) located 33 cm above the floor on the wall opposite the port served as the 20 s continuous light cue. A high-frequency "tweeter" speaker (ENV-224BM) located 25 cm from the floor on the wall opposite the port was used to deliver the auditory cues. Footshock was delivered through a grid floor via a constant current aversive stimulator (ENV-414S). A video camera located at the top of the sound-attenuating shell recorded the rat's behavior for offline video analysis.

# Discriminative Conditioning

The three cues signifying reward, fear or safety were a 20 s continuous 3 kHz tone (70 dB), a 20 s pulsing 11 kHz tone (200 ms on, 200 ms off; 70 dB) or a 20 s continuous light (28 V, 100 mA), counterbalanced across subjects, with the caveat that the light cue was reserved for the safety cue in most subjects, 12 out of 14 rats. Training first consisted of five reward sessions (**Figure 1A**; R1–5), in which a 20 s reward cue was paired with 3 s delivery of a 10% sucrose solution (100µL) into a port accessible to the rat (3 s sucrose delivery commenced pseudorandomly between 10 and 20 s after reward cue onset for 25 trials, ITI 90– 130 s). This was followed by a single session of habituation (H) to the future fear cue and safety cue during a session in which reward cue training continued (25 reward trials, ITI 90–130 s). The future fear cue and safety cue were presented separately five times each for 20 s without reinforcement to allow subjects to habituate to their presentation thereby reducing any baseline freezing to these novel cues. Four sessions of discriminative conditioning followed (DC1–4): reward cue training continued

FIGURE 1 | (A) Summary of experimental design. S, surgical implantation of electrodes into the BA bilaterally followed by 10 d surgical recovery. R1–5, reward sessions in which the reward cue was paired with sucrose delivery. H, habituation in which, in addition to the reward cue-sucrose pairings, rats also received unreinforced presentations of the future fear and safety cues. DC1–4, discriminative conditioning in which reward cue-sucrose pairings continued as well as the addition of trials where the fear cue was paired with footshock, the fear cue was paired with the safety cue without footshock, or the safety cue was presented alone without footshock. E1–2, extinction in which the fear and reward cues were presented unreinforced. (B) Locations of each electrode tip from 14 rats. All 111 recorded neurons were in the BA. (C) Mean (±SEM) percentage of time spent freezing during each cue comparing early vs. late DC sessions (DC1 vs. DC3+4). During late DC, *(Continued)*

#### FIGURE 1 | Continued

rats froze significantly more to the fear cue compared to the fear+safety cue, reward cue or safety-alone cue, demonstrating discriminatory fear behavior (\**p* < 0.05). (D) Mean (±SEM) percentage of time spent freezing for each fear cue trial during E1 and E2. Freezing was significantly suppressed compared to the first trial beginning at trial 7 and remained significantly suppressed for the remainder of trials during E1 and E2 (\**p* < 0.05). (E) Summary of fear cue unresponsive and responsive neurons before extinction and during late extinction. Above, neurons were assigned to one of four groups based on their response to the fear cue and fear+safety cue during late DC (DC3+4); i.e., before extinction. A neuron was considered responsive if there was a significant change in firing frequency during the first 200 ms of the cue compared to pre-cue baseline. Below, a summary of the subset of neurons from each of the four groups to switch their response to the fear cue during late extinction (trials 10–20 of E1 and trials 1–5 of E2 in which freezing behavior was significantly lowered). From left to right, before extinction, one group (*n* = 27) showed no response to the fear cue but did show a significant change in firing frequency in response to the fear+safety cue. During late extinction, 11 of these neurons switched to being fear cue responsive. The next group (*n* = 48) showed no response to either the fear or fear+safety cue before extinction. But during late extinction, 21 of these neurons became fear cue responsive. In contrast, the next group (*n* = 19) showed a significant change in firing frequency in response to both the fear and fear+safety cue before extinction and nine of these neurons became fear cue unresponsive during late extinction. The last group (*n* = 17) showed a significant change in firing frequency in response to the fear cue but not the fear+safety cue before extinction. Of these neurons, eight became fear cue unresponsive. (F) Comparison of the number of neurons that were fear cue responsive, irrespective to its responding to the other cues, before extinction (DC3+4) to late extinction. The number of neurons being fear cue responsive increased from 36 before extinction to 50 during late extinction, a 39% increase.

(3 s sucrose delivery commenced 18 s after reward cue onset; 15 trials), along with the additional presentation of the 20 s fear cue followed by a mild 0.5 s footshock at the offset of the fear cue (0.4 mA; four trials). On separate trials this same 20 s fear cue was simultaneously paired with a 20 s safety cue resulting in no footshock (fear+safety cue; 15 trials). Trials in which the 20 s safety cue was presented alone without any footshock were also included (safety-alone cue; 10 trials) to assess if any freezing developed to the safety cue as a result of being paired to the fear cue as well as providing the animal with additional trials that contained a safety cue-no shock contingency. Trials were presented pseudorandomly (ITI 100–140 s). Two sessions of extinction followed (E1–2), in which the fear and reward cues were presented unreinforced (E1: 20 trials each of the fear and reward cues, E2: five trials each of the fear and reward cues; trials were presented pseudorandomly, ITI 90–130 s).

#### Behavioral Analyses

Fear behavior was assessed, offline from videos, by measuring freezing, defined as complete immobility with the exception of respiratory movements, which is an innate defensive behavior (Blanchard and Blanchard, 1969; Fendt and Fanselow, 1999). The total time spent freezing was quantified during the entire 20 s of each cue presentation and expressed as percent time spent freezing. Calculating the percent time spent in the port assessed reward-seeking behavior. Behavioral data were analyzed using one- or two-way repeated measures ANOVA, followed by Tukey's post-hoctest when indicated by significant (p < 0.05) main effects or interactions.

#### Surgery

Rats were anesthetized with isoflurane and stereotaxically implanted bilaterally with fixed eight-electrode arrays (NeuroBiological Laboratories) directed at the BA (relative to bregma: AP: −2.04 to −2.92 mm posterior, ML: 4.1–4.9 mm, DV: 6.6–7.5 mm ventral from brain surface (Paxinos and Watson, 2007) (**Figure 1B**). Rats were allowed 7–10 d to recover in which they had ad libitum access to food and water.

#### In Vivo Single Unit Recordings

Neuronal activity was recorded with commercial hardware and software, including headstage amplifiers and programmable amplifiers, filters (0.4 and 5 KHz), and multichannel spikesorting software (Plexon). Implanted rats were connected to the recording apparatus via a swivel commutator. Discrimination of individual units was performed offline by using principal component analysis of waveform shape. Single cells were identified by constancy of waveform shape, cross-correlograms, and interspike intervals (Janak, 2002). In addition, quantitative J3 and Davies Bouldin validity index (DB) statistics were calculated. High J3 values and low DB values are indicative of good single unit isolation (Davies and Bouldin, 1979; Nicolelis et al., 2003; Herry et al., 2008; Sangha et al., 2013). Stability of units across sessions was assessed by calculating principal component space cylinders using WaveTracker (Plexon). In addition, linear correlation values between time-shifted average waveforms were calculated (Jackson and Fetz, 2007; Herry et al., 2008; Sangha et al., 2013). As a control, the r-values from average waveforms of randomly paired neurons and sessions were computed. Only units deemed stable across sessions using these procedures were included in the analysis.

#### Classification of Neurons

For each neuron, significance of cue-evoked firing rates was determined as previously published (Sangha et al., 2013), using a 10,000-round paired permutation test (Hesterberg et al., 2005) comparing the averaged 20 s pre-cue baseline period to the first 200 ms after cue onset during the last two DC sessions and during late extinction (trials 10–20 of E1 and trials 1– 5 of E2). That is, the 20 s pre-cue baseline firing rates and the 200 ms post-cue firing rates for a given cue were shuffled and redistributed independently 10,000 times. The differences between the baseline and post-cue firing for the single real case and the 10,000 reshuffled cases were used to create a distribution. In accordance with the permutation test, if the actual mean difference was within <2.5% of either tail, it was considered significant. P-values were then adjusted for multiple corrections using the Benjamini-Hochberg correction with a corrected cutoff of 0.05 (Benjamini and Hochberg, 1995). To avoid false positives, neurons that showed a significant cue-evoked inhibition using this permutation test were only included in the final analyses if the baseline firing frequency was >0.05 Hz. Neurons were classified as "fear cue responsive" if there was a significant increase or decrease in firing rate to the fear cue during late DC. These neurons were then segregated based on whether there was also a significant change in firing rate to the fear+safety cue. Neurons that did not show a significant change in firing rate to the fear cue during late DC (i.e., before extinction) were classified as "fear cue unresponsive." A subset of these neurons did however show a significant increase or decrease in firing rate compared to baseline to the fear+safety cue and were analyzed separately. Similarly, neurons were classified as "reward cue responsive" if there was a significant increase or decrease in firing rate to the reward cue during late DC.

#### Histology

Rats were deeply anesthetized with sodium pentobarbital. A 10 s 19µA DC current was passed through each wire to mark each electrode tip. Rats were then perfused with formalin containing 3% potassium ferrocyanide. Sections (50µM) were stained against acetylcholinesterase and only units recorded from electrode wires verified to be in the BA were included in the analyses.

# RESULTS

In a previous study neurons of the BA were tracked over the course of a discriminative conditioning task (Sangha et al., 2013). In this task rats learn to discriminate among fear, safety, and reward cues. In the present study, the same BA neurons were followed into fear and reward cue extinction to assess the plasticity of neurons that were fear cue responsive and fear cue unresponsive before extinction.

Recordings were made during each behavioral training session (**Figure 1A**, see Materials and Methods). A total of 111 single neurons located in the BA from 14 rats (**Figure 1B**) were isolated from recordings made during discriminative conditioning and extinction. Most neurons had low mean firing rates (Median = 0.83 Hz, Max = 20.35 Hz, Min = 0.06 Hz), suggesting the sample was predominantly putative projection neurons (Likhtik et al., 2006).

# Fear Behavior

#### Discriminative Conditioning

The percent time spent freezing during each cue was averaged across early (first DC session, DC1) and late (final two DC sessions, DC3+4) discriminative conditioning (**Figure 1C**). A Two-way repeated-measures ANOVA on percent time spent freezing revealed a significant interaction between phase of training and cue type [F(3, 39) = 8.575, p < 0.001] and a main effect of phase of training [F(1, 13) = 5.118, p < 0.05] and cue type [F(3, 39) = 29.331, p < 0.001]. Freezing to the fear cue was significantly greater than the fear+safety cues, safety-alone cue, and reward cue during late DC (post-hoc Tukey's, p < 0.001 each comparison), demonstrating discriminatory fear behavior by these animals.

#### Fear Extinction

The percent time spent freezing during each fear cue trial of E1 and E2 was averaged across animals (**Figure 1D**). A One-way repeated-measures ANOVA on percent time spent freezing revealed a main effect of trial [F(24, 336) = 6.35, p < 0.0001] and Dunnett's multiple comparisons test showed freezing was significantly lower (p < 0.05) during trial 7 and each subsequent trial compared to the first trial. Thus, freezing was significantly suppressed compared to the first trial beginning at trial 7 and remained significantly suppressed for the remainder of trials during E1 and E2.

# Neural Recordings of Fear and Safety Neurons

In order to compare neuronal responding during discriminatory fear behavior to significant fear suppression during extinction, neuronal responding was analyzed during late DC (DC3+4) and compared to late extinction (**Figure 1E**). Late extinction consisted of the last 10 trials of E1 and the 5 trials of E2; freezing behavior during each of these trials was significantly lower than the beginning of extinction (trial 1 of E1, **Figure 1D**). Z-scores were calculated for each neuron's response to the first 200 ms of each cue (see Materials and Methods) and used to make comparisons among different neuronal populations.

#### Neurons Unresponsive to the Fear Cue Before Extinction

Neurons classified as "fear cue unresponsive" before extinction had no significant change in firing rates to the fear cue compared to baseline during DC3+4 (permutation tests, p > 0.05). These neurons were then segregated based on whether or not they showed significant changes in firing rates to the fear+safety cue compared to baseline during DC3+4 (**Figure 1E**). This was done in an effort to assess if the "safety" neurons become "extinction" neurons. In other words, does one subpopulation preferentially switch to being fear cue responsive?

Before extinction, 27 neurons were fear cue unresponsive but fear+safety cue responsive (**Figures 1E**, **2A**), showing a discriminative response to the fear+safety cue vs. fear cue. This subpopulation showed either an excitatory (n = 15, **Figure 2A** upper) or inhibitory (n = 12, **Figure 2A** lower) response to the fear+safety cue. Five of these fear cue unresponsive neurons developed an excitatory response to the fear cue in late extinction and six developed an inhibitory response (permutation tests, p < 0.05). The remaining 16 neurons remained unresponsive to the fear cue (permutation tests, p > 0.05).

Before extinction, 48 neurons were both fear cue and fear+safety cue unresponsive (**Figures 1E**, **2B**). Of these 48 unresponsive neurons, five developed an excitatory response to the fear cue in late extinction and 16 developed an inhibitory response (permutation tests, p < 0.05). The remaining 27 neurons remained unresponsive to the fear cue (permutation tests, p > 0.05).

Together, of all the neurons that were unresponsive to the fear cue before extinction (n = 76), 43% (32 of 76 neurons) switched to being responsive during late extinction (**Figure 1E**). Contrary to the hypothesis, neurons that responded to the fear+safety cue, but not the fear cue, before extinction did not appear to preferentially switch to being fear cue responsive during late extinction compared to neurons unresponsive to both cues.

each cue. Mean (±SEM) Z-scores are shown to the fear and fear+safety (f+s) cues before extinction and to the fear cue during late extinction. \**p* < 0.05, firing frequency during first 200 ms of cue of a given neuron compared to its pre-cue baseline firing frequency. Significant positive Z-score values indicate an excitatory response and significant negative Z-score values indicate an inhibitory response. Non-significant values indicate unresponsive to the cue. (A) Neurons that were fear cue unresponsive but fear+safety cues responsive *(Continued)*

#### FIGURE 2 | Continued

before extinction. Five of these fear cue unresponsive neurons developed an excitatory response to the fear cue in late extinction and six developed an inhibitory response. (B) Neurons that were both fear cue and fear+safety cue unresponsive before extinction. Of these 48 unresponsive neurons, five developed an excitatory response in late extinction and 16 developed an inhibitory response to the fear cue. The remaining 27 neurons remained unresponsive to the fear cue.

#### Neurons Responsive to the Fear Cue Before Extinction

Neurons classified as "fear cue responsive" before extinction had significant increases or decreases in firing rates to the fear cue compared to baseline during DC3+4 (permutation tests, p < 0.05). These neurons were then segregated based on whether or not they also showed significant changes in firing rates to the fear+safety cues compared to baseline during DC3+4 (**Figure 1E**). This was done to assess if one subpopulation preferentially switched to being fear cue unresponsive.

Before extinction, 19 neurons were both fear cue and fear+safety cue responsive (nondiscriminative; **Figures 1E**, **3A**). This subpopulation showed either an excitatory (n = 6, **Figure 3A** upper) or inhibitory (n = 13, **Figure 3A** lower) response to the fear+safety cue. All neurons showing an excitatory response to both types of cues before extinction maintained their response through late extinction (n = 6; permutation tests, p < 0.05). Nine neurons showing an inhibitory response to both cues before extinction lost their inhibitory response in late extinction (permutation tests, p > 0.05). The remaining four neurons maintained their inhibitory response through late extinction (permutation tests, p < 0.05). That is, within this subpopulation of neurons, all excitation responses were maintained through extinction but the majority of inhibition responses were lost through extinction.

Before extinction, 17 neurons were fear cue responsive but fear+safety cue unresponsive (**Figures 1E**, **3B**), showing a discriminative response to the fear cue vs. fear+safety cue. Only 1 neuron showed an excitatory response to the fear cue (**Figure 3B**) while the remaining 16 neurons showed an inhibitory response to the fear cue. Eight neurons that showed significant inhibition to the fear cue before extinction lost the inhibitory response in late extinction (permutation tests, p > 0.05) and one neuron switched its inhibitory response to the fear cue before extinction to an excitatory response to the fear cue in late extinction. The remaining one excitatory response and seven inhibitory responses were maintained through late extinction (permutation tests, p < 0.05).

Together, of all the neurons that were responsive to the fear cue before extinction (n = 36), 47% (17 of 36 neurons) switched to being unresponsive during late extinction (**Figure 1E**).

In summary, extinction induced a gain in response to the fear cue in 43% of fear cue unresponsive neurons and a loss in response to the fear cue in 47% of fear cue responsive neurons. The number of fear cue responsive neurons before extinction was also compared to late extinction (**Figure 1F**) to determine if there was an overall increase or decrease in the absolute number of neurons being fear cue responsive as a result of extinction. Before

#### FIGURE 3 | Continued

extinction maintained their response through late extinction (*n* = 6). Nine neurons showing an inhibitory response to both cues before extinction lost their inhibitory response in late extinction. The remaining 4 neurons maintained their inhibitory response through late extinction. (B) Neurons that were fear cue responsive but fear+safety cue unresponsive before extinction. Eight neurons that showed significant inhibition to the fear cue before extinction lost the inhibitory response in late extinction; one neuron switched its inhibitory response to the fear cue before extinction to an excitatory response to the fear cue in late extinction. The remaining one excitatory response and seven inhibitory response neurons maintained their responses through late extinction.

extinction, 75 neurons were fear cue unresponsive and 36 were fear cue responsive (**Figure 1F**, upper). During late extinction, 61 neurons were fear cue unresponsive and 50 were fear cue responsive (**Figure 1F**, lower). Thus, there was a 39% increase in the number of fear cue responsive neurons as a result of extinction. However, a Fisher's exact test revealed this increase was not significant (p = 0.073).

# Reward Behavior and Neural Recordings of Reward Responsive Neurons

Since reward cue extinction occurred concurrently to fear cue extinction, neuronal responding to the reward cue before extinction (DC3+4) and during late extinction was also assessed.

#### Discriminative Conditioning

The percent time spent in the reward port during each cue was averaged across early (first DC session, DC1) and late (final two DC sessions, DC3+4) discriminative conditioning (**Figure 4A**). A Two-way repeated-measures ANOVA on percent time spent in port revealed a significant main effect of cue type [F(3, 39) = 71.56, p < 0.0001]. Reward seeking during the reward cue was significantly greater than the fear+safety cues, safety-alone cue, and fear cue during both early and late DC (post-hoc Tukey's, p < 0.001 each comparison), demonstrating discriminatory reward seeking behavior by these animals.

#### Reward Extinction

The percent time spent in the reward port during each reward cue trial of E1 and E2 was averaged across animals (**Figure 4B**). A One-way repeated-measures ANOVA on percent time spent in port revealed a main effect of trial [F(24, 336) = 2.858, p < 0.05] and Dunnett's multiple comparisons test showed reward seeking was significantly lower (p < 0.05) during trial 2 and each subsequent trial compared to the first trial with the exception of trials #8, 9, and 10 of E1 and trials #3 and 5 of E2. Thus, compared to the first trial of E1, reward seeking was successfully extinguished by the end of E1 and maintained into E2.

#### Neural Recording

Similar to the analyses completed for the fear responsive neurons, the number of reward cue responsive neurons before extinction was compared to late extinction (**Figure 4C**) to determine if there was an overall increase or decrease in the absolute number of neurons being reward cue responsive as a result of extinction. Before extinction, 62 neurons were reward cue

**15**

unlearning. Fear conditioning also induces synchronization at theta frequencies within the amygdala-hippocampal-prefrontal cortex (PFC) network (Sangha et al., 2009; Lesting et al., 2011). After extinction the synchronization between the amygdala and hippocampus is lost but theta synchronization is maintained between the amygdala and PFC, and between the hippocampus and PFC. A similar effect has been reported in the PFC-BA circuit in which fear extinction decreases excitatory transmission from PFC to BA while maintaining inhibitory transmission (Cho et al., 2013). These data demonstrate both reversal and maintenance of learning-induced network activity occurring in parallel during extinction.

This suggests that both new learning and unlearning mechanisms may occur in parallel during extinction. Both processes are active processes. During extinction, a new association regarding the CS is learned; i.e., a CS-no US association. And, similarly to learning the original association, long-term retention of extinction training requires both RNA and

cue trial during E1 and E2. Reward seeking was significantly suppressed (\**p* < 0.05) compared to the first trial beginning at trial 2 and remained significantly suppressed for the remainder of trials during E1 and E2 with the exception of trials #8–10 of E1 and trials #3 and 5 of E2. (C) Comparison of the number of neurons that were reward cue responsive before extinction (DC3+4) to late extinction. The number of neurons being reward cue responsive decreased from 62 before extinction to 49 during late extinction.

responsive and 49 were reward cue unresponsive. During late extinction, 49 neurons were fear cue responsive and 62 were fear cue unresponsive. This decrease in the number of reward cue responsive neurons as a result of extinction was not significant (Fisher's exact test, p > 0.05).

#### DISCUSSION

This study examined how neurons classified as discriminative, nondiscriminative or unresponsive during discriminative conditioning (DC), based on their responses to the fear and fear+safety cues, responded to the fear cue during extinction training and recall as fear behavior decreased. The hypothesis tested was that there is a bias for the neurons that were safety cue responsive during DC to become responsive to the fear cue as extinction progresses.

Although 41% of "safety" neurons became fear cue responsive as a result of extinction, the data revealed that there was no bias for these neurons to become preferentially responsive during fear extinction compared to the other identified subgroups. In addition to the plasticity seen in the "safety" neurons, 44% of neurons unresponsive to either the fear cue or fear+safety cue during DC became fear cue responsive during extinction. Together these emergent responses to the fear cue as a result of extinction support the hypothesis that new learning underlies extinction. The overall increase in fear cue responsive neurons in response to extinction also implies that these changes in neuronal responding during extinction are not a result of simple exposure to the sensory stimuli. If the shift were a result of repeated sensory exposures, one would expect the neurons across all groups to show decreased responding to sensory stimuli after multiple exposures as a result of sensory habituation. In contrast, 47% of neurons responsive to the fear cue during DC, regardless of its response to the fear+safety cue, became unresponsive to the fear cue during extinction. These findings are consistent with a suppression of neural responding mediated by inhibitory learning, or, potentially, by direct unlearning. Together, the data support extinction as an active process involving both gains and losses of responses to the fear cue.

The prevalent view in the extinction field is that extinction

is an active process, not a passive one (reviewed in Myers and Davis, 2002, 2007). There is ample evidence that extinction does not erase fear memories. In particular, it has been demonstrated by others (Repa et al., 2001; Herry et al., 2008; An et al., 2012), and here in this study (**Figure 3A**, upper), that amygdala neurons maintain increased responsiveness to the CS, even after extinction. However, there is also evidence that extinction reverses the changes induced by fear learning. For example, fear conditioning-induced potentiation is reversed with extinction in both the thalamo-lateral amygdala and cortico-lateral amygdala pathways (Kim et al., 2007; Hong et al., 2009). The data in the current study are in agreement with both views. There was both a gain of response to the fear cue (**Figures 2A,B**), which supports extinction as new learning, and a loss of response to the fear cue (**Figure 3A**, lower and **Figure 3B**), which may be due to

protein synthesis across several learning paradigms and species (reviewed in Lattal et al., 2006). But, since extinction also involves reactivation of the original memory, the integrity of the original memory is vulnerable to disruption through reconsolidation mechanisms. When the original CS-US association is reactivated during extinction, it can be updated via reconsolidation mechanisms resulting in a weakening/reversal of the memory. Extinction and reconsolidation have been demonstrated to occur in parallel in the basolateral amygdala complex during reactivation of a fear memory that is no longer reinforced with shock (Duvarci et al., 2006), supporting a view that both new learning and unlearning mechanisms are at play during extinction. This view is also consistent with reports that briefly reactivating a fear memory before employing fear extinction training results in persistent attenuation of fear in both rats (Monfils et al., 2009) and humans (Schiller et al., 2010). In this case, the brief reactivation of the fear memory may induce unlearning via reconsolidation mechanisms and the extinction training results in the learning of a new CS-no US association.

The unlearning phenomena may be caused by reversal of learning-induced changes at the synapse and within the network, or it may be caused by suppression of neural responding mediated by increased inhibition. Several neurons reported here had decreased firing rates in response to the fear cue during extinction. It is not clear what the source of cue-evoked inhibition, nor its downstream effects, might be. However, it has been shown that the balance between excitation and inhibition in the PFC-BA pathway is shifted toward inhibition after extinction (Cho et al., 2013), suggesting that the upstream source for the inhibitions seen in the data presented here may be the PFC. This would be consistent with the requirement of the infralimbic region of the prefrontal cortex to discriminate between the fear and fear+safety cues in this task (Sangha et al., 2014), and to

REFERENCES


successfully recall fear extinction (Quirk et al., 2000; Laurent and Westbrook, 2009; Chang and Maren, 2010; Fontanez-Nuin et al., 2011; Sierra-Mercado et al., 2011; Santini et al., 2012; Sangha et al., 2014, but see Do Monte et al., 2015).

In summary, the data implicate multiple levels of plasticity in response to fear extinction that most likely interact with multiple microcircuits within the BA. It also indicates that there may be a general remapping of these neuronal microcircuits within the BA in response to extinction. Together it suggests the final output of the integrated BA circuit to influence fear behavior is a balance of excitation and inhibition, and perhaps reversal of learning-induced changes. Further exploration of the intricacies of upregulating or downregulating these BA microcircuits on downstream targets and their effects on fear behavior will lead to greater understanding of the mechanisms contributing to successful fear inhibition which is compromised in individuals suffering from PTSD and similar disorders.

#### AUTHOR CONTRIBUTIONS

SS: experimental design, performed research, analyzed data, wrote manuscript.

#### ACKNOWLEDGMENTS

This work was supported by funding from the State of California for Medical Research on Alcohol and Substance Abuse through the University of California at San Francisco and NIH grant AA014925 to Dr. P. H. Janak. The author greatly appreciates Dr. P. H. Janak for discussion and comments on the manuscript. The author also thanks R. Reese and I. Grossrubatscher for technical assistance, Dr. T. M. Gill for programming and recording guidance and Dr. J. Z. Chadick for programming assistance.

signals: a mini-symposium review. J. Neurosci. 32, 14118–14124. doi: 10.1523/JNEUROSCI.3340-12.2012


and extinction of Pavlovian fear. Eur. J. Neurosci. 26, 3631–3644. doi: 10.1111/j.1460-9568.2007.05970.x


macaque monkeys. Proc. Natl. Acad. Sci. U.S.A. 100, 11041–11046. doi: 10.1073/pnas.1934665100


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer Stefan Herlitze and handling Editor Denise Manahan-Vaughan declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

Copyright © 2015 Sangha. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Memory System Engaged During Acquisition Determines the Effectiveness of Different Extinction Protocols

#### Jarid Goodman and Mark G. Packard\*

Department of Psychology, Texas A&M Institute for Neuroscience, Texas A&M University, College Station, TX, USA

Previous research indicates that extinction of rodent maze behavior may occur without explicit performance of the previously acquired response. In latent extinction, confining an animal to a previously rewarded goal location without reinforcement is typically sufficient to produce extinction of maze learning. However, previous studies have not determined whether latent extinction may be successfully employed to extinguish all types of memory acquired in the maze, or whether only specific types of memory may be vulnerable to latent extinction. The present study examined whether latent extinction may be effective across two plus-maze tasks that depend on anatomically distinct neural systems. Adult male Long-Evans rats were trained in a hippocampus-dependent place learning task (Experiment 1), in which animals were trained to approach a consistent spatial location for food reward. A separate group of rats were trained in a dorsolateral striatum-dependent response learning task (Experiment 2), in which animals were trained to make a consistent egocentric body-turn response for food reward. Following training, animals received response extinction or latent extinction. For response extinction, animals were given the opportunity to execute the original running approach response toward the empty food cup. For latent extinction, animals were confined to the original goal locations with the empty food cup, thus preventing them from making the original running approach response. Results indicate that, relative to no extinction, latent extinction was effective at extinguishing memory in the place learning task, but remained ineffective in the response learning task. In contrast, typical response extinction remained very effective at extinguishing memory in both place and response learning tasks. The present findings confirm that extinction of maze learning may occur with or without overt performance of the previously acquired response, but that the effectiveness of latent extinction may depend on the type of memory being extinguished. The findings suggest that behavioral treatments modeled after response extinction protocols may be especially useful in alleviating human psychopathologies involving striatum-dependent memory processes (e.g., drug addiction and relapse).

Keywords: extinction, memory, hippocampus, striatum, memory systems

#### Edited by:

Oliver T. Wolf, Ruhr University Bochum, Germany

#### Reviewed by:

Armin Zlomuzica, Ruhr University Bochum, Germany Michael E. Ragozzino, University of Illinois at Chicago, USA

> \*Correspondence: Mark G. Packard markpackard@tamu.edu

Received: 18 September 2015 Accepted: 04 November 2015 Published: 24 November 2015

#### Citation:

Goodman J and Packard MG (2015) The Memory System Engaged During Acquisition Determines the Effectiveness of Different Extinction Protocols. Front. Behav. Neurosci. 9:314. doi: 10.3389/fnbeh.2015.00314

# INTRODUCTION

Mammalian memory is not a unitary phenomenon, but rather it transpires through distinct systems. These ''memory systems'' differ in terms of not only the type(s) of memory they mediate, but also the brain regions that subserve them. Although a variety of memory systems have been dissociated in the mammalian brain (Squire, 2004; White et al., 2013), significant attention has been devoted to anatomical dissociations between a spatial/cognitive memory system mediated by the hippocampus and a stimulus-response (S-R)/habit system mediated by the dorsolateral striatum (DLS; Packard et al., 1989; Packard and McGaugh, 1992, 1996; McDonald and White, 1993; Packard and Teather, 1997; Chang and Gold, 2003; Iaria et al., 2003; Compton, 2004).

Research from our laboratory indicates that multiple memory systems may not only be implicated in the initial acquisition of a task, but also in its extinction (Gabriele and Packard, 2006). Extinction constitutes a new, dissociable type of learning that occurs when a subject is placed in the original learning situation but with the reinforcer—or the stimulus event that motivated initial learning—removed. Extinction is deemed to have occurred when the behavioral response or responses that indicated initial learning decrease. Learned behavior in the straight alley maze, a maze in which rats learn to traverse a runway for food reward located at the opposite end of the maze, may be extinguished using two distinct protocols. In a typical ''response extinction'' protocol, rats are placed in the same starting position as during training, but with the food reward at the opposite end of the maze removed. Thus, during response extinction trials, animals can execute the running approach response, only now this response leads to an empty food well. In ''latent extinction,'' rats are confined to the original goal location with the empty food well. Thus, during latent extinction, animals cannot execute the running approach response. Historically the effectiveness of latent extinction figured prominently in learning theory, because it demonstrated that—in contrast to the Hullian S-R view of extinction (Hull, 1943, 1952)—a subject does not need to make the previously acquired response for extinction to occur (Seward and Levy, 1949; Deese, 1951; Moltz, 1955; Denny and Ratner, 1959; Dyal, 1962; Clifford, 1964).

Although the behavior of the rat is ostensibly similar following both extinction protocols, investigators have suggested that response and latent extinction might be achieved through distinct learning mechanisms. The effectiveness of typical response extinction is easily explained through classical S-R models of extinction learning, whereas latent extinction has summoned heated debates between proponents of expectancy theory and proponents of a neo-Hullian view involving the fractional anticipatory approach response (Moltz, 1957; Deese and Hulse, 1967). Although the precise mechanisms underlying latent extinction have yet to be completely elucidated, evidence from our laboratory indicates that latent extinction indeed depends on a dissociable neural system. In the straightalley maze inactivation of the hippocampus, but not the DLS, impairs latent extinction (Gabriele and Packard, 2006; Gabriele, 2008). In contrast, inactivation of the DLS, but not the hippocampus, impairs response extinction (Gabriele and Packard, 2006; Gabriele, 2008). A corollary to the contention that these extinction protocols depend on operatively and anatomically distinct learning systems is that response and latent extinction may not be equally effective across all learning situations. For instance, if a critical feature needed for latent extinction mechanisms to occur is absent from the learning situation, then it is reasonable to hypothesize that latent extinction would not be effective, whereas response extinction could still work.

One potential limitation to examining learning and memory mechanisms using the straight-alley maze is that we do not know what type of memory is being encoded during initial task acquisition. Initial learning in the straight alley maze may involve acquisition of at least two distinct types of memory: (1) a habit-like running approach response to the opposite end of the maze and/or (2) the spatial location of the food reward, which in turn triggers a goal-directed running approach to the rewarded location at the opposite end of the maze. Consequently, when using the straight alley maze to examine extinction mechanisms, the type of memory being extinguished remains unknown. Moreover, studies using the straight alley maze cannot determine whether latent extinction is effective at extinguishing all types of memory or whether latent extinction may only be effective for certain types of memory. Considering that latent extinction may partially operate by producing a new inhibitory spatial memory (see Gabriele and Packard, 2006), it is possible that latent extinction may only be effective in tasks whereby the spatial location of the goal is an integral part of the to-be-extinguished memory, such as in spatial memory tasks. In contrast, latent extinction may not be effective in tasks whereby the spatial location of the goal is irrelevant, such as in S-R/habit memory tasks.

To examine whether only certain types of memory may be vulnerable to latent extinction, the present study utilized two distinct versions of the plus-maze. In a ''place learning'' version of the plus-maze dependent on the hippocampus (Schroeder et al., 2002; Compton, 2004), rats were reinforced to approach a consistent spatial location. In a ''response learning'' version of plus-maze dependent on the DLS (Chang and Gold, 2004; Palencia and Ragozzino, 2005; Asem and Holland, 2015), rats were reinforced to make a consistent egocentric body-turn at the maze choice point. Thus, these place and response tasks tap into dissociable neural systems, a hippocampus-dependent spatial/cognitive memory system and a DLS-dependent S-R/habit memory system, respectively. Following initial learning in these tasks, animals were given response extinction, latent extinction, or no extinction. It was hypothesized that latent extinction would be selectively effective at extinguishing memory in the place learning task, but not the response learning task. Moreover, we hypothesized that typical response extinction would be effective at extinguishing memory in both place and response learning tasks.

# MATERIALS AND METHODS

#### Subjects

The subjects were 46 male Long-Evans rats approximately 90 days old and weighing 375–425 g upon arrival. Animals were subsequently food-restricted and maintained at 85% of the their ad lib weight throughout all behavioral procedures. Water was provided ad libitum. Animals were housed individually in a temperature-controlled vivarium with a 12 h light-dark cycle (lights on at 7 AM), and all behavioral procedures were conducted during the light phase of this cycle. Age, weight, and housing conditions did not differ between animals in Experiments 1 and 2. Animal use in this study was carried out in accordance with the ethical guidelines of the Institutional Animal Care and Use Committee (IACUC) at Texas A&M University. The protocol was approved by IACUC.

# Apparatus

An eight arm radial maze was modified by removing four of the original arms to create a plus-maze configuration consisting of north, south, east, and west arms. The arms of the cross maze measured 60 × 9 cm, and the center platform of the maze connecting the four arms measured 40 cm in diameter. At the end of each arm was a recessed food well. A clear Plexiglas crossshaped structure was placed in the center of the cross maze, serving as the intersection of the four arms. A separate Plexiglas divider was used to block off the arm opposite to the start arm for each trial, creating a T-maze configuration that could be adjusted between trials. The maze was situated in a room with multiple extra maze cues, including posters, a door, a cabinet, and a table.

## Behavioral Procedures

#### Maze Habituation

Before maze training, animals in Experiments 1 and 2 were given 2 days of habituation to the maze. For each day of habituation, a rat was placed on the maze apparatus (from the north arm on day 1 and from the south arm on day 2) and was given 5 min to explore the maze. No food was located on the maze at this time. Immediately after the 5 min, each rat was removed from the maze and placed in a holding container with three Froot Loops cereal pieces (Kellog's). Rats were monitored to confirm consumption of the Froot Loops.

#### Maze Training

Maze training began 24 h following the last day of habituation and lasted 8 days. For the first 2 days of training, animals were given six trials per day, and for the remainder of training animals were given 15 trials per day. The maze was rotated 90º after every two trials to discourage the use of intramaze cues. A wideangle digital camera was fixed over the maze and attached to a computer monitor (only visible to the experimenter) allowing for a clear aerial view of arm entries, and a stopwatch was used to record latencies during task performance.

In Experiment 1, animals (N = 21) received training for 8 days in a place learning version of the plus-maze task whereby animals were reinforced to approach a consistent spatial location. At the start of each training trial, the animal was placed on the north or south arm facing the outside of the maze (the start arm sequence was counterbalanced across training), and the food reward (1/2 Froot Loop) was always located in the recessed food well of the east arm. This place learning protocol presumably compelled rats to acquire a cognitive map of the learning environment that enabled them to guide behavior from different starting positions to the correct spatial location. Extensive evidence indicates that spatial learning in the plus-maze critically involves hippocampal function (Packard and McGaugh, 1996; Packard, 1999; Schroeder et al., 2002; Colombo et al., 2003; Compton, 2004; Jacobson et al., 2012).

In Experiment 2, animals (N = 25) received training in a response learning version of the plus-maze task whereby animals were reinforced to make a consistent egocentric bodyturn response at the maze choice point (Leong et al., 2012, 2015; Goodman and Packard, 2014; Wingard et al., 2015). Animals were released from north and south starting positions (counterbalanced) throughout training. When animals began in the north arm, the food reward (1/2 Froot Loop) was located in the recessed food well of the east arm. When animals began in the south arm, the food reward was located in the west arm. Thus, regardless of the starting position, animals were reinforced to make a left body-turn response at the choice point to receive food reward. Learning in this task constitutes an exemplar of egocentric/S-R learning mediated by the DLS (Packard and McGaugh, 1996; Chang and Gold, 2004; Palencia and Ragozzino, 2005; Asem and Holland, 2015; for reviews, see Packard, 2009; Goodman and Packard, in press).

For each training trial in Experiments 1 and 2, if the animal made an initial full-body entry into the correct arm (i.e., the arm containing the food), the trial was scored as correct. If the animal made an initial full body entry into the incorrect arm, the trial was scored as incorrect. A trial ended once the animal found the food or after 120 s had elapsed. When finding the food, the animal was allowed to finish eating before being removed from the maze and placed in an opaque holding container for a 30 s intertrial interval (ITI). The percentage of correct trials and the latency to reach the correct food well were used as measures of acquisition.

#### Extinction

Extinction was conducted 24 h after the last day of maze training and lasted 3 days. No food was located in the maze throughout extinction training. The maze was rotated 90º after every two trials to prevent the use of intramaze cues.

In Experiment 1, rats that were previously given place learning were subsequently assigned to response extinction (n = 7), latent extinction (n = 7), or ''no extinction'' control (n = 7) groups. Groups were matched on average latency and percent correct responses during the last 3 days of acquisition. Response extinction was conducted over 3 days (10 trials per day). For each trial of response extinction, animals were started from the north or south arm and were given the opportunity to run to the previously correct food well. An animal was removed from the maze after reaching the previously correct food well or after 120 s had elapsed. For each trial, if the animal made an initial full-body entry into the previously correct arm and ran directly to the food well, the trial was identified as ''perseverative.'' A trial was not considered perseverative if the animal at any point made an entry into the incorrect arm or failed to enter either the correct or incorrect arm within 120 s. After each trial the animal was removed from the maze and placed in an opaque holding container for a 30 s ITI. The behavioral procedure for latent extinction was adapted from previous work from our laboratory indicating the effectiveness of latent extinction in the straight alley maze (Gabriele and Packard, 2006, 2007; Gabriele et al., 2009). For each trial of latent extinction, an animal was confined to the previously correct goal arm (i.e., the east arm for the place learning task) for 60 s using a Plexiglas shield secured 20 cm from the end of the maze arm. After each trial, the animal was placed in an opaque holding container for a 30 s ITI. For the ''no extinction'' control group, animals were not placed in the maze for the 3 extinction days, but rather remained in their holding containers for the duration of an extinction session, i.e., while animals in the latent and response extinction groups were receiving extinction training.

In Experiment 2, animals that previously received response learning were subsequently assigned to response extinction (n = 6), limited latent extinction (n = 6), extended latent extinction (n = 6), or ''no extinction'' control (n = 7) groups. Groups were matched on average latency and percent correct responses during the last 3 days of acquisition. The behavioral procedures for response extinction and no extinction control groups were identical to that described for Experiment 1. For limited and extended latent extinction (conducted over 3 days), animals were confined to the east or west goal arm for 60 s for each trial with the sequence of goal arm confinements mimicking the counterbalanced sequence of food locations throughout initial response learning. For each day of limited latent extinction, animals received 10 trials (five trials on each arm). The parameters for limited latent extinction were chosen based on previous evidence indicating that 10 latent extinction trials per day produced extinction in the straight alley (Gabriele and Packard, 2006). However, given that latent extinction trials had to be divided between east and west goal arms, this only permitted five trials on each arm per day. In order to allow for 10 trials on each arm, an additional group was given extended latent extinction, in which animals received 20 trials (10 trials on each arm) per day.

#### Extinction Probes

Twenty four hours following the last day of extinction, all animals in Experiments 1 and 2 were given four probe trials. No food was located in the maze for the extinction probe trials. For each probe trial, an animal was released from the north or south arm (start arm sequence: SNNS), and after reaching the previously correct food well or after 120 s had elapsed, animals were removed from the maze and placed in an opaque holding container for a 30 s ITI. The maze was rotated 90º after every two trials. Latency to reach the previously correct food well and the number of perseverative trials (see above) were recorded and used as measures of extinction. The experimenter conducting the probe trials and scoring the animals was blind to the experimental conditions.

# RESULTS

# Experiment 1

#### Initial Acquisition

Initial acquisition of the place learning task is depicted in **Figure 1**. A two-way repeated measures 3 × 8 ANOVA (Group × Day) computed on percentage of correct turning responses over the course of training (**Figure 1A**) indicated a significant main effect of Day (F(7,126) = 22.22, p < 0.001), but no effect of Group (F(2,18) = 0.15, p = 0.860) and no Group × Day interaction (F(14,126) = 1.51, p = 0.118). Likewise, a 3 × 8 ANOVA (Group × Day) computed on latency (**Figure 1B**) indicated a significant effect of Day (F(7,126) = 52.41, p < 0.001), but no effect of Group (F(2,18) = 0.00, p = 1.00) and no Group × Day interaction (F(14,126) = 1.47, p = 0.131). Together, these results indicate that all groups acquired the task about equally over the course of training, and any subsequent differences between groups during extinction may not be readily attributed to differing rates of initial task acquisition.

#### Response Extinction

**Figure 2** depicts learning rates over the course of extinction training for animals in the ''response extinction'' group. Tests of within-subjects contrasts computed on number of perseverative trials (**Figure 2A**) revealed a significant linear effect of Day (F(1,6) = 39.06, p = 0.001), indicating a decrease in number of perseverative trials during response extinction training. In addition, within-subjects contrasts computed on latency for extinction training days 1–3 (**Figure 2B**) also revealed a linear effect of Day (F(1,6) = 113.56, p < 0.001), indicating that latency increased over the course of response extinction training.

#### Extinction Probes

The results from the extinction probe trials are depicted in **Figure 3**. To assess the effectiveness of the different types of extinction training for each group, comparisons were made between the probe day and the last day of initial acquisition. The first four trials (vs. the last four trials) of the last acquisition day were selected for this comparison based on the observation that during initial acquisition, animals were typically slower and more likely to make errors for the first few trials of each training day vs. the final training trials of the previous day (see **Figures 1C,D**). Therefore, it was reasonable to expect that the extinction probe trials would also have higher latencies and more errors than the terminal trials of the last acquisition day, regardless of whether an extinction protocol was effective. Thus, for a more accurate measurement of the effectiveness of each extinction protocol, we compared the extinction probe trials with the first four trials of the final acquisition day.

A two-way repeated measures 3 × 2 ANOVA (Group × Day) was computed for number of perseverative trials on the last acquisition day (i.e., training day 8; first four trials) and the extinction probe day (**Figure 3A**). Results indicated no significant main effect of Group (F(2,18) = 1.79, p = 0.195), but there was a significant effect of Day (F(1,18) = 10.89, p = 0.004) and a significant Group × Day interaction (F(2,18) = 5.37, p = 0.015). Multiple pairwise comparisons using Fisher's LSD test indicated that there were no significant differences in number of perseverative trials between groups on the last acquisition day. This is consistent with data presented above indicating that the groups did not differ during initial task acquisition. For animals in the latent extinction group, Fisher's LSD test indicated that there was a significant decrease in the number of perseverative trials from the last acquisition day (M = 3.57) to the probe day (M = 2.43), p = 0.007. In addition, the response extinction group showed a significant decrease in number of perseverative trials between the last acquisition day (M = 3.29) and the probe day (M = 2.00), p = 0.003. Animals given no extinction did not show a significant change in number of perseverative trials from the last acquisition day (M = 3.14) to the probe day (M = 3.43), p = 0.456. On the extinction probe day, Fisher's LSD test indicated that the latent extinction group (M = 2.42) displayed a significantly

lower number of perseverative trials than animals in the no extinction control group (M = 3.42), p = 0.026. Similarly, number of perseverative trials during probe day for the response extinction group (M = 2.00) was also significantly lower than perseverative trials for the no extinction group, p = 0.002. In contrast, perseverative trials for the latent extinction group and

FIGURE 3 | Extinction probe trials in the hippocampus-dependent place learning task. (A) There were no between-group differences in perseveration during the first few trials of the last training day (i.e., training day 8). Response and latent extinction groups, but not the "no extinction" group, displayed a decrease in number of perseverative trials from the last acquisition day to the probe day. On the probe day, the latent and response extinction groups displayed lower perseveration than the no extinction group, but the latent and response extinction groups did not differ from each other in perseveration. (B) There were no differences in latency between groups on the last training day. Response and latent extinction groups, but not the "no extinction" group, increased latency from the last acquisition day to the probe day. On the probe day, the latent and response extinction groups had higher latency than the no extinction group. Latency was also higher in the latent extinction group vs. the response extinction group on the probe day. Results indicate the effectiveness of latent and response extinction protocols in extinction of hippocampus-dependent place learning.

response extinction group did not differ on the probe day, p = 0.327.

A two-way repeated measures 3 × 2 ANOVA (Group × Day) was computed for latency on the last acquisition day (i.e., training day 8; first four trials) and the extinction probe day (**Figure 3B**). Results indicated a significant main effect of Group (F(2,18) = 5.48, p = 0.014), a significant effect of Day (F(1,18) = 36.84, p < 0.001), and a significant Group × Day interaction (F(2,18) = 17.92, p < 0.001). Multiple pairwise comparisons using Fisher's LSD test indicated that there were no significant differences in latency between groups on the last acquisition day. For animals given latent extinction, there was a significant increase in latency from the last acquisition day (M = 14.77) to the probe day (M = 40.71), p = 0.002. There was also a significant increase in latency between the last acquisition day (M = 11.61) and the probe day (M = 65.00) for animals given response extinction, p < 0.001. Animals given no extinction did not show a significant change in latency from the last acquisition day (M = 17.39) to the probe day (M = 11.61), p = 0.419. On the probe day, Fisher's LSD test indicated that latency for the latent extinction group (M = 40.71) was significantly higher than latency in the no extinction control group (M = 11.61), p = 0.002. In addition, probe day latency for animals in the response extinction group (M = 65.00) was significantly higher than latency in the no extinction control group, p < 0.001. Latency in the response extinction group was also significantly higher than latency in the latent extinction group, p = 0.009.

Taken together, the results of Experiment 1 indicate that following acquisition in a place learning task animals given latent or response extinction displayed higher latency and lower perseveration during the extinction probe trials, relative to animals given no extinction. These results suggest that either a latent or response extinction protocol may be effective at extinguishing hippocampus-dependent place learning in the plus-maze.

## Experiment 2

#### Initial Acquisition

Initial acquisition of the response learning task is depicted in **Figure 4**. A two-way repeated measures 4 × 8 ANOVA (Group × Day) computed on percentage of correct turning responses over the course of training (**Figure 4A**) indicated a significant main effect of Day (F(7,147) = 23.74, p < 0.001), but no effect of Group (F(3,21) = 0.224, p = 0.878) and no Group × Day interaction (F(21,147) = 0.753, p = 0.771). Similarly, a two-way repeated measures 4 × 8 ANOVA (Group × Day) computed on latency (**Figure 4B**) also indicated a significant effect of Day (F(7,147) = 95.52, p < 0.001), no effect of Group (F(3,21) = 0.330, p = 0.800), and no Group × Day interaction (F(21,147) = 0.88, p = 0.620). These results indicate that all groups acquired the task about equally. Therefore, any subsequent differences between groups during extinction may not be readily attributed to differing rates of initial task acquisition.

#### Response Extinction

**Figure 5** depicts learning over the course of extinction training for animals in the ''response extinction'' group. Tests of withinsubjects contrasts computed on number of perseverative trials (**Figure 5A**) for extinction days 1–3 revealed a significant linear effect of Day (F(1,5) = 24.98, p = 0.004), indicating that the number of perseverative trials decreased over the course of response extinction training. In addition, within-subjects contrasts computed on latency (**Figure 5B**) also revealed a significant effect of Day (F(1,5) = 23.90, p = 0.005), indicating that latency increased over the course of response extinction training.

#### Extinction Probes

The results from the extinction probe trials are depicted in **Figure 6**. The rationale for comparing extinction probe performance with the first four trials of the final training day was described in the results for Experiment 1 (see above). A two-way repeated measures 4 × 2 ANOVA (Group × Day) was computed for number of perseverative trials on the last acquisition day (i.e., training day 8; first four trials) and the extinction probe day (**Figure 6A**). Results indicated a significant main effect of Group (F(3,21) = 3.73, p = 0.027), a significant effect of Day (F(1,21) = 7.66, p = 0.012), and a significant Group × Day interaction (F(3,21) = 4.48, p = 0.014). Multiple pairwise comparisons using Fisher's LSD test indicated that there were no significant differences in number of perseverative trials between groups on the last acquisition day. This is consistent with data presented above indicating that the groups did not differ during initial task acquisition. For animals in the ''response extinction'' group, Fisher's LSD test indicated that there was a significant decrease in the number of perseverative trials from the last acquisition day (M = 3.50) to the probe day (M = 1.33), p < 0.001. No other groups showed a significant change in number of perseverative trials between the last acquisition day and the probe day. On the extinction probe day, Fisher's LSD test indicated that the response extinction group (M = 1.33) displayed a significantly lower number of perseverative trials than animals in the no extinction control group (M = 3.23), p < 0.001. Number of perseverative trials for the limited latent extinction group (M = 3.00) did not differ from the no extinction group, p = 0.642. In addition, perseverative trials for the extended latent extinction group (M = 3.17) did not differ from the no extinction group, p = 0.790. There was a significantly lower number of perseverative trials in the response extinction group vs. the limited latent extinction group, p < 0.001, and the extended latent extinction group, p < 0.001.

A two-way repeated measures 4 × 2 ANOVA (Group × Day) was computed for latency on the last acquisition day (i.e., training day 8; first four trials) and the extinction probe day (**Figure 6B**). Results indicated a significant main effect of Group (F(3,21) = 22.00, p < 0.001), a significant effect of Day (F(1,21) = 183.9, p < 0.001), and a significant Group × Day interaction (F(3,21) = 81.57, p < 0.001). Multiple pairwise comparisons using Fisher's LSD test indicated that there were no significant differences in latency between groups on the last acquisition day. Comparing the mean latencies between the last acquisition day and the probe

day for each group indicated a significant increase in latency between the 2 days for all groups: no extinction (last acquisition day M = 8.46, probe day M = 16.32, p = 0.049), limited latent extinction (last acquisition day M = 7.58, probe day M = 16.67, p = 0.037), extended latent extinction (last acquisition day M = 10.29, probe day M = 19.96, p = 0.027), and response extinction (last acquisition day M = 11.00, probe day M = 92.92, p < 0.001). On the probe day, Fisher's LSD test indicated that latency for the response extinction group (M = 92.92) was significantly higher than latency in the no extinction control group (M = 16.32), p < 0.001. Latency did not differ significantly between limited latent extinction (M = 16.67) and the no extinction control group, p = 0.957, and latency also did not differ between extended latent extinction (M = 19.96) and the no extinction control group, p = 0.567. Response extinction latency was significantly higher than latency in limited latent extinction, p < 0.001, and extended latent extinction groups, p < 0.001.

Taken together, the results of Experiment 2 indicate that following acquisition in the response learning task, animals given response extinction displayed higher latency and lower perseveration during the extinction probe trials, relative to animals given no extinction. In contrast, animals given limited or extended latent extinction protocols did not differ significantly in latency or perseveration from animals given no extinction. The results suggest that in contrast to typical response extinction, latent extinction protocols may not be effective at extinguishing memory in a DLS-dependent response learning task.

### DISCUSSION

The present findings indicate a dissociation regarding the effectiveness of latent extinction across two learning and memory tasks. Latent extinction was effective at extinguishing memory in a hippocampus-dependent place learning task, but not in a DLSdependent response learning task. In contrast, typical ''response extinction'' was effective in both place and response learning tasks.

In Experiment 1, following acquisition of the place learning task, animals given latent or response extinction displayed greater latency and fewer perseverative trials than animals given no extinction. Interestingly, animals given response extinction displayed higher latencies than animals given latent extinction, suggesting response extinction may have had greater efficacy than latent extinction in the place learning task. However, there was no difference in number of perseverative trials between latent and response extinction groups. It is possible that, relative to latent extinction, response extinction was more efficient at slowing the running approach response, but not necessarily more effective at extinguishing the location of food reward.

In Experiment 2, following acquisition of a response learning task, animals given response extinction displayed higher latencies and fewer perseverative trials than animals given no extinction, indicating the effectiveness of response extinction in this task. In contrast, animals given limited or extended latent extinction did not differ in latency or perseveration from animals given no extinction, suggesting that these latent extinction protocols were not effective at producing extinction in the response learning task. Even though latencies in the limited and extended latent extinction groups showed a slight increase from the last acquisition day to the probe day, a comparable increase was also observed for animals in the ''no extinction'' control group. Therefore, this increase in latency from the last acquisition day to the probe day may not be readily attributed to the latent extinction protocols. In addition, latent extinction and no extinction control groups did not show a decrease in number of perseverative trials across the 2 days.

A finding secondary to the differential effects of the extinction protocols, but of considerable relevance to classical learning theories, pertains to the initial acquisition curves in the place and response learning tasks. During most days of initial acquisition, the first few trials were accompanied with greater latencies and more errors than the last few trials of the previous training day (see **Figures 1C,D**, **4C,D**). However, this rise in latency and inaccuracy on the first few training trials of a given day became progressively less pronounced on subsequent training days. The present finding is consistent with early principles in learning theory pertaining to decay theory (e.g., Ebbinghaus, 1913; Thorndike, 1913). Thorndike (1913) proposed that following acquisition, a memory begins to fade as a function of its disuse over time (i.e., decay). However, some traces of the memory survive this decay, and thus relearning not only proves faster than initial learning, but also results in a stronger memory that is less sensitive to memory decay. Although the precise mechanisms of memory decay have

FIGURE 6 | Extinction probe trials in the DLS-dependent response learning task. (A) There were no differences between groups in perseveration during the first few trials of the last acquisition day (i.e., training day 8). Only the response extinction group displayed a decrease in number of perseverative trials from the last acquisition day to the probe day. On the probe day, the response extinction group displayed lower perseveration than all other groups. The latent extinction groups (limited and extended) did not differ in perseveration from the no extinction control group on the probe day. (B) There were no between-group differences in latency on the last training day. All groups increased latency from the last acquisition day to the probe day. On the probe day, the response extinction group had higher latency than all other groups. Latency was not higher in the latent extinction groups (limited and extended), relative to the no extinction group. Results indicate that response extinction was effective and latent extinction was ineffective at extinguishing memory of DLS-dependent response learning.

results in a stronger memory that is less sensitive to decay. Therefore, decay becomes progressively less pronounced following each subsequent period of disuse (From Thorndike, 1913, p. 283; axis labels added).

been disputed (McGeoch, 1932), the general predictions of Thorndike's model (see **Figure 7**) resemble the acquisition curves obtained in the present study. It is possible that some decay (or, more generally, forgetting) occurred in between daily training sessions, but that with each subsequent session of relearning the memory became more firmly ingrained and less sensitive to decay.

The principal finding that latent extinction was effective in the place learning task but not the response learning task may be related to differences between the memories acquired in each task. That is, latent extinction might only be effective when the to-be-extinguished memory contains certain critical features. The tasks selected for the present experiments depended on distinct neural systems, and solving each task hinged on different learning requirements. The hippocampus-dependent place learning task presumably required animals to encode the spatial location of the food reward to guide behavior to the correct arm, whereas the DLS-dependent response learning task only required that animals encode a left body-turn response at the maze choice point. Although animals being trained in the response learning task could also encode the spatial locations of the food reward, this information was not necessary for acquisition and ongoing performance in this task. In fact, extensive evidence indicates that spatial information might interfere with acquisition in the response learning task (for reviews, see Poldrack and Packard, 2003; Packard and Goodman, 2013).

Latent extinction in maze learning tasks might only be effective when the spatial location of the reinforcer is a critical part of the to-be-extinguished memory. Previous studies examining latent extinction have typically employed maze tasks, such as the straight alley maze, that could be solved adequately using either spatial or non-spatial learning strategies. In ''dualsolution'' tasks such as these, animals typically employ spatial learning strategies when the learning environment constitutes a heterogeneous visual surround, whereas animals employ response learning strategies when the task is conducted in a homogeneous visual surround (for reviews, see Restle, 1957; Packard and Goodman, 2013). Interestingly previous studies have indicated that latent extinction was only effective in heterogeneous visual surrounds conducive to allocentric spatial learning (e.g., Seward and Levy, 1949; Denny and Ratner, 1959; Dyal, 1962). Latent extinction was not effective in homogenous visual surrounds that prevented the use of allocentric spatial learning (e.g., Bugelski et al., 1952; Scharlock, 1954; Denny and Ratner, 1959). These previous findings are consistent with the suggestion that in maze learning tasks, latent extinction might be selectively effective at extinguishing allocentric spatial memory.

The finding that latent extinction might only be successful at extinguishing certain types of memory could be attributed to the distinct learning mechanisms through which latent extinction operates. Unlike response extinction, latent extinction does not conform to classical models of extinction that suggest the animal must make the previously acquired response for extinction to occur (e.g., Hull, 1943, 1952). Proponents of the Hullian S-R view of learning have suggested that latent extinction, although it may not be readily explained by Hull's traditional responseinhibition theory of extinction, could still be accounted for through a Hullian fractional anticipatory response mechanism (Hull, 1931; Spence, 1951). According to this view (Moltz, 1957), an unobservable component of the consumatory goal response is elicited by cues throughout the maze during initial acquisition of the task, and this partially guides behavior to the correct goal location. When an animal is confined to the goal box during latent extinction, this fractional goal response is elicited and over time, becomes extinguished to the goal box cues. To the extent that the goal box cues might resemble earlier sections of the maze, extinction of the fractional goal response will generalize to other parts of the maze, resulting in increased latency and incorrect turns during extinction probe trials. Several cogent arguments have been raised indicating the inadequacy of this potential S-R mechanism in explaining latent extinction (Gleitman et al., 1954; Treisman, 1960). In addition, this putative mechanism is not supported by the present findings. If latent extinction were to operate by extinguishing a fractional response in the goal box that generalizes to other parts of the maze, then it would be reasonable to predict that latent extinction would be effective across both place and response learning tasks, which presently was not observed.

Previous evidence from our laboratory suggests that latent extinction may involve spatial memory mechanisms (Gabriele and Packard, 2006). Temporary inactivation of the dorsal hippocampus with bupivacaine blocks the effectiveness of latent extinction in the straight alley maze (Gabriele and Packard, 2006). Considering that a principal function of the hippocampus involves spatial memory formation (O'Keefe and Nadel, 1978; Morris et al., 1982), it is possible that hippocampal inactivation blocked latent extinction by disrupting hippocampus-dependent spatial memory processing. That latent extinction might depend in part on spatial memory processing is largely consistent with previous behavioral evidence. As mentioned previously, latent extinction is selectively effective in heterogeneous visual environments conducive to spatial memory formation, but not homogenous visual environments that prevent spatial memory formation (Seward and Levy, 1949; Bugelski et al., 1952; Scharlock, 1954; Denny and Ratner, 1959; Dyal, 1962).

Latent extinction may involve spatial memory processing insofar as confining an animal to a previously rewarded spatial location without food (i.e., latent extinction) might allow the animal to acquire a new memory in which the spatial location becomes associated with absence of food. Thus, for latent extinction to be successful, a rat must be confined to the previously rewarded spatial location. Confining a rat to an empty goal box located in a different room (Iwahara et al., 1953) or a different spatial location in the same room (Clifford, 1964) does not produce extinction. This proposed mechanism for latent extinction is consistent with its dependance on hippocampal function, i.e., in addition to acquiring information about food rewarded locations, the hippocampus is similarly involved in linking spatial locations with the absence of food reward (Gaskin and White, 2006).

This putative spatial mechanism could also explain why latent extinction was effective in the place learning task, but not the response learning task. In the place learning task, memory performance was presumably guided by a learned association in which a spatial location had been associated with the food reward. Thus, if the same spatial location were subsequently associated with the absence of food reward, which putatively occurs during latent extinction, we should expect memory performance in the place learning task to decline. In contrast, memory performance in the response learning task was presumably not guided by the spatial locations of the food reward, and therefore associating spatial locations with the absence of food reward should not affect later retrieval of the previously acquired response.

Given the effectiveness of typical response extinction across both place and response learning tasks, it is tempting to speculate that response extinction might depend on a distinct learning mechanism. Previous evidence from our laboratory indicates that in contrast to latent extinction, the effectiveness of response extinction in the straight alley maze is not impaired following hippocampal inactivation (Gabriele and Packard, 2006). Rather, response extinction in the straight alley maze is attenuated following lesion or temporary inactivation of the DLS (Dunnett and Iversen, 1981; Thullier et al., 1996; Gabriele, 2008). Considering that the DLS is a chief neural substrate implicated in S-R learning and memory processes (Packard and Knowlton, 2002), one possibility is that during response extinction the DLS forms S-R associations between visual cues in the learning situation (i.e., the stimuli) and the inhibition of a behavior (i.e., the response). Several investigators have proposed similar S-R mechanisms to account for extinction across maze learning, operant lever pressing, and Pavlovian conditioning paradigms (Guthrie, 1935; Hull, 1943; Rescorla, 1993; Delamater, 2004). Importantly, the learned inhibition of behavior during response extinction could potentially explain the effectiveness of this protocol in both place learning and response learning tasks.

Aside from the direct involvement of multiple memory systems, another potential mechanism underlying the selective effectiveness of latent extinction pertains to the immediate differences between the two tasks. Although the place and response learning tasks were identical in terms of their motivational, sensory, and motoric requirements, it was necessary that the tasks differed slightly in some respects so that each task invoked a different memory system. We cannot rule out the possibility that slight differences between the two tasks (e.g., in the place learning task, animals received food in one location; in the response learning task, animals received food in two locations) may have partially influenced the effectiveness of latent extinction.

In sum, the present findings indicate that whereas response extinction successfully extinguished memory in hippocampusdependent place learning and DLS-dependent response learning tasks, latent extinction was selectively effective in the place learning task and not the response learning task. The suggestion that the principal learning mechanisms underlying latent extinction involve an acquired association between the spatial location and the absence of food reward may provide an

#### REFERENCES


explanation for the selective effectiveness of latent extinction across these learning tasks. Future studies utilizing a wider variety of spatial and non-spatial memory tasks are required to further examine this hypothesis.

### AUTHOR CONTRIBUTIONS

JG contributed ideas, manuscript writing, and conducted research. MGP contributed ideas and manuscript writing.

# ACKNOWLEDGMENTS

The authors would like to thank Gizelle Leal and Chelsey Thibodeaux for their assistance on this project. This study was supported by Texas A&M University PESCA grant (MGP).


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Goodman and Packard. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Role of the Medial Prefrontal Cortex in the Conditioning and Extinction of Fear

Thomas F. Giustino and Stephen Maren\*

Department of Psychology and Institute for Neuroscience, Texas A&M University, College Station, TX, USA

Once acquired, a fearful memory can persist for a lifetime. Although learned fear can be extinguished, extinction memories are fragile. The resilience of fear memories to extinction may contribute to the maintenance of disorders of fear and anxiety, including post-traumatic stress disorder (PTSD). As such, considerable effort has been placed on understanding the neural circuitry underlying the acquisition, expression, and extinction of emotional memories in rodent models as well as in humans. A triad of brain regions, including the prefrontal cortex, hippocampus, and amygdala, form an essential brain circuit involved in fear conditioning and extinction. Within this circuit, the prefrontal cortex is thought to exert top-down control over subcortical structures to regulate appropriate behavioral responses. Importantly, a division of labor has been proposed in which the prelimbic (PL) and infralimbic (IL) subdivisions of the medial prefrontal cortex (mPFC) regulate the expression and suppression of fear in rodents, respectively. Here, we critically review the anatomical and physiological evidence that has led to this proposed dichotomy of function within mPFC. We propose that under some conditions, the PL and IL act in concert, exhibiting similar patterns of neural activity in response to aversive conditioned stimuli and during the expression or inhibition of conditioned fear. This may stem from common synaptic inputs, parallel downstream outputs, or corticocortical interactions. Despite this functional covariation, these mPFC subdivisions may still be coding for largely opposing behavioral outcomes, with PL biased towards fear expression and IL towards suppression.

#### Edited by:

Onur Gunturkun, Ruhr University Bochum, Germany

#### Reviewed by:

Cyril Herry, University of Bordeaux, France Miguel Angel Fullana, Hospital del Mar, Spain

> \*Correspondence: Stephen Maren maren@tamu.edu

Received: 29 August 2015 Accepted: 26 October 2015 Published: 09 November 2015

#### Citation:

Giustino TF and Maren S (2015) The Role of the Medial Prefrontal Cortex in the Conditioning and Extinction of Fear. Front. Behav. Neurosci. 9:298. doi: 10.3389/fnbeh.2015.00298 Keywords: prelimbic, infralimbic, freezing, fear, extinction

# INTRODUCTION

Pavlovian fear conditioning is a form of learning that serves as a robust model to explore the neurobiological underpinnings of disorders of fear and anxiety, including post-traumatic stress disorder (PTSD). In a typical rodent experiment, an innocuous conditioned stimulus (CS; e.g., an auditory tone) is paired with an aversive unconditioned stimulus (US; e.g., a mild electric footshock). After one or more conditioning trials, presentation of the CS alone comes to elicit a conditioned fear response (CR) that includes freezing behavior (i.e., immobility except that necessary for respiration), changes in heart rate and respiration, and potentiated acoustic startle (Davis, 1992; LeDoux, 2000; Maren, 2001). Importantly, these fear CRs can be extinguished by repeated presentations of the CS in the absence of the US. In rodents and humans alike, CRs to an extinguished CS tend to return under a number of conditions including the passage of time (spontaneous recovery), when the CS is presented outside the extinction context (renewal), or with exposure to an unsignaled US (reinstatement; Bouton, 2000, 2002; Hermans et al., 2006; Maren et al., 2013; Vervliet et al., 2013; Goode and Maren, 2014). These recovery or relapse phenomena suggest that extinction does not erase fear memories, but generates a new safety memory that inhibits the expression of fear. In addition, extinction learning itself is a fragile process, dependent on many factors including timing relative to conditioning (Maren and Chang, 2006; Myers et al., 2006; Maren, 2014) and stress (Maren and Holmes, 2015).

While learned fear serves an adaptive purpose aiding survival, pathological fear states are thought to underlie various stress and trauma-related disorders such as PTSD, which has a lifetime prevalence of nearly 8% in the general population (Kessler et al., 1995, 2005). Not surprisingly, this number increases to as high as 30% in combat-exposed veterans (Koenen et al., 2008), amplifying the need for more effective therapies. PTSD has been described as the only mental health disorder with a known cause (i.e., a traumatic experience; Pitman et al., 2012) and is characterized by heightened arousal and resistance to extinction learning (Rauch et al., 2006). Many have argued that PTSD may, at least in part, be a disorder of the fear circuitry (Shin and Handwerger, 2009) and an enhanced understanding of learned fear is relevant to the psychological processes underlying this disorder (Liberzon and Sripada, 2008; VanElzakker et al., 2014). It is possible that PTSD patients exhibit exaggerated fear conditioning, resistance to extinction, or both; ultimately, they exhibit persistent fear CRs (Pitman, 1988).

Due to the prevalence and debilitating nature of stress and trauma-related disorders, there has been a surge in interest in understanding the neural processes subserving learned fear and its subsequent extinction (Quirk and Mueller, 2008; Milad and Quirk, 2012; Maren et al., 2013). A triad of brain regions, including the amygdala, hippocampus and medial prefrontal cortex (mPFC) has been heavily studied in relation to fear (Maren and Quirk, 2004; Herry et al., 2010; Dejean et al., 2015). While it is well accepted that the amygdala and hippocampus play a role in conditioned fear and extinction, a dichotomy of function has been proposed within the mPFC in which the prelimbic (PL) and infralimbic (IL) cortices regulate the expression and suppression of fear, respectively (Quirk and Mueller, 2008; Sotres-Bayon and Quirk, 2010; Milad and Quirk, 2012; Maren et al., 2013). Here, we critically review the anatomical and physiological evidence that has led to this proposed dichotomy of function within mPFC, comparing results from rodents with those in humans.

# THE FEAR CIRCUIT

It is well established that both the acquisition and extinction of fear memories requires synaptic plasticity within the amygdala, however a comprehensive discussion of the amygdala circuitry is beyond the scope of this review (Fanselow and LeDoux, 1999; LeDoux, 2003; Maren and Quirk, 2004; Herry et al., 2010; Pape and Pare, 2010; Lee et al., 2013; Duvarci and Pare, 2014). The amygdala is a node of highly interconnected nuclei; the basolateral complex of the amygdala (BLA; consisting of the lateral, basal and basomedial nuclei) and the central nucleus of the amygdala (CeA; consisting of lateral and medial components) play critical roles in the acquisition of both fear and extinction memories. It has been suggested that inhibitory neurons within the amygdala play a role in regulating fear output. These include: (1) the intercalated cell masses (ITCs) positioned between the BLA and CeA (Nitecka and Ben-Ari, 1987; McDonald and Augustine, 1993; Paré and Smith, 1993; Royer et al., 1999; Lee et al., 2013; Duvarci and Pare, 2014); (2) local inhibitory interneurons within the BLA (Spampanato et al., 2011; Wolff et al., 2014); and (3) inhibitory interneurons in CeL that project to CeM (Ciocchi et al., 2010; Haubensak et al., 2010).

How one structure supports the formation and storage of opposing memories is not fully understood, although it appears that distinct cell populations within the BLA may preferentially encode low and high fear states (Goosens et al., 2003; Hobin et al., 2003; Herry et al., 2008; Senn et al., 2014). For example, lesions of the lateral amygdala (LA), a locus for CS and US convergence, or the CeA disrupt fear conditioning (LeDoux et al., 1990; Goosens and Maren, 2001; Wilensky et al., 2006). Similarly, reversible inactivation of the BLA prevents the acquisition and expression of conditioned fear (Helmstetter and Bellgowan, 1994; Muller et al., 1997), suggesting a large degree of overlap between the subnuclei of the amygdala. Studies using overtraining procedures have demonstrated that amygdala lesions disrupt fear memories, not the ability of animals to emit conditioned fear responses (Maren, 1998, 1999). Single-unit recordings have demonstrated learning-related changes in shortlatency (less than 15 ms) CS-evoked responses in the LA after fear conditioning, suggesting that these changes are mediated by direct thalamo-amygdala projections (Quirk et al., 1995; Maren, 2000). Moreover, these conditioning-induced changes in spike firing are specifically related to the associative nature of the CS, indicating that the LA is a crucial site of plasticity for fear memories independent of freezing behavior (Goosens et al., 2003). In contrast, the CeA is primarily thought of as an output station, relaying information to the brain stem, hypothalamus and periaqueductal gray (PAG) to initiate fear responses such as freezing (Paré et al., 2004). Whereas the CeL is necessary for fear acquisition, CRs are mediated by CeM output (Ciocchi et al., 2010; Haubensak et al., 2010). Curiously, while the LA encodes CS-US information, there are no direct connections between the LA and CeA to directly mediate fear output, suggesting that the BL or BM or both may act as an interface (Amano et al., 2011). Interestingly, post-conditioning lesions of the basal nuclei block fear expression while leaving learning intact (Anglada-Figueroa and Quirk, 2005; Amano et al., 2011). Selective inactivation of either BM or BL alone was not sufficient to mimic this effect, whereas inactivation of both BM and BL was sufficient. This implies that some level of functional overlap exists between these two regions (Amano et al., 2011).

Additionally, several studies have shown that BLA synaptic plasticity is crucial for the acquisition of extinction (Falls et al., 1992; Lu et al., 2001; Herry et al., 2006, 2008; Kim et al., 2007; Sotres-Bayon et al., 2007). Upon extinction learning, LA neurons typically show a reduction in CS-evoked neural activity (Quirk et al., 1995; Repa et al., 2001). However, a distinct population of LA cells maintain CS-evoked responding throughout extinction learning (Repa et al., 2001). Interestingly, after extinction, patterns of CS-evoked neural activity in LA are mediated by the context and reflect the level of freezing (i.e., larger responses occur when fear renews; Hobin et al., 2003). In summary, there is compelling evidence to support the notion that the amygdala is a crucial locus for the acquisition and extinction of learned fear with both ''fear'' and ''extinction'' neurons existing within the same subnuclei whose CS-evoked activity strongly correlates with the level of fear expression (Quirk et al., 1995; Repa et al., 2001; Goosens et al., 2003; Herry et al., 2008; Senn et al., 2014).

The hippocampus has also been identified as a key mediator of learned fear. Given the role of the hippocampus in encoding contextual and spatial information it is not surprising this region plays a substantial role in the fear circuit. Numerous studies have shown that hippocampal lesions dampen fear to a context previously associated with a shock US (Selden et al., 1991; Kim and Fanselow, 1992; Phillips and Ledoux, 1992). Importantly, hippocampal lesions produce larger deficits when made soon after context conditioning, suggesting that recent memories rely more heavily on the integrity of the hippocampus (Maren et al., 1997; Anagnostaras et al., 1999). Interestingly, hippocampal lesions do not necessarily interfere with context conditioning when damage is made prior to training (Maren et al., 1997; Frankland et al., 1998), although deficits in the acquisition of contextual fear can be obtained with single-trial procedures (Wiltgen et al., 2006). Collectively, these results suggest that the hippocampus is required for forming and storing memories of the context, but not necessarily context-US associations (Young et al., 1994). These findings support the notion that the hippocampus plays a key role in both the acquisition and expression of conditioned fear to a particular context.

As mentioned above, the extinction of fear is highly context-dependent; that is, fear returns or ''renews'' when the CS is presented outside the extinction context. Considerable evidence indicates that the renewal of fear is mediated by the hippocampus (Bouton, 2000, 2002; Bouton et al., 2006; Hermans et al., 2006; Maren et al., 2013; Vervliet et al., 2013; Goode and Maren, 2014). For example, many studies have shown that hippocampal inactivation dampens fear renewal when the CS is presented outside of the extinction context (Holt and Maren, 1999; Corcoran and Maren, 2001; Hobin et al., 2006; Maren and Hobin, 2007; Zelikowsky et al., 2012). In addition, disconnections of the hippocampus from the amygdala or prefrontal cortex impair renewal (Orsini et al., 2011), amygdala neurons engaged during fear renewal receive hippocampal and prelimbic input (Knapska et al., 2012) and individual hippocampal neurons expressing Fos after fear renewal preferentially project to both the amygdala and prefrontal cortex (Jin and Maren, 2015). These data suggest that the hippocampus integrates contextual information during conditioning and likely regulates the context dependent recall of fear after extinction learning.

Fear regulation must be tightly controlled and this is thought to depend on the mPFC. Two subdivisions of mPFC in rodents, and their human homologs, have been identified as having distinct roles within the fear circuit. The prelimbic cortex (PL) is thought to regulate fear expression, whereas the infralimbic cortex (IL) mediates fear suppression (Quirk and Beer, 2006; Sotres-Bayon and Quirk, 2010; Milad and Quirk, 2012; Riga et al., 2014). A similar division of labor has been proposed in humans, indicating that the neural mechanisms of extinction learning may be conserved across species (Phelps et al., 2004; Schiller et al., 2008; Sehlmeyer et al., 2009; Milad and Quirk, 2012; Vervliet et al., 2013). Below we review the extant literature that has led to this proposed dichotomy of function.

# ANATOMY OF THE RODENT mPFC

Initially, the PFC was defined by a granular layer IV; this criterion excluded lower level mammalian species, including rodents (Brodmann, 1909). This classification was challenged by Rose and Woolsey, who suggested that projections from the mediodorsal nucleus of the thalamus were the defining feature of the PFC. This re-definition of the PFC was inclusive of all mammalian species (Rose and Woolsey, 1948) and it is now generally accepted that rodents have a PFC with some homology to that of higher-order species (Uylings and van Eden, 1990; Uylings et al., 2003). These homologies are based on several criteria including cytoarchitectonics, connectivity patterns, electrophysiological properties, protein expression, and changes in behavior following damage (Campbell and Hodos, 1970; Uylings and van Eden, 1990; Uylings et al., 2003). Indeed, the rodent PFC like that in humans plays a role in an array of complex behaviors (Heidbreder and Groenewegen, 2003; Kesner and Churchwell, 2011).

# Laminar Organization and Cell Types

In rodents, the mPFC is identified as the agranular portion of the frontal lobe and is divided into three subdivisions: the anterior cingulate (ACC), the PL and the IL. Here, we will primarily focus on PL and IL. The rodent PFC exhibits laminar organization with deep and superficial layers (Caviness, 1975; Yang et al., 1996; Uylings et al., 2003; van de Werd et al., 2010), although a granular layer IV is less well defined when compared to humans and non-human primates (Krettek and Price, 1977b; Uylings and van Eden, 1990; Uylings et al., 2003). PL and IL are neighboring structures, with PL lying just dorsal to IL, which can be distinguished based on cytoarchitectonic features and laminar organization. For example, layer V of PL is less well organized compared to more dorsal regions (i.e., ACC), whereas layer VI cells are arranged in a horizontal fashion in both rats and mice (van de Werd et al., 2010). Due to the relatively large size of PL, layer II and III appear broad compared to neighboring subdivisions. Interestingly, there is evidence for a changing organization along the dorsal-ventral axis of PL, which may transition into IL (Heidbreder and Groenewegen, 2003; Perez-Cruz et al., 2007; van de Werd et al., 2010). This distinction is mainly based on the expansion of layer II at the expense of layers III and V along this axis. In contrast to PL, IL layer II neurons innervate layer I at a much higher rate, making IL layer II appear broad (Krettek and Price, 1977b; van de Werd et al., 2010). While the more superficial layers II and III are easily discernible from a lighter layer V in PL, IL layers are less distinct (Krettek and Price, 1977b). In general, IL layers II-VI have a relatively homogenous layout in terms of cell size and density, with smaller cell bodies compared to PL (van Eden and Uylings, 1985; van de Werd et al., 2010). The contribution of different layers and functional changes along the dorsal-ventral axis of PL and IL are largely unknown, but may be differentially engaged in the fear circuit, similar to the findings noted above regarding distinct populations within amygdala nuclei regulating opposing fear states.

Cortical processing of information requires complex interactions between a number of distinct cell types that fall into two broad categories: principal cells (80–90%) and interneurons (10–20%; DeFelipe and Fariñas, 1992; Gabbott et al., 2005). Neurons are typically classified based on unique characteristics including cell size and shape, dendritic arborization, molecular markers, and connectivity. Pyramidal cells are typically thought to communicate to long-distance targets and are found in layers II-VI (DeFelipe and Fariñas, 1992), although there are noted differences in the firing properties, cell body size and dendritic morphology within and across layers (Yang et al., 1996; Barthó et al., 2004; Molnár and Cheung, 2006; Wang et al., 2006; Otsuka and Kawaguchi, 2008; Brown and Hestrin, 2009; Dembrow et al., 2010; van Aerde and Feldmeyer, 2015). In addition, a number of molecular markers have been identified to categorize specific subclasses of pyramidal cells (Gong et al., 2003; Hevner et al., 2003; Gray et al., 2004; Molnár and Cheung, 2006; Watakabe et al., 2007). The complexity and organization of cortical pyramidal neurons makes the PFC well suited to regulate several functions and an array of behaviors (Heidbreder and Groenewegen, 2003; Kesner and Churchwell, 2011).

Similar to pyramidal cells, interneurons of the cortex are separated into several classes based on unique physiological, morphological, and immunocytochemical markers (Kawaguchi and Kubota, 1993, 1997; Kawaguchi, 1995; Gupta et al., 2000; Ascoli et al., 2008; Povysheva et al., 2008). While sparse in number relative to pyramidal cells, interneurons nonetheless serve to modulate cortical function. Broad classes of interneurons, based on the heterogeneous expression of calcium-binding proteins and neuropeptides such as parvalbumin (PV), somatostatin, vasoactive intestinal polypeptide and cholecystokinin, have been observed in most layers of rodent PFC, although this distribution may not be uniform (DeFelipe, 1993; Kawaguchi and Kubota, 1993, 1997; Kawaguchi, 1995; Gabbott et al., 1997). These distinct classes of interneurons exhibit unique firing patterns, synapsing on specific morphological subregions of pyramidal cells. For example, somatostatin-positive interneurons typically innervate pyramidal cell dendrites to modulate the gain of inputs terminating within those subregions (Kawaguchi and Kubota, 1997; Gupta et al., 2000; Freund and Katona, 2007). In contrast, fast-spiking parvalbumin-positive interneurons (PVINs) target the perisomatic region of pyramidal cells, thereby influencing firing rate and action potential synchronization (Cobb et al., 1995). Interestingly, PV expression in mPFC is generally similar between PL and IL, suggesting the mechanisms for modulating mPFC output are similar between these two brain regions (Gabbott et al., 1997; van de Werd et al., 2010). As with the vast array of principal neurons, the differential contribution of specific subtypes of interneurons within and between mPFC layers within the fear circuit are questions of high interest that remain to be resolved.

#### Inputs

It is well established that PL and IL receive excitatory inputs from regions including, but not limited to, the midline thalamus, BLA, hippocampus and contralateral mPFC (Krettek and Price, 1977a; Little and Carter, 2012, 2013). The posterior portion of the amygdala strongly projects to both PL and IL with sparse innervation from the anterior regions (Krettek and Price, 1977a). Some studies however, have shown strong connectivity from anterior regions, especially from the BLA (Sarter and Markowitsch, 1984; McDonald, 1987). BLA projections synapse on layers II–VI with a small percentage of these projections targeting PVINs (Gabbott et al., 2006). Thus, BLA projections can functionally modulate mPFC output via feed-forward inhibitory mechanisms. In addition, dorsal and ventral hippocampus (CA1/subiculum) exhibit robust excitatory projections to PL and IL (Swanson, 1981; Jay et al., 1989; Jay and Witter, 1991; Azuma and Chiba, 1996; Hoover and Vertes, 2007). These projections have been reported to terminate in all layers of mPFC, although this may shift in density along the dorsalventral axis (Jay et al., 1989; Jay and Witter, 1991). In addition, a population of ventral CA1 neurons innervates IL layers I and V and these same hippocampal neurons also synapse on entorhinal neurons, which may be important for integrating contextual and spatial information (Swanson, 1981). Similar to the amygdala, some hippocampal projections may preferentially target mPFC interneurons, inhibiting mPFC output to downstream targets (Sotres-Bayon et al., 2012). In summary, PL and IL receive many similar input patterns, suggesting that these two subdivisions of mPFC integrate incoming information from multiple sources to drive appropriate behavioral responding.

#### Outputs

The regulation of fear is thought to rely heavily on the integrity of the mPFC, which functions to exert top down control over subcortical structures, coding for appropriate behavioral responses. The most widely accepted view is that PL and IL project broadly to the same region (e.g., the amygdala) but to distinct populations of cells that ultimately dictate CRs. To this end, PL and IL both strongly innervate the BLA and these glutamatergic projections originate from layers II, V and VI (DeFelipe and Fariñas, 1992; Pinto and Sesack, 2000, 2008; Gabbott et al., 2005; Hoover and Vertes, 2007). In terms of their potential functional opposition, PL projections terminate in the BLA whereas IL projects to the ventral region of the LA, the basomedial nucleus, and the lateral central nucleus (McDonald et al., 1996; McDonald, 1998; Vertes, 2004). Although many have proposed that IL projections to the ITCs gate CeA output (Royer et al., 1999; Royer and Paré, 2002; Likhtik et al., 2005), recent data challenge this possibility (Cassell and Wright, 1986; Gutman et al., 2012; Pinard et al., 2012; Strobel et al., 2015). Pinard et al. (2012) have suggested that if this indeed is the pathway mediating fear inhibition, it must work via sparse connections. These weak connections may partially explain why extinction learning is not always robust and prone to relapse. Similar results using diffusion tensor imaging and structural tract-tracing techniques in mice further demonstrate largely indistinguishable amygdalar projections from PL and IL (Gutman et al., 2012), although little is known about the functional aspects of PL innervation of the ITCs. One possibility is that IL mediated excitation of the ITCs is disynaptic, acting through the BLA (Strobel et al., 2015). In addition, PL and IL have direct projections the PAG (Hardy and Leichnetz, 1981; Beitz, 1982; Sesack et al., 1989; Floyd et al., 2000; Vianna and Brandão, 2003; Hoover and Vertes, 2007). Floyd et al. (2000) have suggested that rostral PL/IL preferentially innervate the ventrolateral PAG, whereas more caudal portions of PL/IL innervate the dorsolateral PAG. It remains possible that mPFC projections can bypass the amygdala to directly influence freezing behavior. In summary, recent anatomical evidence suggests that PL and IL display overlapping connections, especially to the amygdala and very weakly innervate the ITCs. The majority of these findings are from behaviorally naïve animals however. It would be advantageous to explore the functional outcome of these overlapping projections throughout stages of aversive learning.

#### mPFC Intrinsic Connectivity

A key question in mPFC function revolves around corticocortical interactions, which originate from superficial layers II and III (Hoover and Vertes, 2007). While this has not been studied extensively in fear, in slice preparations IL has higher frequency local field potential (LFP) components than PL, and these differ when the two regions are disconnected—implying some level of functional connectivity regulating basal activity (van Aerde et al., 2008). In addition, optogenetic activation of IL inhibits PL pyramidal cells in vivo (Ji and Neugebauer, 2012). This feed-forward inhibition may be a necessary component of extinction learning, although this has not been tested. Difficulty arises when addressing these questions simply due to the physical proximity of PL and IL, and the trouble of restricting infusions solely to one region.

# EARLY EVIDENCE FOR A DIVISION OF LABOR

#### Lesion Studies

One of the first studies to examine the role of mPFC in defensive behaviors showed that damage to this structure had no effect on flight, biting or reactivity to handling in wild rats, although these lesions primarily encompassed more dorsal regions than PL and IL (i.e., ACC; Divac et al., 1984). In contrast to this report, dmPFC lesions (encompassing ACC/dorsal PL) in laboratory rats increased reactivity to an aversive stimulus and it was shown that these animals were capable of maintaining long-term fear, suggesting that dmPFC is not necessary for memory formation and retention or fear expression (Holson, 1986). More recent work, however, has shown that pretraining ACC lesions impair fear acquisition, while leaving fear expression intact in laboratory rats, although this deficit could be overcome with additional training (Bissière et al., 2008). In a separate study, Morgan et al. (1993) demonstrated that pre-conditioning mPFC lesions (encompassing ACC, PL, and IL) did not have an appreciable effect on the rate of acquisition or level of fear expression to either context or cued fear conditioning. However, these animals took longer to reach extinction criterion, suggesting that mPFC neural activity plays a role in extinction learning (Morgan et al., 1993). In a follow up study, selective PL lesions (damage was mainly restricted to dorsal PL) produced a general increase in both cued and context fear during acquisition and extinction phases, suggesting that dmPFC lesions yield a general increase in fear (Morgan and LeDoux, 1995). The authors suggest that these findings revealed a differential contribution of PL vs. IL to the expression of conditioned fear. However, based on the extent of the lesions presented in each study, an alternative interpretation is that behavioral differences reflected gross differences in functions mediated by the dorsal-ventral axis of mPFC and not specifically PL vs. IL. In support of this, some studies have reported decreased freezing and differential cardiovascular responses to a CS as a function of the dorsalventral extent of mPFC lesions, suggesting that the functional contribution of mPFC may differ along this axis rather than being exclusively confined to PL vs. IL (Frysztak and Neafsey, 1991, 1994).

On the basis that animals with mPFC damage display extinction impairments (Morgan et al., 1993), a subsequent study sought to directly compare the effects of damage restricted to different mPFC subregions and better define their contribution to extinction learning. It was found that while vmPFC lesions (encompassing IL and to some extent PL) do not impair extinction learning per se, they disrupted extinction recall. Importantly, this effect was not observed in sham operated animals or animals with lesions that spared the majority of IL. The authors suggest that IL neural activity in particular is involved in the consolidation of extinction learning (Quirk et al., 2000).

Many of these studies have formed the basis for the proposed dichotomy of function in the mPFC in which PL regulates fear expression and IL fear suppression. However, these findings are largely discrepant in nature with reports indicating increases, decreases, or no changes in learning following mPFC damage. Moreover, of particular interest, Holson (1986) and Morgan and LeDoux (1995) demonstrate that dorsal PL lesions produce a generalized increase in fear expression, indicating that an intact dorsal PL may actually function to suppress fear, which is at odds with the current view. In addition, while Quirk et al. (2000) suggest that IL neural activity is importantly involved in the consolidation of extinction memories, similar experiments have not replicated these effects insofar as mPFC lesions do not yield deficits in either conditioned inhibition or extinction learning under some conditions (Gewirtz et al., 1997; Garcia et al., 2006). Thus, it appears the mPFC is not necessary for the formation or retrieval of extinction memories under some circumstances and this may be partially influenced by factors such as the strain of the animals used in these experiments (Chang and Maren, 2010).

As noted above, it has been shown that both behavioral and autonomic responses to a CS are differentially modulated as a function of the dorsal-ventral extent of mPFC damage (Frysztak and Neafsey, 1991, 1994). These findings leave open the possibility that cell populations with overlapping function exist in PL and IL. A more general interpretation of these lesion studies may be that the observed functional differences are a product of the lesion technique and size. It is possible that the behavioral effects reflect a shift in function along the dorsalventral axis, although this may not be solely interpreted as a functional opposition between PL and IL. It is worth noting that PL shows changes in laminar organization and cytoarchitectonic features along this axis which transitions into IL (Heidbreder and Groenewegen, 2003; Perez-Cruz et al., 2007; van de Werd et al., 2010). Hippocampal input to the mPFC is not uniform along this axis (Jay et al., 1989; Jay and Witter, 1991) and these differences may influence the behavioral outcome of localized damage. Overall, despite the controversies around the conclusions one can draw from these lesion studies, they have been instrumental to our understanding of the fear circuit and have led to a rapid increase in additional studies examining the mPFC in fear.

# Pharmacological and Microstimulation Studies

In an attempt to further characterize the role of PL and IL in fear, many studies have used pharmacological agents to temporarily inactivate the mPFC during behavioral tasks. These methods allow for circuit manipulation at discrete time points. For example, intra-PL infusion of the Na+ channel blocker tetrodotoxin prior to fear conditioning does not disrupt the acquisition of conditional fear, but reduces fear expression to a CS or context previously paired with shock (Corcoran and Quirk, 2007). Consistent with PL activity being necessary for fear expression, inactivation of PL, with the GABA-A receptor agonist muscimol, prior to extinction training also impairs fear expression (Laurent and Westbrook, 2009; Sierra-Mercado et al., 2011). However, this manipulation has no long-term effect on extinction recall, suggesting PL inactivation does not interfere with the acquisition of extinction (Laurent and Westbrook, 2009; Sierra-Mercado et al., 2011). Collectively, these findings suggest that PL activity underlies fear expression, but not learning per se.

There is some evidence to support the idea that PL signaling plays a role in aversive learning, beyond its role in fear expression, however, and this may extend to more dorsal regions, including ACC. For example, PL microstimulation increases fear expression while preventing successful extinction (Vidal-Gonzalez et al., 2006), implying that PL signaling shunts extinction learning by elevating fear. In addition, transient inactivation of rostral ACC impairs fear learning whereas activation enhanced fear acquisition and expression (Bissière et al., 2008). Interestingly, in a study in which rats were trained in a contextual bi-conditional discrimination task (in context A, one CS is paired with shock while a second CS is not, and this contingency is reversed in a second context) PL inactivation interfered with both the encoding and expression of appropriate CS responding. This suggests that PL may integrate contextual information to inform both learning and responding to conditioned stimuli (Sharpe and Killcross, 2014). Moreover, PL inactivation disrupts both recent and remote contextual fear memories after brief memory retrieval, indicating that PL signaling may be involved in reconsolidation. This reconsolidation blockade also prevented reinstatement, further showing that PL activity may subserve the reactivation of fear memories and contribute to their long-term maintenance (Stern et al., 2014), expanding the role of PL in the fear circuit. In summary, PL signaling appears to be a key component encoding the acquisition and expression of learned fears and this may vary based on specific task parameters.

While the PL appears to be involved in the expression of fear, it is widely believed that IL is involved in the suppression of fear during extinction learning and retrieval. IL inactivation increases freezing to conditioned tones while impairing withinsession extinction and retrieval in both rats and mice (Sierra-Mercado et al., 2006, 2011; Laurent and Westbrook, 2009; Morawska and Fendt, 2012; Sangha et al., 2014). Additionally, conditioned tones paired with IL electrical stimulation enable low levels of freezing in rats that had not been previously extinguished, suggesting that IL activation is sufficient to mimic extinction training (Milad and Quirk, 2002; Milad et al., 2004). Interestingly, IL stimulation paired with presentation of a CS in anesthetized rats mimics the behavioral experience of extinction training (Park and Choi, 2010). These effects are frequencydependent: high-frequency IL stimulation immediately after fear memory retrieval reduces freezing at a later time point, whereas low-frequency stimulation impairs extinction learning (Maroun et al., 2012; Shehadi and Maroun, 2013). This may reflect IL potentiation vs. depression with high- and lowfrequency stimulation, respectively. In line with these studies, IL activation, via infusion of the GABA-A receptor antagonist picrotoxin, rescues extinction learning in extinction-deficient mice (Fitzgerald et al., 2014). Others have shown that IL activation prior to an extinction session dampens the expression of fear (Chang and Maren, 2011) and subsequently enhances extinction recall (Thompson et al., 2010; Chang and Maren, 2011).

Extinction learning produces a labile suppression of fear that is susceptible to relapse when a previously extinguished cue is presented outside the extinction context (i.e., fear renewal; Bouton, 2000, 2002; Bouton et al., 2006; Hermans et al., 2006; Maren et al., 2013; Vervliet et al., 2013; Goode and Maren, 2014). This process is likely mediated by hippocampalprefrontal circuits (Corcoran and Maren, 2001; Maren et al., 2013). In addition, the timing of extinction trials relative to conditioning is also a key factor governing the longterm success of extinction training (Maren and Chang, 2006; Myers et al., 2006; Maren, 2014). Extinction trials delivered soon after conditioning often result in a failure to retain this memory long-term, which may reflect impaired mPFC signaling. Using an immediate extinction paradigm, intra-IL picrotoxin abolished conditioned freezing during extinction training and promoted a faster reduction of conditioned responding the following day (Chang and Maren, 2011). In a separate study, IL electrical stimulation paired with CS presentations limited the spontaneous recovery of fear the following day, rescuing the immediate extinction deficit (Kim et al., 2010). Collectively, these findings support the idea that IL signaling promotes extinction learning and suppresses conditional fear.

Overall, the findings discussed above generally lend support to a division of labor in which PL and IL are functionally opposed. However, due to the physical proximity of PL and IL, it is difficult to restrict infusions or electrical stimulation to only one subdivision. Moreover, pharmacological manipulations lack cell specificity, affecting both principal cells and interneurons in a similar fashion. Additionally, electrical stimulation results in ortho- and antidromic signaling which clouds the interpretation of directionality and localization of these effects. Given these experimental limitations, it is not surprising that there is evidence that challenges the dichotomous role of PL and IL in fear expression and suppression, respectively. For example, if PL activity underlies fear expression to associative stimuli, then PL activation at any time point of associative fear learning should increase freezing behavior whereas inactivation should impair freezing. Curiously, PL inactivation does not affect freezing under some conditions (Bravo-Rivera et al., 2014; Sharpe and Killcross, 2015) suggesting that ongoing freezing behavior is not solely dependent on PL activity and that other neural structures can compensate in its absence. Similarly, if IL is a necessary component of fear suppression, then IL activation should serve to promote extinction learning and subsequently reduce fear responding while inactivation should have the opposite effect. Interestingly, some studies have reported facilitated extinction learning with IL inactivation in both aversive and appetitive conditions (Akirav et al., 2006; Mendoza et al., 2015) making it possible that cell populations within IL exist that can bi-directionally modulate extinction learning. These findings challenge existing models of PL and IL function in fear and leave open the possibility that there is some functional overlap between PL and IL that allows one structure to compensate for the other under some conditions.

# mPFC NEURAL CORRELATES OF FEAR AND EXTINCTION

#### Immediate Early Genes

Immediate early genes (IEGs) such as c-fos, Arc and Zif268 are activated in response to cellular stimulation, providing an indirect measure of neural activation and have been implicated in learning and memory (Davis et al., 2003; Plath et al., 2006). Interestingly, patterns of mPFC gene expression may be context-dependent, possibly as a result of feed-forward information being integrated from the hippocampus. In line with the idea that mPFC IEG expression may be partly modulated by context, PL and IL exhibited opposing patterns of Fos expression in a renewal paradigm in which an extinguished CS is presented in the extinction context (low fear) and in a different context (high fear). PL showed robust increases in Fos expression during fear renewal whereas presentation of the extinguished CS in the extinction context induced increased Fos expression in IL (Knapska and Maren, 2009). Similarly, in a separate set of studies, levels of Zif268 were greater in PL upon contextual fear recall (Stern et al., 2014), whereas increased IL Zif268 expression has been reported in animals recalling a remote cued fear memory; this effect was not observed in PL (Fitzgerald et al., 2015b). In addition, prefrontal levels of Arc mRNA expression show context specificity, with higher levels in BA, LA and IL of extinguished rats (Orsini et al., 2013). Further supporting a role for IL in extinction learning, extinction-deficient mice display reduced Fos and Zif268 expression in IL, implying that reduced IL activity may underlie this behavioral deficit (Hefner et al., 2008). In summary, these data suggest that PL and IL IEG expression displays context specificity with PL being primarily activated in a high fear state whereas IL is activated in a low fear state. These findings indicate that the mPFC may integrate contextual cues to process the meaning of the CS and inform conditioned responding.

The above IEG studies mainly suggest opposing roles for PL and IL in the expression or suppression of fear, respectively, while having little influence on learning per se. However, it has been shown that both PL and IL exhibit increased levels of Fos after conditioning, implying that PL and IL activity may underlie new learning. Interestingly, conditioning induced greater activation of PL and IL compared to extinction learning, and Fos expression following each session was indistinguishable between brain regions (Morrow et al., 1999; Herry and Mons, 2004). This conditioning-induced increase in Fos expression may partly be a response to the unconditioned footshock, rather than associative learning per se. However, an antisense oligonucleotide against c-fos mRNA, injected simultaneously into both PL and IL 12 h prior to conditioning, attenuated fear responses during an extinction session (Morrow et al., 1999). Thus, PL and IL appear to be involved in the acquisition of conditioned fear and to a lesser extent, are activated following extinction learning. It is worth noting that this effect was seen with simultaneous manipulations to PL and IL (Morrow et al., 1999), implying that there is some level of functional overlap between the two regions. However, the authors did not manipulate PL or IL alone, leaving the possibility that the decreased fear responding during extinction may be preferentially driven by one of these two regions. In support of the idea that PL and IL may covary at times, a separate study has shown that Fos and Zif268 expression were similar after the retrieval of both a recent and remote contextual fear memory (Frankland et al., 2004). These studies suggest that PL and IL can fluctuate similarly during the acquisition, extinction and expression of conditional fear.

As mentioned previously, animals subjected to extinction trials soon after conditioning often spontaneously recover high levels of freezing the following day which may result from impaired mPFC function (Maren and Chang, 2006; Maren, 2014). In support of this hypothesis, rats extinguished 15 min after conditioning displayed a general decrease in Fos expression in both PL and IL when compared to animals extinguished 24 h after conditioning (Kim et al., 2010; but see Stafford et al., 2013). This suggests that some basal level of activity in both regions is necessary for extinction learning. Additionally, others have shown that the spontaneous recovery of fear after extinction is associated with reduced Fos and Zif268 induction in both PL and IL of rats (Herry and Mons, 2004). Collectively, these studies further demonstrate that neuronal activity in PL and IL are positively correlated under some conditions. The observed similarities may stem from similar synaptic inputs and cortico-cortical interactions, although this remains an open question.

# Electrophysiology

#### Single-Unit Recordings

Electrophysiological methods also provide insight into the function of PL and IL neurons during the conditioning and extinction of fear. Using in vivo single-unit recordings in awake, behaving rats, Milad and Quirk (2002) provided the first evidence that CS-evoked responses in IL correlate with successful extinction recall. This study showed that IL neurons preferentially responded to a CS when rats successfully retrieve an extinction memory, but not during conditioning or the initial extinction session. This effect was specific to IL, as it was not seen in neurons recorded in PL or the medial orbital cortex. The authors suggested that extinction consolidation may enhance IL activity and this subsequently reduces fear the following day (Milad and Quirk, 2002). In agreement with this, successful extinction correlates with high-frequency IL bursting (Burgos-Robles et al., 2007), and under conditions in which extinction fails (i.e., immediate extinction) IL bursting is diminished (Chang et al., 2010). These in vivo findings have been complemented by in vitro studies, which have also provided support that IL signaling is altered upon extinction learning. For example, in slice preparations, the intrinsic excitability of IL neurons was decreased for up to 4 h after conditioning and this can be reversed with extinction training (Santini et al., 2008; Cruz et al., 2014). This reversal suggests the acquisition of extinction induces a ramping upward of spike firing during the consolidation phase, although this inhibition returned in rats that spontaneously recovered fear (Cruz et al., 2014).

How extinction learning and recall are precisely computed at the circuit level is not fully understood, although this was previously thought to be mediated by a direct IL→ITC pathway (Royer et al., 1999; Royer and Paré, 2002; Pape and Pare, 2010; Duvarci and Pare, 2014). In support of this idea, the ITCs are strongly responsive to IL stimulation in anesthetized rats (Amir et al., 2011). Interestingly, at basal levels of activity, ITC neurons actively inhibit each other; however, with brief IL stimulation the ITCs display increased firing rates which diminishes CeA output, a potential mechanism for reduced fear output (Li et al., 2011). Recent evidence, however, has suggested that IL exhibits low levels of connectivity to the ITCs (Gutman et al., 2012; Pinard et al., 2012; Strobel et al., 2015) bringing question to this proposed mechanism of extinction learning. These findings have prompted an updated hypothesis that posits disynaptic projections from IL to the ITCs via the BLA serve to engage inhibitory processes involved in extinction (Strobel et al., 2015). These disynaptic projections may be necessary for IL to overcome the inter-ITC inhibitory network in order to promote extinction learning and reduce fear. Overall, these data support a role for IL excitability in successful extinction learning.

Given that the PL has been implicated in the acquisition and expression of conditioned fear, it follows that this should be reflected in single-unit activity in awake, behaving animals. It has been reported that sustained spike firing in the PL during aversive CSs correlates with ongoing freezing behavior (Burgos-Robles et al., 2009). Consistent with this, extinction-deficient 129/S1 mice show elevated CS-evoked responses in PL, although this effect was also mirrored in IL (Fitzgerald et al., 2014). In contrast, others have reported that the expression of freezing behavior is associated with robust CS-evoked responses in IL (Chang et al., 2010; Fitzgerald et al., 2015b). Interestingly, Chang et al. (2010) also found that, in contrast to IL, CS-evoked PL activity was attenuated during fear expression, revealing a reciprocal relationship between PL and IL activity in the opposite direction to that predicted by prevailing models. In a recent study, we examined the pattern of spontaneous firing in simultaneously recorded PL and IL neurons immediately after fear conditioning (Fitzgerald et al., 2015a). In this post-conditioning period, rats exhibit sustained and high levels of fear that persisted for the duration of the 1 h recording session. During this transition from a low-fear to a high-fear state, spontaneous firing rates some neurons in PL and IL were transiently excited in the minutes following conditioning, but returned to basal levels soon after, despite ongoing freezing behavior. Interestingly, spontaneous firing rates of other neurons in IL were persistently suppressed over the duration of the post-conditioning period (Fitzgerald et al., 2015a). Collectively, these data suggest that PL spike firing alone is unlikely to mediate sustained freezing behavior; indeed, the expression of fear may be due, at least in part, to suppression of IL activity (Chang et al., 2010; Fitzgerald et al., 2015a).

Interestingly, similar to IEG studies, there is evidence for positively correlated single-unit activity in PL and IL after the conditioning or extinction of fear. For example, during the expression of conditioned fear (high fear), spontaneous firing rates are suppressed in both IL and PL, although IL suppression was more robust (Fitzgerald et al., 2015a). Additionally, Holmes et al. (2012) reported no differences in PL vs. IL CS-evoked responses throughout extinction learning as well as extinction retrieval. In a separate study, comparable conditioning-induced increases in CS-evoked activity were observed in the PL and IL of extinction-deficient 129/S1 mice (Fitzgerald et al., 2014). This provides further evidence that PL and IL may covary in their response properties at the single-neuron level, at least under some conditions. Other experiments have found that PL and IL neurons exhibit similar firing patterns in response to CSs or contexts associated with shock (Baeg et al., 2001) or in relation to the types of behavioral responses animals emit (e.g., freeze or move) in response to aversive CSs (Halladay and Blair, 2015). Hence, single-unit activity in IL and PL fluctuates similarly under a number of conditions, which is not surprising given their similar afferent inputs.

#### Local Field Potentials

In addition to single-unit recordings, LFP recordings suggest a high degree of synchrony between the mPFC, amygdala, and hippocampus throughout different stages of aversive learning. LFPs are generated by finely tuned synaptic input patterns, and recent studies have focused on LFPs at the circuit level as a mechanism by which distant brain regions effectively communicate. The coupling and synchronization of brain regions within the fear circuit are likely involved in memory formation and retrieval. Importantly, theta oscillations act to coordinate regional synchronization, providing a means of timely and efficient transmission of information. For example, the BLA and mPFC show enhanced theta synchrony during sleep after conditioning, which plays a role in memory consolidation (Popa et al., 2010). In line with this, increased BLA-mPFC theta synchrony has been observed in animals that successfully learned to differentiate between safe and aversive conditions (Likhtik et al., 2014). During learned safety, BLA firing activity was entrained to theta input from mPFC, suggesting that the BLA is selectively tuned to mPFC input, a potential mechanism underlying memory recall and thus behavioral responding (Likhtik et al., 2014). mPFC projections excite BLA neurons, indicating that inhibition of CeM output may be mediated by an active gating mechanism downstream of BLA (Likhtik et al., 2005). The directionality of this effect supports the role of mPFC in regulating amygdala activity, although it is well known that amygdala output influences mPFC function as well (Senn et al., 2014) and inactivation of BLA decreases PL activity (Sotres-Bayon et al., 2012). One study has shown that in male mice, PL and IL display opposing patterns of theta power across extinction, which may reflect new learning. Given their physical proximity and similar input it is somewhat surprising that LFPs would be drastically different between the two regions. Interestingly, this effect was not seen in females as they displayed heightened freezing and persistently increased mPFC theta in both PL and IL (Fenton et al., 2014). In addition, PL gamma power is elevated in extinction-deficient mice compared to mice that successfully extinguished (Fitzgerald et al., 2014). Moreover, other work has reported theta synchrony of an expanded network involving CA1-LA-IL during the retrieval of conditioned fear. Theta synchronization declined with extinction training, but was partially restored upon extinction recall (Lesting et al., 2011). In summary, LFPs may importantly affect the fear circuit at a global level and theta interactions might provide a mechanism for the fine-tuned organization of neural pathways underlying memory formation and recall.

# OPTOGENETICS AND CHEMOGENETICS: CAUSAL MECHANISMS OF FEAR

The acquisition and retrieval of memories depend on complex patterns of neural activity from distinct neuronal populations defined by their genetic markers. Whereas much of the above evidence convincingly demonstrates a role of mPFC in fear, electrophysiology is only correlative and inactivation methods lack cellular specificity. As such, the fear-related causal mechanisms of precise neural activity and the contribution of different cell types remain largely unknown. Optogenetics and chemogenetics are virally-mediated techniques allowing for cell and circuit specific manipulations to selectively excite or suppress specific neuronal populations. Briefly, optogenetics requires the expression of exogenous light-sensitive ion channels to modulate neuronal activity with high temporal precision (Boyden et al., 2005; Fenno et al., 2011). One chemogenetic approach makes use of Designer Receptors Exclusively Activated by Designer Drugs (DREADDs), which are synthetic Gprotein coupled receptors that respond selectively to the systemic injection of an inert ligand, clozapine N-oxide (CNO; Dong et al., 2010; Urban and Roth, 2015). These technologies provide an in vivo mechanism to control cellular physiology in intact neural circuits and delineate the causal contribution of specific neuronal subtypes to learning and memory.

Recently, optogenetic methods have been used to explore plasticity in prefrontal projections to the amygdala after fear conditioning. Combining optogenetics and ex vivo electrophysiology, Arruda-Carvalho and Clem (2014) have shown that in behaviorally naïve mice, the synaptic connectivity of IL and PL projections onto BLA principal neurons were similar. However, fear conditioning led to a decrease in inhibitory-excitatory balance in PL, but not IL. These data suggest that a PL→BLA pathway is crucial for encoding fear memories and may be engaged when encountering the CS at a later time point to promote a high fear state (Arruda-Carvalho and Clem, 2014).

As discussed above, extinction learning is thought to involve feed-forward inhibition that blunts CeA output via the ITCs (Royer et al., 1999), with IL synaptic transmission regulating this pathway via the BLA. The direct role of mPFC, however, had not previously been tested, including differences between PL and IL. One possibility is that, while weak in number, direct IL projections to the ITCs increase in strength with extinction training to inhibit the CeA, or IL projections to the BLA are modulated which ultimately influences ITC output. If so, the synaptic strength of this pathway may be causally linked to both the acquisition and recall of extinction. Using ex vivo electrophysiology and the excitatory optogenetic virus channelrhodopsin restricted to principal cells under control of the CAMKII promoter, Cho et al. (2013) demonstrated that extinction learning reduced synaptic efficacy in BLA projecting mPFC neurons. Interestingly, mPFC synaptic transmission to ITCs was unchanged and thus the overall balance in the mPFC-BLA pathway shifted towards inhibition following extinction. This effect may stem from monosynaptic connections to BLA interneurons. The authors note that PL and IL projections were nearly indistinguishable in terms of location and evoked current amplitudes downstream in BLA, with the most robust projections terminating in the anterior subdivision of BLA, and to a lesser extent on the ITCs. It could be that the weak IL→ITC projections can dampen amygdala output, without a measurable change in synaptic strength. The relative shift in balance towards BLA inhibition may in turn promote ITC activity, thus impeding CeA output and dampening fear (Cho et al., 2013). These findings suggest a high degree of similarity between both the structural and functional components of PL and IL, lending support to the hypothesis that these regions may covary as noted in several other reports (Baeg et al., 2001; Herry and Mons, 2004; Kim et al., 2010; Holmes et al., 2012; Halladay and Blair, 2015).

In a similar fashion, Hübner et al. (2014) explored functional connectivity between mPFC and the amygdala using retro-bead tracing and excitatory optogenetic techniques in behaviorally naïve mice. They further confirm that mPFC sends monosynaptic excitatory projections to both principal cells and interneurons in the basomedial nucleus of the amygdala (BM). Activating these inputs resulted in feed-forward inhibition of both principal cells and more frequently interneurons, promoting a disinhibition of BM principal cells. PL and IL similarly excited principal BM neurons, consistent with previous work (Cho et al., 2013) and received comparable feedforward inhibition from amygdala feedback loops. However, this study suggested that IL inputs target mainly non-fast spiking interneurons (Hübner et al., 2014). This discrepancy may be explained by the fact that these findings were in behaviorally naïve mice as compared to mice undergoing extinction training in Cho et al. (2013). As noted, the basal levels of synaptic strength in mPFC-BLA circuits may shift significantly after behaviorally relevant events making it difficult to interpret these current findings in regard to fear. Nonetheless, these data further contribute to a growing body of evidence surrounding structural and functional similarities between PL and IL.

Optogenetic manipulations of specific monosynaptic pathways have provided evidence for a revised hypothesis of IL-mediated signaling in extinction. As mentioned above, it was previously believed that the ITCs were a major target of IL projections. A more recent model has proposed that this pathway is disynaptic with BLA serving as the interface between IL and the ITCs (Strobel et al., 2015) given that the direct IL-ITC connections are weak and not modulated upon extinction training (Gutman et al., 2012; Pinard et al., 2012; Cho et al., 2013). It has previously been demonstrated that pharmacological activation of the IL during extinction enhances long-term retention (Thompson et al., 2010; Chang and Maren, 2011) and that CS-evoked activity correlates with extinction recall (Milad and Quirk, 2002). While it was assumed that these findings were a product of enhanced synaptic transmission of pyramidal cells, this had not been tested directly in vivo. In a recent study it was shown that optogenetically activating IL projection neurons during extinction reduces fear expression and enhances extinction recall the next day, in the absence of optical stimulation (Do-Monte et al., 2015). Silencing the same neuronal population during extinction had no within-session effect, but impaired retrieval the following day, consistent with the idea that IL activation during extinction learning predicts the extent of retrieval. Curiously, optogenetically inhibiting IL during extinction retrieval had no behavioral effect (Do-Monte et al., 2015), in contrast with what the findings of Milad and Quirk (2002) would predict.

A similar study, examining the pathway specificity of this effect has found evidence in support of the idea that IL signaling is important for the formation, but not the recall of extinction memories (Bukalo et al., 2015). In this study, the authors selectively expressed either the excitatory opsin (ChR2) or inhibitory opsin (ArchT) in glutamatergic vmPFC neurons (restricted primarily to IL). Optogenetic activation of vmPFCamygdala projecting neurons during a ''partial'' extinction session (10 CS alone trials) was sufficient to promote long-term facilitation of extinction learning, yielding low levels of freezing the following day in the absence of optogenetic stimulation. In contrast, inhibiting this pathway during extinction training yielded long-term deficits in extinction memory formation, providing evidence that activation of the vmPFC→BLA pathway is a necessary component underlying extinction. Interestingly, optogenetic activation or inhibition of this pathway during extinction retrieval did not alter freezing behavior relative to controls, suggesting that vmPFC afferents in the amygdala do not regulate memory retrieval (Bukalo et al., 2015). It is worth noting that in both of these studies (Bukalo et al., 2015; Do-Monte et al., 2015), the retrieval tests were conducted with very few (4–5) test trials. This test procedure would be expected to yield substantial spontaneous recovery and limit IL engagement. It is possible that inhibiting IL or its BLA afferents over a longer (multi-trial) test session would reveal an effect of vmPFC inactivation on extinction retrieval.

A key question of interest that can be addressed with viral technologies lies with the ability to selectively target and modulate neuronal subtypes based on protein expression. Parsing the role of genetically defined interneurons can inform us about local modulatory mechanisms and how this impacts the extended fear network. For example, optogenetic inhibition of dmPFC (encompassing ACC/PL) PVINs causally initiated freezing behavior in unconditioned animals and also modulated fear expression in previously conditioned animals (Courtin et al., 2014). These interneurons can be further subdivided into fast-spiking and non-fast spiking interneurons based on firing rate properties. Fast-spiking PVINs target the perisomatic region of pyramidal cells, thereby dictating the timing and synchronization of action potentials (Cobb et al., 1995; Freund and Katona, 2007). Thus, inhibiting dmPFC PVINs can disinhibit and synchronize the firing of projection neurons. This synchronization is crucial to regulating timely and efficient transmission of information to drive the appropriate behavioral response. These data indicate a key role for PVINs in determining freezing behavior by disinhibiting dmPFC (Courtin et al., 2014). It is unknown, however, if this mechanism is specific to dmPFC regulating conditioning and fear recall. For instance, would activating these neurons induce renewal in an extinction context? A second question to address lies at the circuit level: what influences the state of dmPFC PVINs? Gabbott et al. (2006) have demonstrated that BLA output monosynaptically innervates mPFC PVINs—could this effect be driven by feed-forward disinhibition from amygdala projections? Additionally, ventral hippocampal projections also alter firing patterns of putative mPFC interneurons (Sotres-Bayon et al., 2012), so perhaps amygdala and ventral hippocampal projections to mPFC act to synchronously disinhibit PL output. Alternatively, is direct optical activation of PL pyramidal cells sufficient to induce freezing behavior and is this local modulatory mechanism conserved between brain regions? For example, would disinhibiting IL pyramidal cells induce locomotor behavior? While currently unknown, optogenetics provide the ability to answer such questions by controlling neural activity in a cell and circuit specific manner.

Chemogenetic technology is also beginning to contribute to our understanding of mPFC physiology. By expressing an excitatory DREADD virus in dmPFC (encompassing ACC/PL), Yau and McNally (2015) have recently shown that increased activation of this region is causally involved in prediction error. In fear conditioning, animals must use information from the past to predict the meaning of a CS. If the animal expects the US to be delivered and it is not, this produces a large prediction error. Using a blocking design in which animals are trained to fear one CS and then later given compound training (CS1 and a novel CS2), learning about CS2 will be blocked under normal conditions. However, dmPFC activation with a virus infecting all cell types or a virus restricted to pyramidal neurons was sufficient to promote learned fear to the second CS. Thus, dmPFC activation promotes the acquisition of conditioned fear under circumstances where learning would not otherwise occur. Importantly, this was not simply due to increased fear expression independent of learning (Yau and McNally, 2015). Given the results discussed above, it is somewhat surprising that this manipulation alone did not induce freezing behavior. If disinhibiting dmPFC optogenetically was sufficient to increase freezing, then directly activating it should have an even greater effect. This may be due to differences in the level of viral expression at the time of testing or to differences in activating neuronal activity directly through ion channels vs. G-protein coupled receptors. In summary, optogenetic and chemogenetic technologies have only begun to add to our understanding of the role of mPFC in the fear circuitry, and are primed to contribute further.

# NEUROIMAGING AND HUMAN HOMOLOGS

The neural circuits underlying fear conditioning and extinction in rats have also been identified in humans. For example, the dorsal anterior cingulate cortex (dACC) and the ventromedial prefrontal cortex (vmPFC) have been proposed to regulate the expression and suppression of fear in humans, respectively. While the temporal and spatial resolution of neuroimaging techniques cannot provide fine anatomical details for cross species comparison, they have provided a broad look at the human fear circuit and insight into PTSD. Using functional imaging with a standard fear conditioning paradigm, Phelps et al. (2004) reported activation of the vmPFC that corresponded with the expression of fear during extinction learning. Interestingly, individuals with PTSD often display decreased mPFC blood flow upon recalling a traumatic experience which likely disrupts extinction learning (Semple et al., 1996; Bremner et al., 1999; Shin et al., 1999). In humans, vmPFC has an inhibitory influence over the amygdala similar to that in rodents (Delgado et al., 2008). The vmPFC-amygdala pathway may be dysregulated in some cases of PTSD (Gilboa et al., 2004; Garfinkel et al., 2014) and patients with bilateral vmPFC damage present heightened amygdala activation to aversive images (Motzkin et al., 2015). Thus, vmPFC regulation of amygdalar output may be a common circuit underlying fear extinction.

Another possibility is that those who suffer from PTSD fail to use contextual cues to appropriately guide behavioral responding, resulting in a greater degree of generalized fear (Maren et al., 2013; Garfinkel et al., 2014). This is more likely mediated by vmPFC-hippocampal networks and indeed, individuals with PTSD often have decreased hippocampal volume (van Rooij et al., 2015). Studies in healthy volunteers show that vmPFC-hippocampal activation correlates with extinction success and that this activation is context dependent (Kalisch et al., 2006; Milad et al., 2007b). This network displays diminished activity in PTSD patients, further contributing to extinction deficits (Milad et al., 2009). Structural studies have shown that cortical thickness of vmPFC correlates with the degree of extinction retention in healthy individuals (Milad et al., 2005), providing evidence that neural mechanisms of extinction may be conserved across species, although this has not been replicated in a related study (Hartley et al., 2011). It is unclear if these potential structural differences precede the development of PTSD or if they are a consequence of the traumatic experience. A recent study suggests the former in that combat-exposed veterans who did not develop PTSD showed no differences in hippocampal volume compared to healthy controls (van Rooij et al., 2015). In summary, dysregulated vmPFC activity may be a common biomarker of fear and disrupted extinction learning across species.

The dACC has received considerable attention for regulating fear expression. In healthy subjects, cortical thickness of dACC is positively correlated with skin conductance responses during fear conditioning and this brain region is activated by a CS (Milad et al., 2007a). Interestingly, in a separate study, during extinction training, amygdala metabolism positively predicted vmPFC activation while negatively predicting dACC activation, and resting dACC metabolism predicted fear expression (Linnman et al., 2012a,b). dACC-amygdala networks have also been reported during fear memory consolidation (Feng et al., 2013, 2014) and dACC shows sustained activity increases when shock delivery was expected (Linnman et al., 2012b). Thus, dACC signaling may correspond to ongoing fear responses and it has been shown that PTSD patients display a greater activation of dACC during extinction recall (Milad et al., 2009). This hyperactivity was larger in men with PTSD, implicating the mPFC in sex differences underlying the disorder (Shvil et al., 2014). Overall, there is a growing body of evidence supporting distinct roles within the mPFC regulating emotional learning and memory in humans. However, many of these brain imaging studies do not directly report data comparing vmPFC and dACC, leaving the possibility of covariation of these two brain regions virtually unexplored at the level of human fear conditioning and PTSD.

# PARALLELS WITH REWARD AND DRUG SEEKING BEHAVIOR

Given the recent challenges to the precise role of the mPFC in fear, it is worth turning to the appetitive literature to draw parallels and perhaps provide a more integrated view on mPFC function. In both food- and drug-motivated instrumental tasks, the PL and IL have been posited to play different roles in conditional responding (Peters et al., 2009). Specifically, the PL has been posited to drive drug seeking behavior (McFarland and Kalivas, 2001; Capriles et al., 2002), whereas the IL may suppress conditional responding after extinction (Peters et al., 2008; Moorman et al., 2014). In other words, the PL is believed to be required for the execution of goal-directed behavior (''go''), whereas the IL is believed to regulate behavioral inhibition (''stop''). This view of medial prefrontal cortical function in appetitive instrumental conditioning paradigms has considerable homology with the canonical view of mPFC function in the fear conditioning and extinction (Peters et al., 2009).

In addition to regulating goal seeking and response inhibition, the PL and IL appear to regulate different forms of instrumental responding over the course of conditioning. During instrumental conditioning, performance early in training typically reflects goal-directed behavior (i.e., actions), but this shifts to outcomeindependent (e.g., habitual) performance after extended training. Interestingly, rats with PL lesions exhibit habitual responding that is insensitive to outcome value both early and late in training, whereas rats with IL lesions exhibit goal-directed responding even after extended training (Killcross and Coutureau, 2003). These data suggest that PL promotes flexible, goal-directed responding, whereas the IL inhibits flexibility and promotes behavioral rigidity and perseveration. In line with this idea, IL inactivation reinstates goal-directed responding in rats with extensive training and reduces habitual responding in a responseconflict task (Coutureau and Killcross, 2003; Haddon and Killcross, 2011).

However, recent evidence has surfaced that challenges the canonical view in which PL and IL serve opposing functions for reward/drug seeking behavior (Moorman et al., 2014). For example, there is emerging evidence that PL lesions or inactivation have no effect on reward seeking (Weissenborn et al., 1997; Capriles et al., 2002), and several investigators have shown that PL may serve an inhibitory role in reward/drug seeking under some conditions (Ishikawa et al., 2008; Jonkman et al., 2009; Hayton et al., 2010, 2011; Mihindou et al., 2013; Martín-García et al., 2014). For instance, cocaine self-administration decreases PL pyramidal cell excitability and optogentically activating PL pyramidal cells reduces drug seeking behavior, whereas optical inhibition of this same population of cells increases this behavior (Chen et al., 2013).

Similarly, conflicting results regarding the precise function of IL have also surfaced. IL inactivation has been shown to decrease the maintenance of responding as well as reinstatement of lever pressing for cocaine (Di Ciano et al., 2007; Pelloux et al., 2013; Vassoler et al., 2013). In addition, it has recently been shown that the vmPFC (encompassing IL) plays a role in the expression of cocaine seeking behavior (Koya et al., 2009), a role previously thought to rely primarily of PL signaling. The fact that IL can both activate (Koya et al., 2009) and inhibit (Peters et al., 2008) drug seeking behavior suggests a more complex role for the mPFC, which is not yet fully appreciated. In support of this, recent work has shown that the vmPFC plays a time-dependent role in both the expression and extinction of cocaine seeking (van den Oever et al., 2013). Moreover, a recent study that recorded single-unit activity in PL and IL found cue-evoked activity in both areas during reward seeking and extinction. The authors show that neurons in both areas encoded contextually appropriate behavior (initiation during reward seeking vs. withholding during extinction), suggesting that PL and IL integrate contextual information to regulate behavior, rather than opposing each other to encode go vs. nogo behaviors (Moorman and Aston-Jones, 2015). Despite similar response properties, it remains possible that PL and IL signaling may be coupled to different response outcomes regarding goaldirected vs. habitual behavior. This may partially explain the tendency of PL and IL neural activity to covary, but lesion and inactivation studies suggest some functional bias. Overall, these recent findings support the idea that cell populations within both PL and IL can serve to either activate or inhibit drug seeking behavior and suggest a more complicated interplay of PL and IL than previously thought.

One interesting point about the possibility of overlapping circuits for fear and addiction is the striking difference in behavior that has been suggested to be controlled by PL and IL. In fear, PL activation is thought to underlie fear expression, and in drug seeking PL is thought to encode the expression of drug seeking activity. The nature of these behaviors is quite different. That is, in a high fear state animals exhibit robust freezing (inhibition of movement) whereas the expression of drug seeking behavior corresponds to a rapid activation of movement. However, the associative structure and psychological processes underlying these behaviors may be similar. It has been shown that ''sign-trackers'' (rats who approach a food predictive cue) also show increased auditory fear (compared to context fear), suggesting that these animals are ''cue-directed'' (Morrow et al., 2015). These data suggest that overlapping circuits may be engaged independent of the behavioral outcome. In summary, emerging evidence suggests a more complex role for the mPFC in reward/drug seeking behavior, similar to that in fear, insofar as it remains possible that distinct subpopulations exist within both PL and IL that subserve similar function to either promote or inhibit behavior, which is likely biased by context. It seems unlikely that an entire region of PFC would be necessary for any given function; rather neuronal populations within the mPFC may ultimately underlie a particular behavior through similar afferent and efferent connections.

# CONCLUSIONS

Overall, the majority of the work summarized above has focused on a division of labor within mPFC, where its subregions work largely independently to bidirectionally regulate fear output. These mechanisms appear to be conserved across species. In particular, the canonical view has been that dorsal regions (PL/dACC) of mPFC regulate fear expression and ventral regions (IL/vmPFC) fear suppression. However, findings from recent studies challenge the underlying assumptions of this model. For example, a number of recent anatomical and electrophysiological studies have shown that PL and IL project similarly to the amygdala (Gutman et al., 2012; Pinard et al., 2012; Cho et al., 2013; Hübner et al., 2014) and that neuronal activity (IEG, LFPs, single-units) in IL and PL covary during the conditioning and extinction of fear (Morrow et al., 1999; Baeg et al., 2001; Frankland et al., 2004; Herry and Mons, 2004; Kim et al., 2010; Holmes et al., 2012; Fitzgerald et al., 2014, 2015a; Halladay and Blair, 2015). Moreover, there are conditions under which IL and PL activity show functionally dichotomous activity patterns during the expression or suppression of conditioned fear, but in a direction opposite to that predicted by the canonical model (Chang et al., 2010).

However, even when IL and PL activity covary, it remains possible that the downstream effect of this activity is functionally opposed due to the different efferent targets of each area. Moreover, PL and IL have known structural and functional interactions with each other (Hoover and Vertes, 2007; van Aerde et al., 2008; Ji and Neugebauer, 2012; Little and Carter, 2012, 2013) and these interactions may bias the output of either area despite similar engagement of both regions in a particular task. Another possibility that has been largely unexplored is that distinct neuronal populations within PL or IL may show

# REFERENCES


functional redundancy, where some neurons within each area modulate fear output differentially (e.g., Halladay and Blair, 2015). Given the similar connectivity of PL and IL, this possibility cannot be excluded.

While a wealth of research has explored the role of the mPFC in fear, it is clear that the precise contributions and function of the IL and PL in fear conditioning and extinction are not yet fully understood. Additional experiments coupling electrophysiology with cell and circuit specific techniques are primed to further delineate the complex roles of PL and IL within the fear circuit. A more sophisticated approach looking at simultaneously recorded single-units and oscillatory processes in PL and IL may help to better parse the expanding role of the mPFC in fear. Furthermore, an advanced understanding of the functional input and output patterns of PL and IL can help disambiguate many of the discrepant results. It is likely that PL and IL serve to integrate contextual information to inform behavioral responding and that context greatly impacts the response properties of these two regions, as well as the complexity of the tasks, with more complex tasks requiring greater cortical input. Continued work will likely shed light on unresolved issues, providing translational value for the treatment of trauma-related disorders such as PTSD. An enhanced understanding of the fear circuit at the level of rodents and humans may provide novel insight to improve current therapeutic outcomes and dampen inappropriate fear responding.

#### AUTHOR CONTRIBUTIONS

TFG and SM wrote and edited the manuscript.

#### FUNDING

Supported by a grant from the National Institutes of Health (R01MH065961) and a McKnight Memory and Cognitive Disorders Award to SM.


based on bacterial artificial chromosomes. Nature 425, 917–925. doi: 10. 1038/nature02033


contextual stimuli. Behav. Neurosci. 111, 683–691. doi: 10.1037/0735-7044.111. 4.683


rat: an anterograde tract-tracing study with phaseolus vulgaris leucoagglutinin. J. Comp. Neurol. 290, 213–242. doi: 10.1002/cne.902900205


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Giustino and Maren. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Involvement of Dopamine D1/D5 and D2 Receptors in Context-Dependent Extinction Learning and Memory Reinstatement

#### Marion Agnès Emma André1,2 and Denise Manahan-Vaughan1,2 \*

<sup>1</sup> Medical Faculty, Department of Neurophysiology, Ruhr University Bochum, Bochum, Germany, <sup>2</sup> International Graduate School for Neuroscience, Ruhr University Bochum, Bochum, Germany

Dopamine contributes to the regulation of higher order information processing and executive control. It is important for memory consolidation processes, and for the adaptation of learned responses based on experience. In line with this, under aversive learning conditions, application of dopamine receptor antagonists prior to extinction result in enhanced memory reinstatement. Here, we investigated the contribution of the dopaminergic system to extinction and memory reinstatement (renewal) of an appetitive spatial learning task in rodents. Rats were trained for 3 days in a T-maze (context "A") to associate a goal arm with a food reward, despite low reward probability (acquisition phase). On day 4, extinction learning (unrewarded) occurred, that was reinforced by a context change ("B"). On day 5, re-exposure to the (unrewarded) "A" context took place (renewal of context "A", followed by extinction of context "A"). In control animals, significant extinction occurred on day 4, that was followed by an initial memory reinstatement (renewal) on day 5, that was, in turn, succeeded by extinction of renewal. Intracerebral treatment with a D1/D5-receptor antagonist prior to the extinction trials, elicited a potent enhancement of extinction in context "B". By contrast, a D1/D5-agonist impaired renewal in context "A". Extinction in the "A" context on day 5 was unaffected by the D1/D5-ligands. Treatment with a D2-receptor antagonist prior to extinction had no overall effect on extinction in context "B" or renewal in context "A", although extinction of the renewal effect was impaired on day 5, compared to controls. Taken together, these data suggest that dopamine acting on the D1/D5-receptor modulates both acquisition and consolidation of context-dependent extinction. By contrast, the D2-receptor may contribute to context-independent aspects of this kind of extinction learning.

#### Edited by:

John D. Salamone, University of Connecticut, USA

#### Reviewed by:

Martín Cammarota, Federal University of Rio Grande do Norte, Brazil John J. Wagner, University of Georgia, USA

#### \*Correspondence:

Denise Manahan-Vaughan denise.manahan-vaughan@rub.de

Received: 25 June 2015 Accepted: 24 December 2015 Published: 21 January 2016

#### Citation:

André MAE and Manahan-Vaughan D (2016) Involvement of Dopamine D1/D5 and D2 Receptors in Context-Dependent Extinction Learning and Memory Reinstatement. Front. Behav. Neurosci. 9:372. doi: 10.3389/fnbeh.2015.00372 Keywords: extinction learning, dopamine, rodent, spatial learning, hippocampus, behavior

# INTRODUCTION

During extinction learning, conditioned responses become diminished during exposure to the conditioned stimulus (CS) in the absence of the unconditioned stimulus (US; Bouton, 2004; Myers and Davis, 2007). Extinction learning does not eliminate or erase the original memory, but rather mediates the creation of a new representation that allows the animal to ignore its behavioral responses to the previously learned conditioned stimuli (Rescorla, 2001). This process is reinforced by a change of context (Bouton, 2004), even under non-aversive (appetitive) conditions (Wiescholleck et al., 2014; André et al., 2015a,b). Reinstatement, or renewal, of the original conditioned response is typically reactivated upon reexposure to the CS in the original context, or to conditions that are sufficiently dissimilar to the extinction context (Rachman, 1989; Bouton, 2004; Craske et al., 2008). Neuromodulators such as dopamine play a crucial role in memory processes and regulate synaptic information storage mechanisms such as synaptic plasticity (Hansen and Manahan-Vaughan, 2014). Dopamine is particularly important for the emotional weighting of experiences, but also for memory consolidation (Huang and Kandel, 1995; Bissière et al., 2003; Sajikumar and Frey, 2004; Lisman et al., 2011). It regulates cue-dependent fear conditioning (Fadok et al., 2010), and the consolidation of extinction of fear memory (Holtzman-Assif et al., 2010). This may be related to the role of the dopaminergic system in processing prediction errors as a component of associative learning (Schultz, 2006), or to the role of dopamine in reinforcing encoding of aversive experience. Less is known about the role of dopamine in appetitive context-dependent extinction learning processes that are supported by the hippocampus, and the precise role of dopamine receptor subtypes in this phenomenon is unclear.

The vast majority of studies on the role of dopamine in extinction and renewal have been conducted with regard to fear extinction (Abraham et al., 2014). Where appetitive processes have been explored, the focus has been on addiction (Di Chiara, 2002), rather than extinction of more benign appetitive processes such as the coupling of food-seeking behavior to specific non-aversive contexts. In the areas of fear memory and drug addiction, it is believed that the dopamine reward circuitry influences the encoding of the original aversive or appetitive experience (Lauzon et al., 2013) and extinction learning with regard to these experiences (Schultz and Dickinson, 2000). Strikingly, infusion of Levodopa (L-DOPA) a dopamine precursor, strongly promotes extinction regardless of the context and prevents fear memory from re-emerging (Haaker et al., 2013).

Recently, we reported that neurotransmitter receptor manipulations that are known to directly influence hippocampal synaptic plasticity and hippocampus-dependent learning, also modulate context-dependent extinction learning (André et al., 2015a,b). It has also been shown that the hippocampus contributes to context-dependent extinction learning and renewal of fear memory (Good and Honey, 1991; Ji and Maren, 2005; Hobin et al., 2006; de Carvalho Myskiw et al., 2014; Portugal et al., 2014; Tan et al., 2014). Dopamine receptors are expressed throughout the brain within regions that are key for the encoding and retrieval of long-term memory, such as the hippocampus, as well as in reward circuitry structures (Mansour and Watson, 1995). Whereas dopamine D1/D5-receptors are critically required for multiple forms of hippocampal synaptic plasticity, D2-receptors appear to contribute less to hippocampal plasticity processes, serving rather, to regulate hippocampal basal excitability tonus (Hansen and Manahan-Vaughan, 2014). Both dopamine D1/D5 (Hikind and Maroun, 2008) and dopamine D2-receptors (Mueller et al., 2010) have been implicated in extinction learning, however. Whereas dopamine D2-receptors positively couple to adenylyl cyclase, dopamine D1/D5-receptors are negatively coupled to this enzyme (Hansen and Manahan-Vaughan, 2014). Intuitively, one would expect that this means that dopamine D1/D5 and D2-receptors mediate opposing excitatory and inhibitory cellular responses, but whether this occurs or not depends on the relative activation of these receptors in specific brain regions, and the kind of associative learning event to be stored or retrieved. Evidence exists that dopamine D1/D5-receptors support fear acquisition and extinction (Inoue et al., 2000; El-Ghundi et al., 2001). Whether dopamine D2-receptors support these processes is less clear. Transgenic mice that lack dopamine D2-receptors exhibit a normal fear-potentiated startle response (Fadok et al., 2010). Others have shown that fear extinction is impaired (Holtzman-Assif et al., 2010; Mueller et al., 2010) or enhanced (Ponnusamy et al., 2005) by D2-receptor antagonists. The role of dopamine D1/D5 and D2-receptors in non-aversive appetitive extinction learning is also unclear.

In this study, we explored the role of dopamine D1/D5 and dopamine D2-receptors in extinction and renewal of a context-dependent appetitive spatial learning task. We observed that whereas dopamine D1/D5-receptor manipulation altered context-dependent extinction learning, dopamine D2-receptor manipulation affected context-independent aspects of this form of extinction learning. These data suggest, that with regard to appetitive experience, a differentiation may exist as to the contribution of dopamine D1/D5 and dopamine D2-receptors to key components of extinction learning that is supported by a context-change.

# MATERIALS AND METHODS

The present study was carried out in accordance with the European Communities Council Directive of September 22nd, 2010 (2010/63/EU) for care of laboratory animals. All experiments were performed according to the guidelines of the German Animal Protection Law and were approved by the North Rhine-Westphalia State Authority (Bezirksamt, Arnsberg). All efforts were made to reduce the number of animals used.

# Animals

Male Wistar rats (7–8 weeks old) underwent implantation of guide cannulae, whilst under anesthesia (52 mg/kg sodium pentobarbital via intraperitoneal (i.p.) injection), as described previously (Manahan-Vaughan, 1997). One cannula was implanted into the lateral cerebral ventricle of each hemisphere (0.5 mm posterior to bregma, 1.6 mm lateral to the midline; size: 5.6 mm length, 0.8 mm diameter, 4.5 mm depth).

Animals were allowed 2 weeks to recover, before any behavioral experiment took place. They were housed singly and maintained on a 12 h light/12 h dark cycle with food and water ad libitum.

Two days prior to behavioral training, animal weight was determined and food availability was reduced to achieve 85% of this predetermined body weight. The animal's weight was subsequently maintained at this level until the end of the experiment. Before beginning the experiment, animals were handled individually for 20 min per day.

# T-Maze and Extinction Task

Experiments were conducted in a T-maze that was composed of a starting box (25 × 20 cm) that was separated from the main corridor (100 × 20 cm) by a sliding door and two side corridors (40 × 10 cm) positioned perpendicular to the other end of the main corridor. The maze design and the protocol followed was as described previously (Wiescholleck et al., 2014; André et al., 2015a,b). The context of the maze was changed by exchanging the plastic floor of the maze (zebra stripes, checkered patterns, or geometric lines), odor cues that were placed at the end of the goal arms, and exchanging the extra-maze cue cards that were placed 40 cm above the end of the main corridor (Wiescholleck et al., 2014).

Every day, rats engaged in a learning session that comprised 20 consecutive trials, that were split into two data blocks (1st ten, 2nd ten trials), for analysis purposes (see below, and Wiescholleck et al., 2014; André et al., 2015a,b). The trial commenced when the door to the starting box was opened and the animal entered the maze. It ended when the animal entered a goal arm of the T-maze or when a predetermined time-limit (30 s to 2 min) had elapsed without arm entry (see below). Animals learned to search for a food pellet (Dustless Precision Pellets 45 mg, BioServ, USA) that was placed at the end of a predetermined goal arm. From day 1 through three reward probability was decreased from 100 to 25%. In conjunction with this, the time allowed to reach the arm was decreased in a stepwise manner from 2 min to 30 s. Learning criterion was reached when the animal successfully entered the correct arm on 8 of the last 10 trials of a 20 trial run. Failure to reach criterion by day 3 resulted in exclusion if the animal from subsequent trials (days 4 and 5). Its data from days 1–3 were not integrated into the data analysis for the study.

On day 4, extinction learning was assessed, whereby the animals explored the T-maze for 20 trials, during which time no reward was given (absence of the US). Here, the context was changed (novel floor, novel odors, novel cue cards). On day 5, renewal (RN) was assessed by re-introducing the animal to the original T-maze context (context ''A'') for 20 trials with no food reward. Typically, animals respond to re-exposure to the ''A'' context by showing renewal in the 1st 10 trials followed by extinction in the 2nd set of 10 trials (resulting from the realization that no food reward is provided; Wiescholleck et al., 2014).

## Analysis of Decision Time

To assess choice confidence we measured the time taken by the animal to move from the departure area in the T-Maze to its arm of choice (Wiescholleck et al., 2014). As the confidence of the animal increases during the acquisition of the task, decision-time declines (Luce, 1986; Avila and Lin, 2014; Wiescholleck et al., 2014). We assessed this for every choice (not just correct choices) in order to determine the confidence of the animal in knowing which arm to enter.

# Pharmacological Treatment

All compounds were applied via a cannula that had been implanted into the lateral cerebral ventricle (see ''Animals'' Section). The D1/D5-receptor antagonist SCH 23390 (Tocris, Ellisville, MO, USA) was applied at a dose of 5.94 µg/µl. The D1/D5-receptor agonist, Chloro-PB (Sigma Aldrich St.Louis, MO, USA), was given at a dose of 8.33 µg/µl. The D2-like receptor antagonist, (S)-(–)-3-bromo-N-[(1-ethyl-2 pyrrolidinyl)methyl]-2,6-dimethoxybenzamide (remoxipride), (Tocris, Ellisville, MO, USA), was administered at a dose of 10 µg/µl. These doses were chosen because they are effective in preventing hippocampal synaptic plasticity (Kulla and Manahan-Vaughan, 2000; Manahan-Vaughan and Kulla, 2003; Lemon and Manahan-Vaughan, 2006; Wiescholleck and Manahan-Vaughan, 2014). All compounds were dissolved in double-distilled water and given in an injection volume of 5 µl. Drugs were applied via the guide cannula at a rate of 1 µl/min and given 30 min prior to the commencement of the extinction learning trials on day 4.

At the doses used, the compounds elicited no general changes in behavioral state, such as state-dependent effects. These properties had been assessed as part of previous studies (Kulla and Manahan-Vaughan, 2000; Manahan-Vaughan and Kulla, 2003; Lemon and Manahan-Vaughan, 2006). To additionally verify this, we assessed locomotion (in m/s) from the time of exit from the start box to the end of the main arm (100 cm) for all trials of each animal on day 4, after treatment with a dopamine ligand or vehicle. In addition we assessed stereotypy in the form of head-weaving (total number) for the entire duration of all 20 trials on day 4.

# Data Analysis

Correct answers were defined as trials in which the animal moved directly to the predetermined goal arm. For analysis purposes, each 20 trial session was divided into two sets of 10 trials (first 10 and last 10 trials). The time taken to reach the end of the first arm visited was calculated for each trial. To analyze decision time, the time required to move from the departure box in the T-Maze to the first chosen arm was recorded for each trial, and data were segregated into four sets of five trials for each day, of which the times were averaged (Wiescholleck et al., 2014). Extinction learning effects were assessed by comparing animal performance during the first, or second, set of trials on day 4 with performance during the second set of trials on day 3. Renewal effects were assessed by comparing animal performance during the first set of trials on day 5 with performance during the second set of trials on day 4. To examine if renewal performance was equivalent to learning performance at the end of the acquisition training (extinction efficacy), animal performance during the first set of trials on day 5 with performance during the second set of trials on day 3.

Data were analyzed using analysis of variance (ANOVA) with repeated-measures including two within-subject factors (Day and Session) and two between-group factors (Treatment and Experimental Design) to assess for differences between control and propranolol-treated animals. Differences between trial blocks or between trials days of a specific group (control or ligand-treated animals) were assessed using Bonferroni post hoc tests. Except where ''ANOVA'' is mentioned explicitly, all p values in the results section correspond to values determined from the Bonferroni test. The level of significance was set at p < 0.05.

# RESULTS

# Context-Dependent Extinction is Enhanced by Antagonism of Dopamine D1/D5-Receptors. Renewal is Unaffected

In the first 3 days of acquisition training, the animals successfully met the learning criterion. Thus, by the last 10 trials of day 3, animals made at least 8 out of 10 possible correct goal arm choices despite the reward probability having been reduced to 25% at this stage of acquisition training. A significant increase in correct choices was apparent between day 1 and day 2 (**Figure 1**; within-subject ANOVA: for animals subsequently treated with vehicle, F(1,6) = 14.427; p = 0.009, n = 7; for animals subsequently treated with a dopamine D1 agonist, F(1,7) = 9.215; p = 0.019, n = 8).

No significant difference was evident in performance within the first and second 10 trial blocks on day 3, signifying that the learning criterion had been achieved in both animal cohorts (**Figure 1**, p = 0.324). No significant difference in the animals' learning behavior was found when the two animals cohorts were compared on days 1, 2 or 3. (Between-subject ANOVA: F(1,13) = 0.029; p = 0.868).

Thirty minutes before commencing the extinction learning trials on day 4, either vehicle, or the dopamine D1/D5-receptor antagonist, SCH 23390 was applied. To facilitate extinction, the context of the environment was altered (context ''B'': see ''Materials and Methods'' Section, and Wiescholleck et al., 2014). In vehicle-treated control animals, a significant attrition of correct choices became apparent that was significant in the last 10 trials of this session, when compared to the last 10 trials on day 3 (p < 0.001; **Figure 1**). Within-subject ANOVA confirmed that between day 3 and day 4, significant extinction learning occurred in vehicle-treated animals (F(1,6) = 44.824; p < 0.001).

FIGURE 1 | Antagonism of dopamine D1/D5-receptors enhances extinction, but does not affect renewal. Agonist activation of dopamine D1/D5-receptors has no effect on context–dependent extinction, but impairs renewal. Animals participated in 20 trials per day. Bar charts represent the number of correct arm choices in the first and second set of 10 trials on each test day. Three days of acquisition training (day 1–5), in context "A" were followed by extinction learning in a new context (day 4, context "B") and re-exposure to the original context (context "A") on day 5. Extinction of the learned conditioned stimulus (CS)-unconditioned stimulus (US) response occurred in the "A" context in control animals on day 5 (second 10 trials). No food was available on days 4 and 5. The arrow signifies the time of antagonist/vehicle-injection. The vehicle data for the antagonist group are labelled as "SCH 23390 vehicle" (yellow bars) and for the agonist group are labelled as "ChloroPB vehicle" (blue bars) Treatment of the animals with the dopamine D1/D5-receptor antagonist, SCH 23390 (dark gray bars), prior to the extinction learning trials on day 4 resulted in a significant enhancement of extinction (in the "B" context) compared to vehicle-treated controls (yellow bars). On day 5, renewal in context "A" was equivalent in both treatment groups (first 10 trials). Extinction of the CS-US response that had been learned in context "A" (2nd set of trials on day 5) was also equivalent in both treatment groups. Animals treated with the dopamine D1/D5-receptor agonist Chloro-PB (white bars) prior to exposure to the "B" context on day 4 showed significant extinction was evident by the 2nd set of 10 trials on day 4, that was not different from controls (blue bars). Upon returning to the same context on day 5, renewal of the conditioned behavior occurred in control animals (first 10 trials), whereas renewal was impaired in animals that had been treated on day 4 with the agonist. Extinction of the CS-US response that had been learned in context "A" (2nd set of trials on day 5) was equivalent in both treatment groups.

Treatment of the animals with the dopamine D1/D5 antagonist, 30 min prior to the extinction trials, significantly accelerated extinction (compared to controls, p < 0.001) and resulted in a better extinction effect overall (F(1,7) = 124.096; p < 0.001; **Figure 1**).

On day 5, the animals were re-exposed to the context in which they had undergone acquisition training on days 1–3 (context ''A''), with the exception that no food reward was available. Control animals and animals that had previously been treated with the dopamine D1/D5-antagonist responded immediately with renewal of the learned behavior (comparison of first 10 trials on day 5 with last 10 trials on day 4: F(1,13) = 64.594; p < 0.001; **Figure 1**). During the last 10 trials of day 5, a significant deterioration of correct arm choices became apparent both animal groups (p < 0.001; **Figure 1**). This corresponds to extinction of the behavior learned in context ''A'', as the animals realize that no reward can be expected. The profile of renewal and extinction in context ''A'' on day 5 was equivalent in vehicle-treated and antagonist-treated animals (F(1,13) = 0.343; p = 0.568). These data suggest that the D1/D5 receptor may modulate contextdependent extinction. To clarify this, we examined the effects of agonist activation of D1/D5 receptors prior to extinction learning.

# Context-Dependent Extinction is not Affected by Agonism of Dopamine D1/D5-Receptors. Renewal is Impaired

Strikingly, animals that had been exposed to the dopamine D1/D5-receptor agonist, Chloro-PB (n = 8), exhibited extinction learning on day 4 (F(1,17) = 13.68; p = 0.002: all trials day 4 vs. last 10 trials on day 3) that was equivalent to controls (n = 7; F(1,17) = 0.646; p = 0.432; **Figure 1**). The treatment group showed impaired renewal on day 5, however (**Figure 1**). Here, the number of correct arm choices in the first 10 trials of day 5 was significantly fewer than during the last 10 trials of day 3 (F(1,9) = 24.511; p < 0.001). In fact, performance was at the same level that had been apparent following successful extinction learning in these animals on day 4 (F(1,9) = 2.295; p = 0.164, comparison of first 10 trials on day 5 with last 10 trials on day 4). No further deterioration of performance levels occurred during the second 10 trials on day 5 (F(1,9) = 0.474; p = 0.509). Overall, a significant difference in choice behavior on days 4 and 5 was found when performance in vehicle-treated animals was compared with agonist-treated animals (F(1,9) = 34.211; p < 0.01: all trials, day 4 vs. all trials, day 5).

# Context-Dependent Extinction and Renewal are Unaffected by Antagonism of Dopamine D2 Receptors. Context-Independent Extinction is Impaired

We then tested the effects of a dopamine D2 receptor antagonist on context-dependent extinction learning (**Figure 2**). Animals were treated with remoxipride 30 min before starting

learning trials on day 4 had no effect on extinction learning compared to vehicle-treated controls. Both groups exhibited significant extinction in the second set of 10 trials on day 4. On day 5, renewal in context "A" was equivalent in both treatment groups (first 10 trials). Extinction of the CS-US response that had been learned in context "A" (2nd set of trials on day 5) was impaired in the remoxipride-treatment group however.

the trials on day 4. Here also, we first verified that the animal cohorts that were subsequently treated with vehicle (n = 9) or remoxipride (n = 10) exhibited an equivalent learning performance during the acquisition days 1–3 (**Figure 2**; F(1,16) = 1.441 ; p = 0.247). On day 4, following a change of T-maze context (context ''B'') we assessed extinction learning. Here, although extinction was slightly better in the first 10 trials of day 4 in remoxipride-treated animals, overall no effect on animal behavior was apparent when performance in control and antagonist-treated animals was compared for the first and second trial blocks on day 4 (**Figure 2**; F(1,17) = 0.646; p = 0.432). When the animals were returned to context ''A'' on day 5 no difference in their renewal performance was apparent, either (**Figure 2**; F(1.17) = 0.284; p = 0.601, between-subject comparison of first 10 trials on day 5).

However, when performance within the antagonist-treated animals was assessed, a significant increase towards extinctionresistance in context ''A'' was observed (2nd trial block on day 5). Thus, extinction in context ''A'' was significantly poorer than that seen in vehicle-treated animals (**Figure 2**; F(1,17) = 6.608; p = 0.02, between-subject comparison of last 10 trials on day 5).

# Antagonism of Dopamine D1/D5-Receptors Increases Decision-Time During Context-Dependent Extinction

We have reported in the past that a gradual improvement in time to enter the first arm becomes evident as the animals acquire the task and become more confident as to the arm

choice they should make (Wiescholleck et al., 2014). During extinction learning, decision-time increases once more in association with a decrease in the number of correct arm choices (Wiescholleck et al., 2014). The same performance profile was observed in the current study in vehicle—treated animals (**Figure 3**), whereby no performance differences were evident between the treatment groups on days 2 and 3 (F(1.538,20) = 0.187; p = 0.774).

On day 4 (extinction learning), an increase in decision-time became evident, as the animals lost confidence in their choices (no arm was rewarded; p < 0.001). This was less apparent on day 5 (p = 1).

The animal cohort that was subsequently treated with SCH 23390 exhibited poorer decision times on day 1 of the study compared to controls, but by day 2, and extending through day 3 performance was equivalent in both animal cohorts (**Figure 3**). A clear learning effect occurred on days 1 through 3 (within-subject ANOVA: F(1,9) = 14.961, p = 0.004).

A significant increase in decision-time was evident during extinction learning (in the presence of the antagonist) on day 4 (**Figure 3**; p < 0.001). Furthermore, the decision-time increase was significantly different to that observed in controls 3 (F(1,13) = 31.992; p < 0.001).

On day 5, decision times were equivalent in both cohorts (**Figure 3**; F(1,13) = 2.697; p = 0.125).

# Agonist Activation of D1/D5 Receptors Increases Decision-Time During Renewal and Subsequent Extinction of Context "A"

Animals that were treated with the D1/D5-agonist Chloro-PB on day 4 showed equivalent decision times in the period of days 1–4 (**Figure 4**; Days 1–3: F(2.803,50.459) = 1.899; p = 0.145; Day 4: p = 0.085).

On day 5, a significant increases in decision-time was evident in agonist-treated animals (p = 0.016; **Figure 4**). This aligns with our observation that renewal was impaired in the Chloro-PB group on day 5.

### Antagonism of Dopamine D2-Receptors has no Effect on Decision-Times

The animal cohorts that were subsequently treated on day 4 with the D2-receptor antagonist, Remoxipride showed equivalent decision-times, as their vehicle-treated counterparts on days 1–3 (**Figure 5**). Performance on days 1 through 3 was equivalent in both groups (within-subject ANOVA: F(1,16) = 0.079, p = 0.797). Although a tendency towards improved decision-time was evident on day 4, effects were not significant (Between-subject F(1,17) = 0.037; p = 0.85).

Decision-times were also equivalent on both groups on day 5 (Between-subject F(1,17) = 2.079; p = 0.168).

## The Dopamine Receptor Ligands had no Effect on Locomotion or Stereotypy

No significant differences in locomotion behavior were detected on day 4 after treatment with either dopamine receptor ligand or vehicle. In vehicle-treated animals (n = 10) locomotion speed was 0.62 ± 0.034 m/s, in ChloroPB –treated animals (n = 10) it was 0.64 ± 0.032 m/s (ANOVA: F(1,18) = 0.196 p = 0.663), in SCH23390–treated animals (n = 8) it was 0.63 ± 0.031 m/s (ANOVA: F(1,15) = 0.081 p = 0.78), and in Remoxipride–treated animals (n = 8) it was 0.66 ± 0.052 m/s (ANOVA: F(1,16) = 0.539 p = 0.474; data not shown).

Similarly no significant effects with regard to stereotypy were observed. This was assessed as the number of head weavings conducted throughout all trials on day 4. Here, we observed an average of less than 1 head weaving during the total of 20 trials, for each of the animal groups tested

(vehicle, ChloroPB, SCH23390, Remoxipride; data not shown).

#### DISCUSSION

In this study, we show that pharmacological antagonism of D1/D5-receptors enhances context-dependent extinction without affecting renewal or extinction of behavior in the original context. By contrast, agonist activation of D1/D5 receptors does not affect acquisition of extinction learning, but renewal of the conditioned behavior (context ''A'') is impaired. Antagonism of D2-receptors neither has an effect on contextdependent extinction learning, nor does it affect renewal. Strikingly however, it increases resistance to extinction of the learned behavior in the original context. This suggests that under conditions where the fear circuitry cannot be expected to play a significant role in encoding and retrieval, dopamine D1/D5 receptors regulate context-dependent extinction, whereas dopamine D2-receptors may contribute to the learning of context-independent components of this form of extinction.

Our findings with regard to the involvement of dopamine D1/D5-receptors in the extinction of context-dependent appetitive spatial learning in rodents is in contrast to reports with regard to context-dependent fear extinction (Abraham et al., 2014). However, most studies that have addressed the role of these receptors in context-dependent fear extinction have done this by means of receptor antagonism, or transgenic animals that lack the receptor. Studies using a dopamine D1/D5 partial agonist demonstrated that extinction of fear-potentiated startle is impaired (Borowski and Kokkinidis, 1998), whereas prevention of dopamine/noradrenaline re-uptake enhances fear extinction (Abraham et al., 2012). We observed that blockade of D1/D5-receptors enhanced context-dependent extinction (in context ''B''), and receptor activation impaired renewal of the behavior learned in the original ''A'' context. We propose that these differences can be explained by the brain circuitry

FIGURE 5 | Antagonism of dopamine D2-receptors has no effect on decision-time during context-dependent extinction learning, or renewal of the learned response. The graphs represent the amount of time that was needed to reach the end of an arm (both correct and incorrect choices) after door opening. For each day the time for five contiguous trials was averaged (i.e., four time-points per day are shown). Decision times recorded in animals that were treated with the dopamine D2-receptor antagonist, Remoxipride, or vehicle are shown. The vehicle or antagonist solution was injected 30 min prior to extinction learning in day 4. The dopamine D2-receptor antagonist had no significant effect on decision time during extinction learning on day 4, or renewal on day 5.

that contributes to aversive learning and extinction, compared to non-aversive appetitive learning. In the case of fear learning, activation of the mesolimbic pathway and in particular the amygdala, prefrontal cortex and nucleus accumbens can be expected to predominate (Pezze and Feldon, 2004). In the case of appetitive learning, both the mesolimbic and the mesocortical pathways are involved (Abraham et al., 2014), whereby here, the role of the hippocampus in encoding context-dependent associations can be expected to be significant (Hansen and Manahan-Vaughan, 2014). Interestingly, activation of the locus coeruleus, that responds with noradrenaline release to context change (Bouret and Sara, 2005), and mediates heightened attention during appetitive extinction learning (André et al., 2015b), also results in modulation of ventral tegmental area (VTA) neurons (Grenhoff et al., 1993).

Central to both the mesolimbic and the mesocortical pathways is the VTA. Neurones of the dorsal VTA respond to reward-associated stimuli and their activity is suppressed by aversive stimuli, whereas neurons of the VTA increase their firing activity in response to negative or aversive stimuli (Brischoux et al., 2009). This suggests that a segregation occurs in the processing of reward-related and aversion-related information by the VTA. The ventral (ventromedial) VTA is reciprocally anatomically linked to the medial shell of the nucleus accumbens (Hasue and Shammah-Lagnado, 2002; Ikemoto, 2007), and aversive stimuli trigger dopamine release in this structure, as well as in the medial prefrontal cortex (Abercrombie et al., 1989; Kalivas and Duffy, 1995). Furthermore, dopamine receptor antagonists prevent fear learning if infused into the medial shell of the nucleus accumbens (Faure et al., 2008). The dorsal (dorsorostral) VTA, by contrast, projects predominantly to the amygdala, hippocampal formation and entorhinal region (Braak and Del Tredici, 2008). We are not disregarding the fact that the hippocampus is involved in the encoding of associative fear memory (Wen et al., 2015) and that the former circuit also recruits this structure (Abraham et al., 2014), however, the paradigm we implemented in the current study did not include a distinct aversive component, and therefore we assume that encoding of the associative learning experience was mediated by the latter projections from the VTA, thus possibly circumventing an intensive contribution of the nucleus accumbens.

When we applied a D1/D5-receptor antagonist we observed that extinction learning was immediately enhanced. Performance levels during the extinction trials were close to chance. Thus was in contrast to performance during the acquisition trials on day 1, when the animals first acquired the task. Here, however, the difference was that on day 1 in the first 10 trials all correct arms contained a reward, whereas during extinction learning none of the arms were rewarded: thus motivation levels can be expected to have been very different. Effects of the D1 antagonist on extinction learning were quite potent, but interestingly had no bearing on renewal performance one day after extinction learning had taken place. By contrast, D1/D5 receptor activation by means of an agonist had no ostensible effects on extinction learning in context ''B'', but impaired subsequent renewal in context ''A''. Taken together, these data suggest that in the absence of D1/D5-receptor activation, extinction learning in a new context is accelerated, although consolidation of this effect (and a resultant impact on renewal behavior) is not reinforced. By contrast, when D1/D5 receptors are activated, consolidation of extinction learning is reinforced and thus subsequent renewal of the original behavior (in the ''A'' context) is impaired. The lack of effect of the agonist on extinction learning can be explained by the likelihood that during the acquisition phase D1/D5 receptors may already be occupied by an adequate amount of dopamine, or D1/D5 receptors are not critically required for this component of extinction learning. An alternative, or perhaps complementary possibility is that the enhancement of extinction learning that was evident after D1/D5-receptor activation may have resulted from a modulation by the D1/D5-receptors of the saliency of the animal's experience in the new (''B'') context (Hansen and Manahan-Vaughan, 2014). Thus, effects may not have derived solely, or exclusively, from an enhancement of consolidation, but rather from support of pattern separation through D1/D5-receptor activation.

A basal tonus of dopamine release has been described (Grace et al., 2007) that results in a homeostatic background activation of dopamine receptors. Phasic release of dopamine occurs when the VTA becomes activated by reward, aversive or error prediction events (Grace et al., 2007; Abraham et al., 2014). Given the fact that agonist activation of D1/D5-receptors had no ostensible impact on the extinction learning within the time frame of the T-maze trials, we assume that phasic activation may have been less important in the context-dependent extinction paradigm used in the present study. Thus, the antagonist may have prevented the action of tonically active D1/D5-receptors. As mentioned earlier, it is striking that extinction of the contextdependent appetitive task was enhanced by D1/D5-receptor antagonism, as studies with regard to fear extinction report that receptor antagonism impairs extinction (Inoue et al., 2000; El-Ghundi et al., 2001; Fadok et al., 2010). We think the difference relates to the anatomical circuitry mentioned above, and to the dopamine release patterns and brain structures triggered by these profoundly different behavioral experiences. Although the hippocampus is believed to be involved in both context-dependent aversive (Corcoran and Maren, 2001), and appetitive, extinction learning (André et al., 2015a,b), these processes are likely to be mediated by different cellular mechanisms: context-dependent fear memory triggers robust memory encoding through hippocampal long-term potentiation LTP (Whitlock et al., 2006), whereas non-aversive contextdependent learning triggers hippocampal long-term depression LTD (Manahan-Vaughan and Braunewell, 1999; Kemp and Manahan-Vaughan, 2004, 2007, 2008, 2012; Goh and Manahan-Vaughan, 2013). The antagonist treatment had no bearing on renewal. This is not surprising given the fact that acquisition of behavior in the ''A'' context had been consolidated before the antagonist was applied. Furthermore, and the application of the antagonist prior to extinction learning on day 4, might have prevented consolidation of the extinction learning experience in context ''B''. In line with this, the impairment of renewal as a consequence of D1/D5-agonist treatment on day 4, suggests that consolidation of extinction learning, and/or the enhancement of the behavioral saliency of context ''B'' by D1/D5-receptor activation, served to firmly anchor the new memory created in context ''B'' and that this encoding impacted upon retrieval of the behavior previously learned in the ''A'' context. This observation is in line with many reports that support an important role for D1/D5-receptors in memory consolidation (Hikind and Maroun, 2008; Furini et al., 2014), in behavioral saliency (Hansen and Manahan-Vaughan, 2014), and in the longterm persistency of synaptic plasticity (Kulla and Manahan-Vaughan, 2000; Lemon and Manahan-Vaughan, 2006; Hansen and Manahan-Vaughan, 2014; Wiescholleck and Manahan-Vaughan, 2014).

We observed that antagonism of D2-receptors had no ostensible effect on context-dependent extinction learning, and also did not affect renewal in the ''A'' context. By contrast extinction learning within context ''A'' was impaired. Contradictory reports exist as to the involvement of this receptor in fear extinction (Ponnusamy et al., 2005; Fadok et al., 2010; Holtzman-Assif et al., 2010; Mueller et al., 2010). At the level of hippocampal information processing and this receptor plays a subordinate role: unlike the dopamine D1/D5-receptor, it does not critically contribute to the longevity and stability of LTP and LTD (Hansen and Manahan-Vaughan, 2014), rather activation of the D2-receptor serves to suppress synaptic excitability and lower basal tonus in the hippocampus (Manahan-Vaughan and Kulla, 2003). In line with this, a modulatory role for D2-receptors in spatial recognition memory (Léna et al., 2001) and passive avoidance learning (Sigala et al., 1997) have been reported. Dose-dependent beneficial and debilitatory effects of receptor antagonism for spatial reference memory have also been described (Setlow and McGaugh, 1999, 2000). This receptor may also be preferentially involved in the processing of aversive memories (Jocham et al., 2014; Wen et al., 2015).

It has been postulated, that at least at the level of the striatopallidal pathway, the D2-receptor may be important for learning flexibility (Yawata et al., 2012; Hatalova et al., 2014). Our findings suggest that extinction learning in context ''B'' may have recruited the support of information encoding in the hippocampus, to which the D2-receptor contributes little (Manahan-Vaughan and Kulla, 2003). Interestingly, the lack of extinction of the renewal effect on day 5, after application of the D2-receptor antagonist on day 4, suggests that blocking D2 receptors may nonetheless have affected learning flexibility. Thus, antagonism of D2-receptors may have affected the consolidation of context-dependent extinction learning, such that the memory of the original learned experience became more resilient. In this process, that reflects an impairment of extinction behavior in context ''A'', other extra-hippocampal systems may predominate, to which activation of D2-receptors plays a significant part. In light of these findings it will be

### REFERENCES

Abercrombie, E. D., Keefe, K. A., DiFrischia, D. A., and Zigmond, M. J. (1989). Differential effect of stress on in vivo dopamine release in striatum, nucleus accumbens and medial frontal cortex. J. Neurochem. 52, 1655–1658. doi: 10. 1111/j.1471-4159.1989.tb09224.x

of interest to compare the involvement of D1/D5 and D2 receptors in context-independent forms of appetitive extinction learning.

#### CONCLUDING REMARKS

Our data demonstrate that the dopamine D1/D5-receptor contributes to extinction learning of a context-dependent appetitive task by supporting extinction learning and by suppression of renewal. Antagonism of the receptor enhances extinction learning in a new context (in the absence of the US), but has no lasting impact on renewal or subsequent extinction on the original context. This suggests that tonic D1/D5-receptor activation modulates homeostatic processes whereby contextdependent information encoding is optimized. In line with this an interplay has been reported between D1/D5-receptors and the N-methyl-D-aspartate (NMDA) receptor that is critically required for multiple forms of hippocampal synaptic plasticity (Zweifel et al., 2009). Agonist activation of D1/D5-receptors had no ostensible impact during extinction learning per se (i.e., acquisition) but impaired subsequent renewal of the behavior learned in the ''A'' context. This is consistent with the likelihood that D1/D5-receptor promoted the consolidation of, and/or the behavioral saliency of the context change during, extinction learning that in turn, created interference for subsequent renewal behavior.

Under the conditions tested in our study, dopamine D2-receptors were not required for context-dependent extinction learning. An impairment of extinction of the conditioned behavior in the absence of the CS was evident, however, suggesting that antagonism of D2-receptors renders the original memory more resilient to extinction. Taken together, the findings of this study suggest that both D1/D5-receptors and D2-receptors modulate different components of extinction learning and renewal. Furthermore, the involvement of dopamine D1/D5 and D2-receptors in context-dependent appetitive extinction learning is distinct from their involvement in context-dependent fear extinction. We propose that this relates to the distinct neural circuitries that are activated by, and responsible for, the encoding of these different forms of behavioral experience.

## ACKNOWLEDGMENTS

We gratefully acknowledge the technical assistance of Jens Colitti-Klausnitzer, Anne Borkowski, Alina Blusch and Juliane Böge. We thank Nadine Kollosch and Silke Dirken for animal care. This study was supported by a grant from the German Research Foundation (DFG, FOR1581/P2).

Abraham, A. D., Cunningham, C. L., and Lattal, K. M. (2012). Methylphenidate enhances extinction of contextual fear. Learn. Mem. 19, 67–72. doi: 10.1101/lm. 024752.111

Abraham, A. D., Neve, K. A., and Lattal, K. M. (2014). Dopamine and extinction: a convergence of theory with fear and reward circuitry. Neurobiol. Learn. Mem. 108, 65–77. doi: 10.1016/j.nlm.2013.11.007


stimulation of the locus coeruleus in the rat. J. Neural Transm. Gen. Sect. 93, 11–25. doi: 10.1007/bf01244934


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 André and Manahan-Vaughan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The DA antagonist tiapride impairs context-related extinction learning in a novel context without affecting renewal

#### Silke Lissek <sup>1</sup> \*, Benjamin Glaubitz <sup>1</sup> , Oliver T. Wolf <sup>2</sup> and Martin Tegenthoff <sup>1</sup>

<sup>1</sup> Department of Neurology, BG University Hospital Bergmannsheil, Ruhr-University Bochum, Bochum, Germany, <sup>2</sup> Department of Cognitive Psychology, Faculty of Psychology, Institute for Cognitive Neuroscience, Ruhr-University Bochum, Bochum, Germany

#### Edited by:

John D. Salamone, University of Connecticut, USA

#### Reviewed by:

Seth Davin Norrholm, Emory University School of Medicine, USA Christopher Cain, Nathan S. Kline Institute for Psychiatric Research & NYU School of Medicine, USA

#### \*Correspondence:

Silke Lissek, Neurologische Universitätsklinik, BG-Kliniken Bergmannsheil, Ruhr-Universität Bochum, Bürkle-de-la-Camp-Platz 1, 44789 Bochum, Germany silke.lissek@ruhr-uni-bochum.de

Received: 16 July 2015 Accepted: 20 August 2015 Published: 03 September 2015

#### Citation:

Lissek S, Glaubitz B, Wolf OT and Tegenthoff M (2015) The DA antagonist tiapride impairs context-related extinction learning in a novel context without affecting renewal.

Front. Behav. Neurosci. 9:238. doi: 10.3389/fnbeh.2015.00238 Renewal describes the recovery of an extinguished response if recall is tested in a context different from the extinction context. Behavioral studies demonstrated that attention to relevant context strengthens renewal. Neurotransmitters mediating attention and learning such as the dopaminergic (DA) system presumably modulate extinction learning and renewal. However, the role of DA for non-fear-based extinction learning and renewal in humans has not yet been investigated. This fMRI study investigated effects of DA-antagonism upon context-related extinction in a predictive learning task in which extinction occurred either in a novel (ABA) or an unchanged (AAA) context. The tiapride-treated group (TIA) showed significantly impaired ABA extinction learning and a significant within-group difference between ABA and AAA extinction, compared to placebo (PLAC). Groups did not differ in their level of ABA renewal. In ABA extinction, TIA showed reduced activation in dlPFC and OFC, hippocampus, and temporal regions. Across groups, activation in PFC and hippocampus correlated negatively with ABA extinction errors. Results suggest that in context-related extinction learning DA in PFC and hippocampus is involved in readjusting the cue-outcome relationship in the presence of a novel context. However, relating context to the appropriate association during recall does not appear to rely exclusively on DA signaling.

Keywords: context-related extinction learning, renewal effect, fMRI, dopamine, tiapride, hippocampus, prefrontal cortex

## Introduction

The renewal effect of extinction describes the recovery of an extinguished response when extinction learning has been performed in a context different from that present during extinction recall (Bouton and Bolles, 1979). Thus, it highlights the context-dependency of extinction. In a recent imaging study in humans, we demonstrated that renewal is mediated by hippocampus and ventromedial prefrontal cortex (vmPFC) in concert (Lissek et al., 2013). During extinction learning, hippocampal activation is more pronounced in participants who later exhibit renewal than in those who do not, suggesting that their encoding of context is more effective (Lissek et al., 2013). These results are in line with previous findings in human fear extinction that associated hippocampus and vmPFC with context processing (Kalisch et al., 2006; Milad et al., 2007). Behavioral studies of renewal showed that modulation of attention is guided by stimulus relevance (Uengoer and Lachnit, 2012), and that allocation of attention can be controlled by contextual stimuli (Uengoer et al., 2013). Also on the behavioral level, it has been demonstrated that a task designed to focus attention upon context actually strengthens renewal in participants who have implicitly learned that context is relevant (Lucke et al., 2013). This finding is consistent with the notion that the strength of context-specific learning depends on the amount of attention paid to context stimuli (Rosas and Callejas-Aguilera, 2006). In consequence, it is conceivable that the renewal effect is dependent on attentional and encoding processes that occur during extinction learning and thus may be mediated by neurotransmitter systems involved in learning and attention, such as the noradrenergic and dopaminergic systems (Lauzon et al., 2009). Recent studies in humans and rats meanwhile showed that while stimulation of the noradrenergic system actually enhanced extinction learning, it had no impact upon the strength of the renewal effect (André et al., 2015; Lissek et al., 2015).

A general role for dopamine (DA) in Pavlovian and instrumental learning is well-established (Schultz et al., 1997; Schultz, 1998). DA is involved in both the learning and the attentional aspects of conditioning (El-Ghundi et al., 2007), directing attention to salient and novel stimuli, and delivering a teaching and reward signal during associative learning (Reynolds et al., 2001). DA receptor antagonism in prefrontal cortex (PFC) can affect performance in various aspects of tasks that require attention, such as set-shifting and reversal of a learned response (Boulougouris and Tsaltas, 2008). In a number of animal studies, participation of the dopaminergic system in extinction learning was demonstrated for D1 and D2 receptors. In rats, D1 agonists affected fear extinction learning (Fiorenza et al., 2012; Rey et al., 2014), while D1 antagonists (SCH23390) decreased renewal of a Pavlovian-conditioned response (alcoholseeking) (Sciascia et al., 2014), affected contextual fear extinction (Fiorenza et al., 2012) and prolonged extinction of place preference (Fricks-Gleason et al., 2012). Mice deficient in D1 receptors showed delayed fear extinction (El-Ghundi et al., 2001). Moreover, D1 receptor antagonism modulated performance in a task of contextual control of response conflict (Haddon and Killcross, 2011). For fear extinction, in particular infralimbic D2 receptors appear to be necessary, since local inactivation of infralimbic cortex in rats impaired extinction learning (Mueller et al., 2010). D2 antagonism accelerated fear extinction in mice (Ponnusamy et al., 2005; Dubrovina and Zinov'eva, 2010), while D2 agonism blocked fear extinction in rats (Nader and LeDoux, 1999).

While human data on effects of manipulating the dopaminergic system during extinction learning are lacking, there are studies reporting improving effects of DA-agonists upon other forms of human learning (Breitenstein et al., 2004; Flöel et al., 2005; Breitenstein et al., 2006). Moreover, a recent study on fear extinction in humans demonstrated that the dopamine precursor L-Dopa, administered after extinction, made extinction memories context-independent and thus reduced the return (renewal) of fear (Haaker et al., 2013). Animal studies also implicated the dopaminergic system in renewal. Administration of a DA1 antagonist (SCH23390) before extinction recall prevented renewal of an extinguished instrumental response (Hamlin et al., 2006). Pretreatment with D1 and D2 receptor antagonists attenuated context-induced renewal of cocaine seeking (Crombag et al., 2002) or sucrose seeking (Rauhut et al., 2010) in rats. Taken together, studies in animals and humans have delivered ample evidence for the involvement of the dopaminergic system in fear extinction. However, its function for contextual extinction learning and renewal without a fear component has not yet been investigated in humans.

Conceivably, the relevance of the dopaminergic system for fear extinction learning may be associated with its functions in prefrontal and hippocampal regions during learning and processing of context. Both areas are target regions for dopaminergic influences: expression of D1 and D2 receptors was reported for prefrontal cortex of rodents (Vincent et al., 1995) and for hippocampus of rodents and primates (Camps et al., 1990). In humans, mRNA for all types of dopaminergic receptors is expressed in prefrontal cortex (Meador-Woodruff et al., 1996). In human hippocampus, a moderate to high expression of D2 (Hurd et al., 2001), and a low to moderate expression of D3 receptor mRNA (Suzuki et al., 1998) was observed. In general, dopamine in the prefrontal cortex may be important for extinction by gating cognitive and behavioral flexibility (Abraham et al., 2014). Studies in rats and mice demonstrated that dopaminergic modulation of prefrontal regions can also affect attentional performance and working memory (Granon et al., 2000; Chudasama and Robbins, 2004; Glickstein et al., 2005). Accordingly, local infusions of a D1/D2 receptor antagonist into prelimbic cortex of the rat caused impairments in adaptations of instrumental responses to changes in contingency, suggesting a role for this region in actionoutcome associations (Naneix et al., 2009). Dopamine-mediated activity in human ventromedial PFC is involved in evaluating potential choices when learning to guide reinforcement-based decisions (Jocham et al., 2011). DA release in mOFC, vmPFC as well as dACC is important in reinforcement learning in the human brain, as a PET study measuring dopamine during a reward learning task demonstrated (Vrieze et al., 2013). DA infusions into vmPFC of rats influenced outcome sensitivity (Hitchcott et al., 2007), suggesting that the dopaminergic system in vmPFC has a role in response choices. In line with these findings, local infusion of both D1 or D2 antagonists into rat vmPFC impaired fear extinction (Mueller et al., 2010; Fiorenza et al., 2012).

Regarding dopaminergic influences in hippocampus, recent evidence indicates that hippocampal dopamine has a crucial role in memory formation, promoting memory for episodes that are novel and rewarding as well as building memory representations suited to guide later behavioral decisions (Shohamy and Adcock, 2010). Hippocampal D2 receptor activity was found correlated with memory function in humans (Takahashi et al., 2008), while D1 receptor modulation in rat hippocampus has been shown to affect fear extinction (Fiorenza et al., 2012). Furthermore, a PET study demonstrated that D1 receptor activity in hippocampus was positively linked to executive performance and speed (Karlsson et al., 2011).

In the present study, we aimed at investigating the role of the dopaminergic system in humans for context-related extinction learning without a fear component as well as for the renewal effect. We used an associative learning task in which participants were required to learn relations between cues and outcomes presented in particular contexts, which were reversed during the extinction learning phase. This predictive learning task (Ungör and Lachnit, 2006), which we already used in previous studies (Lissek et al., 2013, 2015) features an ABA design suited to reliably evoke a renewal effect, combined with a control AAA condition that does not evoke renewal. We treated healthy participants with a single dose of the D2/D3 antagonist tiapride prior to an extinction learning session of previously acquired associations.

We hypothesized that the DA-antagonist, compared to placebo, would impair extinction learning performance. In addition, we assumed that due to weak extinction associations in DA-antagonist treated participants, a greater number of acquisition associations would be recovered during extinction recall not only in ABA but also in the AAA condition, an outcome that reflects a reduction in actual ABA renewal. Moreover, we expected a concurrent reduction in activation of brain regions participating in extinction learning and attentional processing, such as prefrontal cortex and hippocampus.

# Materials and Methods

#### Participants

Forty healthy right-handed volunteers (19 females, 21 males), mean age 25.60 years ± 5.16 years st.dev., range 20–31 years, without a history of neurological disorders (questionnaire, selfreport), participated in this study. The participants received a monetary compensation for their participation (in the amount of e 60). Participants were randomly assigned to the experimental tiapride (TIA) and placebo control (PLAC) groups. Mean age within the groups was 25.68 years ± 4.92 st.dev., range 20–36 years in TIA and 24.88 years ± 3.20 st.dev., range 20–31 years in PLAC. Participants were assigned to the groups showing (REN) and not showing renewal (NOREN) according to the procedure described in "Behavioral data analysis."

#### Ethics Statement

All subjects participated in this study after giving written informed consent. The protocol was approved by the Ethics Committee of the Ruhr-University Bochum. The study conforms to the Code of Ethics of the World Medical Association (Declaration of Helsinki). Prior to the experiments, participants received handouts informing them about the fMRI procedures and the DA-antagonist Tiapride.

#### Predictive Learning Task

The predictive learning task that we used in this study was originally developed by Ungör and Lachnit (2006) to explore the context-dependency of extinction learning. Its efficiency in evoking a renewal effect was demonstrated in several behavioral studies using this specific design (Rosas and Callejas-Aguilera, 2006; Ungör and Lachnit, 2006; Üngör and Lachnit, 2008; Nelson and Callejas-Aguilera, 2007; Lucke et al., 2013). We adapted this task for use in an fMRI setting and already used it in previous fMRI studies (Lissek et al., 2013, 2015).

In the predictive learning task, participants were asked to put themselves in the position of a physician and predict whether various articles of food served in different restaurants would lead to the aversive consequence of a stomach ache in their patient. The learning process consisted of the three successive phases of (a) acquisition of associations, (b) extinction, and (c) recall phase (see **Figure 1**). During the acquisition phase (80 trials) participants learned to associate an article of food with a specific consequence. In each trial one of eight stimuli (vegetables or fruits) was presented to the participant in one of two different contexts (indicated by the restaurant names "Zum Krug" (The Mug) and "Altes Stiftshaus" (The Dome) and a frame in either red or blue color). The stimulus in its context was first presented for 3 s, then a question asking whether the patient will develop a stomach-ache was superimposed, with the response options "Yes" or "No." Response time was 4 s, participants responded by pressing the respective button on an fMRI-ready keyboard (Lumitouch, Photon Control Inc. Canada). After the response, or in case of a missing response after expiration of the response time, a feedback with the correct answer was displayed for 2 s, i.e., "The patient has a stomach ache" or "The patient does not have a stomach ache." The actual response of the participant was not commented upon. The food stimuli were presented in randomized order, each stimulus was presented 10 times. Four stimuli were presented per context. Stimuli were counterbalanced with regard to their causing the aversive consequence of a stomach ache, with two stimuli per context causing stomach ache during acquisition, while the other two did not.

During the extinction phase (80 trials), half of the stimuli were presented in the same context as during acquisition (condition AAA—no context change—40 trials) and the other half in the other context (condition ABA—context change—40 trials) in randomized order. In addition, stimuli were subdivided into two types: for actual "extinction stimuli," the consequence changed and the new consequence had to be learned, for "distractor stimuli," which were introduced in order to make overall learning more difficult, the consequence remained unchanged. Per context we used two extinction stimuli and two distractor stimuli. In all other respects, trials were identical to those during acquisition.

During the recall phase (40 trials), all stimuli were presented once again in the context of acquisition (five presentations per stimulus). With the exception that during the recall phase no feedback with the correct response was given, trials were identical to those during acquisition.

#### Procedure

In a first fMRI session, participants passed the acquisition phase of the predictive learning task. Immediately after this session, the dopaminergic antagonist tiapride was administered orally in a single dose of 100 mg. Control participants received an identical-looking placebo. One hundred and twenty minutes after administration of the drug/placebo, in accordance with the pharmacokinetic profile of tiapride with peak plasma

concentrations achieved around this time point (Rey et al., 1982; Norman et al., 1987), the second fMRI session was performed, which comprised the extinction learning phase and the extinction recall phase. Tiapride is a selective antagonist of D2 and D3 dopamine receptors (Dose and Lange, 2000), which has previously been shown to impair motor learning in humans (Lissek et al., 2014), as well as taste (Mediavilla et al., 2012) and place (Hurtado et al., 2014) aversion learning in rats. A study in non-human primates showed that tiapride down-regulated dopaminergic D1-receptors in prefrontal cortex, indicating that D2 receptor antagonism may have an impact upon D1 receptors too (Lidow et al., 1997).

#### Imaging Data Acquisition

Functional and structural brain scans were acquired using a whole-body 3T scanner (Philips Achieva 3.0 T X-Series, Philips, The Netherlands) with a 32-channel SENSE head coil. Bloodoxygen level dependent (BOLD) contrast images were obtained with a dynamic T2<sup>∗</sup> weighted gradient echo EPI sequence using SENSE (TR 3200 ms, TE 35 ms, flip angle 90◦ , field of view 224 mm, slice thickness 3.0 mm, voxel size 2.0 × 2.0 × 3.0 mm). We acquired 45 transaxial slices parallel to the anterior commissure—posterior commissure (AC-PC) line which covered the whole brain. High resolution structural brain scans of each participant were acquired using an isotropic T1 TFE sequence (field of view 240 mm, slice thickness 1.0 mm, voxel size 1 × 1 × 1 mm) with 220 transversally oriented slices covering the whole brain.

The task was presented to the participants via fMRI-ready LCD-goggles (Visuastim Digital, Resonance Technology Inc., Northridge, CA, USA) connected to a laptop which ran specific software programmed in Matlab (Mathworks, Natick, MA, USA). Responses were given by means of an fMRI-ready keyboard (Lumitouch response pad, Photon Control Inc., Canada).

#### Imaging Data Analysis

For preprocessing and statistical analysis of fMRI data we used the software Statistical Parametric Mapping (SPM), Version 8 (Wellcome Department of Cognitive Neurology, London, UK), implemented in Matlab R2008a (Mathworks, Natick, MA, USA). Three dummy scans, during which BOLD signal reached steady state, preceded the actual data acquisition of each session, thus preprocessing started with the first acquired volume. Preprocessing on single subject level consisted of the following steps: slice timing correction to account for time differences due to multislice image acquisition; realignment of all volumes to the first volume for motion correction; spatial normalization into standard stereotactic coordinates with 2 × 2 × 2 mm<sup>3</sup> using an EPI template of the Montreal Neurological Institute (MNI), smoothing with a 6 mm full-width half-maximum (FWHM) kernel, in accordance with the standard SPM procedure. The acceptable limit for head motion was 2 mm for translational movements and 0.5◦ for rotational movements.

In a first level single subject analysis, we calculated activation during extinction and recall phases in the conditions ABA and AAA, respectively. The contrasts were calculated within a combined anatomically defined mask which was constructed using the software MARINA (BION Bender Institute of Neuroimaging, University of Giessen, Germany) (Walter et al., 2003). The mask was centered around a priori regions of interest, containing prefrontal cortex, hippocampus, amygdala, insula, and temporal lobe. All data contained in this combined mask were analyzed together in a single analysis. We used an eventrelated design, modeling the events of each trial (stimulus and questions presentation, feedback presentation) using distinct stick functions convolved with the default HRF in SPM, with our analysis based on the stimulus presentation phase of each trial. The contrast images from these analyses were entered into second-level random-effects analyses to calculate in one-sample tests the activation patterns of the experimental and control groups for the different contrasts, using a threshold of p < 0.001 FWE-corrected on cluster level. Moreover, we calculated two-sample tests to directly investigate in which regions the experimental group showed differential activation compared to controls, using a threshold of p < 0.05 FWE-corrected on cluster level for the reported activations.

For additional analyses in which we correlated BOLD signal changes to performance data, we extracted the mean signal intensities (in arbitrary units) of activated clusters derived from the two-sample tests comparing the TIA and PLAC groups, using the MarsBar toolbox (Brett et al., 2002) in SPM 8.

#### Behavioral Data Analysis

For all three learning phases, log files were written that contained information on response latency, response type, and correctness of response. In acquisition, a response giving the wrong prediction was considered an error. Again, in extinction, a response giving the wrong prediction was considered an error. Thus, for extinction trials with a consequence change, a response that was correct during acquisition was considered an error during extinction. For distractor trials (no consequence change), the correct response remained the same as during acquisition.

For calculation of the renewal effect, only responses to stimuli with consequence change (extinction stimuli) during the recall phase were analyzed. The behavioral renewal effect in the predictive learning task is supposed to occur only in the condition ABA, in which extinction is performed in a context different from the context present during acquisition and recall phase. During the ABA recall phase, a renewal response occurs if the answer reports the association that was correct during acquisition, but wrong during extinction (e.g., if in acquisition in context A cherries cause stomach ache, and in extinction in context B they do not cause stomach ache any more, then a renewal effect response during recall in context A states that cherries cause stomach ache.). During the AAA recall phase, a response that reports an association that was correct during acquisition is considered an error, for since extinction occurred in an identical context, recalling the most recent association would be correct. Statistical analyses were performed using the IBM SPSS Statistics for Windows software package, version 22.0 (Armonk, NY; IBM Corp.). We used one-tailed t-tests to test our directional hypotheses regarding performance impairments following the experimental treatment.

In previous studies using the predictive learning task we found that a considerable portion (about 40%) of the participants did not exhibit the renewal effect. This is a typical finding that also appears in this type of task outside an fMRI setting (Lissek et al., 2013). For further evaluation of their behavioral data, participants were grouped according to whether they showed renewal (REN) or did not show renewal (NOREN). Group assignment was based on participants' performance during the recall phase in those trials designed to evoke renewal (i.e., the ABA trials with consequence change). All participants who never showed a renewal effect (0% renewal responses) were assigned to the NOREN group, and all participants who showed a renewal effect (30–100% renewal responses) were assigned to the REN group.

#### Results

#### Behavioral Results

#### Acquisition

We observed no significant differences in acquisition performance (pre-treatment) between the groups: t(38) = 0.042 p = 0.967 two-tailed (percent errors mean ± SE: TIA 16.50% ± 3.37, PLAC 16.69% ± 2.88).

#### Extinction

As hypothesized, we observed extinction learning impairments in the TIA group. For overall extinction learning performance, there was a trend toward a significant difference between groups regarding errors in trials with a consequence change [t(38) = 1.453 p = 0.078; percent errors mean ± SE: TIA 20.87% ± 3.25; PLAC 15.50% ± 1.75]. When considering only extinction learning in a novel context (ABA condition), the TIA group was significantly impaired compared to PLAC [t(38) = 1.989 p = 0.027; TIA 24.00% ± 3.81; PLAC 15.00% ± 2.43], while there was no significant difference in AAA extinction learning between groups [t(38) = 0.673 p = 0.252; TIA 18.25% ± 2.93; PLAC 16.00% ± 1.59—all t-tests one-tailed]. (See **Figure 2A**) Moreover, within the TIA group, we found a significant difference between extinction learning performance in the ABA and the AAA conditions [t(19) = 2.498 p = 0.022], which is absent in the PLAC group [t(19) = 0.462 p = 0.649].

Regarding error rates in distractor trials (no consequence change), we observed no significant differences between TIA and PLAC [t(39) = 0.522 p = 0.605; percent errors mean ± SE: TIA 12.0% ± 3.97; PLAC 9.5% ± 2.69], suggesting a comparable memory for associations learned during acquisition.

#### Learning Curve

In order to evaluate the groups' learning progress, we divided the extinction session into eight blocks with 10 trials each and calculated the percentage of extinction errors in ABA and AAA separately for each of these blocks. (See **Figures 2C,D**) For ABA extinction learning, a repeated measures ANOVA showed a significant main effect of the repeated measures factor learning block [F(7, 39) = 29.998 p = 0.000] upon error rates and a significant interaction of learning block<sup>∗</sup> treatment [F(7, 39) = 2.794 p = 0.008], indicating that learning progressed differently in TIA and PLAC. The factor treatment showed a trend toward a significant main effect upon the overall progress of learning [F(1, 39) = 3.169 p = 0.083]. For further analyses, we grouped the 8 blocks into three phases: initial exposure to changed stimulus-outcome contingencies (1st block), early extinction learning (blocks 2–5) and late extinction

learning (blocks 6–8). While TIA and PLAC showed similar error rates during initial exposure to the changed stimulusoutcome contingencies [t(38) = −0.760 p = 0.226; TIA 34.0% ± 5.25; PLAC 39% ± 3.96], during the following early extinction learning phase the TIA group made significantly more errors than the PLAC group [t(38) = 2.112 p = 0.020; TIA 19.25% ± 3.81; PLAC 9.75% ± 2.39]. In later extinction learning, the performance difference persisted [t(38) = 1.919 p = 0.031; TIA 9.67% ± 3.33; PLAC 2.67% ± 1.48] (all ttests one-tailed). Despite this slower learning progress, the TIA group showed extinction learning also in the ABA condition, with their rate of correct responses exceeding 90% in the final blocks.

For AAA extinction, an ANOVA with repeated measures yielded a significant main effect of the repeated factor learning block [F(7, 39) = 18.597 p = 0.000], while the interaction learning block<sup>∗</sup> treatment [F(7, 39) = 1.327 p = 0.237] and the factor treatment [F(1, 39) = 0.536 p = 0.468] showed no significant effect. In summary these results indicate a comparable learning progress in both groups over the course of AAA extinction learning.

#### Renewal

In both groups, participants who showed or did not show the renewal effect were equally distributed (TIA: χ <sup>2</sup> = 0.800; p = 0.371; REN 40% NOREN 60%; PLAC: χ <sup>2</sup> = 0.000; p = 1.00; REN 50% NOREN 50%). Renewal rates in REN participants ranged from 30 to 100% in both TIA and PLAC groups.

The dopamine antagonist tiapride had no effect upon contextual extinction retrieval: TIA and PLAC did not differ regarding the strength of the renewal effect (i.e., the percentage of renewal responses in the ABA condition): t(38) = −0.218 p = 0.418 one-tailed (mean ± SE: TIA 25.50% ± 7.929; PLAC 28.00% ± 8.29). When comparing only those participants who actually showed a renewal effect, we again observed no significant difference between groups: t(16) = 0.433 p = 0.670 two-tailed. (TIA 70.83% ± 8.76; PLAC 68.33% ± 10.07) (See **Figure 2B**).

On the other hand, TIA participants showed a trend toward impairment in retrieving the proper answer for trials in which extinction was performed in the acquisition context (AAA condition), which in the test phase required to retrieve the most recently acquired, altered association: t(38) = 1.539 p = 0.066 one-tailed (mean percent errors in AAA: TIA 10.5% ± 5.05 s.e.; PLAC 2.5% ± 1.23 s.e.).

#### Imaging Results

#### Activation Patterns of TIA and PLAC during Extinction Learning and Recall **Extinction learning**

We performed one-sample t-tests of TIA and PLAC during extinction learning in the ABA and AAA conditions, respectively. During extinction learning in both the ABA and AAA conditions, both groups show activation in hippocampus, fusiform gyrus, lingual gyrus, and insula. In contrast to PLAC, however, the TIA group shows no activation in dlPFC, lateral OFC, and superior temporal gyrus. The difference in dlPFC activation is particularly prominent for the ABA condition, where the PLAC group activates a number of clusters in bilateral BA 8, 9, and 46, while there is no dlPFC activation in TIA (see **Table 1** and **Figure 3**).

#### **Recall**

During extinction recall, both groups show activation in fusiform gyrus, lingual gyrus, cingulate gyrus, insula, and dlPFC, as well as in hippocampus, which, however, is not activated in PLAC during AAA recall. In addition, PLAC, in contrast to TIA, shows no activation in lateral OFC and the temporal pole. TIA, in contrast to PLAC, does not activate regions in parahippocampal gyrus (see **Table 2**).

#### Direct Comparisons of TIA and PLAC Groups **Extinction learning**

A two-sample t-test showed reduced activation of the TIA group compared to PLAC during ABA and AAA extinction in bilateral dlPFC (BA 9) and OFC (BA 10), fusiform gyrus and temporal pole, as well as in right hippocampus and left lingual gyrus. Moreover, there was reduced activation in right lingual gyrus exclusively in ABA extinction, as well as reduced activation in left vmPFC (BA 10) and hippocampus, and in bilateral insula exclusively in AAA extinction (see **Table 3** and **Figure 4**).

### **Recall**

TABLE 1 | One-sample tests—activated regions in TIA and PLAC during Extinction learning p < 0.001 FWE-corrected, k = 10.

The two-sample t-test did not yield any significant activation differences between the groups in ABA and AAA recall.


Frontiers in Behavioral Neuroscience | www.frontiersin.org September 2015 | Volume 9 | Article 238 |

### Correlations between Activation during ABA Extinction Learning and Performance

Assuming that the reduced activation in extinction-relevant prefrontal and hippocampal regions in the TIA group was related to their learning performance, we performed across groups analyses correlating ABA extinction learning performance with brain activation in PFC and hippocampus during the task. Activation in PFC and OFC (mean activation of clusters in BA 8, 9, 10, 46; MNI coordinates 38 52 2, 44 48 18, 52 22 34, 38 58 2) showed a significant negative correlation with extinction learning performance (i.e., percent errors during ABA extinction learning), indicating that higher activation in these regions was associated with less errors in ABA extinction learning. (Pearson's r = -0.348 p = 0.016). Activation in a cluster comprising right-hemispheric hippocampus, MNI coordinates 18 -32 -4, too was negatively correlated with the number of extinction errors (Pearson's r = -0.286 p = 0.041).

# Discussion

In this study, we investigated the role of the dopaminergic system for extinction learning in changed and familiar contexts and for the renewal effect. While the DA-antagonist tiapride partially impaired extinction learning, it did not affect renewal per se. Associated with the TIA group's impaired extinction learning was a pattern of brain activation that, compared to the PLAC group, showed reduced activation in extinction-relevant brain areas.

## DA-antagonism Impairs ABA Extinction but Not AAA Extinction

According to our hypothesis, we observed extinction learning deficits in the DA-antagonist treated participants which were restricted to extinction learning in the presence of a novel context: in ABA extinction, TIA participants made significantly more errors than PLAC participants, while there was no significant difference in error rate between groups in AAA extinction learning. The TIA group's learning curves for ABA and AAA extinction show that while AAA extinction learning proceeded at a pace comparable to that of the PLAC group, in the ABA condition extinction learning was slowed down in both the early and late phases of learning. Moreover, the percentage of errors in ABA and AAA extinction differed significantly within the TIA group, but not within the PLAC group. The novelty of the context-cue compound in ABA presumably constituted a particular learning challenge for the TIA group which interfered with their learning progress.

The extinction deficit found in ABA extinction corresponds to studies in mice and rats in which manipulation of the dopaminergic system by D2 antagonists affected extinction learning (Ponnusamy et al., 2005; Mueller et al., 2010). Our results also correspond to findings from animal studies which recently demonstrated that local DA D1 or D2 antagonism in monkey prefrontal cortex impaired learning of novel associations while leaving recall of familiar associations intact (Puig and Miller, 2012; Puig et al., 2014). Our study extends these findings by showing that, in humans, D2/D3 receptor antagonism selectively impaired processing of a novel context-cue compound


#### TABLE 2 | One-sample tests—activated regions in TIA and PLAC during Extinction recall p < 0.001 FWE-corrected, k = 10.

together with an altered outcome (ABA), while at the same time the manipulation had no adverse impact upon associating a changed outcome with a familiar context-cue compound (AAA).

#### DA-antagonism does not Affect Renewal

In contrast to the findings for extinction learning, and contrary to our hypothesis, the selective impairment of ABA extinction learning in TIA participants did not affect the level of renewal. In both groups, a similar proportion of participants showed renewal. Furthermore, the REN participants of both groups showed a similar percentage of renewal effect responses in ABA recall, presumably due to the fact that also the TIA group eventually acquired the altered associations during extinction learning. This lack of a tiapride effect upon renewal is in line with findings reporting that recall of previously established associations is not affected by (D2) DA antagonism (Lee et al., 2007).

#### Reduced Prefrontal and Hippocampal Activation in Extinction Learning is Associated with Impaired ABA Extinction

In parallel to the impairment of extinction learning in the ABA condition, the TIA group showed reduced BOLD activation in dlPFC and OFC during extinction learning. Moreover, the level of prefrontal activation was negatively correlated with learning performance across groups, with lower activation being associated with more errors in ABA extinction. These results are in line with findings from an animal study on associative learning in PFC which revealed a role for dopaminergic D1 and



cluster level, minimum cluster size k = 10) for ABA and AAA extinction. Activation in the TIA group is reduced predominantly in prefrontal regions, and also in further areas including hippocampus, insula and temporal pole.

D2-receptors in modulating PFC-dependent learning (Puig et al., 2014; Puig and Miller, 2015). Antagonizing these receptor types impaired learning of new stimulus-response associations as well as cognitive flexibility, but not recall of familiar associations. Moreover, a recent fMRI study showed that DA signaling in human dlPFC was associated with encoding and updating of context information during a working memory task (D'Ardenne et al., 2012). Correspondingly, in our study, the reduced dlPFC activation in the TIA group was related to their deficits in ABA extinction learning, which required the integration of a novel context into an altered association between cue and outcome, an effort that was not necessary in AAA extinction learning. This interpretation is also in line with the findings of an fMRI study reporting a specific role for dlPFC in encoding relational information as opposed to item-specific information, indicating that dlPFC contributes to memory formation by building relationships between items (Blumenfeld et al., 2011).

Not only prefrontal, but also hippocampal activation reduction was correlated with more errors in ABA extinction. These findings correspond to previous research which showed that modulations of the dopaminergic system in hippocampus can affect learning and memory. In healthy humans, working memory-related dopamine release associated with D2 receptor availability was observed in hippocampus (Aalto et al., 2005). In addition, hippocampal D2 receptors were found to contribute to local functions such as long-term memory as well as to modulation of PFC functions, and thus might be involved in human executive function including working memory (Takahashi et al., 2007, 2008). D1 and D2 dopamine dependent negative feedback in the loop of hippocampus—basal gangliathalamus—hippocampus was shown to have a role in extinction of responses (Sil'kis, 2008). Given the role of hippocampus for context processing, our findings add to the existing evidence by suggesting that D2/D3 antagonism in hippocampus presumably affects processing of novel contexts during extinction learning.

#### Further Regions Showing Reduced Activation Associated with the DA-antagonist Treatment

The lower activation observed in the TIA group in bilateral temporopolar regions may also have contributed to impaired extinction learning performance, since the temporal poles have been implicated in attentional processing (Lane et al., 1999), integration of semantic information (Noppeney and Price, 2002), object recognition (Nakamura and Kubota, 1996), and memory retrieval (Maguire et al., 2000). Furthermore, processes subserved by fusiform and lingual gyrus, such as visual encoding (Rombouts et al., 1999; McKenna et al., 2013), may have been compromised in the TIA group due to reduced activity in this region.

# Conclusion

In this study we investigated the role of dopamine for contextrelated associative extinction learning and renewal. Our findings

### References


for ABA extinction learning demonstrate a DA-antagonist related selective impairment in processing the combined load of an altered association together with a novel context, while changing an association between a cue and an outcome in a familiar context and subsequent renewal was not affected. Results suggest that in contextual extinction learning the dopaminergic system is specifically involved in readjusting the cue-outcome relationship in the presence of a novel context, with dopamine in PFC and hippocampus participating in this adjustment process. In contrast, relating context to the appropriate association and choosing the adequate response during extinction recall does not appear to exclusively rely on intact DA signaling.

# Funding

This work was supported by a grant from the DFG Deutsche Forschungsgemeinschaft (FOR 1581 Extinction Learning).

# Acknowledgments

We thank Tobias Otto for programming the stimulus presentation software. We appreciate the continued support of Philips, Germany.


task-tiapride increases cortical excitability and impairs motor learning. Front. Behav. Neurosci. 8:201. doi: 10.3389/fnbeh.2014.00201


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Lissek, Glaubitz, Wolf and Tegenthoff. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Beta-adrenergic receptors support attention to extinction learning that occurs in the absence, but not the presence, of a context change

#### Marion Agnès Emma André<sup>1</sup> , Oliver T. Wolf 1,2 and Denise Manahan-Vaughan1,3 \*

1 International Graduate School for Neuroscience, Ruhr University Bochum, Bochum, Germany, <sup>2</sup> Faculty of Psychology, Department of Cognitive Psychology, Ruhr University Bochum, Bochum, Germany, <sup>3</sup> Medical Faculty, Department of Neurophysiology, Ruhr University Bochum, Bochum, Germany

#### Edited by:

Edo Ronald de Kloet, Leiden University, Netherlands

#### Reviewed by:

Cesar Venero, Universidad Nacional de Educación a Distancia, Spain Benno Roozendaal, Radboud University Nijmegen Medical Centre, Netherlands

#### \*Correspondence:

Denise Manahan-Vaughan, Medical Faculty, Department of Neurophysiology, Ruhr University Bochum, MA 4/150, Universitätsstr. 150, Bochum 44780, Germany denise.manahan-vaughan@rub.de

> Received: 27 February 2015 Accepted: 29 April 2015 Published: 27 May 2015

#### Citation:

André MAE, Wolf OT and Manahan-Vaughan D (2015) Beta-adrenergic receptors support attention to extinction learning that occurs in the absence, but not the presence, of a context change. Front. Behav. Neurosci. 9:125. doi: 10.3389/fnbeh.2015.00125 The noradrenergic (NA)-system is an important regulator of cognitive function. It contributes to extinction learning (EL), and in disorders where EL is impaired NAdysfunction has been postulated. We explored whether NA acting on beta-adrenergicreceptors (β-AR), regulates EL that depends on context, but is not fear-associated. We assessed behavior in an "AAA" or "ABA" paradigm: rats were trained for 3 days in a T-maze (context-A) to learn that a reward is consistently found in the goal arm, despite low reward probability. This was followed on day 4 by EL (unrewarded), whereby in the ABA-paradigm, EL was reinforced by a context change (B), and in the AAA-paradigm, no context change occurred. On day 5, re-exposure to the Acontext (unrewarded) occurred. Typically, in control "AAA" animals EL occurred on day 4 that progressed further on day 5. In control "ABA" animals, EL also occurred on day 4, followed by renewal of the previously learned (A) behavior on day 5, that was succeeded (on day 5) by extinction of this behavior, as the animals realised that no food reward would be given. Treatment with the β-AR-antagonist, propranolol, prior to EL on day 4, impaired EL in the AAA-paradigm. In the "ABA" paradigm, antagonist treatment on day 4, had no effect on extinction that was reinforced by a context change (B). Furthermore, β-AR-antagonism prior to renewal testing (on day 5) in the ABA-paradigm, resulted in normal renewal behavior, although subsequent extinction of responses during day 5 was prevented by the antagonist. Thus, under both treatment conditions, β-AR-antagonism prevented extinction of the behavior learned in the "A" context. β-AR-blockade during an overt context change did not prevent EL, whereas β-AR were required for EL in an unchanging context. These data suggest that β-AR may support EL by reinforcing attention towards relevant changes in the previously learned experience, and that this process supports extinction learning in constant-context conditions.

Keywords: extinction learning, noradrenaline, rodent, spatial learning, beta-blocker, hippocampus, propranolol

# Introduction

Arousal and attention are key factors in effective learning behavior. Attending to experience both facilitates and expedites learning, and one of the key neuromodulators that regulates this process is noradrenaline (Crow, 1968; Kety, 1970, 1972; Aston-Jones and Bloom, 1981a,b; Sara and Segal, 1991). Attending to experience is also a key element in the process of extinction learning, whereby an individual learns that a prior learned experience no longer fulfills its learned function, or is no longer relevant. In biological terms, this means that the response to a conditioned stimulus (CS) declines when the stimulus is presented without reinforcement. In cognitive terms it means learning, for example, that the neighbor's house is no longer frightening, because the dog that bit you has been removed, or because it subsequently desists from biting you.

Extinction learning can thus be expected to occur under two possible conditions: the removal of the neighbor's dog comprises a context change, and substantial evidence exists that this strongly facilitates extinction (Bouton, 2004), whereby the circumstance whereby the neighbors dog remains in residence but never bites you again, amounts to extinction learning in the absence of a context change. Understanding the mechanisms that facilitate extinction is an important goal in understanding how extinction occurs at the cellular level, and in identifying strategies to optimise extinction. The noradrenergic (NA) system has been subjected to considerable attention in this regard, due to its postulated role in impaired extinction learning, for example, in post-traumatic stress disorder (Taylor and Raskind, 2002; Peskind et al., 2003; Griffith, 2005). Although it is clear that NA modulation of the amygdala plays a very important role in the learning and extinction of emotive memories mediated by the amygdala (Debiec and Ledoux, 2004; Roozendaal and McGaugh, 2011), much less is understood about the role of the NA system in extinction learning processes that are supported by the hippocampus. The hippocampus is involved in the assimilation and retrieval of context during novel extinction learning as well as during recall of context-dependent fear extinction (Good and Honey, 1991; Hobin et al., 2006; de Carvalho Myskiw et al., 2014; Portugal et al., 2014; Tan et al., 2014), and as well as during associative learning in humans (Lissek et al., 2013). It is also strongly implicated in context-dependent extinction in the absence of fear-reinforcement (Wiescholleck et al., 2014). Furthermore, the dorsal hippocampus contributes to the renewal of the conditioned response following fear extinction (Ji and Maren, 2005).

Current reports suggest that is that the hippocampus is particularly important for context-dependent extinction (Kalisch et al., 2006). Most studies have examined this with regard to fear-extinction (Alvarez et al., 2008; Lang et al., 2009; Maren et al., 2013), but recently, it was demonstrated that extinction learning in an appetitive context is also likely to involve the hippocampus (André et al., 2015). In rodents, contextdependent spatial learning, as well as hippocampal synaptic plasticity that is triggered by spatial learning, is supported by β-adrenergic receptors (Kemp and Manahan-Vaughan, 2008; Hagena and Manahan-Vaughan, 2012; Goh and ManahanVaughan, 2013). Furthermore, object-context learning triggers β-adrenergic receptor-dependent synaptic plasticity in the hippocampus (Kemp and Manahan-Vaughan, 2008; Hagena and Manahan-Vaughan, 2012; Goh and Manahan-Vaughan, 2013; Hansen and Manahan-Vaughan, 2014). We therefore postulated that NA modulation via activation of β-adrenergic receptors may be important for extinction learning of an associative spatial learning task. To test this possibility, we examined whether β-adrenergic receptors contribute to extinction learning in a T-maze task, when the context remains consistent, or when extinction is facilitated by a context change.

# Materials and Methods

The present study was carried out in accordance with the European Communities Council Directive of September 22nd, 2010 (2010/63/EU) for care of laboratory animals. All experiments were performed according to the guidelines of the German Animal Protection Law and were approved by the North Rhine-Westphalia State Authority (Bezirksamt, Arnsberg). All efforts were made to reduce the number of animals used.

# Animals

Male Wistar rats (7–8 weeks old) underwent implantation of guide cannulae, whilst under anesthesia (52 mg/kg sodium pentobarbital via intraperitoneal (i.p.) injection), as described previously (Manahan-Vaughan, 1997). One cannula was implanted into the lateral cerebral ventricle of each hemisphere (0.5 mm posterior to bregma, 1.6 mm lateral to the midline; size: 5.6 mm length, 0.8 mm diameter, 4.5 mm depth).

Animals were allowed 2 weeks to recover, before any behavioral experiment took place. They were housed singly and maintained on a 12-h light/12-h dark cycle with food and water ad libitum.

Two days prior to commencing the behavioral training, the rats were weighed and food access was reduced to result in a consistent body weight of 85% relative to the animal's weight immediately prior to starting the study. During the habituation phase, the animals were handled individually for 20 min per day.

### T-maze and Extinction Task

Experiments were conducted in a T-maze that comprised a starting box (25 × 20 cm) that was separated from the main corridor (100 × 20 cm) by a sliding door and two side corridors (40 × 10 cm) positioned perpendicular to the other end of the main corridor, as described previously (Wiescholleck et al., 2014). The walls were 40 cm high. At the end of each arm, at a distance of 1 cm from the end wall, a small round cup was placed on the floor equidistant from the walls, in which a reward could be placed. The reward could not be seen from a distance.

The context of the maze was changed in three ways, as described previously (Wiescholleck et al., 2014): (1) the plastic floor of the maze could be exchanged. Typical floor patterns comprised zebra stripes, checkered patterns, or geometric lines; (2) at the end of the 2 arms odors were placed that could be exchanged—1 µl of almond or vanilla (food aroma, Dr. Oetker, Bielefeld, Germany) was used; (3) extra-maze cue cards were used

FIGURE 1 | Antagonism of β-adrenergic receptors prevents extinction learning in the AAA paradigm. Animals underwent 20 contiguous trials per day of training in the AAA paradigm. Bar charts represent the number of correct arm choices in the first and second set of 10 trials on each test day. Animals participated in 3 days of acquisition training in the AAA paradigm, ending on day 3 with a 25% reward probability. Control animals were treated with vehicle prior to re-exposure to the context on day 4, in the absence of reward. Here, by the 2nd set of 10 trials significant extinction was evident. Upon return to the same context on day 5 (without reward) a further extinction of the learned conditioned stimulus (CS)-US response was shown. Treatment of animals, with the β-adrenergic receptor antagonist propranolol, before re-exposure to the A context in the absence of reward on day 4, significantly impaired extinction learning. A return to the same context on day 5 resulted in extinction of the learned response. An asterisk indicates a significant effect of at least p < 0.01 between the trials indicated by the bar. The arrow signifies the time of antagonist/vehicle-injection.

that could also be exchanged (Din A5 white paper with a black cross or a black square). These were placed 40 cm above the end of the main corridor.

On each experiment day, rats participated in a learning session that comprised 20 consecutive trials, that were split into two data blocks (1st 10, 2nd 10 trials) for analysis purposes (see below). The trial commenced with the opening of the door to the starting box, whereupon the animal entered the maze. The trial concluded when the animal entered an arm of the T-maze or when a specific time-limit (see below) had elapsed in the absence of arm entry. Animals learned to locate a food pellet (Dustless Precision Pellets 45 mg, BioServ, USA) that was placed at the end of a predetermined arm. This ''correct'' arm remained constant for a given animal during the training days. The floor and odor context were also kept constant during this time. On days 1 through 3, the reward probability was reduced in a stepwise manner from 100% to 25% to augment extinction resistance, as described previously (André et al., 2015). In conjunction with the reward probability reduction, the time limit for reaching the arm was also reduced from 2 min to 30 s Learning criterion was deemed to be acheived when the animal had successfully entered the correct arm on 8 of the final 10 trials of a given experiment day. Animals that failed to reach criterion by day 3 were excluded from the remainder of the study and their data from days 1–3 were not included in the analysis.

On day 4, extinction learning was assessed, whereupon the animals participated in 20 trials, during which no reward was present at any time. One day later (day 5), renewal (RN) was assessed by re-introducing the animal to the original T-maze (A) context for 20 trials with no food reward.

One animal cohort was tested in an AAA paradigm, where all trials (days 1–5) were conducted in the same context. A second cohort was assessed in an ABA paradigm, in which training was conducted in context A while the extinction session was conducted in context B, whereby the context (floor, odor and cue card) had been changed (André et al., 2015).

On day 5, animals (in both cohorts) were returned to the ''A'' context (in the absence of food reward). Typically, further extinction occurs under control conditions in the AAA group, whereas renewal of the behavior learned in the A context (1st 10 trials) followed by extinction of this behavior due to the lack of food reward (2nd 10 trials) occurs in the ABA groups (Wiescholleck et al., 2014; André et al., 2015).

#### Analysis of Decision Time

Decision-time typically declines, in close alignment with the increase in choice confidence on the part of the animal, during the gradual acquisition of the T-maze task (Luce, 1986; Avila and Lin, 2014; André et al., 2015). We evaluated this by recording the time required to leave the start box and reach the arm chosen by the animal. We evaluated this for every choice (incorrect and correct choices). By this means we obtained a measure of the confidence of the animal as to which arm was the correct choice (André et al., 2015).

#### Pharmacological Treatment

The β-adrenergic receptor antagonist, propranolol (Tocris Bioscience, Bristol, UK), was dissolved in 0.9% NaCl in a dosage of 2 µg/5 µl. This dose does not affect basal synaptic transmission in the hippocampus (Kemp and Manahan-Vaughan, 2008). The bilateral guide cannulae were inserted, and after ca. 5 min, a 5 µl solution volume was injected at a rate of 1 µl/min. The cannulae were left in place for a minimum of 5 min before removal (André et al., 2015). Propranolol, or vehicle, was given 30 min prior to the first trial of the extinction day (day 4) in the AAA and ABA paradigms. In a separate experiment with a third animal cohort, propranolol, or vehicle, was applied 30 min before the 1st trial before renewal testing on day 5 in the ABA group.

#### Data Analysis

Correct answers were defined as trials in which the animal moved first to the target arm. Each 20-trial session was divided into two sets of 10 trials (first 10 and last 10 trials), as described previously (André et al., 2015). The time required to reach the end of the first arm visited was calculated for each trial.

To analyse decision time, the time taken by the animal to move from the departure area in the T-Maze to its arm of choice was recorded for each trial, and data were segregated into 4 sets of 5 trials for each day, of which the times were averaged (André et al., 2015).

Data were analyzed by means of a multifactorial analysis of variance (ANOVA) with repeated-measures including 2 within-subject factors (Day and Session) and 2 between-group factors (Treatment and Experimental Design). Differences between trial blocks or between trials days of a specific group (control or propranolol-treated animals) were assessed using Bonferroni post hoc tests. Except where ''ANOVA'' is mentioned explicitly, all p values in the results section correspond to values determined from the Bonferroni test. The level of significance was set at p < 0.05.

#### Results

#### Extinction in the AAA Paradigm is Prevented by Antagonism of β-Adrenergic Receptors

During the first 3 experiment days, animals learned to take a constant turn (e.g., left) in a T-Maze to obtain a food reward, whereby reward probability was systematically reduced to 25% by the last trial block of day 3. A significant difference in performance was evident between day 1 and day 2 (**Figure 1**), reflecting successful acquisition of the task (ANOVA: for animals subsequently treated with vehicle, p < 0.001, n = 8; for animals subsequently treated with propranolol, p < 0.001, n = 8). No significant difference was evident in performance within the first and second 10 trial block on day 3, at which point, learning criterion had been reached (**Figure 1**). No significant difference in the animals' performance was evident on days 1, 2 or 3 when the two animals cohorts were compared (F(1.06,13.783) = 0.07; p = 0.81).

On day 4 and 5 the animals were returned to the same context but received no reward (AAA paradigm). Thirty minutes prior to commencing the first trial on day 4, animals were treated with either the β-adrenergic receptor antagonist, propranolol (n = 8), or vehicle (n = 8).

In both treatment groups, performance levels were equivalent in the 1st ten trials of day 4 (p > 0.001). Furthermore, performance levels were equivalent during the 1st ten trials of day 4 compared to the last ten trials of day 3 ANOVA: for control animals, p > 0.001; for propranolol-animals, p > 0.001, n = 8).

Differences became apparent in the 2nd trial block on day 4, however (**Figure 1**). Here, vehicle-treated animals exhibited significant extinction of the learned response when performance in the 1st trial block on day 4 was compared to performance in the 2nd trial block (p < 0.001). In contrast, propranolol-treated animals failed to show this extinction effect. Here, performance in the 2nd trial block was equivalent to performance in the 1st trial block (p > 0.001). Furthermore, the performance of the vehicle and propranolol-treated animals during the first and second trial blocks on day 4 was significantly different (ANOVA: F(1,14) = 11.486; p = 0.005). Thus, extinction in the AAA paradigm, in the absence of a context change, is impaired by prior treatment with a β-adrenergic receptor antagonist.

On day 5, animals were re-exposed to the same context in the absence of reward. Here, performance in vehicle-treated animals was equivalent in the 1st set of trials compared to performance in their last trial block on day 4 (p = 0.514). Extinction continued during the trials, with correct arm choices in the 2nd trial block on day 5 being significantly poorer than in the 1st trial block p < 0.001).

Effects were similar in the animals that had been treated on day 4 with propranolol. Here, their performance during the 1st and 2nd trial blocks on day 5 were equivalent to vehicletreated controls (ANOVA: F(1,14) = 1.112; p = 0.311), although their performance in the 1st 10 trials was significantly reduced compared to their performance in the last trial block on day 4 (p < 0.002). Thus, in the absence of propranolol, extinction learning was equivalent.

These data suggest that, antagonism of β-adrenergic receptors impair extinction learning in the absence of a context change.

#### Extinction in the ABA Paradigm is not Prevented by Antagonism of β-Adrenergic Receptors

A context change in the T-maze paradigm has been shown to facilitate extinction (Wiescholleck et al., 2014). Here, the protocol was identical to the AAA paradigm described above, except that on day 4 (''B'' context) the floor pattern was changed, as were the odor-related and extramural cues. On day 5, the animals were reexposed to the ''A'' context that they had experienced on days 1–3. On days 4 and 5, no reward was given, as was the case for the AAA paradigm. Thirty minutes prior to commencing the first trial on day 4, animals were treated with either propranolol (n = 10) or vehicle (n = 10).

In vehicle-treated animals, extinction occurred on day 4 that was significantly better than extinction effects in the AAA paradigm (**Figure 2**) (p < 0.029), in line with previous results

FIGURE 2 | Antagonism of β-adrenergic receptors before the extinction trials in a new context (ABA paradigm) does not prevent extinction learning. Control animals were treated with vehicle prior to exposure to the novel context "B" on day 4, in the absence of reward. Here, by the 2nd set of 10 trials significant extinction was evident that was also significantly better than extinction learning under the same conditions in the "A" context. Upon return to the learning context "A" on day 5 (without reward) an initial recovery (renewal) of the learned CS-US response was evident in the 1st set of 10 trials that was followed by significant extinction of the CS-US response. Treatment of animals with the β-adrenergic receptor antagonist, propranolol, before novel exposure to the "B" context in the absence of reward on day 4, had no effect on extinction learning. A return to the learning context "A" on day 5 resulted in renewal of the learned CS-US response (in the 1st 10 trials), that was followed by an extinction of this learned response during the last 10 trials of the day. Responses were equivalent to this observed in control animals. The arrow signifies the time of antagonist/vehicle-injection.

(Wiescholleck et al., 2014). Performance in the second trial block on day 4, was significantly weaker than in the first trial block (p < 0.006) indicating that significant extinction had occurred.

Treatment of animals with propranolol 30 min prior to entering the ''B'' context on day 4, had no significant effect on extinction learning (**Figure 2**): the performance of the animals was equivalent to that seen in controls (ANOVA: F(1,18) = 0.258; p = 0.618).

These data suggest that antagonism of β-adrenergic receptors does not influence extinction learning that is supported by a change of context.

Re-exposure to context ''A'' on day 5 elicited significant renewal effects in both animal groups (**Figure 2**). Thus, a comparison of the last trial block on day 4 with the 1st trial block on day 5, revealed a significantly improved correct choice performance in both the vehicle-treated animals (p < 0.05), and in animals that had been treated with propranolol on day 4 (p < 0.05). Thus, β-adrenergic receptor-antagonism does not affect renewal of the experience learned in the ''A'' context.

In both animal groups, extinction of this renewal effect became evident during the second set of 10 trials on day 5 (**Figure 2**) (p < 0.001, 1st vs. 2nd 10 trials, for both cohorts). No significant effect was evident when performance on day 5 was compared in the control and propranolol-treated animals (ANOVA: F(1,18) = 0.196; p = 0.663).

#### Antagonism of β-Adrenergic Receptors Prior to the Renewal Test in the ABA Paradigm has no Effect on Renewal but Prevents Extinction of the Old Context

The lack of effect of the β-adrenergic receptor-antagonist could derive from the fact that by 24 h after drug administration, its biological titre is so low as to no longer effectively block β-adrenergic receptors. This likelihood is supported by the finding that on day 5, in the AAA paradigm, no extinction impairment occurs. Thus, to clarify if β-adrenergic receptor antagonism has no bearing on renewal, we applied the antagonist 30 min before trial-begin on day 5 in the ABA paradigm.

Under these circumstances, renewal was also equivalent in vehicle-treated (n = 10) and propranolol-treated animals (n = 10) (**Figure 3**). Here, we observed a significant renewal of the response learned in context ''A'' in both vehicleinjected and propranolol-treated animals (p < 0.001, for both groups), when performance in the 1st trial block on day 5 was compared to performance in the last trial block on day 4. Renewal effects were also equivalent in both animal groups (ANOVA: F(1,18) = 0.181; p = 0.676). Strikingly, although vehicle-treated animals exhibited significant extinction in the last trial block of day 5 (p < 0.001, compared to the 1st trial block on day 5), extinction was impaired in the propranolol-treated group (p = 0.108, 1st vs. 2nd trial block, day 5). Furthermore, the performance of the control and propranolol-treated animals was also significantly different from one another during the 2nd (extinction) trial block on day 5 (ANOVA: F(1,18) = 5.469; p = 0.032).

the exposure to the learned "A" context in the absence of reward, control animals were treated with vehicle. During the first 10 trials, there was a recovery of the learned response that was followed by its extinction during the last 10 trials. Treatment of animals with the β-adrenergic receptor antagonist, propranolol, before the re-exposure to the learned context "A" on day 5, did not have any effect on the renewal effect (1st set of trials), but significantly inhibited subsequent extinction (2nd set of trials). The arrow signifies the time of antagonist/vehicle-injection.

The data confirm that renewal is unaffected by β-adrenergic receptor-antagonism. The data further indicate that β-adrenergic receptors are required for extinction (in the AAA paradigm) and (re-)extinction in the ''A'' context within the ABA paradigm. In other words β-adrenergic receptors are only required when extinction learning takes place in the context in which the original learning occurred.

#### Antagonism of β-Adrenergic Receptors has no Effect on Decision-Time in the "ABA" Paradigm but Improves Decision Time during Extinction Learning in the "AAA" Paradigm

When animals begin to acquire the task, the decision-time decreases in conjunction with an improvement in correct choices (Luce, 1986; Avila and Lin, 2014). Conversely, during extinction learning, decision-time typically increases if an attrition in the number of correct arm choices occurs (André et al., 2015). The latter situation was the case for vehicle-treated and propranololtreated animals in the periods encompassing day 1 and day 3 (task acquisition), under all conditions tested (**Figure 4**). In other words decision-time steadily decreased as the animals acquired the task and reached the learning criterion.

In contrast, in the AAA paradigm, in vehicle-treated animals (n = 8) extinction learning on days 4 and 5 was paralleled by a steady increase in decision-time (p < 0.001) (**Figure 4A**), reflecting the increasing insecurity of the animals as to which arm to choose. Decision-time was equivalent on day 5 in both vehicle-treated and propranolol-treated animals (n = 8). However, a direct comparison of decision time during day 4

revealed significantly faster decision times in propranolol-treated animals (F(1,14) = 11.523; p = 0.004).

In the ABA paradigm during renewal (day 5), decision-time continued to decrease in both animals group (each n = 10, **Figures 4B,C**), whereby decision time increased slightly (n.s.) in the time-frame of the 2nd set of trial blocks, whereupon extinction had occurred.

Antagonism of β-adrenergic receptors using propranolol prior to extinction, did not affect the overall trend towards an increase in decision-time across days 4 and 5 in the AAA paradigm (**Figure 4A**), (F(1,14) = 2.148; p = 0.165) (**Figure 4A**), or on performance in the ABA paradigm where propranolol was given on day 4 (**Figure 4B**), (F(1,18) = 0.02; p = 0.89).

The injection of propranolol prior to the renewal trials in the ABA protocol also didn't influence the decision time (F(1,18) = 1.154; p = 0.297, each n = 10) (**Figure 4C**). Thus, treatment with propranolol did not impair the choice-making confidence of the animals, suggesting that consolidation of the extinction experience was not influenced by propranolol treatment. The fact that propranolol improved decision-times during extinction in the ''AAA'' paradigm, suggests that the impairments of extinction observed under these conditions (**Figure 1**) may relate to a reduction in attention.

#### Discussion

The data of this study indicate that when extinction learning occurs under non-emotive circumstances, release of noradrenaline and subsequent activation of β-adrenergic receptors is a critical factor. We observed that antagonism of β-adrenergic receptors prevents extinction learning if the context remains constant. In contrast, extinction learning is unaffected by β-adrenergic receptor–antagonism, if extinction is reinforced by a context change. This suggests that the βadrenergic receptor is required for extinction learning of a consolidated learned experience, whereby it supports attention to the absence of the CS, and the subsequent adaptation in behavior that results. Where attention to the absent CS is reinforced by a context change, support of extinction by βadrenergic receptors becomes redundant, presumably because the increased arousal triggered by the context change mediates activation of additional neuromodulatory systems that support and reinforce extinction (e.g., dopamine or corticosterone). This likelihood is reinforced by the finding that renewal of the learned experience (in the ''A'' context, on day 5), following extinction in the ''B'' context, is unaffected by β-adrenergic receptor–antagonism (applied prior to testing on day 5), whereas the subsequent (re)-extinction of this behavior is prevented. The finding that decision-time is unaffected by β-adrenergic receptor antagonism in the ABA contexts, but is improved during extinction learning in the AAA context, suggests that it is not learning per se, but rather attention to the salient elements of the experience that is modulated by β-adrenergic receptors during extinction.

Although many studies have addressed the role of noradrenaline and β-adrenergic receptors in extinction of aversive experience (Cain et al., 2004), little is known about its role in extinction of appetitive memory (Mueller and Cahill, 2010), as was the focus of the current study. A role for noradrenaline in both memory consolidation has been reported (Quirarte et al., 1997; Roozendaal et al., 2002). Furthermore, noradrenaline is involved in fear extinction consolidation processes (Ouyang and Thomas, 2005; but see also: Lonsdorf et al., 2014). In the present study we did not see such an effect with regard to consolidation of extinction of appetitive memory, at least in terms of the involvement of β-adrenergic receptors: renewal of the learned response was unaffected by treatment with a β-adrenergic receptor antagonist prior to extinction learning. Two possible explanations spring to mind: on the one hand, the studies, where noradrenaline involvement in extinction consolidation was reported, were predominantly conducted under the conditions of fear extinction (Mueller and Cahill, 2010) and were particularly related to context-dependent extinction (Ouyang and Thomas, 2005), leading to the proposal that noradrenaline release onto β-adrenergic receptors is particularly relevant for context-dependent fear extinction (Mueller and Cahill, 2010). On the other hand, we cannot exclude that consolidation of extinction learning depends on the activation of adrenergic receptors other than the β-adrenergic receptors. In fact, evidence exists that different adrenergic receptors may play different roles in the regulation of extinction learning, and this may relate to their relative sensitivity to noradrenaline and the signaling pathways to which they couple. For example, although we observed that β-adrenergic receptor antagonism prevents extinction learning in an unchanged context, others have reported that antagonism of α2-adrenergic receptors enhances extinction in an unchanged context (Morris and Bouton, 2007). This may relate to the differences in the paradigms used (non-fearful memory vs. conditioned-fear memory), and thus, to the relative release of noradrenaline from the locus coeruleus triggered by these different experiences (Bouret and Sara, 2004; Sara, 2009), as well as and to differences in receptorsensitivity to noradrenaline (Ahlquist, 1948; Molinoff, 1984). Furthermore, whereas β-adrenergic receptors are positively coupled to adenylyl cyclase (Strader et al., 1989) and promote insertion of the AMPA-receptor subunit, GluA1/GluR1, into the postsynapse (Joiner et al., 2010), α2-adrenergic receptors are negatively coupled to adenylyl cyclase and suppress activity of voltage-activated Ca2+-channels and activate receptor-operated K <sup>+</sup>-channels (Limbird, 1988). Thus, these receptors can be expected to mediate opposing effects on neuronal function. Nonetheless, our data suggest that β-adrenergic receptor activation is not required for consolidation of extinction learning. However, we saw clear effects of β-adrenergic receptor antagonism on extinction learning in the absence of a context change. This suggests that activation of β-adrenergic receptors may be required to support attentional focus on the CS to enable effective extinction learning under these circumstances.

In the central nervous system, noradrenaline is released from afferent fibers that originated in the locus coeruleus, the firing of which increases in response to novelty (Sara, 2009), and a variety of behaviorally relevant stimuli such as unexpected events, threats, reward or fear (Sara and Bouret, 2012). The degree of activity of the locus coeruleus is graded according to the saliency of the experience, whereby the slow tonic changes in firing rates that accompany fluctuations in arousal state, can rapidly change into burst firing upon exposure to noxious stimuli, for example (Valentino and Van Bockstaele, 2008). The locus coeruleus also exhibits a very specific activity profile in response to conditioned stimuli, whereby firing can become persistent and intensify if a stimulus is followed by a salient event (Aston-Jones et al., 1994; Sara et al., 1994; Bouret and Sara, 2004), and firing is also triggered during extinction of appetitive and aversive learning (Sara and Segal, 1991). Furthermore, emotionally arousing experiences reinforce the acquisition emotional experiences via activation of β-adrenergic receptors (Liang et al., 1986; Cahill et al., 1994; Ji et al., 2003; Grillon et al., 2004). In an interesting parallel to the ability of the locus coeruleus to engage in noradrenaline release that is graded according to the saliency of the experience, the hippocampus exhibits graded sensitivity to NA (Loy et al., 1980). The dentate gyrus is the most sensitive, followed by the CA3 region and the CA1 region (Loy et al., 1980). The hippocampus engages in the very precise sorting of learned associative experiences, such that the discrimination of stored experiences from novel similar experiences occurs (pattern separation), presumably at the level of the dentate gyrus (Kesner, 2013a,b). By contrast, retrieval of associative memories based on exposure to a fragment of that memory (pattern completion) is enabled by the CA3 and possibly the CA1 region (Kesner, 2013a,b). In the present study, we saw that extinction learning in the absence of a context change is supported by β-adrenergic receptors. This process is arguably supported by pattern separation mechanisms in the hippocampus. Learning under these conditions would not be expected to trigger intense noradrenaline release from the locus coeruleus, but this may be sufficient to selectively support information processing and pattern separation in the dentate gyrus.

Many of the effects on cognition and synaptic plasticity of noradrenaline, released from the locus coeruleus, are mediated by β-adrenergic receptors, (Lemon et al., 2009; Lemon and Manahan-Vaughan, 2012; Goh and Manahan-Vaughan, 2013; Hansen and Manahan-Vaughan, 2014), and the T-maze task we used in our study, because included both spatial and contextdependent learning elements, is likely to recruit hippocampal information encoding. For this reason we hypothesized that β-adrenergic receptors would be required for extinction learning in this task. Thus, it was surprising to find that antagonism of β-adrenergic receptors only prevented extinction learning in the AAA paradigm, given the fact that the change in context during extinction learning in the ABA paradigm would be expected to elicit a higher level of locus coeruleus firing and thus, of noradrenaline release. Extinction of context ''A'' was impaired when propranolol was applied prior to extinction learning on day 4 (AAA paradigm), and when applied prior to re-exposure to the (unrewarded) ''A'' context on day 5 (ABA paradigm), suggesting that the robustness of this effect was not compromised by the context-dependent extinction event on day 4 in the ''ABA'' paradigm. We did not see an effect in extinction in the ''A'' context on day 5, when propranolol was given prior to extinction learning in context ''B'' on day 4, however. We propose that this is because propranolol is rapidly metabolized from the animals' system (Bargar et al., 1983; Baughman et al., 2009) and few or no β-adrenergic receptors remained under the influence of the antagonist when behavior was tested 24 h after the antagonist had been applied. Taken together, our data suggest that β-adrenergic receptor activation is an important component for extinction learning in the absence of a context change.

This is not the case, however, for context-dependent extinction. The change of context facilitated extinction in control animals, and this effect was not hindered by antagonism of β-adrenergic receptors. One possibility is that the context change promotes a more intense release of noradrenaline from the locus coeruleus that activates β-adrenergic receptors in the hippocampal CA regions (Loy et al., 1980) and promotes the novel encoding of this new associative experience. Another possibility is that under conditions of increased arousal during the context change, dopamine that is released from the locus coeruleus (Lemon and Manahan-Vaughan, 2012; Smith and Greene, 2012) serves to reinforce the extinction learning process and compensated for the absence of β-adrenergic receptors (that occurred under the experimental conditions of the present study). In contrast, under conditions where no context change accompanied extinction learning, arousal levels can be expected to be comparatively lower, and learning under these conditions was tightly dependent on noradrenaline acting on β-adrenergic receptors. This possibility is supported by observations that depletion of noradrenaline impairs extinction learning of appetitive behavior (Mason and Iversen, 1975, 1978; Mason, 1979; McGaugh, 2002). It may also be the case that a more intense NA release was stimulated by the context change that was not overcome by the antagonist dose used. However, this seems less likely, because treatment of an animal cohort prior to re-exposure to the ''A'' context, following successful extinction learning in the ''B'' context, failed to prevent renewal but significantly prevented subsequent re-extinction of the behavior learned in the ''A'' context.

It was striking that following inhibition of extinction in day 4 in the AAA context (following prior treatment with propranolol), renewal behavior occurred in the ''A'' context on day 5. We believe this effect adds support to our interpretation that β-adrenergic receptor antagonism affected attention but not learning per se. If learning had been impaired by the antagonist, a further, at least initial, suppression of extinction would have been expected on day 5: the animals had not learned (on day 4) that the ''A'' context can no longer be associated with a reward and thus, do not persevere to search for a reward in this context. Our animals showed renewal behavior, however, that refutes this possibility. If attention, and not, learning was affected by the antagonist, then the animal could be expected to fail to notice (on day 4) that the selected arm had previously been entered without reward success. This is not implausible, bearing in mind that reward probability had been reduced to 25% on day 3. Cumulatively, during day 4, the animal could still learn that in total, no food reward at all had been found during the 20 trials, but not bring this behavior into association with the previously learned CS-US response. Under these conditions, the animal would be expected to show normal initial renewal behavior on day 5. This was indeed the case in the present study.

Propranolol did not affect decision-time in the ABA paradigm, but in the AAA paradigm, decision times were slightly, but nonetheless, significantly better in the presence of propranolol during extinction learning on day 4. Despite this, extinction was impaired in the AAA paradigm in the presence of the β-adrenergic receptor antagonist. This further suggests that attention was undermined, and the animals failed to notice that the 25% reward probability had decreased to 0%. Blocking β-adrenergic receptors impairs rodent and human performances in attentional tasks (Hahn and Stolerman, 2005; de Martino et al., 2008). Furthermore, noradrenaline release from the locus coeruleus serves to enhance neuronal responses towards discrete stimuli and thereby to increase the signal-tonoise ratio (Woodward et al., 1979; Sara, 1985; Servan-Schreiber et al., 1990; Lemon and Manahan-Vaughan, 2012). Attentional set-shifting is supported by noradrenaline acting on the medial prefrontal cortex (Lapiz and Morilak, 2006; Tait et al., 2007; McGaughy et al., 2008; Snyder et al., 2012). Moreover, neuronal activity in the locus coeruleus precedes activity in the prefrontal cortex that is triggered by a CS (Snyder et al., 2012). Our observations that propranolol prevented extinction in the AAA paradigm is in line with the likelihood that noradrenaline release from the locus coeruleus is required in circumstances that require enhanced attentional focus and an associated change in behavioral strategy, as proposed by others (Bouret and Sara, 2005; Yu and Dayan, 2005; Dayan and Yu, 2006). In addition, our findings suggest that this kind of neuromodulation is mediated by noradrenaline acting on β-adrenergic receptors. This in turn may enable qualitative control over extinction learning whereby, under specific circumstances, attentional focus is optimised when extinction learning should take place under subtle (constant context) conditions. In line with this, a role for noradrenaline in the neuronal encoding of prediction errors has been proposed (Schultz and Dickinson, 2000). This would support, for example, attentional focus towards and the registration of subtle changes in environmental conditions that could facilitate extinction learning.

# Concluding Remarks

In conclusion, the findings of this study indicate that in an appetitive learning task that includes low reward probability, antagonism of β-adrenergic receptors impairs extinction in the absence of a context change (AAA paradigm), but does not affect extinction that is supported by a change of context (ABA paradigm). The inhibition of extinction that occurred in the AAA paradigm suggests that NA modulation of attentional focus is an important factor for the extinction of appetitive experience. Recent studies conducted in the context of reconsolidation blockage have indicated that propranolol prevents the reconsolidation of emotional memories (Kindt et al., 2009; Schwabe et al., 2012). These studies raise hope for the usage of propranolol as a potential treatment for post-traumatic stress disorder (Pitman and Delahanty, 2005). However, other studies reported that propranolol impairs fear extinction in humans, especially at a cognitive level (Bos et al., 2012). Taken together, with findings obtained under non-emotive/non-fearful conditions, this suggests that the effects of beta-blockade might be harmful, rather than beneficial, if extinction takes place in an appetitive context, and if cognitive rather than affective changes are desired.

# Acknowledgments

We gratefully acknowledge the technical assistance of Jens Colitti-Klausnitzer, Anne Borkowski, Alina Blusch

#### References


and Juliane Böge. We thank Nadine Kollosch and Silke Dirken for animal care. This study was supported by a grant from the German Research Foundation (DFG, FOR1581).


cortex. Eur. J. Neurosci. 29, 823–832. doi: 10.1111/j.1460-9568.2009. 06624.x


Yu, A. J., and Dayan, P. (2005). Uncertainty, neuromodulation and attention. Neuron 46, 681–692. doi: 10.1016/j.neuron.2005.04.026

**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 André, Wolf and Manahan-Vaughan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Blocking NMDA-receptors in the pigeon's "prefrontal" caudal nidopallium impairs appetitive extinction learning in a sign-tracking paradigm

Daniel Lengersdorf <sup>1</sup> \*, David Marks <sup>1</sup> , Metin Uengoer <sup>2</sup> , Maik C. Stüttgen3,4 and Onur Güntürkün<sup>1</sup>

<sup>1</sup> Faculty of Psychology, Department of Biopsychology, Institute of Cognitive Neuroscience, Ruhr University Bochum, Bochum, Germany, <sup>2</sup> Department of Psychology, Philipps-University Marburg, Marburg, Germany, <sup>3</sup> Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University, Mainz, Germany, <sup>4</sup> Focus Program Translational Neuroscience, University Medical Center of the Johannes Gutenberg University, Mainz, Germany

#### Edited by:

Manfred Schedlowski, University of Duisburg-Essen, Germany

#### Reviewed by:

Christian J. Merz, University of Trier, Germany Federico Bermudez-Rattoni, Universidad Nacional Autónoma de México, Mexico

#### \*Correspondence:

Daniel Lengersdorf, Faculty of Psychology, Department of Biopsychology, Institute of Cognitive Neuroscience, Ruhr University Bochum, GAFO 05/624 44780 Bochum, Germany Tel: +49 234 32 24917 daniel.lengersdorf@rub.de

> Received: 20 February 2015 Accepted: 19 March 2015 Published: 13 April 2015

#### Citation:

Lengersdorf D, Marks D, Uengoer M, Stüttgen MC and Güntürkün O (2015) Blocking NMDA-receptors in the pigeon's "prefrontal" caudal nidopallium impairs appetitive extinction learning in a sign-tracking paradigm. Front. Behav. Neurosci. 9:85. doi: 10.3389/fnbeh.2015.00085 Extinction learning provides the ability to flexibly adapt to new contingencies by learning to inhibit previously acquired associations in a context-dependent manner. The neural networks underlying extinction learning were mostly studied in rodents using fear extinction paradigms. To uncover invariant properties of the neural basis of extinction learning, we employ pigeons as a model system. Since the prefrontal cortex (PFC) of mammals is a key structure for extinction learning, we assessed the role of Nmethyl-D-aspartate receptors (NMDARs) in the nidopallium caudolaterale (NCL), the avian functional equivalent of mammalian PFC. Since NMDARs in PFC have been shown to be relevant for extinction learning, we locally antagonized NMDARs through 2-Amino-5-phosphonovalerianacid (APV) during extinction learning in a within-subject sign-tracking ABA-renewal paradigm. APV-injection slowed down extinction learning and in addition also caused a disinhibition of responding to a continuously reinforced control stimulus. In subsequent retrieval sessions, spontaneous recovery was increased while ABA renewal was unaffected. The effect of APV resembles that observed in studies of fear extinction with rodents, suggesting common neural substrates of extinction under both appetitive and aversive conditions and highlighting the similarity of mammalian PFC and the avian caudal nidopallium despite 300 million years of independent evolution.

Keywords: renewal, APV, sign-tracking, context, retrieval

# Introduction

Learning enables organisms to survive in a permanently changing environment. During learning, stimuli that are initially neutral become associated with co-occurring unconditioned stimuli and acquire the ability to elicit conditioned responses. Extinction learning of these conditioned responses is as relevant for adaptive behavior as initial acquisition. During extinction, a conditioned stimulus appears repeatedly without the unconditioned stimulus, and subsequently the conditioned response vanishes. Unlike original acquisition, extinction learning is highly context-dependent. After successful extinction, the transfer to a context other than that where extinction took place results in the reappearance of the conditioned behavior, a phenomenon termed renewal (Bouton and Bolles, 1979; Bouton and Ricker, 1994; Rauhut et al., 2001; Bouton, 2002; Crombag and Shaham, 2002). It illustrates that extinction does not simply erase the old memory trace but entails new learning (Pavlov, 1927; Bouton, 2004). The majority of studies on extinction learning employ fear conditioning experiments in rodents (Quirk and Mueller, 2008; Quirk et al., 2010). Results from both rodent and human studies point to three prominent brain areas as critical for extinction learning: amygdala, prefrontal cortex (PFC) and hippocampus. Contextual information is integrated by the hippocampus while the PFC and its interactions with substructures of the amygdala seem to play a key role in extinction organization and retrieval (Hobin et al., 2003; Peters et al., 2009). In rats, the functionality of the PFC for extinction learning differs between two main subareas, prelimbic and infralimbic PFC, which seem to have opposite functions. While the infralimbic cortex facilitates extinction learning, prelimbic cortex seems to inhibit it (Milad and Quirk, 2012). Pharmacological manipulations of the hippocampus as well as the PFC demonstrate that these structures are involved in contextual coding during renewal and extinction retrieval (Corcoran and Maren, 2004; Burgos-Robles et al., 2007).

Extinction learning is an evolutionary conserved phenomenon that can be studied in vertebrates and invertebrates (Stollhoff et al., 2005). But are the neural mechanisms involved in extinction learning in other species comparable to what we know from mammals? To answer this question, we study pigeons, a species that represents a classic model organism for conditioning tasks (Skinner, 1948; Güntürkün et al., 2014) but is separated from mammals by 300 million years of evolution. The pigeon brain is devoid of a cerebral cortex, but their pallium is partly homologous to mammalian cortex. In addition, there is strong evidence that birds have a specialized pallial area, the nidopallium caudolaterale (NCL) which constitutes a functional equivalent to the mammalian PFC (reviewed in Güntürkün, 2005; Lengersdorf et al., 2014a). Recently, Lengersdorf et al. (2014b) showed that transient NCL inactivation impairs contextspecific extinction memory consolidation. It is possible that the consolidation of extinction memory in the NCL is mediated via N-methyl-D-aspartate receptors (NMDARs). Indeed, Herold et al. (2011) revealed a high density of NMDAR in the pigeon's NCL, and Lissek and Güntürkün (2003) observed that the injection of 2-Amino-5-phosphonovalerianacid (APV), a NMDAR antagonist, in the NCL resulted in impaired extinction learning. Moreover, Lissek and Güntürkün (2005) provided evidence for the role of NCL NMDARs in contextual processing in a conditional discrimination task. In those studies, however, the possible contextual dependency of NMDARs in the NCL for extinction learning was not assessed. Therefore, we adapted this treatment to our established within-subject context-dependent extinction task for pigeons (Lengersdorf et al., 2014b). Bilateral injection of the NMDR antagonist APV in the NCL before extinction training was thus employed to test the hypothesis that the blockade of NMDAR in the NCL impairs extinction learning.

# Materials and Methods

#### Subjects

Adult unsexed pigeons (Columba livia) served in both experiments. Overall twenty-one animals participated in the experiment. Subjects were housed singly in wire-mesh cages (30 cm × 30 cm × 45 cm) in a colony room, with a 12-h light-dark schedule (lights on 8 a.m.), controlled humidity and temperature. The access to water was ad libitum while access to food was restricted (see below). Body weight was monitored daily and maintained around 85% of the free-feeding weight. All experiments were approved by the national authorities of the state of North Rhine-Westphalia, Germany and carried out in accordance with the National Institute of Health Guide for Care for Laboratory Animals.

#### Surgery

Naïve pigeons were prepared for bilateral cannula implantation with the painkiller Dolorex (0.3 ml, 10 mg/ml, Butorphanol, Intervet, MSD Animal Health, Unterschleißheim, Germany). Gas anesthesia (Isoflorane; Forane 100% (V/V), Mark 5, Medical Developments International, Abbott GmbH and Co KG, Wiesbaden, Germany) was initiated 10–15 min after painkiller injection. Feathers on top of the skull were cut, the skin was removed, and 8–10 stainless steel microscrews (Small Parts, Logansports, USA) were placed on the skull to anchor head mounts. Additionally, two small craniotomies were performed above the target areas to provide access to the underlying brain tissue. One double cannula (26-gauge, length 8 mm, spaced 2 mm, Plastics One Inc., Roanoke, USA) was inserted into each hemisphere under visual control at the following coordinates: AP +5.25 mm, L ±5 and 7 mm, V +1.1 mm (Karten and Hodos, 1967) at an angle of 30◦ relative to the coronal plate. Dental cement was used to fixate the cannulas at the defined position. Following surgery, injections of the painkiller Carprofen (0.3 ml, 10 mg/ml, Rimaldyl, Pfizer GmbH, Münster, Germany) were administered twice daily for at least 3 days. Animals were allowed to recover for 7–10 days following surgery before initial training commenced.

#### Behavioral Apparatus

Training was conducted in four similarly shaped experimental chambers (36 cm × 34 cm × 36 cm). Each chamber was placed in a sound-attenuating cubicle. White or brown noise (approximately 80 dB SPL) was played continuously to mask extraneous sounds. The center of the rear wall consisted of a transparent plexiglass pecking key (2 cm × 2 cm; 12 cm above the floor) to measure key pecking responses. Each registered response produced an audible feedback click. Stimuli were presented on LCD flat screen monitors mounted behind the chambers (2 × Belinea Model No.: 101536; Philips Model No. 150S4 and Model No. 150P4CG/00), hence a stimulus on the monitor was visible through the plexiglass pecking key. A

food hopper was positioned at the bottom center underneath the pecking key. The internal illumination of the boxes was provided either by 6W light bulbs or LED bands at the ceiling. Distinct contexts were produced by covering the rear and the side walls of the chambers with different color cards: Either by 2.5 cm wide vertical tan stripes spaced 5 cm apart on red background, or by yellow marbling pattern on white background. Four stimuli with different color patterns were used in each experiment. The hardware was controlled by customwritten Matlab code (The Mathworks, Natick, MA; Rose et al., 2008).

#### Procedure

The complete experiment included five different phases labeled Pretraining I, Pretraining II, Acquisition, Extinction and Test. Details of each experimental stage will be explained below and are illustrated in **Figure 1**, **Table 1**.

#### Pretraining I

Animals were trained on a simple sign tracking task (a Pavlovian conditioning procedure sometimes also referred to as autoshaping; Brown and Jenkins, 1968). A stimulus (''target'') appeared for 5 s. Upon termination of the stimulus, the food hopper was activated to provide grain for 3 s. The trials were separated by a fixed intertrial interval of 45 s. Responses during stimulus presentation were counted. Each session contained 48 target presentations. Training was conducted twice daily (work days only), once in each context. Sessions were spaced 2 h apart, and the context sequence (A→B or B→A) alternated daily. Once an animal exhibited conditioned responding in at least 80% of the TABLE 1 | General training procedure overview. ((+) = rewarded stimulus; (−) = non-rewarded stimulus; CS1 = conditioned stimulus 1;

CS2 = conditioned stimulus 2; — = no stimulus presentation).


trials in both contexts, the subject entered the next training stage (Pretraining II).

#### Pretraining II

The conditions of Pretraining I were extended by introducing 12 presentations of a non-reinforced stimulus (''non-target''). The number of target presentations was reduced to 24, and the duration of the intertrial interval was reduced to 35 s. Each session started with two target presentations, followed by randomized stimulus presentation. Conditioned responding in at least 80% of target and non-responding in at least 80% of nontarget trials was required for the animal to move into the next training phase (Acquisition).

The two stimuli employed in the two separate Pretraining phases served to detect possible non-systematic effects (up- or downregulations of responding) brought upon by pharmacological treatments during extinction. Additionally, the non-target served to discourage pigeons from responding indiscriminately to the visual stimuli. To summarize, the target stimulus was always followed by reward while the non-target was never followed by reward, and these contingencies were maintained throughout the entire experiment.

#### Acquisition

In this phase, three different stimuli (target, non-target, and CS1 or CS2, depending on the context) were presented in random order, each for 12 times. A rewarded CS1 was added in context A and a rewarded CS2 was added in context B. The performance criterion for completion of the acquisition phase was extended to a minimum of 6 days of training and three consecutive days of 80% correctly responded trials.

#### Extinction

Two extinction sessions in which either CS1 or CS2 was not followed by reinforcement anymore were conducted on separate days, spaced 48 h apart: One session with drug infusion and one with saline infusion (sequence counterbalanced). One day off between extinction days was necessary to guarantee complete washout of the drug. To adjust the daily amount of food, subjects were provided with 10 g of grain on days without training. Approximately 10–15 min before extinction commenced, either APV (total volume 2 µl, containing 10 µg of APV; 0.5 µl per cannula, i.e., 2.5 µg of APV per cannula) or saline (total volume 2 µl; 0.5 µl per cannula) was microinjected bilaterally (see Helduser and Güntürkün, 2012 for more procedural details). Irrespective of treatment, each extinction session consisted of 24 non-reinforced CS presentations, as well as 12 nontarget and 24 target presentations. During extinction, CS presentation was never followed by grain and was tested in the context in which it had not been presented during acquisition training: thus, CS1 was presented in context B and CS2 was presented in context A. Since this constitutes a within-subject experimental design, all animals experienced extinction of one CS under saline and extinction of the other CS under drug conditions.

#### Retrieval Test

48 h after the second extinction session, all stimuli were presented 12 times each (randomized sequence) under drug-free conditions on a single day. Testing took place in both contexts with test sessions separated by 2 h. Each test session contained all four stimulus types (target, non-target, CS1 and CS2) and started with two target presentations. CS presentations remained unrewarded, as during extinction training. Since both CSs were presented in both contexts, ABA renewal as well as spontaneous recovery (ABB) of responding could be assessed. The character sequences ABA and ABB refer to the order of contexts in which Acquisition, Extinction and Retrieval were assessed, respectively (**Figure 1**).

#### Histology

After completion of the test session, injection sites were verified with immunohistochemical techniques. Animals received a lethal injection of Equithesin (0.5 µl per 100 g body weight). Once the animal was deeply anesthetized and claw reflexes were completely absent, transcardial perfusion with warm sodium chloride solution (0.9%, 38◦C) and subsequently cold paraformaldehyde (4% in 0.12 M phosphate buffer pH 7.4, PBS, 4◦C) was performed. The brain was removed and postfixed in 4% paraformaldehyde for 2 h. Then the brain was transferred to paraformaldehyde with additional 30% sucrose overnight for cryoprotection and subsequently sliced in 40 µm sections. Sections were stained with cresyl violet to reveal anatomical structures. The position of the cannulas were analyzed under the microscope by means of the brain atlas from Karten and Hodos (Karten and Hodos, 1967).

#### Data Analysis

The main dependent variable was the fraction of trials in which animals showed conditioned responding during the 5 s CS presentation interval (henceforth, ''fractional response count''). This variable was chosen because results from our previous study suggested that this variable is more sensitive for detecting drug effects than the absolute number of conditioned responses. Nonetheless, absolute response counts during CS presentation were also analyzed. Statistical analyses were conducted employing one-way and two-way repeatedmeasures analyses of variance (RMANOVA), along with paired-samples t-tests. All analyses were performed with the Statistics Toolbox of Matlab R2012a (The Mathworks, Natick, USA). Normalized response counts during extinction were calculated by multiplying the average number of responses in a given bin of four consecutive trials by the ratio of target responses under saline and drug in the same bin of four trials, separately for each animal. Since animals almost never responded during presentation of the non-target stimulus, response data for this stimulus are not shown in the result figures.

# Results

### Histology

We tested 21 subjects. Two animals were excluded due to improper cannula position, two animals failed to achieve criterion performance, and another animal was subjected to an incorrect extinction procedure due to a mistake of the experimenter, leaving 16 subjects for analysis. Regarding cannula position, subjects were included if the tip of the lateral cannulas was positioned in the NCL and the medial cannula was either in the NCL or the nidopallium caudocentrale (NCC). Overall 36 cannulas were found to be within the NCL and 28 cannulas were placed in the NCC (**Figure 2**). The NCC is adjacent to the NCL. As judged from the fiber connections (Rehkämper and Zilles, 1991; Husband and Shimizu, 1999; Atoji and Wild, 2009) and a lesion study (Hartmann and Güntürkün, 1998) the NCC is sketched as a tertiary limbic area. Herold et al. (2011) reported that the NMDAR density within the NCC is comparable to that of the NCL. The reported effects therefore result from manipulations of both areas.

cannulas (black: nidopallium caudolaterale (NCL); gray: NCC). Pictures are based on the brain atlas by Karten and Hodos (1967).

#### Acquisition

Mean fractional response rates for individual stimuli during acquisition over the last three sessions were similar (**Figure 3A**) and accordingly did not differ significantly (paired t-test: target vs. CS1: t(15) = 1.7; p = 0.111; target vs. CS2: t(15) = 0.4; p = 0.693; CS1 vs. CS2: t(15) = 1.14; p = 0.27).

Absolute response rates on the stimuli during acquisition over the last three sessions were similar as well (**Figure 3C**, target vs. CS1: t(15) = 1.1; p = 0.281; target vs. CS2: t(15) = 0.49; p = 0.629; CS1 vs. CS2: t(15) = 0.03; p = 0.748).

#### Extinction

Fractional response counts to the target did not differ significantly under saline or APV conditions respectively during extinction training (RMANOVA: saline: F(5,75) = 1.5, p = 0.202, APV: F(5,75) = 0.97, p = 0.442; **Figure 3B**). However, a two-way RMANOVA revealed a block effect (F(5,75) = 2.4, p = 0.049) but neither treatment (F(1,15) = 0.7, p = 0.41) nor interaction effects (F(5,75) = 0.45, p = 0.82). Non-rewarded CS presentations led to decreased response probability under both saline (RMANOVA: F(8,75) = 22, p < 10−14) and drug conditions (RMANOVA: F(5,75) = 4.1, p = 0.002). A two-way RMANOVA revealed a treatment (F(1,15) = 12.92, p = 0.003), block (F(5,75) = 17.65, p < 10−10) and interaction effect (F(5,75) = 5.1, p < 10−<sup>4</sup> ). Paired t-tests showed significant differences in blocks 4–6 between CSsal and CSAPV (t(15) = 2.76; p = 0.014; block 5: t(15) = 4.5; p = 0.0004; block 6: t(15) = 3.56; p = 0.004). Importantly, fractional response counts for the target differed between drug conditions in the last block of extinction (paired t-test: t(15) = 2.24, p = 0.04), hinting at an unspecific effect of APV on conditioned responding. Therefore, we proceeded to investigate this possibility using absolute response counts.

**Figure 3D** depicts the mean absolute response rates to the target and the CSs under saline and drug conditions during extinction. A two-way RMANOVA for target responses between the two conditions revealed no treatment (F(1,15) = 1.9, p = 0.188) but a block effect (F(5,75) = 5.7, p < 10−<sup>3</sup> ), as well as a significant interaction of treatment and block factors (F(5,75) = 6.8, p < 10−<sup>4</sup> ). Follow-up RMANOVAs indicated that target responses increased significantly under APV (F(5,75) = 10, p < 10−<sup>6</sup> ) but not under saline (RMANOVA: F(5,75) = 1.7, p = 0.143). Regarding responding to the CSs, a two-way RMANOVA yielded both significant treatment (F(1,15) = 13.1, p = 0.003) and significant block effects (F(5,75) = 14.6, p < 10−<sup>9</sup> ), accompanied by a significant interaction (F(5,75) = 2.8, p = 0.021). Follow-up RMANOVAs revealed significant response decrements to the CS in both conditions (CSAPV: F(5,75) = 3.5, p = 0.007; CSsal: F(5,75) = 16, p < 10−10).

These results from fractional and absolute responses suggest that blocking NMDA-receptors of the NCL delays extinction learning. However, APV injection also increased responding to the (non-extinguished) target, indicating that the drug effect was not specific to the CS. To disentangle the non-specific response disinhibition from a potential addition effect on extinction learning, we conducted a series of pairwise comparisons to identify the time point at which a drug effect on target and CS responses could be demonstrated. Indeed, a paired t-test showed that absolute responding to the CS already differed between saline and drug conditions in block 4 (trials 13–16, t(15) = 2.83, p = 0.03), while at that time responses to the target did not differ

response counts and additionally indicate unspecific disinhibition of conditioned responding. (E) Normalized response counts reveal prolonged extinction for APV treated subjects.

significantly between conditions (t(15) = 1.86, p = 0.083). The lack of statistical significance was not due to a ceiling effect, as target responding for APV still increased significantly beyond this point (block 4 vs. block 6: t(15) = 3.4, p = 0.004).

In another attempt to disentangle these two effects (slowed extinction and disinhibition), we calculated normalized response rates to the CS (**Figure 3E**). Normalization was performed by multiplying CS response counts by the ratio of target responses under saline to target responses under APV (see methods), with the intention to statistically remove the unspecific effect of APV on conditioned responding, as measured by the target control stimulus. Importantly, even when the non-specific increase in responding as measured by increased target responses was factored out through normalization of CS responses, differences

between APV and saline remained: while the time course of the response decrement is highly similar between conditions, responding under APV is stronger than under saline, as indicated by a significant treatment effect (two-way RMANOVA: F(1,15) = 10, p = 0.006; block: F(5,75) = 14.3, p < 10−<sup>9</sup> ; interaction: F(5,75) = 0.3, p = 0.919). Similar to the previous analysis, responses to the CS under both conditioned started to differ after block 3 (paired t-test: block 4: F(15) = 2.84; p = 0.012; block 5: F(15) = 3.04; p = 0.008; block 6: F(15) = 2.47; p = 0.03).

Taken together, the analyses of fractional response counts, absolute response counts, and normalized response counts support the hypothesis that APV, in addition to an unspecific enhancement of conditioned responding, specifically delays extinction learning.

#### Retrieval

Retrieval of extinction memory was tested by presenting all stimuli in both contexts. Two-way ANOVA analysis for fractional CS responding in ABB and ABA revealed a main effects of (prior) treatment (F(1,15) = 8.1, p = 0.01) and of testing context (ABB vs. ABA, F(1,15) = 65.5, p < 10−<sup>6</sup> ) in the absence of a significant interaction (F(1,15) = 0.2, p = 0.68). Post hoc tests indicated that fractional CS response counts in the context of extinction differed significantly between drug conditions (ABB, extinction under drug vs. saline: t(15) = 2.5, p = 0.025) while ABA renewal was unaffected (ABA: t(15) = 1.7, p = 0.111) (**Figure 4A**).

For absolute response rates (**Figure 4B**) a two-way RMANOVA showed no significant main effect of treatment (F(1,15) = 3, p = 0.105) or interaction (F(1,15) = 0.1, p = 0.774), but there was a significant main effect of test context (block ABA vs. ABB: F(1,15) = 37, p < 10−<sup>4</sup> ). In contrast to fractional response counts, responding to the CS extinguished under saline was not significantly different from responding to the CS extinguished under APV when tested in the context of acquisition (ABA; t(15) = 1.1, p = 0.297) or when tested in the context of extinction (ABB; t(15) = 1.6, p = 0.132). Thus, fractional response counts again turned out to be more sensitive for detection of pharmacological manipulation than absolute response counts, as was found in our earlier study (Lengersdorf et al., 2014b).

Unimpaired renewal could in principle be due to a ceiling effect, i.e., that animals responded maximally during ABA testing under both drug and saline and therefore a possible effect on associative strength is masked. However, inspection of **Figures 4A,B** shows that ABA response counts to the CSs were somewhat lower than to the target, and statistical analyses indicated that the differences in response counts between CS and target were statistically significant in some cases and marginally significant in the others (fractional response counts: target vs. CSAPV in ABA: t(15) = 2.06, p = 0.057; target vs. CSsal in ABA: t(15) = 4.39, p < 10−<sup>4</sup> ; absolute response counts: target vs. CSAPV in ABA: t(15) = 2.12, p = 0.051; target vs. CSsal in ABA: t(15) = 9.44, p < 10−<sup>8</sup> ). We conclude that a ceiling effect is unlikely to have masked differential responding between APV and saline treatments in ABA testing.

# Discussion

The present study investigated the role of NMDARs in the NCL for extinction memory by pharmacologically modulating these receptors with the antagonist APV during extinction. In our previous study (Lengersdorf et al., 2014b) we reported that transient ''prefrontal'' NCL inactivation with the sodium channel blocker Tetrodotoxin (TTX) during extinction learning impairs extinction memory consolidation. Now, in APVinjected subjects, several analyses showed that extinction learning was slowed down through NMDAR antagonism injection. This effect was accompanied by general behavioral disinhibition, as evidenced by subjects' enhanced responding to the continuously reinforced target stimulus. Context-dependent extinction memory retrieval revealed that the APV-treated animals did not exhibit a retrieval deficit as such but merely continued responding at the level of the last trials of extinction training.

Regarding the effects of APV, the present findings mostly align well with previous work from our laboratory. Lissek et al. (2002) demonstrated that NMDAR blockade in the NCL slows down color reversal learning due to prolongation of extinction. Our study likewise mostly accords with Lissek and Güntürkün (2003) who demonstrated that APV in the NCL retards extinction learning. However, Lissek and Güntürkün (2003) could not see a concomitant behavioral disinhibition of responding to a non-rewarded stimulus. This stimulus corresponds to our non-target and our results for this stimulus are identical to what was described by these authors (Lissek and Güntürkün, 2003). However, we additionally included a stimulus which was always followed by reward (target) and therefore consistently produced conditioned responding. Importantly, responding to this stimulus did increase under APV (during the last third of extinction training), suggesting that some of the effects of APV on responding to the extinguished CS are indeed due to behavioral disinhibition. However, fractional and normalized CS response counts indicated that disinhibition does not explain the full extent of the retardation of extinction. This pattern of results highlights the necessity to include appropriate control stimuli when applying pharmacological agents to animals, as unspecific effects on responding might otherwise be mistakenly attributed to specific learning mechanisms. Importantly, the presence of a significant difference between ABB CS response counts during retrieval reinforces our conclusion that APV does not merely disinhibit conditioned responding, but affects the encoding or the consolidation of extinction memory as well, because retrieval testing was conducted after any drug effects had dissipated.

It might seem counterintuitive that blocking NMDARs results in an increase rather than a decrease of behavioral output, since NMDAR activation depolarizes neurons due to influx of cations. However, blockade of NMDARs in PFC indeed does not dampen neural excitability but rather enhances it. For example, systemic MK-801 injections in rats impair working memory and, at the same time, increase motor activity, and the magnitude of these effects correlates with firing rate potentiation and burst activity reduction in the PFC (Jackson et al., 2004). MK-801 seems to act through decreased inhibitory interneuron activity, thereby disinhibiting prefrontal pyramidal cells (Homayoun and Moghaddam, 2007). It is conceivable that a similar mechanism might be at work in the pigeon NCL since electrophysiological and morphological analyses of NCL neurons indicate the existence of fast spiking neurons which resemble GABAergic interneurons of the mammalian telencephalon (Kröner et al., 2002) and which project to principal neurons. The absence of disinhibition for the non-target (see also Lissek and Güntürkün, 2003) could be due to a floor effect or might be related to the much stronger appetitive associative strength of the target that was constantly rewarded. Taken together, locally blocking NMDARs during extinction learning in the limbic and ''prefrontal'' caudal nidopallium slows down extinction learning, and disinhibits responses to rewarded stimuli.

Finally, extinction memory retrieval was tested under conditions of spontaneous recovery and renewal. Blocking NMDARs in the caudal nidopallium during extinction did not affect renewal but significantly increased spontaneous recovery when using fractional rather than absolute response rates. Impaired spontaneous recovery is readily explained by the impairment of extinction learning under APV. The fact that fractional but not absolute response rates yielded significant effects (although the analysis using the latter measure pointed into the same direction) was already observed in our previous study using TTX inactivation of the NCL (Lengersdorf et al., 2014b). This is somewhat puzzling since absolute response counts reflect the subject's valuation of a given CS in a graded manner (Honig, 1962; Starosta et al., 2013), while fractional response counts omit the valuation but detect more sensible if extinction memory can be retrieved in general. Fractional response counts in addition largely omit this information by reducing a continuum of responding to a dichotomous measure. This could be explained if absolute response counts were a very coarse measure of variation which would largely reflect non-specific factors and therefore merely represent noise, which would be reduced by dichotomizing responses into presence or absence of conditioned responding.

But why did we observe a result pattern with APV that deviates from the TTX-results that were obtained with the identical design by Lengersdorf et al. (2014b)? In this first study, we found that TTX-injections into NCL do not impair extinction learning but rather impair extinction memory retrieval (Lengersdorf et al., 2014b). This accords with similar experiments on the PFC in mammals which make it likely that extinction learning can proceed without prefrontal involvement in various downstream neural structures (Burgos-Robles et al., 2007; Milad and Quirk, 2012). However, the retrieval of extinction memory requires that the PFC had modified its synaptic contacts with neurons that had undergone extinction learning (Milad and Quirk, 2002; Vertes, 2004; Herry et al., 2008). Consequently, impaired NCL/PFC-functions during extinction learning perturb subsequent extinction memory retrieval from downstream structures (Sierra-Mercado et al., 2006; Lengersdorf et al., 2014b). Here, using APV, we observe impaired extinction learning but no impaired extinction memory retrieval. As outlined above, our APV-injections possibly increased excitability of caudal nidopallial principal neurons. The NCL is one of the largest hubs of the bird forebrain and is connected to a very large number of sensory-associative, limbic and motoric areas (Shanahan et al., 2013). Possibly, an APV-induced increase of excitation of nidopallial principal neurons interferes with extinction learning in this wide forebrain

#### References


network, resulting in slowed down extinction. At the same time, an increased excitation of nidopallial principal neurons could easily explain the selective disinhibition of responses to a reward-associated stimulus as observed in our study.

A large number of rodent studies suggest that blocking NMDARs results in a retardation of extinction learning (Baker and Azorlosa, 1996; Santini et al., 2001; Lee et al., 2006; Hsu and Packard, 2008). These results match our findings for context-specific extinction learning. Additionally, we could show that blockade of NMDA receptors results in behavioral disinhibition on top of its effects on extinction learning, and that our paradigm allows disambiguating these two effects.

To conclude, our results support the notion that NMDARs in the pigeon's limbic and ''prefrontal'' caudal nidopallium is implicated in extinction learning as well behavioral inhibition. The comparative approach underscores the shared functionality of the NCL and the prefrontal areas of mammals and shows that the neurochemical architecture of extinction learning shows some invariant properties in vertebrates that are separated by 300 million years of independent evolution.

# Acknowledgments

Supported by the DFG through FOR 1581.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Lengersdorf, Marks, Uengoer, Stüttgen and Güntürkün. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Effects of a Flavonoid-Rich Fraction on the Acquisition and Extinction of Fear Memory: Pharmacological and Molecular Approaches

Daniela R. de Oliveira1, 2, Claudia R. Zamberlam1, 2, Gizelda M. Rêgo<sup>3</sup> , Alberto Cavalheiro<sup>4</sup> , Janete M. Cerutti <sup>2</sup> and Suzete M. Cerutti <sup>1</sup> \*

<sup>1</sup> Cellular and Behavioral Pharmacology Laboratory, Department of Biological Science, Universidade Federal de São Paulo, São Paulo, Brazil, <sup>2</sup> Genetic Bases of Thyroid Tumor Laboratory, Division of Genetics, Department of Morphology and Genetics, Universidade Federal de São Paulo, São Paulo, Brazil, <sup>3</sup> Department of Forestry Colombo, Brazilian Agricultural Research Corporation, Colombo, Brazil, <sup>4</sup> Institute of Chemistry, Nuclei of Bioassay, Biosynthesis and Ecophysiology of Natural Products, São Paulo State University, Universidade Estadual Paulista, Araraquara, Brazil

The effects of flavonoids have been correlated with their ability to modulate the lutamatergic, serotoninergic, and GABAergic neurotransmission; the major targets of hese substances are N-methyl-D-aspartic acid receptor (NMDARs), serotonin type1A eceptor (5-HT1ARs), and the gamma-aminobutyric acid type A receptors (GABAARs). everal studies showed that these receptors are involved in the acquisition and extinction f fear memory. This study assessed the effects of treatment prior to conditioning with flavonoid-rich fraction from the stem bark of Erythrina falcata (FfB) on the acquisition nd extinction of the conditioned suppression following pharmacological manipulations nd on gene expression in the dorsal hippocampus (DH). Adult male Wistar rats were reated before conditioned fear with FfB, vehicle, an agonist or antagonist of the 5-HT1AR, ABAARs or the GluN2B-NMDAR or one of these antagonists before FfB treatment. The ffects of these treatments on fear memory retrieval, extinction training and extinction etrieval were evaluated at 48, 72, and 98 h after conditioning, respectively. We found hat activation of GABAARs and inactivation of GluN2B-NMDARs play important roles n the acquisition of lick response suppression. FfB reversed the effect of blocking luN2B-NMDARs on the conditioned fear and induced the spontaneous recovery. locking the 5-HT1AR and the GluN2B-NMDAR before FfB treatment seemed to be ssociated with weakening of the spontaneous recovery. Expression of analysis of DH amples via qPCR showed that FfB treatment resulted in the overexpression of Htr1a, rin2a, Gabra5, and Erk2 after the retention test and of Htr1a and Erk2 after the extinction etention test. Moreover, blocking the 5-HT1ARs and the GluN2B-NMDARs before FfB reatment resulted in reduced Htr1a and Grin2b expression after the retention test, but layed a distinct role in Grin2a and Erk2 expression, according session evaluated. We how for the first time that the serotoninergic and glutamatergic receptors are important argets for the effect of FfB on the conditioned fear and spontaneous recovery, in which he ERK signaling pathway appears to be modulated. Further, these results provide mportant information regarding the role of the DH in conditioned suppression. Taken ogether, our data suggest that FfB represents a potential therapy for preventing or reating memory impairments. g t r S o a a a t G e r t i G B a s G r t p s t t i t t

#### Keywords: flavones, fear memory, GABAAR, 5-HT1AR, GluN2B-NMDAR

#### Edited by:

Onur Gunturkun, Ruhr-Universität Bochum, Germany

#### Reviewed by:

Carsten T. Wotjak, Max-Planck-Institute of Psychiatry, Germany Hongjoo J. Lee, University of Texas, USA

> \*Correspondence: Suzete M. Cerutti smcerutti@unifesp.br

Received: 19 June 2015 Accepted: 23 November 2015 Published: 05 January 2016

#### Citation:

de Oliveira DR, Zamberlam CR, Rêgo GM, Cavalheiro A, Cerutti JM and Cerutti SM (2016) Effects of a Flavonoid-Rich Fraction on the Acquisition and Extinction of Fear Memory: Pharmacological and Molecular Approaches. Front. Behav. Neurosci. 9:345. doi: 10.3389/fnbeh.2015.00345

# INTRODUCTION

Several studies have investigated the effects of the extracts of flavonoid-rich plants or flavonoid molecules as potent modulators of brain structure and function, including their neuroprotective and chemopreventive properties and their beneficial effects on memory and cognition. The effects of flavonoids have been correlated with their ability to modulate the phosphorylation state of intracellular proteins via the activation or inhibition of protein kinases and phosphatases (Gamet-Payrastre et al., 1999; Wang et al., 1999; Schroeter et al., 2002; Li et al., 2003; Hoffman et al., 2004; Joseph et al., 2005; Maher et al., 2006; Nakajima et al., 2007; Spencer, 2007; Vauzour et al., 2008; Williams et al., 2008; Lovera et al., 2012; Mansuri et al., 2014), to increase the level of 5-HT and its metabolites (Zhang et al., 2012) or to alter expression of GABA<sup>A</sup> receptors (GABAARs) and/or glutamatergic N-methyl-D-aspartic acid (NMDA) receptors (NMDARs) (Wang et al., 2005, 2008; Rendeiro et al., 2014). In addition, studies addressing the effects of specific flavonoid subgroups, including flavanols, anthocyanins, flavanones, and flavones, have shown that these constituents display potential to act as cognition-enhancing and neuroprotective agents (Vauzour et al., 2008; Kehr et al., 2012; Rendeiro et al., 2013, 2014; Vauzour, 2014), to prevent many forms of cerebrovascular disease, or to function as anti-anxiety drugs (Hasenöhrl et al., 1998; Spencer, 2008; Zhang et al., 2012). Although studies ex vivo, in vivo, and in vitro have provided evidence supporting the effects of flavonoids on the central nervous system, the cellular, and molecular pathways through which these compounds modulate memory formation are not completely elucidated (Youdim et al., 2004; Nakajima et al., 2007; Spencer, 2007; Williams et al., 2008; Ballesteros et al., 2014; Kimura et al., 2014; Rendeiro et al., 2014). However, several studies have established that the hippocampus, which plays a central role as a substrate of fear memory and anxiety (Fendt and Fanselow, 1999; Sanders et al., 2003) and which is a component of the Behavioral Inhibition System (McNaughton and Gray, 2000), appears to be a target for the mnemonic effects of flavonoid metabolites (Bannerman et al., 2004; Wang et al., 2006, 2009; Williams et al., 2008; Rendeiro et al., 2012, 2014; Oliveira et al., 2013; Vauzour, 2014).

Previous studies from our laboratory have demonstrated the role of flavonoid-rich plant extracts, such as a standardized extract of Ginkgo biloba L. (EGb), in the modulation of fear memory (Oliveira et al., 2009, 2013) by inducing differential CREB-1, GAP-43, and GFAP gene and protein expression in the dorsal hippocampus (DH), the prefrontal cortex and the amygdaloid complex. Further, we have established that crude extracts, fractions, and flavonoid molecules isolated from the stem bark of Erythrina falcata (CE) improved the acquisition of conditioned fear as evaluated by single-trial, step-down inhibitory avoidance (IA) (de Oliveira et al., 2014). Additionally, we used an IA procedure to show for the first time that treatment with flavones produces another well-established conditioning phenomenon, spontaneous recovery (de Oliveira et al., 2014). These findings corroborate with the results described in the literature and expand the understanding that flavonoids act as cognition-enhancing agents. However, these results raise new questions, which are highlighted below.

The first question concerns the anti-anxiety properties and cognitive effects of the flavonoid-rich fraction from CE, given the various actions of flavonoids on the central nervous system. Despite the close relationship between fear memory and anxiety, these functions are dissociable at the behavioral, pharmacological, molecular, and neuroanatomical levels (McNaughton and Corr, 2004; Kalueff, 2007; Nakajima et al., 2007). The conditioned emotional response (CER) is a suitable animal model for studying the behavioral, pharmacological, and molecular mechanisms underlying fear memory and anxiety. To assess these phenomena, our lab has used the conditioned suppression of the lick response, in which the conditional stimulus (CS, tone), when associated with a noxious unconditioned stimulus (US, footshock), ultimately suppresses the licking response reinforced by water; i.e., the CS leads to the suppression of the ongoing behavior (Blanchard and Blanchard, 1969; Bolles and Collier, 1976; Fanselow, 1980; Sotty et al., 1996; Sanders et al., 2003). Fear responses (flight/fight/freezing) increase systematically as fear memory is acquired and decrease as fear memory is extinguished (Sotty et al., 1996; Liu et al., 2004; Apergis-Schoute et al., 2005; Davis, 2006; Erlich et al., 2012; Furini et al., 2014). Conditioned fear responses are insensitive to anxiolytic drugs (McNaughton and Corr, 2004), but several works show that treatment with diazepam, an anxiolytic drug that is widely used in the clinic, prior to the conditioning session disrupts the initial acquisition of learned fear (Jensen et al., 1979; Izquierdo and Medina, 1991; Makkar et al., 2010), decreases the occurrence of freezing responses in a dose-dependent manner in rats (Fanselow and Helmstetter, 1988; Decker et al., 1990; Beck and Fibiger, 1995; Malkani and Rosen, 2000; Isoardi et al., 2004; Yeh et al., 2015) and impairs the acquisition of conditioned suppression (Oliveira et al., 2009). Anxiolytic compounds were effective in reducing the inhibitory response of animals to an aversive stimulus, which alleviated the suppression of the CER (McNaughton and Gray, 2000; Miyamoto et al., 2000; George et al., 2009). McNaughton and col. showed that anxiolytic drugs reduced theta frequency in the hippocampus (Coop et al., 1991; Munn and McNaughton, 2008). In this sense, the sensitivity of the CER to anxiolytic drugs, such as benzodiazepines and agonists of 5-HT1A receptors (5-HT1ARs) (Millenson and Leslie, 1974; Davis, 1990; Stanhope and Dourish, 1996; Mirza et al., 2005; George et al., 2009; Oliveira et al., 2009), substantiates the use of this model to investigate the fundamental mechanisms underlying the effects of anti-anxiety drugs in addition to their function in alleviating conditioned fear in rodents.

The second question concerns the neurochemical mechanisms underlying both the acquisition and the extinction of conditioned suppression, as well as the role of the flavonoidrich fraction from the stem bark of Erythrina falcata (FfB) in modulating these processes. We primarily focused on the molecular events underlying the acquisition of fear memory and the modulatory effects of FfB. Further, we were interested in determining whether treatment with flavonoids prior to conditioning can modulate the extinction process. Studies demonstrating the involvement of glutamatergic, serotoninergic and GABAergic neurotransmission in the acquisition of fear memory have been accumulating in past decades; the major targets of these neurotransmitters are NMDARs, 5-HT1ARs, and GABAARs, respectively (Santini et al., 2001; Davis and Myers, 2002; Lin et al., 2003; Quirk and Mueller, 2008; Kim and Richardson, 2010), and the modulation of these receptors in the hippocampus is essential for the acquisition and consolidation of fear memory (Izquierdo, 1997; Cammarota et al., 2000; Alonso et al., 2002; Milad et al., 2007). Similarly, these changes are essential to consolidation of fear extinction (Myers and Davis, 2002). These effects are mediated by the activity of kinases and phosphatases, and ERK1/2 activation has been described to be involved in several cellular changes associated with long-term memory (LTM) (Atkins et al., 1998; Cammarota et al., 2000). Blocking NMDARs in the prefrontal cortex and the hippocampus is known to result in a deficit in the acquisition of fear extinction (Lissek and Güntürkün, 2003) and the retrieval of fear extinction (Lengersdorf et al., 2014). Evidence from in vitro and in vivo studies showed that flavones modulate GABAARs and GluN2B-NMDARs, but few studies have been conducted on the mechanisms underlying the modulatory effects of flavonoids on these processes. Therefore, in our study, we sought to elucidate the neurochemical systems involved in the acquisition of fear memory in the presence or absence of FfB treatment and to determine whether FfB treatment prior to conditioning modulates the extinction of fear memory. Further, we evaluated how these changes may control or be controlled by the activation or inhibition of specific receptors using pharmacological agonists or antagonists.

Therefore, the contributions of the glutamatergic, serotoninergic, and GABAergic systems, as well as the interactions between these systems, to the effects of FfB on the acquisition and extinction of conditioned suppression were assessed for the first time by administering agonists, antagonists of receptors for GABA, glutamate (NMDA) and 5-HT or one these antagonists before FfB prior the conditioning session. Additionally, we evaluated the mRNA expression levels of the GluN2A and GluN2B subunits of the NMDAR, the receptor subunits GABAAR and 5-HT1AR and ERK1/2 in the DH of controls and treated rats subjected to acquisition and extinction of conditioned fear.

This combination of molecular, behavioral and pharmacological analyses advances our understanding of the role of flavones in fear memory and anxiety. The findings regarding the molecular mechanisms of flavone action appear to be promising with respect to the development of new therapeutic strategies for the treatment of cognitive deficits or anxiety disorders. Moreover, we assessed the contribution of the hippocampus to these processes. In particular, we focused on the suppression of the licking response as a behavioral model and the hippocampus as a key component of the neural circuitry involved in the acquisition, consolidation and extinction of fear memory in animals and humans, as the hippocampus may represent a target for the action of FfB.

# EXPERIMENTAL PROCEDURES

# Drugs and Reagents

Methanol (HPLC grade) was obtained from Merck (Darmstadt, Germany). Formic acid, ethanol, n-butanol, and Tween <sup>R</sup> -80 were obtained from Synth (Diadema, Brazil). Vitexin and isovitexin standards (99.99%) were purchased from Sigma-Aldrich (São Paulo, Brazil). The 6-C-glycoside-diosmetin and vicenin-2 standards were generated in our laboratory according to the methods described by de Oliveira et al. (2014). Valium <sup>R</sup> (diazepam) was purchased from Roche (São Paulo, Brazil). Sintocalmy <sup>R</sup> (standardized extract of Passiflora incarnate L. extract ACH 06) was obtained from Aché (Guarulhos, Brazil). Ro25-6981, picrotoxin and (S)-WAY100135 were purchased from Tocris Biosciences (Ellisville, MO, USA). NMDA was obtained from Sigma-Aldrich (São Paulo, Brazil). Buspirone hydrochloride was obtained from LIBBS Pharmaceutical Ltd (São Paulo, Brazil).

#### Standardized FfB Preparation

FfB was obtained by flash chromatography, as previously described by de Oliveira et al. (2014). Additionally, the FfB was analyzed using high-performance liquid chromatography (HPLC) combined with electrospray ionization tandem mass spectrometry (HPLC-ESI/MS<sup>n</sup> ) using a Thermo LCQ Fleet System mass spectrometer (Thermo Scientific, San Diego, CA, USA) equipped with an electrospray interface (ESI) and an HPLC (model Accela, Thermo Scientific). FfB separation was performed using a Luna <sup>R</sup> C18 column (250 × 4.60 mm; Phenomenex, Torrance, CA, USA) at room temperature. The mobile phase consisted of 0.1% aqueous formic acid-water (A) and methanol (B). A gradient elution method of A/B (from 64:36 to 1:1, v/v) was applied over 50 min. Ultraviolet (DAD) detection was performed at 330 nm; the flow rate was maintained at 0.8 mL/min; the sample concentrations were 1 mg.mL−<sup>1</sup> ; and the injection volume was 10µL. The column effluents were analyzed by ESI-MS in negative ion mode in the mass-to-charge ratio (m/z) range of 50–2000, with a scan time of 0.3 s in the centroid mode. The ESI conditions were as follows: nebulizer gas (nitrogen), 30 psi; drying gas, 60 L.min−<sup>1</sup> ; drying temperature, 280◦C; capillary voltage, 4000 V; collision gas, nitrogen; and collision energy,1 V. The data were acquired in the MS and MS<sup>n</sup> scanning modes. The CE was dissolved in H2O: MeOH (1:1v/v) and was infused directly via a syringe pump (flow rate 5µL.min−<sup>1</sup> ) in the ESI source. The data were analyzed using Xcalibur 2.0 Software <sup>R</sup> (Thermo Scientific).

The flavonoids present in the FfB were quantified by HPLC-DAD using a Luna <sup>R</sup> C18 column (Phenomenex, Torrance, CA, USA; 250 mm × 4.60 mm, 5µm). The mobile phase consisted of 0.1% aqueous formic acid (A) and methanol (B). An isocratic elution method of A/B (64:36, v/v) was applied for 50 min. UV spectra were recorded from 200 to 400 nm, and the chromatogram was monitored at 254, 280, and 330 nm. The flow rate was maintained at 1 mL.min−<sup>1</sup> ; the sample concentration was 1 mg.mL−<sup>1</sup> ; and the injection volume was 20µL. Analytical curves were obtained for vitexin, isovitexin, vicenin-2, and 6-C-glycoside-diosmetin (1 mg.mL−<sup>1</sup> of each compound in 80:20 methanol/water), which peaked at concentrations ranging from 100 to 1.000 mg.mL−<sup>1</sup> . The sample peak areas were integrated at 254 nm. All of the procedures were performed in triplicate.

# Behavioral and Pharmacological Effects of Acute Treatment with FfB Before Conditioning on the Acquisition and Extinction of Conditioned Suppression Subjects

A total of 470 adult male Wistar rats (±250–300 g) were obtained from the Center for the Development of Experimental Medicine and Biology (CEDEME, Federal University of Sao Paulo, SP, Brazil). The rats were housed 5 animals/cage. For 15 days, the animals had free access to food and water under a 12 h:12 h dark:light cycle (lights on at 6:00–18:00 h) at a controlled temperature (21◦C ± 2 ◦C) and relative humidity (53 ± 2%). These conditions were maintained throughout the experimental period. One minute prior to the experimental sessions, each rat was placed in an individual cage for transportation to the testing room. All of the procedures for manipulation of the animals were consistent with the Ethical Principles in Animal Research adopted by the Brazilian College for Animal Experimentation (COBEA) and were performed as suggested by the APA Guidelines for Ethical Conduct in the Care and Use of Animals. The protocol was approved by the Committee on the Ethics of Animal Experiments of the Federal University of Sao Paulo (Permit Number: 840560). After completion of the behavioral experiments, the animals were decapitated, and the DH was extracted within 40–60 s using a magnifying glass, immediately frozen on dry ice, and maintained at −80◦C until gene expression analysis. All behavioral procedures were conduced during the light phase of the dark:light cycle, and all efforts were made to minimize suffering.

#### Systemic Administration

Diazepam and buspirone hydrochloride (a GABAAR and a 5-HT1AR agonist, respectively), Sintocalmy <sup>R</sup> (a standardized extract of Passiflora incarnata L.-extract ACH 06, containing 7% (21 mg) total flavonoids expressed as vitexin) and three different concentrations of FfB were dissolved in 12% Tween <sup>R</sup> - 80 and administered orally via intragastric gavage (IG) 30 min before each conditioning session. The GABAAR, 5-HT1AR, and GluN2B-NMDAR antagonists (picrotoxin, S-WAY 100135 and Ro25-6981, respectively) and NMDA (an NMDAR agonist) were dissolved in saline and injected intraperitoneally (i.p.) 20 min before each conditioning session or prior to treatment with FfB. When an antagonist was administered before with FfB, the drugs were administered −50 or −30 min before conditioning, respectively. No drugs were administered before the retention test, extinction training or the extinction retention test. The drugs were administered i.p. or IG in a volume of 1 mL. The doses, administration routes and vehicles used to dissolve of the antagonists and agonists were chosen based on previous reports (Aguilar et al., 1997; Risbrough et al., 2003; Oliveira et al., 2009).

# Experimental Procedure

Rats were randomly assigned to the control group or the FfB group (n = 20 per subgroup) (**Table 1**). The control group was subdivided into 12 subgroups as follows: (i) the paired stimulus conditioned/unconditioned stimulus (CS-US) subgroup; (ii)

TABLE 1 | Experimental groups.

the unconditioned subgroup [no footshock, i.e., only tone (CS); as such, these animals were used as controls for learning]; (iii–iv) the negative control subgroups (12% Tween <sup>R</sup> -80 or saline); (v–xi) the positive control subgroups (4.0 mg.Kg−<sup>1</sup> diazepam; 10.0 mg.Kg−<sup>1</sup> NMDA; 10.0 mg.Kg−<sup>1</sup> buspirone hydrochloride; 600 mg.Kg−<sup>1</sup> Sintocalmy <sup>R</sup> ; 0.75 mg.Kg−<sup>1</sup> picrotoxin; 3.0 mg.Kg−<sup>1</sup> Ro25-6981; or 0.3 mg.Kg−<sup>1</sup> (S)-WAY 100135; these animals were used as controls for treatment with the respective drug together with FfB); and (xvii) a naïve subgroup (n = 10), which was used as a control for gene expression. The FfB groups were also divided into 12 subgroups as follows: (xiii–xiv) FfB alone (0.15 mg.Kg−<sup>1</sup> FfB, 0.30 mg.Kg−<sup>1</sup> FfB or 0.65 mg.Kg−<sup>1</sup> FfB); (xv–xvii) picrotoxin+FfB (Picro+ 0.15 mg.Kg−<sup>1</sup> FfB; Picro+0.30 mg.Kg−1FfB; or Picro+0.65 mg.Kg−<sup>1</sup> FfB); (xviii–xx) Ro25-6981+FfB (Ro+0.15 mg.Kg−<sup>1</sup> FfB; Ro+0.30 mg.Kg−<sup>1</sup> FfB or Ro+0.65 mg.Kg−<sup>1</sup> FfB); and (xxi– xxiii) (S)-WAY+FfB [(S)-WAY+0.15 mg.Kg−<sup>1</sup> FfB; (S)-WAY+ 0.30 mg.Kg−<sup>1</sup> FfB or (S)-WAY+0.65 mg.Kg−<sup>1</sup> FfB]. Half of the rats (n = 10/subgroup) were sacrificed after the retention test ended. The remaining half (n = 10/subgroup) were subjected to extinction training and an extinction retention test of the CER and were sacrificed 3 h after the conclusion of the extinction retention test.

#### Behavioral Apparatus

Rats were fear conditioned in a lick-operant chamber. Briefly, the experimental chambers consisted of an aluminum (side walls) and Plexiglas (ceiling and hinged front door) box measuring 25×25 × 20 cm set inside a sound-attenuation cabinet (53×65 × 50 cm). Three identical chambers and cabinets were used in all experiments. The floor consisted of stainless steel rods connected to grid shockers (model EP 107R, Insight, Ribeirão Petro, Brazil) set to deliver 0.4 mA, 0.5 s scrambled shocks, which were used as the US. A speaker positioned on top of the square, which produced a 2 kHz, 85 dB sound for 30 s, was used as the CS. A licking spout was slipped into the cage through a hole in the middle of the wall of the chamber; this hole protruded from the lateral wall 5.0 cm above the grid floor. Stimulus presentation and data recording were controlled using software (Refor II Software <sup>R</sup> , Insight) and a central controller box (Insight). The chambers were cleaned with 10% ethanol before each test.

#### Behavioral Procedure

The behavioral procedure was conducted for 8 or 10 days, according to the experimental design, to assess the acquisition or extinction of a CER, respectively. All rats, except for those in the CS and naïve subgroups, were subjected to a procedure to induce acquisition of fear memory (n = 20/group) (**Figure 1**). Three hours after the completion of the fear acquisition test (8th day), half of the rats were decapitated. Then, the DH was extracted within 40–60 s using a magnifying glass, immediately frozen on dry ice, and maintained at −80◦C until gene expression analysis (acquisition analysis) (n = 3/subgroup). The remaining half of the animals (n = 10/subgroup) were subjected to extinction training (9th day) and an extinction retention test (10th day) performed on each of the two consecutive days following the acquisition test. Three hours after completing the extinction retention test, these rats were decapitated, and the DH was extracted as described above (n = 3/group).

#### **Suppression of the licking response**

The animals were deprived of water on a daily basis for 12–16 h before all experimental sessions. For five consecutive days, the rats were placed individually in the chamber once a day for 20 min sessions with free access to the drinking spout to obtain a stable baseline of drinking behavior, but no other stimuli were presented (**Figure 1A**). After the administration of drugs or vehicle, each rat was gently placed in the experimental chamber, and after 5 min, the animal was submitted to four toneshock (CS-US) pairings (fear conditioning, 6th day; **Figure 1B**). Twenty-four hours after fear conditioning, the animals were subjected to reacquisition of the licking response sessions (7th day) as performed during the acquisition of the licking response to re-establish drinking behavior after conditioning and to reduce

#### FIGURE 1 | Schematic outline of the experimental procedure and drug administration time, common to all animals, except for the CS-US, and CS groups, in which did not receive the drug or vehicle. (1A) The animals were submitted to acquisition of the licking response for 5 days (baseline behavior). (1B) On day 6, the animals were submitted to four associations of CS-US (conditioning). (1C) Twenty-four hours later (day 7), the animals were submitted to re-acquisition of licking behavior, in conditions identical to those of the acquisition period (1–5 days). Retention Test (1D), Extinction Training (1E), and Extinction Test (1F) were performed on days 8, 9, and 10, respectively. Ten CS trials were presented at these times. No drugs were administered during the tests and extinction training sessions.

contextual cues (**Figure 1C**). The retention test was performed 48 h after acquisition (8th day) to evaluate the acquisition of fear memory as well as the effects of drug treatment. Here, each rat was placed in the experimental chamber with free access to the water spout and was subjected to the CS on 10 consecutive trials, in which the time to complete 10 licks pre-tone (no CS) and during the tone (CS) were recorded, and the suppression ratio (SR) was calculated for each trial. The tone was presented immediately after the animal completed its 90th lick and was switched off after its 100th lick (**Figure 1D**). The latency to complete licks 0–80 was recorded to ensure that the rats were licking when the tone was presented, but this value was not used to calculate the suppression of the lick response. The latency to complete licks 81–90 was measured as a control for time in the absence of a tone and was used to calculate the SR.

Therefore, the SR was calculated as the ratio of B/(A+B) for each rat, where A is the time to complete 10 licks pre-tone (pre-CS), i.e., time to complete licks 81–90 and B is the time to complete 10 licks during the CS, i.e., time to complete licks 91–100.

#### **Extinction of suppression of the licking response**

Analysis of the effects of FfB on extinction was performed using the behavioral protocol described for acquisition. All rats were subjected to tests of adaptation (1st–5th days), acquisition (6th day), reacquisition (7th day), and retention of the CER (8th day). Seventy-two hours (9th day) and ninety-six hours (10th day) after fear conditioning, the rats were placed in the experimental chamber for extinction training and extinction test sessions, respectively (**Figures 1E,F**). In both sessions, the latencies to complete licks before the tone and during the tone for 10 consecutive CS presentations were recorded as described for the 8 th day.

#### Data Analysis

The data from the first CS presentation indicated whether the association was learned. An SR approaching 1.0 indicates total suppression (high fear), whereas an SR ≤0.5 indicates no suppression (low fear), i.e., failure to learn the tone-shock relationship. The data are reported as the means ±SEM. A Twoway analysis of variance (ANOVA) was used to test for the presence of the effects of group and trial and the interaction between these variables; two fixed factors (group and trial), one random factor (rat), and repeated measurement of the trials were considered. P < 0.05 were considered significant. Graph Pad 6.0 Software <sup>R</sup> (version 6.0; Graph Pad Inc., San Diego, CA, USA) was used for data analysis.

# Expression of Gabra5, Htr1a, Grin2a, Grin2b, and Mapk1/Erk2 by Quantitative PCR (qPCR) Following Treatments Before Conditioning and Behavioral Analysis

The analysis of gene expression in the DH samples was extracted 3 h after the completion of the retention test or the extinction retention test as previously described. The candidate genes gamma-aminobutyric acid receptor subunit alpha-5 (Gabra5), 5-hydroxytryptamine (serotonin) receptor, subunit 1A (Htr1a), glutamate receptor ionotropic, NMDAR subunit GluN2A (Grin2a), glutamate receptor ionotropic, NMDAR subunit GluN2B (Grin2b), and extracellular signal-regulated kinase 2 (Erk2) were investigated. To this end, total RNA was isolated using Trizol reagent (Invitrogen Corp., Carlsbad, CA, USA) according to the manufacturer's recommendations. One microgram of total RNA was subjected to DNA-free DNase treatment (AMBION, Austin, TX, USA) and reverse-transcribed into cDNA using the SuperScript <sup>R</sup> III Reverse Transcriptase kit (Invitrogen Corp.) together with oligo12−<sup>18</sup> primer and 10 units of an RNase inhibitor (Invitrogen Corp.). Reverse transcriptasenegative samples were prepared for each individual reaction and were used as controls for assay contamination. Aliquots of 1µL of cDNA were used in 12µL reactions containing SYBR Green Master Mix (PE Applied Biosystems, Foster City, CA) and 3pM of each primer for the target genes and the reference gene (RS8) as described previously (Cerutti et al., 2004). The primer sequences are displayed in (Table S1). The qPCR reactions were performed in triplicate, and the threshold for each cycle (Ct) was obtained using Applied Biosystems software (Applied Biosystems) and averaged [standard deviation (SD) ≤ 1]. Relative expression (RE) levels were calculated using the 2−△△CT method (ddCt formula) as described previously (Cerutti et al., 2004). The vehicle (12% Tween <sup>R</sup> -80 or saline) was used as a control.

The analyses were performed using Graph Pad 6.0 Software <sup>R</sup> (version 6.0; Graph Pad Inc., San Diego, CA, USA). For candidate gene expression analysis, normality of the data was verified using the Shapiro-Wilk normality test. One-way ANOVA followed by a post-hoc Bonferroni test was performed to evaluate the relationships between the expression levels of Gabra5, Htr1a, Grin2a, Grin2b, and Erk2 across groups. P < 0.05 were considered significant.

# RESULTS

### Identification of Flavonoids in FfB

The spectroscopic and chromatographic data of the peaks (1– 6) of the FfB are summarized in **Figures S1A,B**. The identities, fragmentation patterns and UV spectra were confirmed as follows: (1) vicenin-2:λmax = 334, 271 nm, [M-H]<sup>−</sup> = m/z 593; (2) vicenin-1:λmax = 332, 271 nm, [M-H]<sup>−</sup> = m/z 563; (3) vitexin:λmax = 269, 235 nm, [M-H]<sup>−</sup> = m/z 431; (4) isovitexin:λmax = 335, 271 nm, [M-H]<sup>−</sup> = m/z 431; (5) 6-Cglycoside-diosmetin:λmax = 342, 270 nm, [M-H]<sup>−</sup> = m/z 461; and (6) apigenin:λmax = 305, 265 nm, [M-H]<sup>−</sup> = m/z 269. These results were consistent with those previously reported by de Oliveira et al. (2014). The identification of 6-C-glycosidediosmetin, vicenin-2, vitexin and isovitexin was supported by the co-injection of the standards and FfB. The flavones (1, 3, 4, and 5) found in the FfB were quantified by HPLC-DAD, and the concentrations contained in the FfB were 0.15 mg/g vicenin-2, 0.20 mg/g vitexin, 0.30 mg/g isovitexin, and 0.25 mg/g 6-C-glycoside-diosmetin.

Despite the evidence from our studies, few studies have examined the effects of a flavonoid fraction on fear memory. Further, previous data from our group suggest that the FfB may modulate different neurochemical systems.

# Behavioral, Pharmacological, and Molecular Analysis

The timelines illustrating the time points of drug administration and of brain removal are shown in **Figures 2A**, **3A**, **4A** and **5A**. The effects of treatment with FfB and with agonists and antagonists specific to 5-HT1ARs, GluN2B-NMDARs, and GABAARs or antagonists before FfB on the acquisition and extinction of the suppression of the licking response were assessed according to the mean SR for each tone, measured across 10 trials **Figures 2B**, **3B**, **4B** and **5B**. Every figure shows the mean SRs for the CS on the first trial and each three-trial block from the retention test, extinction training, and extinction retention test sessions. The first trial is presented independently because it represents the first presentation of the CS after conditioning, extinction training, or retrieval of extinction; thus, the results from this trial can characterize the level of fear of the animal in each situation. In addition, the results from the first trial can show (i) the duration of fear memory expression and (ii) the occurrence of spontaneous recovery. The means (±SEM) for each first trial

FIGURE 2 | (A) Timeline illustrating the time points of drug administration and brain removal. (B) Mean SR of licking behavior in the retention test session (8th day, n = 20/group), extinction training (9th day, n = 10/group) and the extinction retention test (10th day, n = 10/group). The first point indicates the mean SR for the CS, learning, Sintocalmy®, Tween®, 0.15 mg.Kg−<sup>1</sup> FfB, 0.30 mg.Kg−1FfB, and 0.65 mg.Kg−1FfB subgroups. The subsequent data points represent the mean of nine trials in blocks of three trials. The drugs and vehicle were administered orally 30 min before the fear conditioning session; the CS and CS-US groups received no treatment. The data are reported as the means (±SEM). A repeated measures ANOVA was employed for the intra-group comparison of the retention test, extinction training and extinction retention test (CS presentation) results. This analysis was performed considering two fixed factors (group and trial) and one random factor (rat) using GraphPad Prism software. he relative Htr1a, Grin2b, Grin2a, Gabra5, and Erk2 mRNA expression levels in the DH after acute treatment with Sintocalmy®, Tween®, 0.15 mg.Kg−1FfB, 0.30 mg.Kg−1FfB, or 0.65 mg.Kg−1FfB (n = 3/subgroup) followed by the retention test (C) or the extinction retention test (D). The CS-US, CS, and naïve subgroups did not receive treatment (n = 3/subgroup). The values are expressed as the means (± SEM). \*P < 0.05 and \*\*\*P < 0.0001. ###P < 0.0001 according to ANOVA followed by post-hoc Bonferroni tests, when necessary.

fear conditioning session. The data are reported as the means (±SEM). A repeated measures ANOVA was employed for the intra-group comparison of the retention test, extinction training and extinction retention test (CS presentation) results. This analysis was performed considering two fixed factors (group and trial) and one random factor (rat) using GraphPad Prism software. The relative Htr1a, Grin2b, Grin2a, Gabra5, and Erk2 mRNA expression levels in the DH after acute treatment with 0.3 mg.Kg−<sup>1</sup> (S)-WAY100135, 10 mg.Kg−<sup>1</sup> buspirone, 0.3 mg.Kg−<sup>1</sup> (S)-WAY100135 + (FfB 0.15 mg.Kg−<sup>1</sup> FfB, 0.30 mg.Kg−<sup>1</sup> FfB, or 0.65 mg.Kg−<sup>1</sup> FfB) or saline (n = 3/subgroup) followed by the retention test (C) orthe extinction retention test (D). The values are expressed as the means (± SEM). \*P < 0.05 and \*\*\*P < 0.0001. ###P < 0.0001 according to ANOVA followed by post-hoc Bonferroni tests, when necessary.

and block of three trials are presented in Tables S2, S4, S6 and S8.

To investigate the molecular mechanisms involved in modulating the suppression of the licking response by FfB, the expression levels of Grin2a, Grin2b, Gabra5, Htr1a, and Erk2 in the DH were assayed by qRT-PCR. The effects of FfB, agonists and antagonists specific to the glutamatergic, serotoninergic and GABAergic systems were evaluated 3 h after the retention test session (8th day; **Figures 2C**, **3C**, **4C** and **5C**) and the extinction retention test session (10th day; **Figures 2D**, **3D**, **4D** and **5D**). The mean ± SEM values for the RE of candidate genes (ddCt) are available in Tables S3, S5, S7, S9.

Further, we have made statistical comparison between control groups, which received the vehicle solutions (Saline and Tween <sup>R</sup> ). To comparison of SR means during all trial of presentation of CS we used Paired t-test. No statistically significant difference was found between-session or intra-session (P = 0.1450). To comparisons of differential gene expression from samples of DH, we have used unpaired T-test. Comparisons

FIGURE 4 | (A) Timeline illustrating the time points of drug administration and brain removal. (B) Mean SR of licking behavior in the test session (8th day, n = 20/group), extinction training (9th day, n = 10/group) and the extinction retention test (10th day, n = 10/group). The first point indicates the mean SR for the 0.9% saline, 3.0 mg.Kg−<sup>1</sup> Ro25-6981, 10 mg.Kg−<sup>1</sup> NMDA, and Ro25-6981+FfB (0.15 mg.Kg−<sup>1</sup> FfB, 0.30 mg.Kg−<sup>1</sup> FfB, or 0.65 mg.Kg−<sup>1</sup> FfB) subgroups. The subsequent data points represent the mean of nine trials in blocks of three trials. The drugs and vehicle were administered orally 30 min before the fear conditioning session. The data are reported as the means (±SEM). A repeated measures ANOVA was employed for the intra-group comparison of the retention test, extinction training and extinction retention test (CS presentation) results. This analysis was performed considering two fixed factors (group and trial) and one random factor (rat) using GraphPad Prism software. The relative Htr1a, Grin2b, Grin2a, Gabra5, and Erk2 mRNA expression levels in the DH after acute treatment with 3.0 mg.Kg−<sup>1</sup> Ro25-6981, 10.0 mg.Kg−<sup>1</sup> NMDA, 3.0 mg.Kg−<sup>1</sup> Ro25-6981+FfB (0.15 mg.Kg−<sup>1</sup> FfB, 0.30 mg.Kg−<sup>1</sup> FfB, or 0.65 mg.Kg−<sup>1</sup> FfB), or saline (n = 3/subgroup) followed by the retention test (C) or the extinction retention test (D). The values are expressed as the means (±SEM). \*P < 0.05, \*\*P < 0.01 and \*\*\*P < 0.0001. ###P < 0.0001 according to ANOVA followed by post-hoc Bonferroni tests, when necessary.

between Saline and Tween <sup>R</sup> groups, after retention test (8th day), were made to each gene evaluated. No significant difference was observed in the RE levels of Htr1a (P = 0.9737), Grin2b (P = 0.9691), Gabra5 (P = 0.9592), Grin2a (P = 0.7358), or Erk2 (P = 0.0962). Comparisons between Saline and Tween <sup>R</sup> groups, after extinction retention test (10th day), were made to each gene evaluated, similarly to aforementioned, no significant difference between groups was observed in the RE levels of Htr1a (P = 0.5834), Grin2b (P = 0.9208), Gabra5 (P = 0.9982), Grin2a (P = 0.9628), or Erk2 (P = 0.1469).

#### Effects of FfB on the Acquisition and Extinction of Suppression of the Licking Response

The effects of FfB on the acquisition and extinction of suppression of the licking response are shown in **Figure 2B** and Table S2. A Two-way ANOVA revealed a significant group × trial interaction [F(66, 756) = 2.785, P < 0.0001], a main effect of group [F(66, 756) = 24.56, P < 0.0001] and a main effect of trial [F(11, 756) = 17.27, P < 0.0001].

Comparisons of the results for the first trial in the retention test sessions between groups revealed elevated SR in the

n = 20/group), extinction training (9th day, n = 10/group), and the extinction retention test (10th day, n = 10/group). The first point indicates the mean SR of the 0.9% saline, 0.75 mg.Kg−<sup>1</sup> picrotoxin, 4 mg.Kg−<sup>1</sup> diazepam, and 0.75 mg.Kg−<sup>1</sup> picrotoxin+FfB (0.15 mg.Kg−<sup>1</sup> FfB, 0.30 mg.Kg−<sup>1</sup> FfB, or 0.65 mg.Kg−<sup>1</sup> FfB) subgroups. The subsequent data points represent the mean of nine trials in blocks of three trials. The drugs and vehicle were administered orally 30 min before the fear conditioning session. The data are reported as the means (±SEM). A repeated measures ANOVA was employed for the intra-group comparison of the retention test, extinction training and extinction retention test (CS presentation) results. This analysis was performed considering two fixed factors (group and trial) and one random factor (rat) using GraphPad Prism software. The relative Htr1a, Grin2b, Grin2a, Gabra5, and Erk2 mRNA expression levels in the DH after acute treatment with 0.75 mg.Kg−<sup>1</sup> picrotoxin, 4 mg.Kg−<sup>1</sup> diazepam, 0.75 mg.Kg−<sup>1</sup> picrotoxin+FfB (0.15 mg.Kg−<sup>1</sup> FfB, 0.30 mg.Kg−<sup>1</sup> FfB, or 0.65 mg.Kg−<sup>1</sup> FfB), or saline (n = 3/subgroup) followed by the retention test (C) or the extinction retention test (D). The values are expressed as the means (±SEM). \*P < 0.05, \*\*P < 0.01 and \*\*\*P < 0.0001. ###P < 0.0001 according to ANOVA followed by post-hoc Bonferroni tests, when necessary.

subgroups treated with FfB, CS-US or Tween and reduced SR in the subgroups treated with Sintocalmy <sup>R</sup> or CS (P < 0.0001; left panel of **Figure 2B**). The analysis of SR in the first threetrial block (2nd–4th trials) showed a significant decrease in mean SR relative to the first trial in the Tween <sup>R</sup> , CS-US, and FfB subgroups (P < 0.0001); these results indicated the acquisition of extinction of fear memory within the session. An ANOVA comparing the three-trial blocks revealed no differences within sessions (P > 0.05). This finding indicated a reliable decrease in suppression and a reduction of fear after each session.

The middle panel of **Figure 2B** depicts the data from the extinction training session conducted 24 h after the retention test. Treatment with FfB at all doses promoted spontaneous recovery, as demonstrated by the results for the first trial in each subgroup, compared to treatment with Tween <sup>R</sup> or Sintocalmy <sup>R</sup> or to CS or CS-US alone. However, in subsequent trials, rats treated with FfB acquired fear extinction within the session (P < 0.0001). The Tween <sup>R</sup> and CS-US subgroups exhibited a similar SR mean across successive exposures to the CS during the extinction training session, as observed in the retention test. The

Sintocalmy <sup>R</sup> and CS subgroups showed mean SRs during all CS presentations that were similar to those measured in the retention test (P > 0.05). This result indicated no conditioned fear and, as a consequence, no acquisition of fear extinction.

The right panel of **Figure 2B** shows the mean SRs in the extinction retention session conducted 24 h after extinction training. The subgroups treated with FfB exhibited spontaneous recovery in the first trial, similar to the behavior observed during the extinction training session. However, all subgroups exhibited similar behavior by the end of the session. Notably, for the first trials, the mean SRs for each subgroup treated with FfB were significantly different from those for the Tween <sup>R</sup> and CS subgroups (P < 0.0001). Comparisons between the first trial and the first three-trial block (2–4) showed reduced suppression of the licking response in the Tween <sup>R</sup> , CS-US and all FfB subgroups (P < 0.0001). An ANOVA comparing the results for the three-trial blocks within the session demonstrated no significant differences in the mean SR between the subgroups (P > 0.05). In addition, no significant differences in the mean SR were observed in the CS subgroup across all sessions (P > 0.05).

In summary, our data show for the first time that FfB does not impair the conditioned fear. However, rats treated with FfB showed spontaneous recovery of fear conditioning, as observed in the extinction training and extinction retention test sessions, although FfB did not prevent the acquisition within-session extinction. Furthermore, acute treatment with Sintocalmy <sup>R</sup> , a standardized extract containing 7% of the total flavonoids expressed in vitexin, impaired the conditioned fear and, consequently, resulted in no acquisition of the extinction of fear conditioning.

#### FfB Treatment Modulates the Spontaneous Recovery of Fear Memory via Htr1a and Erk2 Expression Within the DH

FfB treatment at three different doses resulted in the overexpression of Htr1a [F(7, 16) = 173.0, P < 0.0001], Grin2a [F(7, 16) = 165.2, P < 0.0001], Gabra5 [F(7, 16) = 40.82, P < 0.0001], and Erk2 [F(7, 16) = 155.5, P < 0.0001] in the DH after the retention test session (8th day) compared with the control treatments (Tween <sup>R</sup> , Sintocalmy <sup>R</sup> , CS-US, CS, and naïve; **Figure 2C**). The Htr1a, Grin2b, Grin2a, Gabra5, and Erk2 expression levels were increased in the control subgroups (Tween <sup>R</sup> and CS-US subgroups) compared with the CS and naïve subgroups (P < 0.0001). No difference was observed in Grin2b [F(7, 16) = 13.97] expression after treatment with FfB compared to the treatment with FfB and CS-US groups (P > 0.05; **Figure 2C** and Table S3), and Grin2a expression decreased after treatment with Sintocalmy <sup>R</sup> (P < 0.0001).

In the extinction retention test, Htr1a [F(7, 16) = 96.39] and Erk2 [F(7, 16) = 388.9; P < 0.0001] expression was significantly increased after FfB treatment in the DH compared with all other treatments (P < 0.0001; **Figure 2D** and Table S3). No significant difference in the RE levels of Grin2b [F(7, 16) = 12.20], Gabra5 [F(7, 16) = 16.11], or Grin2a [F(7, 16) = 181.6] was observed in the FfB-treated subgroups compared to the CS-US or Tween <sup>R</sup> subgroups (P > 0.05), although these expression levels were increased compared to the CS and naïve subgroups (P < 0.05). Furthermore, Grin2a expression was reduced following the extinction retention test due to treatment with Sintocalmy <sup>R</sup> compared with all other treatments (Tween <sup>R</sup> and CS-US treatments; P < 0.0001).

In summary, the acquisition and extinction of the suppression of the licking response modulated Htr1a, Grin2b, Grin2a, Gabra5, and Erk2 expression, and FfB treatment altered Htr1a, Grin2a, Gabra5, and Erk2 expression after the retention test. Furthermore, the spontaneous recovery of fear memory appears to correlate with the overexpression of Htr1a and Grin2a in the DH.

#### Effects of FfB on Fear Memory After Blocking 5-HT1ARs

**Figure 3B** illustrates the specific effects of blocking 5- HT1ARs before FfB treatment, which was administered before conditioning, on the results for the retention test, extinction training, and the extinction retention test. A Twoway ANOVA revealed a significant trial × group interaction [F(55, 648) = 1.365, P = 0.0453], a main effect of group [F(5, 648) = 2.792, P = 0.0166] and a main effect of trial [F(11, 648) = 9.116, P < 0.0001]. Similar mean SRs were observed between the Tween and saline groups across sessions (P > 0.05). Therefore, saline was used to compare the effects of antagonists and agonists together with FfB.

The left panel of **Figure 3B** shows the mean SRs in the CS, negative control (saline), positive control (0.30 mg.Kg−<sup>1</sup> (S)-WAY100135 and 10.0 mg.Kg−<sup>1</sup> buspirone), and treated subgroups [0.30 mg.Kg−<sup>1</sup> (S)-WAY+FfB (0.15 mg.Kg−<sup>1</sup> ; 0.30 mg.Kg−<sup>1</sup> ; or 0.65 mg.Kg−1FfB)] for the retention test session. The analysis of mean SR for the first trial showed that treatment with (S)-WAY100135 or (S)-WAY100135 before FfB resulted in reduced suppression of the licking response compared with saline (P < 0.0001) and buspirone treatment (P < 0.0001). Analysis of the SR for the first three-trial block (2nd–4th trials) showed significant differences in the mean SRs compared to the first trial in the saline and buspirone subgroups (P < 0.0001). This result demonstrates acquisition within-session extinction to these subgroups. Alternatively no such differences were observed in the (S)-WAY100135 or (S)-WAY+FfB subgroups (P > 0.05). An ANOVA comparing the first three-trial block (2–4) with the subsequent three-trial blocks (5–7 and 8–10) demonstrated no significant differences in mean SR on the extinction retention session between the subgroups (P > 0.05; see Table S4).

The data from the extinction training tests are shown in the middle panel of **Figure 3B**. Comparisons between groups showed that the groups treated with (S)-WAY+FfB, at all doses, did not demonstrate a difference in the mean SR (P > 0.05). Analysis of the mean SR during the first three-trial block (2–4) showed that rats treated with (S)-WAY+FfB, saline, buspirone or (S)- WAY100135 exhibited a similar mean SR to that in the first trial (P > 0.05). Moreover, similar mean SRs were observed with in all groups for the subsequent three-trial blocks (5–7 and 8–10; P > 0.05; see also Table S4).

The data from the extinction retention tests are shown in the right panel of **Figure 3B**. The analysis of the SR showed that the subgroups treated with (S)-WAY100135+FfB did not exhibit spontaneous recovery. Furthermore, the analysis of the mean SR showed no significant difference between the first three-trial block (2nd–4th trial) and the first trial among the (S)-WAY100135+FfB subgroups at all doses (P < 0.0001). Therefore, no significant differences in mean SRs were found among the subgroups between the first three-trial block (2–4) and the subsequent three-trial blocks (5–7 and 8–10; P > 0.05; see Table S4).

A Two-way ANOVA comparison between groups treated with FfB vs. (S)-WAY100135+FfB revealed a significant groups × trial interaction [F(55, 495) = 2.018, P < 0.0001] and main effects of trial [F(11, 99) = 21.25, P < 0.0001] and groups [F(5, 45) = 21.41, P < 0.0001]. Treatment with (S)-WAY100135+ 030 mg.Kg−<sup>1</sup> FfB and (S)-WAY100135+ 065 mg.Kg−<sup>1</sup> FfB, resulted in reduced of licking response compared with FfB group in the first trial during extinction training. No significant difference was observed among subsequent three-trial blocks. Furthermore, the analysis of the mean SR showed significant difference between (S)-WAY100135 + 0.65 mg.Kg−<sup>1</sup> FfB and 0.65 mg.Kg−<sup>1</sup> FfB to the first trial during extinction retention test (P < 0.0001). No significant difference was observed among subsequent three-trial blocks.

In summary, our data demonstrate that (S)-WAY+FfB, at all doses, reduces the suppression of the licking response compared with the control treatment, as demonstrated by the results from the retention test. These data suggest for the first time that the spontaneous recovery observed in the FfB subgroups is modulated by 5-HT1ARs.

#### (S)-WAY100135 Treatment Prevents the Overexpression of Htr1a and Erk2 Within the DH Caused by FfB

We used treatment with (S)-WAY100135 prior to FfB administration to assess the role of the 5-HT1AR in the acquisition and extinction of fear memory. In addition, the roles of NMDARs, GABAARS,and ERK2 were evaluated.

**Figure 3C** shows the levels of Htr1a, Gabra5, Grin2a, Grin2b, and Erk2 expression in the DH after the retention test session (8th day). Consistent with the results of Htr1a and Erk2 expression after FfB administration, treatment with (S)- WAY100135+FfB, at all doses, resulted in the downregulation of Htr1a expression [F(5, 12) = 449.9, P < 0.0001]. Treatment with (S)-WAY100135+FfB resulted in the downregulation of Htr1a expression when compared with saline and buspirone treatment [F(5, 12) = 40.05, P < 0.0001]. Although Erk2 expression was similar across all groups [F(5, 12) = 3.071, P = 0.0516]. A ANOVA comparison between the groups treated with (S)-WAY100135+FfB vs. FfB revealed that the overexpression of Erk2 observed after FfB treatment was reversed by (S)-WAY100135 pretreatment, at all doses [F(5, 12) = 57.79, P < 0.0001]. Furthermore, (S)-WAY100135+FfB induced the downregulation of Grin2a [F(5, 12) = 124.8] and Grin2b [F(5, 12) = 8.794;P = 0.001] expression compared with saline and buspirone (P < 0.0001). Moreover, (S)-WAY100135 treatment decreased the expression of Grin2a, but not Grin2b, and buspirone treatment reduced Grin2b expression compared with saline treatment (P < 0.0001). No significant changes in Gabra5 expression were observed [F(5, 12) = 2.505, P = 0.0894]. These statistics are shown in Table S5.

**Figure 3D** shows Htr1a, Gabra5, Grin2a, Grin2b, and Erk2 expression in the DH after the extinction retention test (10th day). These data show that treatment with (S)-WAY100135 before FfB administration, at three different doses, resulted in the overexpression of Grin2a [F(5, 12) = 278.4, P < 0.0001] compared with the control treatments [saline, buspirone, and (S)- WAY100135] and in the downregulation of Gabra5 compared to treatment with 0.15 mg.Kg−<sup>1</sup> FfB [F(5, 12) = 4.338, P = 0.0174]. Additionally, treatment with (S)-WAY100135+FfB resulted in the downregulation of Htr1a [F(5, 12) = 31.18, P < 0.0001], Erk2 [F(5, 12) = 119.9, P < 0.0001] and Gabra5 expression [F(5, 12) = 20.48, P < 0.0001] and overexpression of Grin2a in the DH [F(5, 12) = 82.00, P < 0.0001]. Furthermore, treatment with buspirone resulted in the upregulation of Htr1a expression [F(5, 12) = 72.92; P < 0.001] compared with all other treatments. Nevertheless, no significant differences were observed in the RE of Grin2b [F(5, 12) = 3.039, P = 0.0010] or Erk2 [F(5, 12) = 2.94, P = 0.0580]. Treatment with (S)-WAY100135 prior to FfB administration, at all three doses, prevented the upregulation of Htr1a expression observed after FfB treatment (**Figure 2D**; see also Table S5).

In summary, our data show that treatment with (S)- WAY100135 prior to FfB administration decreases Htr1a, Grin2b, and Grin2a expression in the DH after the retention test and prevents the increase in Htr1a and Erk2 expression after the extinction retention test in relation to observed after treatment with FfB alone. Conversely, Grin2a expression in the DH was increased after (S)-WAY100135+FfB treatment after the extinction retention test compared with FfB treatment.

#### Effects of FfB on Fear Memory After Blocking GluN2B-NMDARs

**Figure 4B** shows the effects of specifically blocking GluN2B-NMDARs with Ro25-6981 before FfB treatment, which was administered before conditioning, on the results from the retention test, extinction training and the extinction retention test. A Two-way ANOVA revealed a significant trial × group interaction [F(55, 648) = 2.170, P < 0.0001], a main effect of group [F(5, 648) = 3.356, P < 0.0001] and a main effect of trial [F(11, 648) = 11.57, P < 0.0001].

The left panel of **Figure 4B** shows the mean SRs for the retention test session. Comparisons of the mean SR on the first trial revealed a difference between the Ro25-6981 treated subgroup and all other subgroups (P > 0.0001); this result indicated that blockade of GluN2B impaired the acquisition of fear memory. However, treatment with Ro25- 6981 before FfB administration did not affect fear memory. FfB treatment reversed the learning impairment observed in the subgroup treated with Ro25-6981 alone. Analysis of the SR for the first three-trial block (2nd–4th trials) showed significant differences in mean SR for the saline, NMDA, Ro25-6981, andRo25-6981+FfB groups (0.15 mg.Kg−<sup>1</sup> ;0.30 mg.Kg−<sup>1</sup> ; or 0.65 mg.Kg−1FfB) compared with the first trial (P < 0.0001). Furthermore, an ANOVA comparing the subsequent three-trial blocks (5–7 and 8–10) with the first three-trial block of the test revealed no differences (P > 0.05). The Ro25-6981 subgroup showed similar SR values across all trials of CS presentation (statistics shown in Table S6).

However, comparisons of the first trial of the extinction training test between the subgroups showed that the subgroups treated with Ro25-6981+FfB, at all doses, did not demonstrate differences in mean SR compared with the saline and NMDA subgroups (P > 0.05) or the Ro25-6981 alone subgroup, which showed no acquisition of conditioned fear (middle panel of **Figure 4B**). Analysis of the SR for the first three-trial block (2nd–4th trials) compared with the first trial showed no significant differences in the mean SR for the subgroups treated with saline, NMDA, Ro25-6981, or Ro25-6981+FfB (P > 0.05). Nevertheless, no significant difference in SR was found within the groups for the first three-trial block (2–4) compared with the subsequent three-trial blocks (5–7 and 8–10; P > 0.05; see also Table S6).

The subgroups treated with Ro25-6981+FfB, at all doses, exhibited higher mean SRs for the first trial than the saline subgroup on the extinction retention test (P > 0.05; right panel of **Figure 4B**) and remained similar throughout the trials of the extinction training test. Comparisons between the first trial and the first three-trial block (2–4) showed reduced suppression of the licking response for all groups treated with Ro25-6981+FfB (P < 0.05). An ANOVA comparing the threetrial blocks demonstrated no significant differences in the mean SRs throughout the extinction retention session (P > 0.05; Table S6).

A Two-way ANOVA comparison between groups treated with FfB vs. Ro 25-6981+FfB revealed a significant groups × trial interaction [F(55, 495) = 2.094, P < 0.0001] and main effects of trial [F(11, 99) = 31.20, P < 0.0001] and groups [F(5, 45) = 2.873, P = 0.0247). Analysis of the mean SR showed significant difference between Ro 25-6981 + 0.65 mg.Kg−<sup>1</sup> FfB vs. 0.65 mg.Kg−<sup>1</sup> FfB to the first trial during extinction training (P < 0.0001). No significant difference was observed among subsequent three-trial blocks (P > 0.05). Furthermore, similar means SR were found to groups treated with Ro 25-6981+FfB to the first trial during extinction retention test. No significant difference was observed among subsequent three-trial blocks (P > 0.05).

In summary, our data show for the first time that treatment with Ro25-6981, an antagonist of the GluN2B-NMDAR, impairs the acquisition of suppression of the licking response. Conversely, treatment with after FfB after Ro25-6981 administration, at all doses, reverses the learning impairment associated with the GluN2B-NMDAR antagonist. In this sense, GluN2B is involved in the acquisition of suppression of the licking response, but the disruptive effects of Ro25-6981 appear to be offset by treatment with FfB. Additionally, we showed that the spontaneous recovery observed in the FfB subgroups may be modulated by GluN2B because rats treated with Ro25-6981 before FfB administration seems to decrease the spontaneous recovery observed during the extinction training sessions compared with the rats treated with FfB alone (see **Figure 2B**).

#### Ro25-6981 Treatment does not Prevent the Overexpression of Grin2a and Erk2 Caused by FfB, Although it Reduces Htr1a Expression

We used treatment with Ro25-6981 prior to FfB to evaluate the roles of NMDARs, 5-HT1ARS, GABAARS, and ERK2. **Figure 4C** shows the Htr1a, Grin2a, Grin2b, Gabra5, and Erk2 expression levels in the DH after the retention test (8th day). Treatment with Ro25-6981 before FfB treatment, at all three doses, resulted in the overexpression of Grin2a [F(5, 12) = 107.1, P < 0.0001] and Erk2 [F(5, 12) = 90.89, P < 0.0001] and the decreased expression of Htr1a [F(5, 12) = 32.67, P < 0.0001] and Gabra5 [F(5, 12) = 12.44, P = 0.0002] compared with the control treatment. No change in Grin2b was observed after Ro25-6981 or Ro25-6981+FfB treatment [F(5, 12) = 20.18]. A ANOVA analysis revealed that treatment with Ro25-6981 before FfB, resulted in the overexpression of Grin2a [F(5, 12) = 134.8, P < 0.0001] and Erk2 [F(5, 12) = 47.98, P < 0.0001] and the decreased expression of Htr1a [F(5, 12) = 361.7, P < 0.0001] and Gabra5 [F(5, 12) = 32.57, P < 0.0001] in relation to groups treated with FfB alone. Additionally, NMDA treatment resulted in the overexpression of Grin2b compared with all other treatments (P < 0.0001) and in the overexpression of Erk2 compared with saline or Ro25-6981 treatment (P < 0.001; see Table S7).

**Figure 4D** shows that treatment with Ro25-6981+FfB increased the Grin2a [F(5, 12) = 14.47, P = 0.0001], Erk2 [F(5, 12) = 44.78, P < 0.0001], Htr1a [F(5, 12) = 158.6, P < 0.0001], and Grin2b [F(5, 12) = 5.37, P = 0.008] expression levels in the DH after the extinction retention test (10th day) compared with saline treatment. No significant differences in Gabra5 expression were observed between the subgroups treated with Ro25-6981+FfB and the saline subgroup [F(5, 12) = 1.169, P = 0.3790]. Furthermore, comparison among groups treated with Ro25-6981 before FfB treatment resulted in the overexpression of Grin2a [F(5, 12) = 32.68, P < 0.0001] and Erk2 [F(5, 12) = 34.58, P < 0.0001] and Htr1a [F(5, 12) = 299.0, P < 0.0001]. Further, significant difference tog Grin2b was seeing to groups treated with Ro25-6981 before 0.65 mg.Kg−<sup>1</sup> FfB in relation to 0.65 mg.Kg−<sup>1</sup> FfB group [F(5, 12) = 4.727, P = 0.0128].

In summary, treatment with Ro25-6981 reduced the expression of Gabra5, Erk2, and Ht1ra in the DH after the retention test, although treatment with FfB reduced the effects of Ro25-6981 on Gabra5 and Htr1a expression and increased Grin2a and Erk2 expression. Conversely, treatment with Ro25- 6981+FfB increased Htr1a expression after the extinction retention test. Furthermore, treatment with FfB after Ro25-6981 administration increased the Htr1a, Grin2b, Grin2a, and Erk2 expression levels in the DH after the extinction retention test (see Table S7).

#### Effects of FfB on Fear Memory After Blocking GABAARs

The effects of specifically blocking GABAARs prior to FfB treatment before conditioning on the results of the retention test, extinction training, and the extinction retention test are shown in **Figure 5B**. A Two-way ANOVA revealed a significant trial × group interaction [F(55, 648) = 2.695, P < 0.0001], a main effect of group [F(5, 648) = 6.416, P < 0.0001] and a main effect of trial [F(11, 648) = 11.77, P < 0.0001].

Analysis of the mean SRs for the first trial of the retention test showed that treatment with picrotoxin, an antagonist of GABAARs, or picrotoxin+FfB did not prevent the acquisition of conditioned fear. These subgroups showed a similar mean SR to the saline subgroup (P > 0.05). Conversely, animals treated with diazepam exhibited reduced suppression of the licking response compared with the animals treated with saline, picrotoxin or picrotoxin+FfB (P < 0.001).

Analysis of the SR for the first three-trial block (2nd–4th trials) compared with the first trial showed significant differences in the mean SR for the subgroups treated with saline, picrotoxin, or picrotoxin+FfB (0.15 mg.Kg−<sup>1</sup> , 0.30 mg.Kg−<sup>1</sup> , or 0.65 mg.Kg−1FfB; P < 0.0001). An ANOVA comparing the first three-trial block (2–4) of extinction with the other three-trial blocks (5–7 and 8–10) demonstrated no significant differences in the mean SR between the subgroups (P > 0.05; see Table S8).

The data from the extinction training session are shown in the middle panel of **Figure 5B**. Comparisons between the groups showed that the subgroups treated with picrotoxin+FfB, at all doses, demonstrated differences in mean SRs in the first trial compared to the saline, picrotoxin and diazepam subgroups (P < 0.0001). In addition, rats treated with picrotoxin+FfB showed spontaneous recovery similar to that observed in rats treated with FfB alone (see **Figure 2B**). Furthermore, no differences in SR on the first trial were observed between the saline and picrotoxin subgroups (P > 0.05). Analysis of the mean SR during the first three-trial block (2–4) showed that rats treated with picrotoxin+FfB at all doses demonstrated reduced suppression of the licking response compared with the mean SR for the first trial (P < 0.0001). The saline and picrotoxin subgroups exhibited a similar mean SR across successive exposures to the CS during extinction training. Similar mean SRs were observed for all groups across the subsequent three-trial blocks (5–7 and 8–10; P > 0.05; see Table S8).

Similar to the previous sessions, on the extinction retention test, rats treated with picrotoxin+FfB, at all doses, showed spontaneous recovery on the first trial, as demonstrated by the higher SR means in the picrotoxin+FfB subgroups (P > 0.001; **Figure 5B**, right panel). A reduced mean SR was observed in the picrotoxin+FfB subgroups on the first three-trial block compared to the first trial (P < 0.0001). No significant difference in the SR was found within the groups for the first three-trial block (2–4) compared with the subsequent three-trial blocks (5–7 and 8–10; P > 0.05; see also Table S8).

A Two-way ANOVA comparison between groups treated with FfB vs. Picrotoxin+FfB revealed a significant groups × trial interaction [F(55, 495) = 1.091, P = 0.3116] and main effects of trial [F(11, 99) = 45.11, P < 0.0001] and groups [F(5, 45) = 5.375, P = 0.0006]. Analysis of the mean SR showed no significant difference between groups treated with FfB vs. Picrotoxin+FfB, at all doses, to the first trial during retention test (P > 0.05), extinction training (P > 0.05) and extinction test sessions (P > 0.05). Significantly difference were observed among groups treated with FfB, at a dose 0.15 mg.Kg−<sup>1</sup> and 0.65 mg.Kg−1FfB (P < 0.05). Furthermore, analysis of SR means for the first trial showed significant difference among rats treated with different doses of FfB groups during extinction training and extinction retention test (P < 0.001). No significant difference was observed among subsequent three-trial blocks (P > 0.05).

In summary, our data demonstrated that each group treated with picrotoxin prior to FfB administration acquired fear memory. Conversely, diazepam treatment impaired the acquisition of fear memory. Furthermore, we showed that treatment with picrotoxin+FfB resulted in spontaneous recovery in the first trial of extinction training and the extinction retention test, although the suppression gradually decreased over the trials. Therefore, rats treated with FfB showed within-session extinction of fear memory. These data suggest that spontaneous recovery is not modulated by GABAARs.

#### Picrotoxin Prevents the Overexpression of Gabra5 and Grin2a Caused by FfB

Treatment with picrotoxin prior to FfB treatment, at a dose of 0.15 or 0.65 mg.Kg−<sup>1</sup> FfB, did not prevent the increase in the expression of Htr1a [F(5, 12) = 28.02, P < 0.0001] or Erk2 [F(5, 12) = 84.48, P < 0.0001] in the DH after the retention test compared with saline, picrotoxin, or diazepam treatment (**Figure 5C**), as observed in the subgroups treated with FfB alone (**Figure 2C**). A ANOVA analysis revealed that treatment with picrotoxin before FfB, resulted in the downexpression of Htr1a [F(5, 12) = 128.7 P < 0.0001], Grin2b [F(5, 12) = 15.22, P < 0.0001] and Erk2 [F(5, 12) = 22.81, P < 0.0001], at all doses, and Gabra5 [F(5, 12) = 28.02, P < 0.0001] and Grin2a [F(5, 12) = 8515, P < 0.00002], at a higher doses in relation to groups treated with FfB alone. Additionally, picrotoxin+FfB treatment did reduce Grin2b expression [F(5, 12) = 14.42, P < 0.0001], but no change was observed in the expression of Grin2a [F(5, 12) = 12.24, P = 0.0001] or Gabra5 [F(5, 12) = 7.580, P = 0.0020]. Furthermore, picrotoxin increased Gabra5, and Erk2 expression in the DH (P < 0.0001). Conversely, diazepam treatment decreased Htr1a, Grin2a, Erk2, and Grin2b expression in the DH (see Table S9).

The data shown in **Figure 5D** demonstrate the upregulation of Erk2 [F(5, 12) = 90.76, P < 0.0001] and Grin2a [F(5, 12) = 67.51] expression in the DH after the extinction retention session for the subgroups treated with picrotoxin+FfB compared to those treated with saline, picrotoxin or diazepam (P > 0.0001). Upregulated Htr1a [F(5, 12) = 23.98, P < 0.0001] and Erk2 [F(5, 12) = 26.24, P < 0.0001] expression was observed in the picrotoxin+FfB in relation to FfB group. Similarly, upregulated Htr1a [F(5, 12) = 10.75] expression was observed in the picrotoxin+FfB compared with the saline and diazepam subgroups (P < 0.0001). Moreover, we showed that picrotoxin resulted in the overexpression of Htr1a, Grin2b [F(5, 12) = 31.61, P < 0.0001], and Gabra5 [F(5, 12) = 12.13, P = 0.0020] compared with saline. Downregulated Grin2b [F(5, 12) = 22.08, P < 0.0001], Grin2a [F(5, 12) = 22.04, P < 0.0001] and Gabra5 [F(5, 12) = 14.46, P < 0.0001] expression was observed in the picrotoxin+FfB in relation to FfB group. Furthermore, diazepam increased Htr1a expression and decreased Grin2a and Gabra5 expression (P < 0.0001; see Table S9).

In summary, treatment with picrotoxin before FfB administration, at all doses, increased Htr1a and Erk2 expression in the DH after the acquisition and extinction of fear memory and reduced Grin2b expression and prevented the increase in Grin2a and Gabra5 expression after the retention test. Furthermore, this treatment decreased Grin2b and Gabra5 expression after the extinction retention test.

# DISCUSSION

The major findings of our study are as follows. (i) Rats treated with FfB acquired suppression of the licking response, and FfB upregulated the expression of Htr1a, Grin2a, Gabra5, and Erk2 in the DH after the acquisition of conditioned fear, compared to rats exposed to the CS alone, naïve rats and Sintocalmy <sup>R</sup> -treated rats. (ii) Rats treated with FfB, at all doses, showed spontaneous recovery when subjected to the extinction training and extinction retention test sessions; these observations were correlated with Htr1a and Erk2 overexpression in the DH. (iii) These findings were confirmed by data from treatment with (S)-WAY100135, which reduced the lick SR and inhibit spontaneous recovery. Further, data from DH samples obtained from rats treated with (S)-WAY100135 prior to FfB resulted in the downregulation of Htr1a expression and no modulation of Erk2 expression after the retention test and the extinction retention test. (iv) Our data are in line with previous findings concerning the requirement of GluN2B for fear memory formation (Sotres-Bayon et al., 2007, 2009). In particular, we present evidence that treatment with Ro25-6981 disrupts the acquisition of suppression of the licking response. Nevertheless, treatment with FfB after Ro25- 6981 reversed the dose-dependent deficit in the acquisition of fear memory caused by Ro25-6981, which was associated with upregulation of Grin2a and Erk2 expression and downregulation of Htr1a and Gabra5 expression in the DH after the retention test. The occurrence of spontaneous recovery to group treated with Ro25-6981 before FfB during extinction retention test seems to be associated with increase of Grin2b, Grin2a, and Erk2 expression. (v) Treatment with picrotoxin prior to FfB administration no inhibits the spontaneous recovery of fear. This observation was correlated with overexpression of Htr1a and Erk2 and no modulation of Gabra5 expression in the DH. This result suggested that spontaneous fear recovery is not modulated by inactivation of GABAARs; however, the data concerning Gabra5 expression in the DH indicated that FfB modulated the expression of the α5-subunit, which is particularly important for mediating the process of memory formation in the hippocampus (Bannerman et al., 2004; Rudolph and Möhler, 2006; Atack, 2011). Additionally, treatment with diazepam and Sintocalmy <sup>R</sup> disrupt the acquisition of fear memory, in which was associated with downregulation of Grin2a expression in the DH. Several pharmacological studies have indicated that the administration of diazepam before training impairs LTM, as evaluated in a behavioral model such as IA (Izquierdo and Ferreira, 1989), contextual fear conditioning (Harris and Westbrook, 1998), or conditioned suppression (Oliveira et al., 2009). Consistent with this evidence, our results show that acute treatment with 4.0 mg.Kg−<sup>1</sup> diazepam impaired fear memory acquisition and highlight the role of GABAAR in this process. Together with previous data, our current data further support the concept that flavonoid fractions do not prevent fear memory extinction within a session (de Oliveira et al., 2014). In addition, these data suggest an important role of the DH in mediating the acquisition and extinction of conditioned suppression of the lick response.

The roles of the hippocampus in the acquisition, consolidation, and retrieval of fear memory (Kim and Fanselow, 1992; Cammarota et al., 2008) and in fear extinction have been extensively studied in different rodent paradigms (Izquierdo, 1997; Ji and Maren, 2008). Further, the involvement of a circuit including the hippocampus, the pre-frontal cortex and the amygdala in these processes has long been established (Vinogradova, 2001; Fanselow and Dong, 2010). However, the present data suggest an important role of the hippocampus in conditioned suppression, whereas hippocampal plasticity may represent another function of the hippocampus in addition to contextual fear memory modulation and executive and integrative functions (McNaughton and Gray, 2000; Anagnostaras et al., 2001; Vinogradova, 2001; Sanders et al., 2003). Further, many theories have attempted to explain both the neurochemical processes that occur during the acquisition and extinction of fear memory and in the mechanism by which new drug, which are designed to enhance the consolidation or facilitate the extinction of fear memories, might modulate these neurochemical systems (Ji and Maren, 2007; Dalton et al., 2008). However, much less is known about drugs that modulate the brain substrates of extinction, conditioned inhibition, and other inhibitory processes involved in the suppression of a motivated response or the basis of spontaneous recovery. In addition, very few studies have shown the effects of drug treatment prior to conditioning training on fear extinction or spontaneous recovery, i.e., the relationship between the strength of fear memory acquisition and spontaneous recovery. In contrast, the majority of the existing data show the effects of pre-extinction treatment on spontaneous recovery.

Our data suggest that the role of the hippocampus in the acquisition and extinction of lick suppression is dependent on the interaction between glutamatergic, serotoninergic and GABAergic neurotransmission via the activation or inactivation of specific NMDARs, GABAARs, and 5-HT1ARs, as demonstrated by the results from pharmacological manipulation and differential gene expression of Grin2a, Grin2b, Gabra5, Htr1a, and Erk2. The reappearance of a conditioned response after acquisition and training for extinction of fear memory, as shown in our subgroups treated with FfB, has been previously described (Bouton, 1993; Rescorla, 2004; Leung and Westbrook, 2008; Quirk and Mueller, 2008). Specifically, it is thought that the persistence of a fear response after extinction training is associated with anxietyrelated disorders (Davis et al., 2006). However, we showed that FfB enabled the acquisition of extinction within a session despite the occurrence of spontaneous recovery. Although these findings may seem paradoxical, our current findings raise the hypothesis that the original memory was somewhat enhanced, i.e., better preserved; therefore, the flavones from Erythrina falcata may be studied as a novel pharmacotherapy for the treatment of cognitive impairment. Furthermore, we believe that the reappearance of the original memory (spontaneous recovery) observed after FfB treatment is associated with the expression of Htr1a, Erk2, and Grin2a in the DH.

# 5HT1ARs as a Potential Target for the Effects of FfB on Spontaneous Recovery

Drugs that modulate the serotoninergic system are important for cognitive and emotional functions, and 5-HT1ARs are involved in this process. The heteromeric 5-HT1AR is highly expressed in the hippocampus (Barnes and Sharp, 1999), where it modulates GABA- and glutamate-mediated activities (Jacobs and Azmitia, 1992; Barnes and Sharp, 1999; Meneses and Perez-Garcia, 2007). Activation of post-synaptic 5-HT1ARs (heteroreceptors) in the hippocampus is a central component of conflict resolution and anti-anxiety effects. Alternatively, reduced 5-HT1AR expression results in a deficit in hippocampal-dependent memory (Bert et al., 2005, 2006; Altieri et al., 2013). However, the effect of the activation of 5-HT1AR on the modulation of Erk2 expression remains controversial and may depend on neuronal origin and maturation states. Treatment with a 5-HT1AR agonist increased ERK phosphorylation and activity in the hippocampal neuronderived cell line HN2-5 and in hippocampal slices cultured from postnatal day-15 animals (Adayev et al., 1999). In addition to these effects, the activation of 5-HT1ARs alters the dynamics of other neurotransmission systems.

The serotonergic regulation of NMDAR function in the DH was described in pyramidal neurons in the prefrontal cortex (Yuen et al., 2005). Additionally, the activation of 5-HT1ARs resulted in disruption of the transport of GluN2B subunitcontaining vesicles in dendrites, and this transport is regulated by the CaMKII and ERK signaling pathways (Yuen et al., 2005). However, further investigations of the adaptive changes in receptor functions and their specific localization are needed to elucidate the precise role of flavonoids.

Intra-hippocampal treatment with (S)-WAY100135 alone did not affect the punished response in rats (Przegalinski et al., 1995). Therefore, our data suggest that treatment with (S)- WAY100135 reduced lick suppression and that treatment with FfB was unable to reverse this effect. Moreover, the treatment with (S)-WAY100135 modulated Grin2a and Grin2b expression. In this sense, heteromeric 5-HT1ARs in the DH appear to be related to the acquisition of conditioned fear in addition to anti-conflict functions because rats treated with (S)-WAY100135 before FfB administration did not show spontaneous recovery.

The reduced Grin2a, Grin2b, and Htr1a expression in the DH in groups treated with (S)-WAY100135 or (S)- WAY+FfB may underlie the reduced lick suppression and lack of spontaneous recovery. This result suggests an interaction between neurochemical systems. Therefore, the 5-HT1AR represents an additional potential target for the regulation of emotion and cognition in the DH.

# Activation of the GluN2B-NMDARs is Required for Acquisition of Conditioned Suppression and Their Inactivation Before FfB Treatments Modulates the Spontaneous Recovery

Since the discovery of the involvement of NMDARs in longterm potentiation (LTP) at CA1 synapses in the hippocampus, it has become evident that NMDARs are critical for a variety of cognitive processes, such as the acquisition and extinction of fear conditioning (Morris et al., 1986; Bliss and Collingridge, 1993). GluN2A and GluN2B are the predominant subunits of NMDARs. Furthermore, both of these subunits are expressed in the adult brain, predominantly in forebrain regions such as the amygdala, the prefrontal cortex, and the hippocampus, which are involved in the signaling pathways required for aversive memory formation (Schenberg et al., 2006; Mathur et al., 2009; Sotres-Bayon et al., 2009; Morris, 2013).

The hippocampal functions of NMDARs, particularly the GluN2B and GluN2A subunits, in fear memory have been reported (Zhang et al., 2008; Brigman et al., 2010). Several works have suggested that the NMDAR subunit composition could be responsible for the induction of the two forms of plasticity: LTP and long-term depression (LTD) (Shipton and Paulsen, 2014). The contribution of each subunit to ERK2 activation appears to be related to the localization and population of these receptors as well as the behavioral paradigm evaluated (Traynelis et al., 2010). NMDARs either produce weak ERK2 activation or do not activate ERK2 (Gao et al., 2010). Myung et al. (2005) showed that the GluN2B-NMDAR is coupled to the inhibition, rather than the activation, of ERK1/2. Furthermore, differences between behavioral data and gene expression data may explain the different effects of the GluN2B-NMDAR on downstream pathways according to regional localization. Our data showed that Ro25-6981 downregulated the expression of Erk2 in the DH, which resulted in the impairment of conditioned suppression. Alternatively, pharmacological activation of NMDARs increased Grin2b and Erk2 expression but did not affect Grin2a expression in the DH after the acquisition of conditioned suppression. Furthermore, treatment with FfB after Ro25-6951 administration increased Grin2a and Erk2 expression in the DH. Thus, Erk2 activity is closely related to the acquisition of conditioned suppression, as well as extinction and spontaneous recovery.

The increase in Erk2 expression, in response to the acquisition of fear memory or to NMDAR stimulation, has been consistently related to memory-dependent plasticity in the hippocampus (Atkins et al., 1998; Cammarota et al., 2000). The first evidence for the involvement of MAPK in LTP and fear memory originated from studies by English and Sweatt (1996) and Atkins et al. (1998), which showed that ERK2 is required for the formation of LTM in a fear conditioning paradigm in the hippocampus. The levels of ERK2 are elevated following the activation of NMDARs and during the influx of calcium (Impey et al., 1999) but are decreased by 5-HT1A-receptor activation or infusion of an agonist of the serotonergic 5-HT1AR in the hippocampus as Erk1/2 plays an important role in neuroprotection and synaptic activity.

In addition to hippocampal NMDARs and 5-HT1ARs, GABAARs play an important role in synaptic plasticity and therefore contribute to the acquisition of fear memory. Accordingly, drugs that modulate GABAergic transmission have been shown to interfere with fear acquisition and extinction (Chhatwal et al., 2005; Delamater et al., 2009; Oliveira et al., 2009).

# Activation of GABAARs Impairs Acquisition of Conditioned Suppression. Their Inactivation, Before FfB Treatment, However, didn't Prevent the Spontaneous Recovery

We observed that conditioned suppression was impaired in the subgroups treated with Sintocalmy <sup>R</sup> or diazepam, and this impairment appeared to be related to the downregulation of Grin2a expression in the DH. The pharmacological properties and behavioral actions of benzodiazepines, such as amnesic, sedative, and antianxiety effects, on GABAARs appear to be mediated by the α1 subunit, which is preferentially located in interneurons of forebrain areas (Collinson et al., 2002). However, evidence has demonstrated that the GABAAR α5 subunit is highest in the hippocampus compared with deep cortical layers and the amygdala (Rudolph and Möhler, 2014), where it mediates memory formation (Yee et al., 2004; Rudolph and Möhler, 2006; Atack, 2011) and is involved in learning and memory tasks (Harris and Westbrook, 1998; Collinson et al., 2002, 2006). Although Gabra5 expression was not modulated in rats subjected to fear conditioning with or without FfB treatment or to the acquisition of conditioned suppression following FfB treatment, rats treated with picrotoxin displayed upregulation of Gabra5 expression and showed acquisition of memory. In addition to the role of the α5 subunit in the acquisition of fear memory, its modulation in the DH after extinction of fear memory is supported by data from the subgroups treated with picrotoxin+FfB; these data suggest that the α5 subunit is not correlated with spontaneous recovery. Thus, our data reveal a central role of the α5 subunit of the GABAAR in the acquisition of conditioned emotional suppression, as evaluated by the lick response. The memory-enhancing effects of benzodiazepine site partial inverse agonists have been shown (Yee et al., 2004).

# CONCLUSION

The major fear memory/treatment-dependent changes observed in our study included the spontaneous recovery of fear memory,

# REFERENCES


which may be related to the enhancement of consolidation of fear memory. No anti-anxiety effects were observed after treatment with FfB. Furthermore, for the first time, we showed that the spontaneous recovery of fear memory may be correlated with the combined activation of GluN2A-containing NMDARs, and 5-HT1ARs in the DH, which, in turn, modulates ERK1/2 activity. Finally, the results from gene expression analysis in the DH and the results showing the modulatory effects of FfB treatment indicate that the DH appears to anatomically and functionally subserve other structures involved in the acquisition and extinction of fear memory formation, such as the amygdala and the prefrontal cortex. Together, our data provide important information concerning the molecular basis of fear-conditioned suppression and the role of the DH in these processes, and our results suggest that FfB may represent a potential therapeutic target for preventing or treating neurocognitive impairments.

# ACKNOWLEDGMENTS

This work is dedicated to Dra Elenice Aparecida de Moraes Ferrari (in memoriam). This study was supported by the São Paulo State Research Foundation (FAPESP) (grant 2009/15229-3 and 2013/20378-8 to SC). DO and CZ are scholars from CAPES. JC is an investigator on the Brazilian Research Council (CNPq).

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnbeh. 2015.00345

Figure S1 | HPLC-DAD-ESI/MS<sup>n</sup> analysis of the flavonoidic fraction (FfB) from the roots of Erythrina falcata using a C18 Luna column (A,B). TIC was performed in negative mode, and MS<sup>2</sup> spectra of deprotonated molecules (A) were obtained as follows: (1) vicenin-2 [M-H]<sup>−</sup> at m/z 593, (2) vicenin-1 [M-H]<sup>−</sup> at m/z 563, (3) vitexin [M-H]<sup>−</sup> at m/z 431, (4) isovitexin [M-H]<sup>−</sup> at m/z 431, (5) 6-C-glycoside diosmetin [M-H]<sup>−</sup> at m/z 461, and (6) apigenin [M-H]<sup>−</sup> at m/z 269. The chromatogram was recorded at 254 nm for the UV spectra of compounds 1-6 (A).

relevance to emotion-related behavior. ACS Chem. Neurosci. 4, 72–83. doi: 10.1021/cn3002174


signaling and AMPA receptor trafficking. Neuron 46, 745–760. doi: 10.1016/j.neuron.2005.04.031


in the lateral amygdala. Neuropsychopharmacology 32, 1929–1940. doi: 10.1038/sj.npp.1301316


a microtubule-dependent mechanism. J. Neurosci. 25, 5488–5501. doi: 10.1523/JNEUROSCI.1187-05.2005


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 de Oliveira, Zamberlam, Rêgo, Cavalheiro, Cerutti and Cerutti. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Modulation of defensive reflex conditioning in snails by serotonin

Vyatcheslav V. Andrianov 1, 2, Tatiana K. Bogodvid1, 3, Irina B. Deryabina<sup>1</sup> , Aleksandra N. Golovchenko<sup>1</sup> , Lyudmila N. Muranova<sup>1</sup> , Roza R. Tagirova<sup>1</sup> , Aliya K. Vinarskaya<sup>4</sup> and Khalil L. Gainutdinov 1, 2 \*

<sup>1</sup> Laboratory of Neurobiology, Institute of Fundamental Medicine and Biology, Kazan Federal University, Kazan, Russia, <sup>2</sup> Group of Biophysics, Zavoisky Physical-Technical Institute, Russian Academy of Sciences, Kazan, Russia, <sup>3</sup> Department of Biomedical Sciences, Volga Region State Academy of Physical Culture, Sport and Tourism, Kazan, Russia, <sup>4</sup> Laboratory of Cellular Neurobiology of Learning, Institute of High Nerve Activity and Neurophysiology, Russian Academy of Sciences, Moscow, Russia

#### Highlights


Edited by:

Onur Gunturkun, Ruhr-University Bochum, Germany

#### Reviewed by:

Katharina Spoida, Ruhr-University Bochum, Germany Tiago Souza Dos Santos, Institut d'Investigacions Biomèdiques de Barcelona, Spain

#### \*Correspondence:

Khalil L. Gainutdinov kh\_gainutdinov@mail.ru

Received: 08 June 2015 Accepted: 02 October 2015 Published: 23 October 2015

#### Citation:

Andrianov VV, Bogodvid TK, Deryabina IB, Golovchenko AN, Muranova LN, Tagirova RR, Vinarskaya AK and Gainutdinov KL (2015) Modulation of defensive reflex conditioning in snails by serotonin. Front. Behav. Neurosci. 9:279. doi: 10.3389/fnbeh.2015.00279 We studied the role of serotonin in the mechanisms of learning in terrestrial snails. To produce a serotonin deficit, the "neurotoxic" analogs of serotonin, 5,6- or 5,7-dihydroxytryptamine (5,6/5,7-DHT) were used. Injection of 5,6/5,7-DHT was found to disrupt defensive reflex conditioning. Within 2 weeks of neurotoxin application, the ability to learn had recovered. Daily injection of serotonin before a training session accelerated defensive reflex conditioning and daily injections of 5-HTP in snails with a deficiency of serotonin induced by 5,7-DHT restored the snail's ability to learn. We discovered that injections of the neurotoxins 5,6/5,7-DHT as well as serotonin, caused a decrease in the resting and threshold potentials of the premotor interneurons LPa3 and RPa3.

Keywords: serotonin, associative learning, identified neurons, membrane potential, threshold potential, snail

# INTRODUCTION

One of the widespread and well-investigated transmitters in the nervous system is serotonin (Kandel and Schwartz, 1982; Sakharov, 1990; Crow, 2004; Gillette, 2006). Within a short period of time serotonin (5-HT) was identified as a neurotransmitter in both mollusks and mammals (Whitaker-Azmitia, 1999; Marinesco et al., 2004; Lee et al., 2009). The serotoninergic system plays an important role in the modulation of stress-induced excitability (arousal) and defensive behavior

**Abbreviations:** 5,6- and 5,7 DHT, 5,6- and 5,7- dihydroxytryptamine; 5-HTP, 5-hydroxytryptophan; SS, saline solution; 5-HT, serotonin.

(Il-Han et al., 2010). The "5-HT neurons dispersed throughout the CNS of lophotrochozoan invertebrates (mollusks and leeches) are analogous to vertebrate 5-HT neurons concentrated in the raphe nuclei of the mid- and hindbrain: they innervate specific central pattern generators and other circuits of the CNS, receive feedback from them, and support general behavioral arousal" (Gillette, 2006). It has been shown that long-term facilitation in connections between sensory and motor neurons of the gill withdrawal reflex is mediated by 5-HT and this form of synaptic plasticity was found to be a critical cellular mechanism in behavioral sensitization (Barbas et al., 2003; Hawkins et al., 2006; Hart et al., 2011). In connection to the discovery of the relationship between the level of 5-HT in the hemolymph of mollusks and the sensitization of reflexes (Levenson et al., 1999; Hernadi et al., 2008), a lot of experiments have been conducted using manipulation of the 5-HT system to investigate cellular analogs of learning (Lent and Dickinson, 1984; Gadotti et al., 1986; Glanzman and Krasne, 1986; Jahan-Parwar et al., 1987; Vehosvzky et al., 1989; Mauelshagen et al., 1996; Kemenes, 1997; Malyshev et al., 1997; Shevelkin et al., 1997; Gainutdinov et al., 1999; Balaban, 2002; Burrell and Sahley, 2005; Jing et al., 2009). It was found that the injection of the neurotoxin 5,7-DHT led to a significant decrease in the withdrawal reflex caused by tail shock and inhibited the heterosynaptic facilitation between the sensory neuron and the subsequent cells in Aplysia (Glanzman et al., 1989). Balaban et al. (1987) showed that the pairing of food presentation and electrical stimulation didn't result in changes in responses to food in 5,7-DHT-injected snails, whereas in vehicle-injected snails defensive reactions were observed. Furthermore, injection of 5,7-DHT led to the disruption of long-term sensitization in Aplysia (Glanzman et al., 1989) and in snails (Balaban and Bravarenko, 1993; Gainutdinov et al., 1999). At the same time, it has been shown that although the intensity of the conditioning strongly decreases in leeches after depletion of serotonin by 5,7- DHT, they didn't lose the ability to learn (Burrell and Sahley, 1999). Thus, a considerable wealth of experimental material has been accumulated which points to the association between the functioning of the 5-HT- system of mollusks and their ability to learn (in behavioral experiments). However, the questions regarding to the specific mechanisms (and/or pathways) of 5- HT participation in associative learning and the role of specific neurons in these processes remain open. These findings and questions motivated us to investigate the role of 5-HT in the mechanisms of learning by behavioral and electrophysiological methods, using the "neurotoxic" analogs of serotonin 5,6/5,7- DHT, and the precursor of 5-HT syntheses, 5-hydroxytryptophan (5-HTP).

# MATERIALS AND METHODS

# Experimental Animals

For the experiments, the terrestrial snails Helix lucorum from the Crimean population, were used. The nervous system of these snails has been well described (Schmalz, 1914; Kilias, 1985; Balaban, 1993).

All experimental procedures are in compliance with the Guide for the Care and Use of Laboratory Animals published by the National Institutes of Health and Directive 2010/63/EU of the European Parliament and of the Council of 22 September 2010. Snails were stored asleep. Prior to the experiments, the snails were kept for no less than 2 weeks in a glass terrarium in a humid atmosphere at room temperature (each group in a separate terrarium) (Article 33 of Directive 2010/63/EU). All groups were housed in separate terrariums which were kept together all the time in the same room under the same conditions. The electrophysiological measurements were carried out in isolated preparations the day after training. Prior to the preparation procedure, snails were anesthetized (Article 14 of Directive 2010/63/EU) by 30 min of immersion in water mixed with ice.

#### Drugs 5,6/5,7-DHT

In the experiments the "neurotoxic" analogs of serotonin 5,6 and 5,7-dihydroxytryptamine (5,6/5,7-DHT) were used. Their effects on defensive reflex conditioning of snails and the electrical characteristics of premotor interneurons were investigated. They selectively destroy the 5-HT elements in the nervous system, particularly in nerve terminals, thus decreasing the level of 5- HT (Gadotti et al., 1986; Glanzman and Krasne, 1986; Hernádi et al., 1992). 5,6-DHT (Sigma) was injected into snails twice at doses of 15 mg/kg with an interval of 7 days for a total dose of 30 mg/kg. 5,7-DHT (Sigma) was injected into snails once at a dose of 20 mg/kg (Balaban et al., 1987). The neurotoxins were dissolved in 0.1 ml of saline solution (SS). In addition, ascorbic acid was added to the solution as an antioxidant to achieve a concentration of 0.1%. The injection of 0.1 ml of SS (with the addition of ascorbic acid to achieve a concentration of 0.1%) was used as a control. The injection of only 5,6-DHT without training served as an alternative control. One month after the injection of 5,6/5,7-DHT the 5-HT-containing neurons were selectively labeled with brown pigmentation, pointing to capture of 5,6/5,7-DHT by these cells. This phenomenon has been very well described earlier (Glanzman and Krasne, 1986; Balaban et al., 1987; Vehosvzky et al., 1989). Therefore, we did not take photos of the off-labeled cells.

#### 5-HT

In another series of experiments the influence of serotonin (5- HT) on defensive reflex conditioning in snails and the electrical characteristics of premotor interneurons was investigated. 5-HT (Sigma) was injected into snails daily 1 h before a training session at a dose of 10 mg/kg. 5-HT was dissolved in 0.1 ml of SS, in addition ascorbic acid was added to the solution as an antioxidant to achieve a concentration of 0.1%. The injection of 0.1 ml of SS (with the addition of ascorbic acid to achieve a concentration of 0.1%) was used as a control.

#### 5-HTP

In a third series of experiments the influence of the precursor to 5-HT synthesis 5-hydroxytryptophan (5-HTP) on defensive reflex conditioning in snails and the electrical characteristics of premotor interneurons was investigated. 5-HTP (Sigma) was injected into snails daily 1 h before a training session at a dose of 10 mg/kg. 5-HTP was dissolved in 0.1 ml of SS, in addition ascorbic acid was added to the solution as an antioxidant to achieve a concentration of 0.1%. The injection of 0.1 ml of SS (with the addition of ascorbic acid to achieve a concentration of 0.1%) was used as a control.

# Experimental Groups

In the experiments we used the following groups of animals:

#### Group 1: Defensive Reflex Conditioning

Snails were trained to execute the defensive reflex on tapping on the shell (Balaban, 1993; Gainutdinova et al., 2003). Tapping on the shell (2 times) was used as a conditioned stimulus, which under normal conditions doesn't produce any defensive reaction in a snail. As an unconditioned stimulus a puff of air into the lung cavity orifice (pneumostome) was used, which produces the defensive reaction of pneumostome closure in animals. Only a complete closure of the pneumostome was taken as a "positive reaction" to the stimulus. Combinations of stimuli were presented with a random interval that ranged from 2 to 4 min (to prevent the elaboration of a conditioned reflex to time). The defensive reflex was trained according to 2 protocols:


#### Group 2: Effects of 5-HT and 5-HT Neurotoxins on Defensive Reflex Conditioning


#### Group 3: Common Effects of 5-HTP and 5,7-DHT on Defensive Reflex Conditioning


#### Group 4: Effects of 5-HT and 5-HT- Neurotoxins on the Electrical Characteristics of Premotor Interneurons in Snails after Defensive Reflex Conditioning

(1) registration of the electrical characteristics of premotor interneurons in snails which were trained to execute the defensive conditioned reflex on tapping on the shell according to the first protocol after injection of 5-HT (n = 7), snails were injected daily 1 h before a training session with 5-HT at a dose of 10 mg/kg in 0.1 ml of SS;


#### Intracellular Recording

The nervous ring was immersed in a saline solution of the following composition: NaCl—80 mM, KCl—4 mM, CaCl2— 10 mM, MgCl2—6 mM, NaHCO3—5 mM (or Tris—5 mM), pH—7.6–7.8. The electrical characteristics of the withdrawal interneurons of the snail's pneumostome closure reflex LPa3 and RPa3 (Balaban, 1993, 2002) were analyzed. The recordings of the electrical characteristics were carried out on the day after training. The measurements were conducted at room temperature (18–21◦C) using intracellular glass microelectrodes with a resistance 10–30 MOm filled with 2.5 M KCl. The following parameters of the nervous cells were studied: resting membrane potential (the initial value prior to the onset of a number of tactile or electrical stimulations)—Vm, and the threshold of action potential generation (threshold potential)— Vt. The measurements of the electrical characteristics of the premotor (withdrawal) interneurons were conducted in preparations of snails from all series of experiments.

Since the premotor withdrawal interneurons LPa3 and RPa3 are silent in normal conditions, to generate action potentials in the isolated preparations we applied a depolarizing square-wave form electrical current through the recording microelectrode into the cell for 1 s. For stimulation the minimal current strength for the generation of action potentials was selected; it varied from 1.7 to 3.5 nA.

#### Data Analyses

The results are shown as mean ± SEM. The unpaired Student's t-test and non-parametric Mann–Whitney test were used for comparison between two groups. One-Way ANOVA followed by the Tukey post-hoc test and a repeated Two-Way ANOVA were used for comparison between three- or more statistical groups. Independent t-tests and the Tukey post-hoc test were used to make specific group comparisons. The statistical software SigmaStat32 was used. The statistical significance criterion was p < 0.05.

# RESULTS

# Conditioning with Neurotoxins

The defensive reflex conditioning on shell tapping was achieved within 6 days with the use of 300–350 repetitions of a combination of shell tapping (conditioned stimulus) and air blowing into pneumostome (unconditioned stimulus) in the case of the second protocol (**Figures 1A,B**, SS + T) or within 3 days with the use of 150 combinations in the case of first protocol (**Figure 2B**, SS + T). In both cases the share of positive responses to a conditioned stimulus during training reached 100%. The results of behavioral experiments showed a reliable maintenance of the conditioned defensive reflex for 40 days after training. Snails from the active control group received an identical amount of the conditioned and unconditioned stimuli, but in an unpaired combination. In this case the share of positive responses to a conditioned stimulus in this experimental series during training reached 25–30% (**Figure 1B**, SS + AC), which proved to be less (p < 0.01) than that in the experimental group (up to 100%). It was found that the conditioned defensive reflex wasn't induced in snails (with the first protocol) which were trained the day after the second injection of the "neurotoxic" serotonin analog, 5,6- DHT, (**Figure 2B**, DHT1 + T). At the same time, in the animals this reflex started to form 1 week after the second injection of the neurotoxin, there was an increased reaction to the conditioned stimulus at the end of the training session and conditioned reflex was successfully elaborated a (**Figure 2B**, DHT2 + T). From **Figure 2A** we see that snails begin to learn on 13th day after the second injection of 5,6-DHT. These results probably indicate that 2 weeks after the application of 5,6-DHT the 5-HT system, required for defensive reflex conditioning, begins to recover. It should be noted that the curves for training in naive snails and the snails injected with saline solution did not reliably differ.

# Conditioning with Serotonin and 5-hydroxytryptophan

In the following series of experiments (**Figure 3A**) the defensive reflex conditioning on shell tapping in snails (the second protocol) was progressed more slowly, so that complete learning was achieved as a result of 350 stimuli combinations (**Figure 3B**, SS + T). The daily injection of 5-HTP before each training session did not reliably accelerate the defensive reflex conditioning during most of the training (**Figure 3B**, 5-HTP + T). However, after injections of 5-HTP the snails learned faster. Injection of 5,7-DHT inhibited learning (**Figure 3B**, DHT + SS + T). From **Figure 3A** we see that even on the 16-th day after injection of 5,7-DHT snails didn't start to learn. However, daily injection of 5-HTP after the injection of 5,7-DHT restored the snail's ability to learn (**Figure 3B**, DHT + 5-HTP + T). A daily injection of 5- HT before the training session accelerated the conditioned reflex elaboration (**Figure 1B**, 5-HT + T).

FIGURE 1 | (A) Protocol of experiments. (B) Dynamics of defensive reflex conditioning on tapping on shell in snails according to the second protocol after daily injections of serotonin (5-HT + T) and saline solution (SS + T) before each training session. N, number of pairings of conditioned and unconditioned stimuli; positive responses (N), %, part of positive responses to conditioned stimulus, as a percentage; 5-HT + T, defensive reflex conditioning during daily injections of serotonin; SS + T, defensive reflex conditioning during daily injections of saline solution; SS + AC, active control—snails received unpaired unconditioned and conditioned stimuli. Vertical axis shows quantity of positive responses of pneumostome (its closure in response to conditioned stimulus), in %; horizontal axis shows numbers of pairs of unconditioned and conditioned stimuli (n). One-Way ANOVA followed by Tukey post-hoc test was performed for each time point. Asterisk (\*) indicates significant difference in the post-hoc test between (SS+T)/(5-HT+T) on 220 pairs. Sharp (#) indicate significant differences (SS+T, 5-HT+T) vs. (SS+AC) after the first 30th pairs. (p < 0.05, the Tukey post-hoc test and independent t-test).

#### Electrophysiological Data

Example traces of recorded electrical characteristics of premotor interneurons in naive and trained snails are given in **Figures 4A,B**. Measurements of electrical characteristics (**Figure 4C**) showed that the initial membrane resting potential in withdrawal interneurons in the naive snails was −60.9±0.3 mV (n = 92), the threshold potential was 19.9 ± 0.4 mV (n = 76) (**Figures 5A,B**—Control). After associative learning (n = 74) a reliable decrease in the membrane resting and threshold potentials by 4 mV was observed in the studied interneurons (**Figures 5A,B**—SS + T). It was found that these changes in the observed electrical characteristics were retained for 1 month. The duration this change in electrical characteristics of premotor interneurons (n = 19) has been shown by us earlier (Gainutdinova et al., 2003).

After injection of 5,6/5,7-DHT the depolarization of the membrane in premotor interneurons was observed during recording, both the next day and a week after the injection of 5,6- DHT. The resting membrane potential decreased from −60.3 ± 0.3 mV in SS-injected snails (n = 37) to −57.2 ± 0.3 mV in 5,6-DHT-injected snails (n = 41), the threshold potential decreased from 19.9 ± 0.3 mV (n = 22) to 17.3 ± 0.3 mV (n = 33) accordingly (**Figures 5A,B**—DHT). In snails trained after the second injection of 5,6-DHT no further decrease of the resting membrane and threshold potentials was observed in comparison with the snails injected with 5,6-DHT without training (**Figures 5A,B**—DHT1 + T, DHT2 + T).

Next, we analyzed the changes in the membrane and threshold potentials of premotor interneurons of snails having received only the injection of 5-HT, and snails, injected with 5-HT before the associative learning. The membrane potential decreased from −60.3 ± 0.6 mV (n = 12) to −55.7 ± 0.4 mV (n = 13) in snails which received 5-HT only and to −55.0 ± 0.4 mV (n = 12) in snails injected with 5-HT before the elaboration of the defensive reflex. In this case in interneurons the value of the threshold potential significantly decreased from 20.0 ± 0.5 mV (n = 12) to 15.9 ± 0.3 mV (n = 10) and to 15.3 ± 0.3 mV (n = 8), respectively (**Figure 6**, 5-HT, 5-HT + T).

#### DISCUSSION

It is well known that learning on the basis of the defensive reflexes of molluscs is mediated by 5-HT (Kandel and Schwartz, 1982; Balaban et al., 2001; Burrell and Sahley, 2005; Gillette, 2006; Il-Han et al., 2010; Hart et al., 2011). The investigations of mechanisms of learning and memory have resulted in new experimental approaches for studying the neurotransmitter and modulator effects of 5-HT, and for studying the mechanisms of participation of corresponding systems in the phenomena of behavioral plasticity (Glanzman et al., 1989; Balaban, 2002, 2008; Barbas et al., 2003; Crow, 2004; Burrell and Sahley, 2005; D'iakonova, 2007; Il-Han et al., 2010; Hart et al., 2011). It is well known that 5-HT induces presynaptic facilitation (Byrne and Kandel, 1996; Lin et al., 2010), it has also been shown that 5-HT can perform integrative functions through its release in the extracellular medium (Sakharov, 1990; Zakharov et al., 1995; Marinesco et al., 2004). These results served as a basis for using 5-HT application to the washing solution as an analog of reinforce stimulus during the formation of cell analogs of learning (Mauelshagen et al., 1996; Liao et al., 1999; Hawkins et al., 2006; Lin et al., 2010; Hu et al., 2011).

## Effects of Neurotoxins Connected with Depletion of Serotonin

Our work is devoted to the investigation of mechanisms of associative learning on the basis of the defensive reflex

DHT; DHT2 + T, defensive reflex conditioning the next day after the second injection of "neurotoxic" analogs of serotonin 5.6- DHT; SS + T, defensive reflex conditioning after injection of saline solution. Vertical axis shows quantity of positive responses of pneumostome (its closure in response to conditioned stimulus), in %; horizontal axis shows numbers of pairs of unconditioned and conditioned stimuli (n). Asterisk (\*) and sharp (#) indicate significant difference (SS+T) vs. (DHT1+T, DHT2+T) and (DHT1+T) vs. (DHT2+T) accordingly. (p < 0.05; Two-Way ANOVA and independent t-test).

of a terrestrial snail. For the analysis of the role of 5- HT, its temporary deficit using the "neurotoxic" analogs 5,6/5,7-DHT was created. The predecessor for the synthesis of serotonin 5-HTP as well as the injection of 5-HT in the hemolymph of the snail Helix pomatia were also used. Our results as in the work of Balaban et al. (1987) show that after depletion of 5-HT the conditioned reflex is not produced. These results correlate with the data of Glanzman et al. (1989) which show the inhibition of the heterosynaptic facilitation in Aplysia by the neurotoxin 5,7-DHT. Data from the literature (Pivovarov and Nistratova, 2003; Abramova et al., 2005) and our results show that premotor interneurons respond to 5-HT, i.e., 5-HT may modulate the behavioral effects. Direct evidence of the possibility of electrophysiological modulatory effects of the serotonergic neuron Pd4 on the premotor interneuron LPa3 was found by Balaban et al. (2001).

One of the possible reasons for such effects is a depletion of 5-HT in the nervous system of mollusks by neurotoxins (Gadotti et al., 1986; Glanzman and Krasne, 1986; Jahan-Parwar et al., 1987; Vehosvzky et al., 1989; Kemenes et al., 1990). It was shown early that the "neurotoxic" analogs 5,7-DHT significantly reduced the immunofluorescence staining of 5-HT in the nervous system of crayfish (Glanzman and Krasne, 1986), analysis using HPLC also shows a decrease in the level of 5- HT in the nervous system of Aplysia after exposure to this neurotoxin (Glanzman et al., 1989). The results of ultrastructural and biochemical studies showed significant depletion of 5- HT by 5,6-DHT in the first week, and after 21 days the levels of 5-HT returned to normal level (Hernádi et al., 1992; Kemenes, 1997). Our results showed that the snails start to learn on the 13th day after the injection of 5,6-DHT, but after injection of 5,7-DHT snails hadn't even started to learn on the 16-th day.

# Excitability of Premotor Interneurons and Learning

Earlier we found that the defensive reflex conditioning in snails is accompanied by a depolarization shift of the membrane potential and a decrease of the threshold potential (Gainutdinov et al., 1998). In recent years there have been a sufficient number of experimental results that demonstrate the membrane correlates of learning (Gillette et al., 1982; Alkon, 1984; Frysztak and Crow, 1997; Cleary et al., 1998; Gainutdinova et al., 2003; Disterhoft and Oh, 2006; Kemenes et al., 2006; Nikitin et al., 2006, 2013; Mozzachiodi et al., 2008; Jing et al., 2009; Debanne and Poo, 2010; Gainutdinov et al., 2011; Sakharov, 2012; Cavallo et al., 2014a,b). These experiments were done on preparations from trained animals as well as within the cell analogs of learning.

A number of questions arise during the analysis of the effects of 5-HT. It is known that one of the main functions of 5- HT both in vertebrates and invertebrates is to facilitate the motor output. For example, the facilitation action on reflex activity and on central pattern generators (Gillette, 2006). The increase in excitability of neurons under the action of 5-HT has been noted by a number of authors (Frysztak and Crow, 1997; Liao et al., 1999; Balaban et al., 2001; Pivovarov and Nistratova, 2003; Abramova et al., 2005; Dumitriu et al., 2006; Hawkins et al., 2006). Jin and co-authors have shown that 5-HT increases the peak amplitude of the complex excitatory postsynaptic potential induced by light, and also increases the internal excitability and the spike activity of type Ie(A) interneurons of the mollusk Hermissenda (Jin et al., 2009). In contrast 5- HT reduces the spike activity and internal excitability of type Ie(B) interneurons. We found two effects in our experiments: the depolarization shift of membrane potential and decrease in the threshold potential of premotor interneurons after training and after the injection of the 5-HT. There is an absence of a summing effect of these two factors. The depolarization shift

of membrane potential and decrease in threshold potential of premotor interneurons in response to injection of 5-HT and its "neurotoxic" analogs to intact snails is a possible consequence of coupling of these substances (5-HT and neurotoxins) with 5-HT receptors.

# Serotonin Receptors in Premotor Interneurons

In our experiments we have shown that injections of the neurotoxins 5,6/5,7-DHT is accompanied by the depolarization of premotor interneurons and a decrease in their threshold potential, as with injections of 5-HT. The question arises, are the depolarization shift of membrane potential and decrease in threshold potential of premotor interneurons after injection of 5-HT and its neurotoxic analogs the result of their interaction with 5-HT- receptors due to the structural similarity between 5- HT and its neurotoxic analogs? These common effects suggest that they are related to the effects of 5-HT on receptors located on the membrane of premotor interneurons and possibly on the intermediate neurons that are presynaptic to premotor interneurons. Pivovarov and Nistratova (2003) analyzed the possible presence of 5-HT receptors on the soma of snail's premotor interneurons. 5-HT, applied locally to the soma, reversibly decreased the input current caused by acetylcholine (local ionophoretical application). They demonstrated that only NAN-190 (a 5-HT1A-receptor antagonist) and methiothepin (a 5-HT1E receptor antagonist) inhibited the development of the 5- HT effect. The results show the presence of 5-HT receptors but

only of the first type on the soma of Helix premotor interneurons (Pivovarov and Nistratova, 2003; Abramova et al., 2005).

It is known that the 5-HT1A receptor is among the most abundant and widely distributed 5-HT receptors in the brain, but is also expressed on 5-HT neurons as an autoreceptor where it plays a critical role in regulating the activity of the entire 5-HT system and over-expression of the 5-HT1A autoreceptor has been implicated in reducing 5-HT neurotransmission

FIGURE 5 | Value of resting membrane potential—(A) and threshold potential—(B) of premotor interneurons LPa3 and RPa3 in snails in various conditions. SS, naïve snails; DHT1 + T, defensive reflex conditioning a week after the second injection of the "neurotoxic" analog of serotonin 5.6- DHT; DHT2 + T, defensive reflex conditioning the next day after the second injection of the "neurotoxic" analog of serotonin 5.6- DHT; SS + T, defensive reflex conditioning after injection of saline solution; DHT, snails injected by the "neurotoxic" analog of serotonin 5.6- DHT. \*The reliable difference (p < 0.001) against active control group (injection of saline solution). Vertical axis shows the value of potential, in mV. ANOVA revealed no T/DHT/5-HT interactions. \*p < 0.05; Two-Way ANOVA and independent t-test.

(Albert et al., 2011). Our results showed that 5-HT and its neurotoxin produced similar effects. It is possible to consider the results obtained with 5-HT1A autoreceptors. It has been shown that most if not all 5-HT1A autoreceptors on the plasma membrane of soma-dendrites from nucleus raphe dorsalis are located extrasynaptically (Riad et al., 2004). Therefore, 5-HT and its neurotoxin may bound with these autoreceptors and decrease the effect of 5-HT. However, the question remains whether the 5-HT1A receptors in the premotor interneurons are autoreceptors?

It has been shown that the broadening of the action potential of Aplysia sensory neurons in response to 5-HT application is mediated by 5-HT receptors of the first type, blocked by methiothepin (Dumitriu et al., 2006). K. Lukowiak et al studied the role of the 5-HT- system in the responses of the mollusk Lymnaea to the danger stimulus. Using mianserin, a 5-HT receptor antagonist they found the disruption of two types of

defensive behavior (increase in the time of exit of freshwater snail from their shell and the shadow reflex), caused by an extract of crab tissue (Il-Han et al., 2010). Methysergide, another 5-HT receptor antagonist had the same effect, blocking the formation of long-term memory after training with an extract of crab tissue. However, importantly, mianserin didn't affect formation of long-term memory after training in water without an extract of crab tissue. These data suggest that the 5-HT- system is activated only with danger detection. These results show the possibility of an extracellular action of 5-HT. The differences in the responses to 5-HT led to the opinion that there are different subtypes of 5-HT receptors in the nervous system of Aplysia (Barbas et al., 2003). The possibility of participation of different types of 5-HT receptors in different signaling pathways has also been demonstrated in the work of Kiss et al. (2003). Since potentiation of S-cells in the leech was blocked by the 5-HT receptor antagonist methylsergide, it was concluded that this metabotropic receptor is involved in the regulation of excitability of S-cells (Burrell and Sahley, 2005). Other researchers have cloned a 5-HT receptor called 5-HT(apAC1), which stimulates the production of cAMP, the inhibition of which blocks synaptic facilitation of the sensorimotor synapse of Aplysia (Lee et al., 2009). A 5-HT receptor, which regulates protein kinase C PKC has also been found, called Apl II (Nagakura et al., 2010).

# Effects of Serotonin and 5-hydroxytryptophan Injections on Learning

It is known that 5-HT is an important mediator of defensive behavior in molluscs (Whitaker-Azmitia, 1999; Gillette, 2006; D'iakonova, 2007; Hernadi et al., 2008; Lee et al., 2009). It has been shown that the 5-HT transmission from modulatory neurons to premotor interneurons includes the release of 5- HT from modulatory neurons into the extracellular medium (Zakharov et al., 1995; Balaban et al., 2001). We found that daily injection of 5-HT accelerated the defensive reflex conditioning in snails. This result is similar to those found by Lee et al. (2008). They found that one pulse of 5- HT produces a transient facilitation mediated by the cAMPdependent protein kinase leading to covalent modifications in the sensory neurons which results in an enhancement of transmitter release and a strengthening of synaptic connections lasting minutes. By contrast, repeated pulses of 5-HT induce a transcription- and translation-dependent long-term facilitation lasting more than 24 h and trigger the activation of a family of transcription factors in the presynaptic sensory neurons including ApCREB1, ApCREB2, and ApC/EBP. Other researchers have also shown that 5-HT-induced long-term facilitation of the Aplysia sensorimotor synapse depends on enhanced gene expression and protein synthesis (Villareal et al., 2007; Hart et al., 2011). There is evidence that one of these proteins could be synapsin (Fioravante et al., 2007; Hart et al., 2011).

5-HTP didn't reliably accelerate the defensive reflex conditioning during training, however, after injection of 5-HTP learning in general was achieved faster. Our results demonstrate that daily injection of 5-HTP before a training session in snails with a 5-HT deficiency, created by the "neurotoxic" analog of serotonin 5,7-DHT, restored the ability of snails to learn. The explanation for this fact can be found in the data of Fickbohm et al. (2005). Using high performance liquid chromatography and immunochemistry they showed a significant increase in 5-HTP content for over 20 h in the brain of a mollusk Tritonia after 30 min standing in a solution of 2 mM 5-HTP and they also showed the increase of 5-HT in specific areas of the brain (Fickbohm et al., 2005). The difference in our experiments is that we not only injected snails with 5-HT and 5-HTP but also elaborated a conditioned reflex, i.e., had to deal with the simultaneous action of two factors. So the question arose by what mechanisms does 5-HT accelerate learning? It is known that learning is the result of changes in presynaptic processes, such as direct modulation of the release of neurotransmitters and postsynaptic processes, such as the properties of receptors (Kandel, 1976, 2001; Hawkins et al., 2006; Balaban, 2008; Mozzachiodi and Byrne, 2010; Vavoulis et al., 2010; Balaban et al., 2014). At the same time the effect of recovery in the ability to learn caused by injections of 5-HTP on the background of deficiency of 5-HT created by 5,7-DHT, demonstrates a partial maintenance of the functioning of the 5-HT synapses.

### CONCLUSION

In conclusion, we have shown that a daily injection of 5- HT before a training session accelerated learning, and daily injection of 5-HTP before a training session in snails with a 5-HT deficiency (caused by the "neurotoxic" analogs of serotonin 5,7-DHT), restored the ability of snails to learn. The results suggest that during learning 5-HT is released into the extracellular medium, which interacts with receptors located

## REFERENCES


on the membrane of premotor interneurons and possibly on the intermediate neurons that are presynaptic to premotor interneurons. Learning is also accompanied by a decrease in membrane and threshold potentials of premotor interneurons.

#### ACKNOWLEDGMENTS

This work was funded by the subsidy of the Russian Government to support the Program of Competitive Growth of Kazan Federal University among World's Leading Academic Centers (agreement No. 02.A03.21.0002), by Russian Foundation for Basic Research (grant No. 15-04-05487) and by Russian Scientific Foundation (grant No. 14-25-00072, in part of experiments of neurotoxins effect on learning).


salivary duct. Neuroscience 116, 775–790. doi: 10.1016/S0306-4522(02) 00754-6


Zakharov, I. S., Ierusalimsky, V. N., and Balaban, P. M. (1995). Pedal serotonergic neurons modulate the synaptic input of withdrawal interneurons of Helix. Invert. Neurosci. 1, 41–52. doi: 10.1007/BF02331831

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Andrianov, Bogodvid, Deryabina, Golovchenko, Muranova, Tagirova, Vinarskaya and Gainutdinov. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Extinction and Retrieval + Extinction of Conditioned Fear Differentially Activate Medial Prefrontal Cortex and Amygdala in Rats

Hongjoo J. Lee<sup>1</sup> , Rebecca P. Haberman<sup>2</sup> , Rheall F. Roquet <sup>1</sup> and Marie-H. Monfils 1,3 \*

<sup>1</sup> Department of Psychology, The University of Texas at Austin, Austin, TX, USA, <sup>2</sup> Department of Psychological and Brain Sciences, The Johns Hopkins University, Baltimore, MD, USA, <sup>3</sup> Center for Learning and Memory, The University of Texas at Austin, Austin, TX, USA

Pairing a previously neutral conditioned stimulus (CS; e.g., a tone) to an aversive unconditioned stimulus (US; e.g., a footshock) leads to associative learning such that the tone alone comes to elicit a conditioned response (e.g., freezing). We have previously shown that an extinction session that occurs within the reconsolidation window (termed retrieval + extinction) attenuates fear responding and prevents the return of fear in Pavlovian fear conditioning (Monfils et al., 2009). To date, the mechanisms that explain the different behavioral outcomes between standard extinction and retrieval + extinction remain poorly understood. Here we sought to examine the differential temporal engagement of specific neural systems by these two approaches using Arc catFISH (cellular compartment analysis of temporal activity using fluorescence in situ hybridization (FISH)). Our results demonstrate that extinction and retrieval + extinction lead to differential patterns of expression, suggesting that they engage different networks. These findings provide insight into the neural mechanisms that allow extinction during reconsolidation to prevent the return of fear in rodents.

#### Edited by:

Oliver T. Wolf, Ruhr University Bochum, Germany

Reviewed by:

Seth Davin Norrholm, Emory University School of Medicine, USA Harald Engler, University Hospital Essen, Germany

> \*Correspondence: Marie-H. Monfils marie.monfils@utexas.edu

Received: 12 October 2015 Accepted: 21 December 2015 Published: 22 January 2016

#### Citation:

Lee HJ, Haberman RP, Roquet RF and Monfils M-H (2016) Extinction and Retrieval + Extinction of Conditioned Fear Differentially Activate Medial Prefrontal Cortex and Amygdala in Rats. Front. Behav. Neurosci. 9:369. doi: 10.3389/fnbeh.2015.00369 Keywords: fear conditioning, Arc catFISH, extinction, reconsolidation, retrieval + extinction

# INTRODUCTION

Fear conditioning is a widely used paradigm in which the pairing of an initially neutral conditioned stimulus (CS) with an aversive unconditioned stimulus (US) leads to associative learning, such that when later presented with the CS alone, individuals will show a conditioned response (e.g., freezing). After conditioning, fear memories become strengthened over time through a process called consolidation (McGaugh, 2000). Once consolidated, fear memories are extremely persistent, and less susceptible to disruption. Two paradigms (blockade of reconsolidation and extinction) have traditionally been used in the laboratory setting to reduce acquired fear (Wolpe, 1969; Nader et al., 2000). In reconsolidation blockade, retrieval of a consolidated memory followed by pharmacological disruption (e.g., protein synthesis inhibition) leads to a sustained decrease in fear expression. In extinction, the repeated presentation of the CS in the absence of a US leads to a progressive decrease in fear expression (Pavlov, 1927; Rescorla and Heth, 1975; Robbins, 1990). The clinical efficacy of these techniques, however, has been limited. Reconsolidation blockade generally requires potentially toxic drugs, and extinction is not typically permanent (Rescorla and Heth, 1975; Bouton and Bolles, 1979a,b; Robbins, 1990). We devised an effective, drug-free paradigm for the persistent reduction of learned fear that capitalizes on the mechanistic differences between reconsolidation and extinction (Monfils et al., 2009). More specifically we applied extinction training during the retrieval-induced labile period to incorporate, during the reconsolidation window, the re-encoding of the CS as less threatening (retrieval + extinction). Using this approach, we were able to prevent the return of fear. Our retrieval + extinction paradigm has since been used successfully to persistently modify aversive and appetitive memories in rodents (Monfils et al., 2009; Rao-Ruiz et al., 2011; Xue et al., 2012; Olshavsky et al., 2013a,b; for a review, Auber et al., 2013; but see also: Chan et al., 2010). The effect has also been observed in humans (Schiller et al., 2010, 2013; Xue et al., 2012). It should be noted however that the phenomenon may be susceptible to boundary conditions which may not be fully understood at this point (for example, see Sevenster et al., 2012, 2014).

We previously showed that fear memory retrieval leads to increased levels of phosphorylated GluR1-containing AMPARs (pGluR1-containing AMPARs). When a second CS is presented 1 h after the initial retrieval, the receptors undergo dephosphorylation, possibly suggesting that destabilization of the memory trace might underlie the lack of fear reemergence in the retrieval extinction manipulation (Ret + Ext; Monfils et al., 2009). Clem and Huganir (2010) found that a central component of Ret + Ext-induced reduction in fear expression is the synaptic removal of CP-AMPARs in the lateral amygdala (LA), a metabotropic GluR1 receptors (mGluR1) dependent mechanism that leads to memory destabilization and subsequent reconsolidation, and an ensuing weakening of pre-existing synapses similarly to what occurs following long-term depression (LTD). Clem and Huganir (2010) thus showed that reconsolidation update and CP-AMPARs-mediated LTD share a requirement for mGluR1 activation. Recently, we observed a differential pattern of Zif268 and rpS6P expression in the amygdala and medial prefrontal cortex (mPFC) following extinction vs. retrieval + extinction. Those data suggested that new information from extinction training applied after the retrieval of a consolidated fear memory led to an updating in a reconsolidation process (Tedesco et al., 2014).

Still, to-date, the precise dynamic mechanisms underlying the different behavioral outcomes of standard extinction vs. extinction applied after an isolated retrieval are not completely understood. Here, we sought to examine the differential temporal engagement of specific neural systems by the initiation of Extinction vs. Retrieval + Extinction mechanisms, using Arc catFISH [cellular compartment analysis of temporal activity using fluorescence in situ hybridization (FISH)]. catFISH provides a brain-wide visualization of the populations of neuron that are selectively involved in two temporally-distinct events as identified by the presence of Arc mRNA either in a cell's nucleus and/or cytoplasm (Guzowski et al., 1999; Vazdarjanova et al., 2002). We specifically quantified nuclear and cytoplasmic Arc expression in the amygdala (lateral and basal), and mPFC (prelimbic and infralimbic), as these regions have been implicated in fear consolidation and extinction (Phillips and LeDoux, 1992; Quirk et al., 1997; Knapska and Maren, 2009).

# MATERIALS AND METHODS

# Animals

Male Sprague Dawley rats (250–300 g at arrival; Harlan Lab Animals Inc., IN, USA) were housed in pairs in clear plastic cages with food and water provided ad libitum. The rats were maintained on a 12 h light-dark cycle (lights on at 7 am) and the behavioral procedures were conducted during the light cycle. Procedures were conducted in compliance with the National Institutes of Health Guide for the Care and Use of Experimental Animals and were approved by the University of Texas at Austin Animal Care and Use Committee.

# Apparatus

All behavioral procedures took place in standard conditioning chambers made with stainless-steel walls and rod floors connected to a shock generator (Coulbourn Instruments, Allentown, PA, USA). Chambers were enclosed in acoustic isolation boxes (Coulbourn Instruments) and lit with a red light. Behavior was recorded with digital cameras mounted on the top of each unit. The chambers were wiped with soap and water between each session. Stimulus delivery was controlled using Freeze Frame software (Coulbourn Instruments). The CS was a tone (5 kHz, 80 dB) 20 s in duration and the US was a 0.7 mA foot-shock 500 ms in duration.

## Behavioral Procedure

The rats were first fear conditioned. After a 10 min habituation period in the chamber, rats received three presentations of the CS co-terminating with the US, with an average of 180 s intertrial intervals (ITI). The next day, the rats were divided to either (1) ''1 then 4 CSs'' or (2) ''10 CSs'' groups. The rats in the 1 then 4 CSs group (n = 6) received a single 20 s CS presentation in the absence of the US and were returned to their home cages in the colony for 15 min. Then, they were returned back to the chambers and received four more 20 s CS presentations without US (150 s ITI). The rats in the 10 CSs group (n = 6) received 10 20 s CS presentations in the absence of the US (150 s ITI). In addition to these two groups, a third group of rats (termed ''1 then 4 tones'', n = 3) underwent mock fear conditioning the first day in which they were exposed to the three presentation of CS but not the accompanying US. The next day, they received an identical procedure as the 1 then 4 CSs group, in which they received one 20 s tone presentation followed by a 15 min period in the home cage, and then four additional presentations of 20 s tone. The behavioral procedures for all three groups lasted 30 min. These behavioral procedures were temporally arranged to detect expression of nuclear and cytoplasmic Arc mRNA, which have time-limited appearance in the activated neurons (Vazdarjanova et al., 2002). As seen in **Figure 2**, neurons activated during the first 5 min of the session should show peak cytoplasmic Arc expression at the time of perfusion (which occurred 30 min later) while neurons activated during the last 5 min of the session should show peak nuclear Arc expression at the time of perfusion which occurred immediately after the session was over. Neurons activated at both time points should have both cytoplasmic and nuclear Arc staining.

An experimenter blind to the overall hypothesis and design of the study scored freezing behavior manually from video recorded during each session. However, it was difficult for the experimenter to remain completely blind to the second day behavioral procedures in which the number of CS presentations differed between groups. Freezing was defined as the absence of any movements, excluding those required for respiration. The total number of seconds spent freezing throughout the CS presentation was expressed as a percentage of CS duration.

#### Histology Procedure

Immediately after the end of the 30 min behavioral procedure on the second day, rats received an overdose of pentobarbital (86 mg/kg) and phenytoin (11 mg/kg) mix (Euthasol<sup>r</sup> by Virbac Animal Health) and then were perfused transcardially with 0.9% saline followed by 4% Paraformaldehyde (PFA) in 0.1 M phosphate buffer (PB). The brains were extracted, placed into a 20% sucrose PFA/PB solution overnight, rapidly frozen using powdered dry ice the next day and stored at −80◦C. The brains were sliced as 25 µm thick coronal sections using a sliding microtome and the sections were immediately mounted on slides. Then, they were vacuum dried overnight at room temperature and stored in an air-tight container with desiccant at −80◦C.

#### Fluorescence in situ Hybridization

Every fifth section containing the medial PFC and the amygdala were processed with FISH for Arc mRNA detection using a modified protocol of Petrovich et al. (2005). Slides were treated with proteinase K and then with acetic anhydride. Then, they were gradually dehydrated through ascending concentrations of ethanol solutions. The sections were then covered with hybridization solution containing cRNA probe and incubated for 20 h at 60◦C. The cRNA riboprobe was generated by using T7 RNA polymerase (Ambion; Grand Island, NY, USA) and by incorporating digoxigenin-UTP (DIG RNA labeling mix; Roche Applied Science, Indianapolis, IN, USA). The riboprobe was then purified using mini Quick Spin Columns (Roche). The plasmid used for generating Arc antisense contained the full length cDNA (∼3.0 kbp) of Arc transcript.

After hybridization, slides were first washed in 4X SSC at 60◦C before being treated with RNase and then washed in descending concentrations of SSC at 60◦C. Then, the slides underwent immunocytochemical process using the PerkinElmer Tyramide Signal Amplification system (NEL704A; PerkinElmer, Waltham, MA, USA). Briefly, the tissue was incubated with antidigoxigenin conjugate for 2 h and with cyanine 3 substrate for 30 min. Then, the tissue was covered slipped using a mounting medium that contained the nuclear stain 4<sup>0</sup> , 6-diamidino-2 phenylinodole, DAPI (Vectashield; Vector Lab, Burlingame, CA, USA).

#### Image Acquisition and Analysis

Images were acquired using a fluorescence laser scanning confocal microscope, Zeiss LSM 710 (Zeiss: Thornwood, NY, USA). First, the correct regions of interest [i.e., prelimbic and infralimbic cortices, and lateral and basal amygdala (BA)] were identified based on nuclear DAPI staining with 10× objective. Then, using 40× oil objective, confocal z-stacks composed of 0.9 µm thick optical sections were collected through the regions of interest. A typical confocal stack had ∼12 optical sections that contained ∼112 cells identified by nuclear DAPI staining. For each of the prelimbic and infralimbic regions, an average of six stacks were collected from the sections that were between 3.20 to 2.80 anterior to Bregma according to Brain Maps v3 (Swanson, 2004). For Amygdala, an average of eight stacks in the lateral nucleus and four stacks in the basal nucleus were collected between 1.78 to 2.45 posterior to Bregma (Swanson, 2004).

Using Imaris software (Bitplane; Concord, MA, USA), an experimenter blind to the behavioral conditions analyzed the acquired images. Only the cells that showed the entire nuclei DAPI staining throughout the z-sections were considered. First, the cells that contained diffused perinuclear Arc staining were counted and classified as ''cytoplasm''. Second, the cells that contained clear two Arc intranuclear foci were counted and classified as ''nucleus''. Then, the cells that contained both the perinuclear and intranuclear foci staining of Arc were classified as ''double''. These Arc+ cells were calculated as percentage of the overall DAPI stained cells for each stack and then averaged across the sampled stacks.

#### Statistical Analysis

Statistical analyses were carried out using SPSS Statistics software. One-way ANOVAs with retrieval group as between subject factors were conducted. Where appropriate, post hoc tests were performed with Tukey's honestly significant difference mean comparison.

#### RESULTS

# Retrieval and Extinction of Conditioned Fear

Our previous work showed that extinction and retrieval + extinction procedures led to different behavioral outcomes. Here, we sought to determine how the two differ at the timepoint where we hypothesize they mechanistically diverge. Two principal groups were run for this experiment. The first group (1 then 4 CSs) was representative of the initiation of the retrieval + extinction memory updating and associated mechanisms. The second group (10 CSs) was representative of the initiation of extinction and associated mechanisms.

The rats in these two groups (''1 then 4 CSs'' and ''10 CSs'') received an identical fear conditioning procedure (i.e., three pairings of tone CS—shock US) on the first day. As expected, there was no difference in freezing during the fear conditioning session between the two groups (**Figure 1** left panel). Oneway ANOVA with repeated measures over three trials show a significant within-subjects effect, F(2,20) = 39.4, p < 0.001,

indicating that rats froze significantly more toward the end compared to the beginning of the session. And there was no main effect of groups, F(1,10) = 0.17, p > 0.5, supporting that both groups of rats displayed comparable freezing.

On the second day, the rats returned to the conditioning chambers and received either: (1) a single presentation of CS followed by 15 min in the homecage and then four additional CS presentations in the chamber (1 then 4 CSs group); or (2) ten CS presentations (10 CSs group). All rats showed significant conditioned freezing to the CS and the levels were similar between the groups (**Figure 1** right panel). One-way ANOVA with repeated measures over the first 4 trials of extinction show no main effect of groups, F(1,10) = 0.12, p > 0.5. Furthermore, we compared conditioned freezing during the very first CS exposure and the last CS exposure of these two groups given that the behavioral procedure was designed to detect two different time points of neuronal activation (i.e., the first and last 5 min) using the catFISH method (**Figure 2**). Thus, for the 1 then 4 CSs group, conditioned freezing is shown from the single CS presentation prior to 20 min homecage time and from the fourth CS presentation given after the homecage time. For the 10 CSs group, conditioned freezing is shown from the first and tenth CS presentations. The freezing levels were not different between the groups neither during the first CS, t(10) = 1.8, p > 0.1, nor during the last CS, t(10) = 0.1, p > 0.5. We expected to see activation of Arc during the initial 5 min with the first CS primarily in cytoplasm and Arc activation during the last 5 min with the last CS primarily in the nucleus.

## Detection of Arc mRNA Activated by Fear CS

Using nuclear DAPI staining as an anatomical guide, four regions of interest (i.e., prelimbic and infralimbic regions of the mPFC, and lateral and basal nuclei of the amygdala) were analyzed via confocal z-stacks. Then, DAPI-stained cells that expressed nuclear and/or cytoplasmic Arc were calculated.

**Figure 3** shows Arc+ cells in the prelimbic **(A)** and infralimbic **(B)** areas between the two groups with representative photomicrographs. For both regions, there was no difference in cytoplasmic Arc expression between the two groups (p's > 0.1 for both). This suggests that there were comparable neuronal activation by the initial CS presentation at the beginning of the session. However, there was a significant difference in nuclear Arc expression in which the rats in the 10 CSs group showed significantly more nuclear staining both in the prelimbic cortex (PL; t(10) = 3.25, p < 0.01) and in the infralimbic cortex (IL; t(10) = 2.72, p < 0.05). This suggests that more neurons were activated by the CS presentation during the last 5 min among the rats in the 10 CSs group. Furthermore, there were also more double labeled cells in the 10 CSs group both in the PL (t(10) = 3.67, p < 0.01) and in the IL (t(10) = 2.14, p = 0.058). This suggests that the neurons initially engaged by the CS presentation at the beginning of the 30 min session were recruited again by the CS presentation at the end of the session.

**Figure 4** shows Arc+ cells in the lateral **(A)** and basal **(B)** nuclei of the amygdala between the two groups with representative photomicrographs. Within the LA, there was no difference in cytoplasmic Arc expression between the two groups (p > 0.5). This suggests that there was comparable neuronal activation by the initial CS presentation. However, there was a significant difference in nuclear Arc expression in which the rats in the 1 then 4 CSs group showed significantly reduced nuclear staining compared to the 10 CSs group (t(10) = 3.05, p < 0.05). This suggests that fewer neurons were activated by the CS presentation during the last 5 min among the rats in the 1 then 4 CSs group. Unlike in the LA, there were no obvious group differences in the BA with the exception of the marginally significant difference seen with double labeled cells, t(10) = 2.19, p = 0.054.

In order to rule out the possibility that these differences might be purely based on the differences in the number of CS presentations, Arc expression of the 1 then 4 CSs group was compared to a third group (i.e., 1 then 4 tones group) that received mock fear conditioning (i.e., 3 CS presentations without US) and an identical procedure as the 1 then 4 CSs group on the second day. **Figure 5** shows the comparisons of Arc expression in all four regions of interest. The Arc expression of the control (1 then 4 tones) group was similar to 1 then 4 CSs group in both subregions of the mPFC as well as the BA. In terms of the LA, there was no significant difference in the cytoplasm Arc (t(7) = 1.97, p = 0.089); however, there was a significant difference in the nucleus Arc staining. There were fewer activated cells in the 1 then 4 CSs group than the control (1 then 4 tones) group (t(7) = 2.55, p < 0.05). Furthermore, there were significantly fewer double-labeled cells in the 1 then 4 CSs (t(7) = 2.64, p < 0.05) suggesting that, relative to the 1 then 4 tones, a fewer portion of the cells that were engaged by the initial CS presentation were recruited again by the CS presentations at the end of the session.

### DISCUSSION

Memories acquired through fear conditioning are extremely persistent. Extinction and reconsolidation blockade are routinely used in laboratory settings to attenuate fear memories, though their clinical efficacy remains limited. Reconsolidation-based

interventions are generally effective in permanently modifying memories, but they often require the use of toxic drugs that are not safe for use in humans (propranolol being a notable exception, Kindt et al., 2009). Extinction-based approaches (e.g., exposure therapy) do not work in all individuals, and for those in which they are effective, fear re-emergence often occurs. We, and others, previously showed that a combination of these two approaches, that is, extinction applied after retrieval of consolidated memories, prevented fear reemergence and drug-seeking relapse in a context-independent way in rats and humans (Monfils et al., 2009; Schiller et al., 2010, 2013; Rao-Ruiz et al., 2011; Xue et al., 2012; Olshavsky et al., 2013a,b; for a review, Auber et al., 2013; see also Sevenster et al., 2012, 2014). To date, cellular and molecular mechanisms underlying extinction applied during reconsolidation remain poorly understood.

The present study examined the differential temporal engagement of specific neural systems by the initiation of Extinction vs. Retrieval + Extinction mechanisms, using Arc catFISH. Two main experimental groups were conducted: an extinction mechanism group (10 CSs) and a retrieval + extinction mechanism group (1 then 4 CSs). The 10 CSs group was chosen to allow us to examine the circuitry engaged as extinction mechanisms begin to be progressively recruited. The 1 then 4 CSs group was chosen to isolate the circuitry engaged as mechanisms associated with retrieval + extinction (extinction applied after an isolated retrieval) are recruited. We examined 4 brain regions: IL, PL, LA, and BA. These regions were selected, because previous studies found them to be engaged during fear extinction (Quirk et al., 1997; Knapska and Maren, 2009). Knapska and Maren (2009) previously showed that reduced fear expression in response to a CS in the extinction context is associated with increased activity in the IL, and the return of fear to a CS presented in a different context is associated with activity in PL and LA.

Importantly, we found that the 10 CSs and the 1 then 4 CSs groups showed significantly different overall patterns of Arc expression as the mechanisms of extinction and retrieval + extinction became progressively initiated. The initial neural engagement in the prefrontal cortex was comparable in our two experimental groups—there was no difference in Arc expression in the cytoplasm in the PL and IL. The involvement of these brain structures intensified in the 10 CSs group, suggesting the continued and increasing engagement of the IL as extinction processes were recruited (in line with previously reported findings from the literature, Quirk et al., 1997; Milad and Quirk, 2002; Knapska and Maren, 2009; Do-Monte et al., 2015). The continued engagement of the PL in our 10 CSs group

is likely reflective of the maintained behavioral fear response at this stage of the extinction protocol. PL has previously been found to be required for fear expression, and not to be required for the maintenance of extinction (Sierra-Mercado et al., 2011). Our results further suggest that a portion of the cells activated during the later phase of our 10 CSs group were newly recruited as extinction progressed, as evidenced by the fact that only a fraction of the cells in these regions expressed double labeling in both nucleus and cytoplasm. A different pattern emerged in the 1 then 4 group. Fewer cells were de novo recruited in the latter phase of this experimental group, as evidenced by the fact that there was significantly less nuclear staining than in the 10 CSs group. Furthermore, only very few cells showed double labeling.

The two groups also differed in their expression in the amygdala. Similarly to what was observed in the prefrontal cortex, the two groups were comparable in cytoplasmic staining for both the LA and the BA at the beginning of their respective experimental window. The 1 then 4 CSs group showed fewer cells with nuclear staining relative to the 10 CSs group, in the face of comparable double expression, suggesting that while the 10 CSs group recruited more cells in the LA during the later phase of the extinction paradigm, the 1 then 4 groups did not. There were no differences in BA.

Together, our results in the prefrontal cortex and amygdala indicated that the initiation of reconsolidation updating (retrieval + extinction) differed from that of the initiation of standard extinction. We next compared the results from our 1 then 4 CSs group to that of a group that received the same pattern of Cue (tone) presentations, but which had not been fear conditioned the previous day. Our results show no difference between our two groups in the IL, PL, and BA, suggesting little engagement of these structures by the retrieval extinction manipulation beyond baseline levels. Interestingly, there were notable differences in the LA, whereby there were fewer cells expressing double and nuclear staining in the 1 then 4 CSs (conditioned group) than its control (unconditioned) counterpart. These results help solidify the notion that fewer cells in the LA were activated as retrieval-extinction mechanisms engaged, compared to what occurs in the case of standard extinction, and that difference observed between the 10 CSs and the 1 then 4 CSs groups was not simply due to a difference in the number of tone presentation.

Arc is thought to play a critical role in synaptic plasticity (Guzowski et al., 2000; Plath et al., 2006; Messaoudi et al., 2007), and in the present study, identifies the neurons that are active in response to different groupings of CS presentations, with the advantage of capturing the neural ensembles that

are involved at two different time points. The LA results appear to be in-line with our previously published findings (Monfils et al., 2009). Effectively, we previously found that either a single CS, or 2 CSs presented with an interval of 3 min (akin to intervals typically used in a standard extinction paradigm) led to an increase in GluR1 expression in the LA. When the CS was applied 1 h after an isolated CS led to a dephosphorylation of GluR1 receptors in the LA (Monfils et al., 2009). Together with the present results, as well as the findings of Clem and Huganir (2010), we propose that the retrieval + extinction may occur through an active reversal of plasticity in the LA.

More recently, we examined the effects of extinction vs. retrieval + extinction on the expression of two different proteins (zinc-finger protein 268 [zif268], and phosphorylated ribosomal protein S6) in the IL, PL, LA and CA1 region of the hippocampus. The experiments from that study revealed that extinction applied after retrieval selectively increased zif268 and phosphorylated ribosomal protein S6 in the prefrontal cortex and amygdala, in a pattern of activity that was distinct from standard extinction (Tedesco et al., 2014).

Taken together, these studies suggest that at the beginning, as well as the end of training, extinction and retrieval + extinction engaged divergent brain mechanisms. In the present study, we used a more dynamic approach (Arc catFISH), which allowed us to identify the networks engaged as a result of extinction vs. retrieval + extinction at the time-point where we believe the two protocols to mechanistically diverge. Effectively, we hypothesized that the two protocols would lead to comparable circuit activation at the beginning of training, which generally corresponds to memory retrieval. For the first time, we were also able to determine which cells, of those that were active near the end of our training paradigms, were also active at the beginning (fear retrieval timepoint), and which were de novo recruited. The latter was crucial in allowing us to determine whether increased activity would be best explained as sustained increased engagement from fear memory retrieval, recruitment of new cells, or a combination of both.

Our data reveal the differential engagement of amygdala, and mPFC subregions during extinction vs. retrieval + extinction, thereby highlighting their specific dynamic contributions at the moment where their mechanistic contributions are thought to diverge. In essence, our results strengthen the notion that extinction applied during the reconsolidation window engages mechanisms distinct from standard extinction, and explains why they lead to drastically different behavioral outcomes.

### REFERENCES


#### AUTHOR CONTRIBUTIONS

HJL and M-HM designed, conducted and wrote the work. RPH provided Arc plasmid and helped with designing the procedure for fluorescence in situ hybridization. RFR helped with behavioral experiment and data analyses.

#### ACKNOWLEDGMENTS

The work was supported by 1R21MH086805 and 1R01MH091147 to MHM.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Lee, Haberman, Roquet and Monfils. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Assessing Fear Following Retrieval Extinction Through Suppression of Baseline Reward Seeking vs. Freezing +

Jason Shumake and Marie H. Monfils \*

Department of Psychology, The University of Texas at Austin, Austin, TX, USA

Freezing has become the predominant measure used in rodent studies of conditioned fear, but conditioned suppression of reward-seeking behavior may provide a measure that is more relevant to human anxiety disorders; that is, a measure of how fear interferes with the enjoyment of pleasurable activities. Previous work has found that an isolated presentation of a fear conditioned stimulus (CS) prior to extinction training (retrieval + extinction) results in a more robust and longer-lasting reduction in fear. The objective of this study was to assess whether the retrieval + extinction effect is evident using conditioned suppression of reward seeking, operationalized as a reduction in baseline licking (without prior water deprivation) for a 10% sucrose solution. We found that, compared to freezing, conditioned suppression of reward seeking was much more sensitive to fear conditioning and far less responsive to extinction training. As in previous work, we found that retrieval + extinction reduced post-extinction fear reinstatement when measured as freezing, but it did not reduce fear reinstatement when measured as conditioned suppression. This suggests that there is still residual fear following retrieval + extinction, or that this procedure only modifies memory traces in neural circuits relevant to the expression of freezing, but not to the suppression of reward seeking.

#### Edited by:

Oliver T. Wolf, Ruhr-University Bochum, Germany

#### Reviewed by:

Susan Sangha, Purdue University, USA Sarah Starosta, Ruhr-University Bochum, Germany

> \*Correspondence: Marie H. Monfils marie.monfils@utexas.edu

Received: 28 August 2015 Accepted: 07 December 2015 Published: 23 December 2015

#### Citation:

Shumake J and Monfils MH (2015) Assessing Fear Following Retrieval + Extinction Through Suppression of Baseline Reward Seeking vs. Freezing. Front. Behav. Neurosci. 9:355. doi: 10.3389/fnbeh.2015.00355 Keywords: fear conditioning, freezing, conditioned suppression, extinction, reconsolidation, retrieval + extinction

# INTRODUCTION

Freezing, or becoming motionless in the presence of fear-evoking stimuli, is one of the innate defensive reactions of rats and other rodents, and it has become the predominant measure—often the only behavioral measure—used in studies of conditioned fear/threat. However, historically this was not always the case. For several decades following its introduction by Estes and Skinner (1941), conditioned suppression was the predominant technique for measuring conditioned fear. In the prototypical conditioned suppression paradigm, food-deprived rats are first trained to press a lever to receive food reward. Then a conditioned stimulus (CS) is paired with a shock unconditioned stimulus (US). Subsequently, rats suppress lever responding when the CS is present, and the magnitude of this suppression offers an indirect measure of conditioned fear.

Eventually, the numerous advantages offered by conditioned freezing led to its dominant use in studies of Pavlovian fear conditioning. Namely, freezing behavior can be directly observed without the need for extensive prior operant training or the accompanying states of physiological deprivation required to elicit robust operant responding. Such a simplified preparation is especially appealing for uncovering the neurobiological mechanisms specific to fear learning because one does not need to control for the neural effects of altered motivational states or concurrent appetitive learning. However, in recent years, there has been increasing interest in translating findings from Pavlovian fear conditioning to the treatment of human anxiety disorders. For this objective, conditioned suppression may be more relevant than freezing to the way that human beings experience fear, i.e., as something that interferes with the enjoyment of pleasurable activities (McDannald and Galarce, 2011).

We recently reported that a modified extinction paradigm, retrieval + extinction (Ret + Ext), resulted in a persistent attenuation of fear memories, leaving them less susceptible to return of fear as evidenced by several measures, including resistance to fear reinstatement following unsignaled shock presentations (Monfils et al., 2009). This finding has been replicated both in rodents and humans (Schiller et al., 2010, 2013; Rao-Ruiz et al., 2011; Olshavsky et al., 2013a; for a review, Auber et al., 2013; but see also: Chan et al., 2010) and extended to appetitive memories (Xue et al., 2012; Olshavsky et al., 2013b; Sarter and Ashton-Jones, 2014). What is not yet known is whether the original fear memory is being erased (we do not believe this to be the case; Tedesco et al., 2014) or updated, and, if the latter, to what extent the CS is still perceived as a threat. This is impossible to assess using freezing alone because the absence of freezing does not necessarily indicate the absence of fear. As discussed by Blanchard and Blanchard (1988), animals do not shift abruptly from freezing to normal behavior (eating, drinking, aggression, and sexual activity); rather, there is a protracted intermediate period of risk assessment, characterized by cautious exploration and the suppression of unnecessary activities. Thus, the main objective of this study was to assess whether the Ret + Ext procedure is successful not only at preventing reinstatement of conditioned freezing, but also at preventing reinstatement of conditioned suppression.

We further introduce in this study a modified version of the conditioned suppression procedure to mitigate its major drawbacks: namely, the need for food or water deprivation and operant response training. We have previously shown that, when placed in an operant box with free access to water sweetened with sucrose, rats will voluntarily spend a substantial percentage of time drinking without the need for prior water deprivation (Shumake et al., 2005; Hamani et al., 2010). Moreover, licking behavior does not require special training and can be automatically and precisely quantified using an optical lickometer. While a deprivation period would no doubt result in more robust drinking behavior, our objective was to simulate conditions under which a human patient with an anxiety disorder might experience dysfunction, i.e., conditions which typically do not involve severe hunger or thirst. In other words, we wanted the response competing with fear to be motivated by pleasure, not survival.

Arguably "pleasure drinking" offers not only greater translational relevance to humans, but also a more sensitive instrument with which to measure fear itself. We submit that "absolute zero" on the fear-measurement scale should be operationalized as a complete return to normal behavior in a safe environment, and that neither 0 freezing nor 0 suppression of "survival drinking" offers sufficient evidence that this has occurred. For reasons already discussed, the CS can be perceived as threatening without evoking freezing. Likewise, eating or drinking under conditions of extreme hunger or thirst does not demonstrate that the CS is no longer threatening; rather, it only demonstrates that the threat of the CS is less severe than the threat of starvation or dehydration. Drinking for the simple pleasure of experiencing a sweet taste, on the other hand, is not a necessary activity. Therefore, if Ret + Ext behavior restored and preserved this behavior, it would offer stronger evidence that the original fear memory had been fundamentally rewritten. We tested this hypothesis by comparing the long-term memory of fear vs. extinction learning in animals who received either standard extinction or Ret + Ext, as assessed by freezing vs. suppression of drinking following acquisition, extinction, and reinstatement.

# METHODS

## Animals

A total of 32 male Sprague-Dawley albino rats (Charles River Laboratories) arrived in our animal facility at approximately 50 days of age and were pair-housed (2 per cage). Rooms were maintained at steady temperature (21 ± 1 ◦C) and a 12-12 lightdark cycle (lights on at 7:00 and off at 19:00). Except for one 24-h period of water deprivation as described below, food and water were provided ad libitum. All procedures followed US National Institutes of Health guidelines and were approved by the Institutional Animal Care and Use Committee at the University of Texas at Austin.

#### Apparatus

Rats were conditioned and tested in a Habitest Modular System (Coulbourn Instruments) equipped with metal rod flooring connected to a shock generator, a speaker connected to a tone generator, and an optical lickometer that continuously monitored licking of an attached water bottle. Rats were videoed by overhead cameras. Graphic state software controlled stimulus presentations and recorded lickometer data. Raw data files from these sessions were exported as text files, and stimulus-dependent changes in licking behavior were quantified using a customwritten R package, "lickometer," which can be downloaded from https://github.com/jashu/graphic-state-munging.

#### Procedure

A schematic of the experimental design is shown in **Figure 1**.

#### Establishment of Baseline Drinking (Days 1–3)

Following 1 week of acclimation after arrival at our facility, rats underwent 3 daily sessions of habituation to the conditioning chamber to establish a baseline rate of drinking (**Figure 2**), each session consisting of 10 min in the conditioning chamber with access to a bottle of drinking water with a 10% concentration of sucrose. Based on pilot data in which we ran subjects under conditions of both restricted and unrestricted access to water before assessing baseline drinking, we found that most rats (80%)

that were unrestricted reached levels of drinking comparable to the restricted rats after 4 days of habituation sessions (15 min per day). In order to shorten the required habituation period, we adopted the hybrid paradigm used in this study, in which rats were water-deprived 24 h prior to the first habituation session in order to motivate them to overcome neophobia for drinking in the novel chamber, but then the restriction condition was removed for all subsequent habituation, training, and testing sessions.

#### Fear Conditioning, Extinction, and Reinstatement (Days 4–9)

On Day 4 (the day following the last habituation session), rats underwent fear conditioning. Rats received 3 conditioning trials of a tone (5 kHz for 20 s) co-terminating with a foot shock (0.7 mA for 0.5 s) separated by a variable intertrial interval (ITI) of 1–5 min. This was followed by 2 days of extinction ("Extinction Session 1" and "Extinction Session 2" in **Figure 3**) in which rats either received a standard extinction protocol (Ext) or a retrievalplus-extinction protocol (Ret + Ext). Assignments to the Ext and Ret + Ext groups (n = 16 per group) were made based on cagewise matching of baseline drinking behavior: after calculating the mean baseline drinking for each cage pair, cages were matched according to these means. This was done in order to assign cage mates to the same experimental condition while minimizing differences in baseline drinking motivation between the Ext and Ret + Ext groups.

On Days 5–6 (Extinction Session 1 and Extinction Session 2), rats were returned to the acquisition context for 4 min, during which time the Ret + Ext group (but not the Ext group) received a single 20 s tone. Rats were then returned to their home cages for 1 h and then reintroduced to the same context for extinction training, consisting of tone-alone presentations of the same duration and ITI as experienced in acquisition until both Ext and Ret + Ext groups had heard a total of 18 tones. On Day 7 ("Post-Extinction" in **Figure 4**), rats received 3 memory-recall trials of tone-alone presentations, again with the same duration and ITI parameters. On Day 8, rats received 2 unsignaled footshocks. On Day 9 ("Post-reinstatement" in **Figure 4**), they were tested for fear reinstatement, assessed by their freezing to the tone alone.

# Data Analysis

#### Units of Measurement

Since its advent by Annau and Kamin (1961), the conditioned suppression ratio (CS responding/CS responding + pre-CS responding) is traditionally used when reporting the results of a conditioned suppression experiment, but we are not using Kamin's ratio in this report. For one reason, there were many trials when there was no drinking during either the CS or pre-CS

interval, which would result in division-by-zero errors if not modified. Moreover, such normalization is not conventionally performed for freezing measures, which are expressed in units of percent time. Therefore, we used percent-time units for reporting both drinking and freezing measures.

#### Analysis of Drinking Motivation

Paired t-tests were used to evaluate the effect of water restriction on drinking behavior during habituation to the test chamber, and to compare the water-restricted rate of drinking to the subsequent maximum rate of ITI drinking after extinction training had begun. In addition, we used independent t-tests to evaluate group differences in maximum drinking rate and the latency to reach it.

#### Analysis of Extinction Curves

Based on previous work, we did not expect the Ret + Ext manipulation to result in significant group differences during extinction learning, but, to be thorough, we included experimental group as an independent variable in these analyses. Extinction data were first analyzed with a 2×2 × 18×2 (Group × Session × Trial × CS) repeated measures ANOVA, with Group (Ext vs. Ret + Ext) as the between-subjects measure and Session, Trial, and CS as within-subject measures. Session was included to evaluate between-session extinction, operationalized as increased drinking between the first and second extinction sessions. Trial was included to evaluate within-session extinction, operationalized as increased drinking over the 18 trials. CS was included to evaluate the specificity of fear acquisition and extinction to the tone CS vs. contextual cues. For the above ANOVA, the effect of CS was operationalized as a difference between drinking during the tone vs. the 20 s preceding the tone, in order to match the CS and pre-CS in terms of temporal proximity and duration. However, one limitation of pleasuremotivated (as opposed to thirst-motivated) drinking is that it is more erratic, i.e., characterized by many spontaneous starts and stops. Thus, aggregating over longer intervals may provide a more reliable index of fear by averaging out this source

the middle 50% of the distribution (the interquartile range between the 25th and 75th percentiles), and the horizontal line indicates the median. The "whiskers" that extend vertically from the box indicate the range of observations that fall within ±1.5 times the interquartile range, and any observations outside the whiskers are graphed as individual points. Note the complete suppression of drinking (A) vs. the large variability in freezing (B) following fear conditioning. A significant mean difference between Ext vs. Ret + Ext groups was only observed for tone-CS freezing following reinstatement (F).

of noise. Therefore, we also conducted a separate 2 × 2 × 18 (Group × Session × Trial) repeated measures ANOVA of contextual extinction alone, in which drinking was averaged over each ITI.

In addition, we used LOESS (LOcal regrESSion) to fit separate extinction curves (drinking as a function of trial number) within each Group × Session × CS cell. This nonparametric method constructs a nonlinear "smooth" of the drinking data over time, using local polynomial regression fitting (which requires no prior assumptions about the distribution of the data or the shape of the curve to be fit) to extract signal (systematic variation) from noise (random variation). Plots of these LOESS smooths provide an elegant way to visually compare differences in extinction curves. Moreover, confidence intervals can be constructed for the curve fits themselves, providing a gauge for when a curve significantly diverges from a reference point (e.g., 0) or from another curve without the need for multiple statistical tests at multiple time points.

#### Analysis of Long-Term Memory (LTM)

Long-term changes in fear expression caused by fear conditioning, Ext vs. Ret + Ext, and reinstatement were assessed using data from the first tone presentation 24 h after the end of each of these training protocols. Separate but parallel ANOVAs were applied to the drinking and freezing data using a 2 × 3 × 2 design (Group × Session × CS), with group as a between-subject variable and session (post-acquisition vs. post-extinction vs. post-reisntatement) and CS (context vs. tone) as within-subject variables. Note that we also performed an analysis that included repeated measures of trial, which did not result in any substantive insights beyond the analysis of just the first trials. Moreover, our experimental manipulation was introduced following the first post-acquisition trial (meaning that the first trial provides the only pure baseline measure of fear acquisition), and the first trial of an LTM session following extinction or reinstatement training is the most likely to show spontaneous recovery or savings, respectively. For all these reasons, we chose to confine our analysis of LTM to the first trial.

#### Permutation and Bootstrap p-Values and Confidence Intervals

As can be seen in **Figures 3**, **4**, fear conditioning suppressed drinking to floor levels. The severity of the floor effect varied as a function of training, time, and CS presentation, but nearly all time points showed distributions heavily skewed toward 0 with very long tails. Since this would appear to severely violate ANOVA assumptions, we did not rely on the theoretical sampling distribution of the F statistic to calculate p-values (i.e., the p-value output of standard analysis software); rather, we used random permutations to generate an empirical F distribution as described by Manly (2007). The observations were permuted (randomly reshuffled) 5000 times, simulating the distribution under which the null hypothesis would be true: under data randomization, any relationship between the independent and dependent variables is due to chance. Each permutation was performed in two stages to maintain the distinction of within- vs. between-subject variance: first, observations were permuted within each subject, and then group assignments were permuted between subjects. The ANOVAs outlined above were recomputed for each permutation, and the F statistic was recorded for each of the main effects and interactions. Thus, for each effect, we obtained a sampling distribution of the F-values that would be expected under the null hypothesis for such an unusually distributed measure. The p-value is then approximately equivalent to the proportion of F-values from this sampling distribution that are more extreme than the one obtained from our original data. The exact formula is (r + 1)/(n + 1), where r is the number of permutations that resulted in an F statistic greater than or equal to the original F statistic, and n is the total number of permutations.

Although the freezing data were not characterized by such severely skewed distributions, for consistency, we calculated empirical p-values for these data as well. Both theoretical and empirical p-values are given in the ANOVA summary tables (**Tables 1**–**4**). In the case of significant interactions involving group, simple effects of group were calculated using analogous permutation tests for group-mean differences within each level of the interacting variables (**Table 5**). Likewise, for consistency, we calculated the 95% confidence intervals surrounding the LOESS curves in **Figure 3** using a bootstrap procedure in which the data for each curve were randomly resampled 5000 times with replacement and a LOESS curve was fit to each resample. The confidence intervals in **Figure 3** correspond to the 2.5–97.5 percentile range of these 5000 different fits.

#### Software

Figures were generated using the ggplot2 package for R (Wickham, 2009). Data were analyzed in RStudio (Version 0.99.467) using R (Version 3.2.2).

#### RESULTS

As described in the methods, the distribution of the drinking data severely violated ANOVA assumptions underlying the theoretical distribution of the F statistic used to calculate p-values. Therefore, TABLE 1 | ANOVA of CS vs. pre-CS drinking time (fear of tone vs. context) during extinction learning.


Statistically significant effects are emphasized in boldface.

TABLE 2 | ANOVA of ITI drinking time (fear of context alone) during extinction learning.


Statistically significant effects are emphasized in boldface.

we generated an empirical distribution of the F statistic using unrestricted permutation of the raw data within subjects and permutation of the group labels (Ext vs. Ret + Ext) between subjects. For consistency across analyses, we did the same for the freezing data as well. Both types of p-values, theoretical and empirical, are given in **Tables 1**–**5**. As expected, the two methods yielded highly consistent p-value estimates for the freezing data, to within rounding error of the hundredth decimal place. To our surprise, the two methods were also highly consistent for the drinking data. We caution that this result should not be taken as evidence that one can always ignore the presence of many zero values, which have the potential to exaggerate or diminish statistical effects depending on the proportion of zeros in each group and the direction of mean differences (see Delucchi and Bostrom, 2004, for a review of this problem and recommendations for analysis).

#### Baseline Drinking

**Figure 2** shows that baseline rates of drinking were considerably higher under water-restricted vs. unrestricted conditions, t(31) = 6.5, p < 0.001. However, approximately 60% of rats

TABLE 3 | ANOVA of drinking time during memory trials.


Statistically significant effects are emphasized in boldface.

#### TABLE 4 | ANOVA of freezing time during memory trials.


Statistically significant effects are emphasized in boldface.

TABLE 5 | Simple effects of group differences in freezing to context vs. CS for each memory test.


Statistically significant effects are emphasized in boldface.

were drinking at a baseline rate within the range shown by rats after water deprivation (not counting one rat that showed zero drinking under water restriction). **Figure 2** also shows that rats achieved even higher rates of drinking following extinction (typically during the second extinction session), reaching a maximum ITI drinking rate that was significantly greater than even the rate observed under water restriction, t(31) = 4.9, p < 0.001. Recall that water restriction was implemented only during the first exposure to the apparatus, so this result does not reflect an effect of water restriction so much as an effect of extensive habituation to the apparatus; in other words, we do not know whether water deprivation would have resulted in higher maximum drinking rates had it been implemented for the entire experiment. However, the important point here is that most rats were spontaneously motivated to drink a sweetened solution at a substantial rate without the need for water restriction. Note that there was no significant group difference in the maximum drinking rate using a Welch two sample t-test, t(29.4) = 0.2, p = 0.82, and no significant group difference in the latency to reach maximum drinking, t(29.3) = 0.6, p = 0.58.

#### Drinking During Extinction

**Table 1** reports the results of a Group × Session × Trial × CS ANOVA of drinking behavior within and between the 2 extinction sessions. The main finding was a significant Session × CS interaction (p < 0.001), indicating that drinking during the 20 s preceding the CS showed a significantly greater between-session increase than did drinking during the tone CS. Plots of extinction curves show that while there was a slight between-session increase in drinking during the tone CS (**Figures 3E,F**), tone-CS drinking remained greatly suppressed relative to the drinking observed during the 20 s before the tone CS (**Figures 3C,D**).

**Table 2** reports the results of a Group × Session × Trial ANOVA of drinking during the entire ITI (**Figures 3A,B**), which, as detailed in the Methods, provides a less noisy (lower withinsubject variance) index of contextual fear. In addition to the significant effect of Session, this analysis revealed a significant effect of Trial (p < 0.001). Plots of extinction curves show a gradual increase from an initially complete suppression of drinking during the first extinction session (**Figure 3A**). Rats began the second extinction session (**Figure 3B**) with an even higher rate of ITI drinking that remained steady for most of the session but declined toward the end, presumably because of satiety.

In summary, the results indicate that pleasure-motivated drinking shows CS specificity and responds to extinction training. The Ret + Ext manipulation was not hypothesized to cause behavioral changes during extinction training, and no significant effects of Group were found (**Tables 1**, **2**).

# Long-Term Memory as Assessed by Freezing vs. Drinking

As the top row of **Figures 4A,B** illustrates, fear memory 24 h after acquisition looks very different when viewed through the lens of freezing vs. drinking behavior. There were large individual differences in conditioned freezing. The majority of rats showed high levels of freezing to tone, but several rats showed low levels of freezing. However, as the drinking data illustrate, low-freezing rats did acquire an aversive association to the tone: not one out of the 32 rats spent even a brief moment drinking during the post-acquisition trial. All rats showed a complete suppression of drinking.

**Table 3** reports the results of the Group × Session × CS ANOVA of drinking during the memory trials. There was no significant effect of Ret + Ext on conditioned suppression. As with the above analysis of the full extinction sessions, the main finding was a significant Session × CS interaction (p = 0.004), indicating that the relative difference in drinking time during tone vs. context changes as a function of training. Both tone and context showed complete conditioned suppression following acquisition (**Figure 4A**), but only context showed extinction of conditioned suppression (**Figure 4C**). Following reinstatement training, complete conditioned suppression to context returned for all but 3 subjects per group (**Figure 4E**).

**Table 4** reports the results of the same ANOVA design for freezing during the same trials. This ANOVA revealed that Ret + Ext training caused significant changes in freezing behavior, as evidenced by a Group × Session interaction (p < 0.05). The Ret + Ext group showed a greater reduction in freezing between post-acquisition (**Figure 4B**) and post-extinction (**Figure 4D**) and post-reinstatement (**Figure 4F**). This effect was specific to the tone CS, as evidenced by a significant Group × CS interaction (p < 0.01). Simple effects tests of the interactions (**Table 5**) showed that a significant group difference (p < 0.05) was only evident for the tone CS following reinstatement. Note that, based on previous findings, this was the only condition in this experiment under which the Ret + Ext group was hypothesized to show less post-reinstatement freezing.

# DISCUSSION

We previously reported that an isolated presentation of a fear CS prior to extinction training (retrieval + extinction) results in a more robust and longer-lasting reduction in fear as measured by conditioned freezing after reinstatement. In the present study, we assessed whether the retrieval + extinction effect is evident when conditioned suppression is used to measure fear. Freezing has become the predominant measure used in rodent studies of conditioned fear, but conditioned suppression may provide a measure that is more relevant to human anxiety disorders; that is, a measure of how fear interferes with the enjoyment of pleasurable activities. As in previous work, we found that retrieval + extinction reduced fear reinstatement after extinction when measured as freezing, but it did not reduce fear reinstatement when measured as conditioned suppression. Our results suggest that there is still residual fear following retrieval + extinction, or that this procedure is only modifying memory traces in neural circuits relevant to the expression of freezing but not the expression of conditioned suppression.

Bouton and Bolles (1980) reported that freezing was reliably correlated with the suppression of several consummatory behaviors, including licking for a sucrose solution as used in our study. However, their study deprived rats of food and water for 48 h prior to testing, and, to our knowledge, all studies utilizing the conditioned suppression paradigm have similarly used food or water restriction to instill an intense consummatory drive to compete with conditioned fear. In studies of fear extinction, this may lead to the false impression that when rats resume consummatory behaviors, they are no longer experiencing fear. But an alternative possibility is that the severe physiological challenge of food or water deprivation creates a survival emergency. Interpreted in this way, rats that resume drinking water in the presence of a conditioned fear stimulus have determined that the threat to their survival from dehydration outweighs the threat imposed by the CS; it implies that the CS has becomes less threatening, but it does not necessarily imply that the CS is no longer perceived as a threat. There may be substantial residual fear—not enough to cause freezing, but enough to suppress the pursuit of rewards for the sake of pleasure as opposed to the sake of survival.

Our results show that, not surprisingly, rats consume sweetened water at a higher rate when they have been water deprived, but most were still motivated to drink without prior deprivation because presumably the sweet solution was rewarding in itself. When the survival imperative associated with thirst was no longer a factor, we observed a sharp dissociation between freezing levels and drinking behavior. Whereas fear acquisition levels in terms of freezing were highly variable, acquisition of conditioned suppression was both absolute and invariant. Two extinction sessions were required before some rats began to drink during the tone CS. However, most responded to the extinction session by drinking substantially during the ITI and drinking minimally during the CS. This behavior may have more translational relevance to fear-related psychopathology, reflecting that fear-associated stimuli can continue to elicit sufficient wariness and vigilance to disrupt normal life. The complete extinction of these fear memories would seem to pose a much greater challenge than the extinction of freezing, which may represent only a diminution of fear.

To this end, we were interested to see if the Ret + Ext paradigm, which has been proposed to lead to a disruption in the reconsolidation of fear memories (Monfils et al., 2009), might be successful in normalizing drinking behavior following conditioned suppression. While we replicated previous work finding that Ret + Ext training inhibits the reinstatement of fear as measured by freezing, there was no evidence that Ret + Ext made any difference in the expression of fear as measured by conditioned suppression. One limitation of the drinking measure is that any given animal on any given trial may take a break from drinking that is unrelated to the presence or absence of the CS. (This is indeed the impetus for prior deprivation, which increases the likelihood of continuous drinking.) However, with a sufficient number of subjects, one should still be able to detect the effects of experimental manipulations from the aggregate data. Indeed, the extinction curves reveal an obvious difference between measures of drinking during the 20 s prior to the CS (**Figures 3C,D**) vs. the 20 s during the CS (**Figures 3E,F**).

There are several ways to interpret this finding in terms of the effect of the Ret + Ext paradigm on memory mechanisms. First, it seems clear that the fear memory is not "erased" in its entirety, but it is still possible that the memory is weakened beyond what is achieved by Ext alone. In this view, the fear memory is weakened enough to prevent reinstatement of freezing for the average animal, but not so much as to remove the wariness of drinking during the CS. Another possibility is that fear conditioning instantiates multiple memory traces in parallel neural circuits, only some of which may be vulnerable to disruption by the Ret + Ext manipulation. For example, despite its prominent and wellestablished role in conditioned freezing, the basolateral amygdala (BLA) has been reported to play a minimal role in conditioned suppression (Killcross et al., 1997; Lee et al., 2005; Petrovich et al., 2009; McDannald and Galarce, 2011). Thus, to the extent that the Ret + Ext manipulation selectively targets BLA neuroplasticity, we would expect it to have a greater impact on conditioned freezing than on conditioned suppression.

Finally, **Figure 4F** illustrates that Ret + Ext shifted the distribution of freezing scores toward floor levels of freezing, but there was still substantial overlap with standard extinction. In other words, for some individuals, two sessions of standard Ext training were sufficient to prevent reinstatement of conditioned freezing while, for others, Ret + Ext was not enough. Thus, it seems likely that individual differences moderate the response to extinction paradigms, and it will be important for future research efforts to uncover the relevant phenotypes (Olshavsky et al., 2013a).

Moreover, when animals are drinking to experience the pleasure of a sweet taste (as opposed to quenching an experimentally induced thirst), fear conditioning appears to cause far more indelible behavioral changes. All of the rats resumed drinking by the end of the second extinction session, but they confined their drinking almost entirely to the time between tones (**Figure 4**, right column). Very few were willing to continue drinking in the presence of the tone CS. This persistent wariness may have far more clinical relevance to how anxiety disorders

# REFERENCES


interfere with the ordinary activities of daily life and may also prove far more difficult to eradicate. We believe that conditioned suppression of baseline reward-seeking behavior offers an animal model for investigating the more pervasive consequences of anxiety disorders, which interfere with important activities in daily life—activities that are not necessary for survival but that nonetheless bring pleasure and fulfillment.

# FUNDING

This project was funded by grants from the National Institute of Mental Health (1R21MH086805 and 1R01MH091147 to MM).

# ACKNOWLEDGMENTS

We thank Elizabeth Chen for her technical assistance.

role of enhanced cue-directed behavior and amygdala central nucleus. Front. Behav. Neurosci. 7:186. doi: 10.3389/fnbeh.2013.00186


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Shumake and Monfils. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Retrieval and Reconsolidation Accounts of Fear Extinction

Ravikumar Ponnusamy \*, Irina Zhuravka , Andrew M. Poulos , Justin Shobe , Michael Merjanian , Jeannie Huang , David Wolvek , Pia-Kelsey O'Neill and Michael S. Fanselow \*

Departments of Psychology, Psychiatry and Biobehavioral Sciences, and The Brain Research Institute, University of California, Los Angeles, CA, USA

Extinction is the primary mode for the treatment of anxiety disorders. However, extinction memories are prone to relapse. For example, fear is likely to return when a prolonged time period intervenes between extinction and a subsequent encounter with the fearprovoking stimulus (spontaneous recovery). Therefore there is considerable interest in the development of procedures that strengthen extinction and to prevent such recovery of fear. We contrasted two procedures in rats that have been reported to cause such deepened extinction. One where extinction begins before the initial consolidation of fear memory begins (immediate extinction) and another where extinction begins after a brief exposure to the consolidated fear stimulus. The latter is thought to open a period of memory vulnerability similar to that which occurs during initial consolidation (reconsolidation update). We also included a standard extinction treatment and a control procedure that reversed the brief exposure and extinction phases. Spontaneous recovery was only found with the standard extinction treatment. In a separate experiment we tested fear shortly after extinction (i.e., within 6 h). All extinction procedures, except reconsolidation update reduced fear at this short-term test. The findings suggest that strengthened extinction can result from alteration in both retrieval and consolidation processes.

#### Edited by:

Oliver T. Wolf, Ruhr University Bochum, Germany

Reviewed by: Marie H. Monfils, University of Texas at Austin, USA Oliver Hardt,

The University of Edinburgh, UK

#### \*Correspondence:

Ravikumar Ponnusamy rponnusamy@gmail.com; Michael S. Fanselow fanselow@ucla.edu

Received: 22 October 2015 Accepted: 26 April 2016 Published: 09 May 2016

#### Citation:

Ponnusamy R, Zhuravka I, Poulos AM, Shobe J, Merjanian M, Huang J, Wolvek D, O'Neill P-K and Fanselow MS (2016) Retrieval and Reconsolidation Accounts of Fear Extinction. Front. Behav. Neurosci. 10:89. doi: 10.3389/fnbeh.2016.00089 Keywords: fear, extinction, reconsolidation, consolidation, memory, anxiety disorders

# INTRODUCTION

Fear extinction creates a new ''safe'' memory that co-exists with the original fear memory (Bouton, 1993). Because the two memories are retrieved by the same cue, extinction presents a retrieval problem because it is not clear which memory will be retrieved in any given situation. Retrieval of the fear memory leads to an undesired return of fear. For example, fear spontaneously recovers when substantial time intervenes between extinction and testing. Return of fear contributes to relapse following exposure-based therapies (Bruce et al., 2005), establishing a need for methods capable of making extinction robust against fear recovery.

Myers et al. (2006) hypothesized that beginning extinction soon after fear acquisition might be such a procedure. Memories undergo time-dependent consolidation and they reasoned that if extinction occurred before consolidation of the fear memory, it would be erased and unrecoverable. Supporting this hypothesis, there was less fear recovery if extinction occurred 1 h, rather than 24 h after acquisition. While Myers et al. (2006) attributed their findings to consolidation failure, a retrieval explanation is also possible. Having acquisition and extinction close in time might result in both memories being encoded into the same episode. If the extinction memory was dominant, retrieval of that episode should not provoke fear. Maren and Chang (2006) reported that when fear levels are very high during extinction there was little evidence of a long-term extinction memory. According to a retrieval interpretation, if acquisition and extinction are part of the same episode, and fear is high during that episode, then a fear memory would be retrieved. In contrast to Myers et al. (2006) and support of Maren and Chang (2006) many studies found failure of immediate extinction on fear memory [rats (Morris et al., 2005; Archbold et al., 2010), mice (Stafford et al., 2013) and human (Alvarez et al., 2007; Norrholm et al., 2008; Schiller et al., 2008; Huff et al., 2009)].

A practical limitation in using immediate extinction is the need to begin extinction close to the time of trauma. Monfils et al. (2009) suggested a way to ease this limitation, by providing a brief reminder to open a window of vulnerability before a typical experimental extinction that is done 24 h or longer after fear conditioning. The logic is that memories are vulnerable to amnestic agents both shortly after fear learning (Schafe and LeDoux, 2000) and after a reminder (Nader et al., 2000). Monfils et al. (2009) found that, indeed, extinction memories were robust against fear recovery when a reminder shortly preceded extinction (for opposite results in both rodents and humans, see Chan et al., 2010; Costanzi et al., 2011; Kindt and Soeter, 2013; Stafford et al., 2013). This effect has become known as reconsolidation update (Monfils et al., 2009). There are several possible explanations as to why this procedure works. One based on the consolidation and reconsolidation literatures is that reopening the ''vulnerability window'' allows the original fear memory to be deconsolidated in much the way a protein synthesis inhibition allows deconsolidation of the original memory. A second, which does not depend on reconsolidation mechanisms at all, suggests that the reminder allows the extinction learning to be incorporated into the original memory and thereby results in a change in the encoded CS-US relationship. It is difficult to reconcile this account of the more durable extinction result with what happens in traditional multi-trial extinction procedures as these also have reminders but do not produce enduring extinction. These two accounts of the reminder-extinction effect do make a differential prediction. The former deconsolidation account suggests that at a short-term test the original fear memory should be intact just as it is in traditional consolidation and reconsolidation experiments. The fear memory should only disappear after a longer-term test interval. The latter interpretation in terms of a degraded CS-US relationship, suggests no difference at a short- or long-term interval as the CS-US relationship would be degraded at either time point. Therefore, we conducted both long and short-term tests to distinguish these accounts.

This reconsolidation update effect is also interpretable from retrieval-based models (Bjork, 1994; Schmidt and Bjork, 1992), which predict that variability in retrieval practice makes memories more retrievable. According to their model, variation increases the storage strength of information to be learned by making retrieval of past learning easier via the availability of cues that were present during prior learning. The retrieval model stresses the importance of variability of exposure but the order of the session types is less important. Variability in extinction training conditions leads to enhanced extinction retrieval in studies of human fear memory (e.g., Rowe and Craske, 1998a,b). The retrieval and extinction sessions can be viewed as two different extinction experiences. This increased retrieval variability may render extinction memories more retrievable and therefore, more resistant to fear recovery. To test this interpretation, we used a procedure that retained the same variability of experience as the procedure thought to generate reconsolidation update but did not give extinction during a window of vulnerability. We simply gave our extinction session prior to the reminder. This lead to a five group design assessing spontaneous recovery following traditional massed extinction, immediate extinction, reconsolidation update and our variability control procedure. Note that our fear extinction—retrieval methods are similar to methods used by Baker et al. (2013) and Millan et al. (2013) that showed enhanced fear extinction retrieval in adolescent rats and enhanced extinction retrieval of alcohol seeking in adult rats respectively.

To further distinguish memory reconsolidation and retrieval accounts, Experiment 2 tested fear shortly after extinction (3.25 h) and at the typical 24 h period after extinction. When memories are tested while consolidation or reconsolidation processes are ongoing, amnestic manipulations typically have no effect. Rather, their effect emerges later when the memory is dependent on that earlier consolidation process (Nader et al., 2000; Schafe and LeDoux, 2000). If these manipulations affect memory consolidation then fear memory should be intact during the early test, but absent during the later test.

# MATERIALS AND METHODS

# Animals

In the present study, we used male adult rats (Long Evans; HsdBlu:LE) initially weighing 250–280 g (Harlan, Indianapolis, IN, USA). After arrival at UCLA, the rats were housed individually in standard stainless steel cages on 12 h light/dark cycle and were provided free access to food and tap water. After being housed, the rats were handled daily (60–90 s per rat) for 7 days to acclimate them to the experimenter. All procedures conformed to the USA National Research Council Guide to the Care and Use of Laboratory Animals and were approved by the UCLA Institutional Animal Care and Use Committee. The number of animals used was the minimum required to ensure reliability of the results, and every effort was made to minimize animal discomfort while achieving the goals of the experiment.

# Behavioral Parameters

All behavioral training was performed using two sets of four identical fear conditioning chambers equipped with a Medassociates VideoFreeze near infrared video tracking system. Chambers were enclosed within sound attenuated chambers in a well-lit room separated from the observers.

#### Contexts

Two contexts that differ on spatial location, odor, interior design (opaque or clear), background noise, lighting and transport were used. All groups were fear conditioned in context A. All retrieval/extinction and testing sessions occurred in context B. However, importantly, all statistical comparisons were made between groups that were tested in the same context after equivalent exposure to that context.

# Context A

The context A environment consisted of aluminum (side walls) and Plexiglas (front, back, and top) chambers (30 × 25 × 25 cm, Med-Associates, Inc. St. Albans, VT, USA) and two white plastic side walls (24 cm × 21 cm) placed at 60◦ to the floor, forming a triangular enclosure. The floor of each chamber had 18 stainless steel rods (4 mm diameter, 1.5 cm apart) connected to a shock scrambler and generator (which, along with internal ventilation fans, supplied background noise of 60 dB, A scale). The context A chambers were cleaned with 7% isopropyl alcohol and scented with 10% Simple Green. Animals were transported to the context in squads of four using a square black tub divided into four compartments with a plastic insert and filled with bedding and covered with a wooden lid.

# Context B

The context B environment consisted of aluminum (side walls) and clear Plexiglas (front and top) chambers (30 cm × 25 cm × 25 cm, Med-Associates, Inc. St. Albans, VT, USA). The rear wall was white opaque plastic and the distinct grid flooring pattern consisted of two planes of up/down ''staggered'' stainless steel rods (4.8 mm thick) spaced 1.6 cm apart (center to center; Med-Associates, Inc. St. Albans, VT, USA). The background fan was turned off. The context B chambers were cleaned with 10% ethanol and scented with 10% Windex. Animals were transported to the context in squads of four in their individual home cages, which were slid onto hanging racks mounted to a portable cart and covered with a white cloth sheet.

# Cues, Training and Testing

For auditory cue fear conditioning, rats received delay conditioning using four tone—shock pairing [Baseline (BL) = 2 min, CS = 2800 Hz; Pure tone; 77 dB; 30 s each, US = 0.8 mA; 2 s each; co-terminating with the tone CS, Inter trial interval(ITI) = 2 min, end period = 2 min]. Freezing was scored during the CS presentations on the fear conditioning day. Based on the fear level to the last CS of the fear conditioning day, rats were then rank ordered and assigned to experimental groups in a randomized block order to match the groups for average freezing. Seventy two hours after fear conditioning the retrieval and/or extinction training procedure was conducted. Seventy two hours interval was used because the original study on memory deconsolidation (Myers et al., 2006) found that extinction trained at an interval of 72 h following fear acquisition (long-interval extinction) was sensitive to disruption through reinstatement, renewal, and spontaneous recovery when compared to that of 24 h.

Rats were divided into five groups: (1) Retrieval before extinction (Ret-Ext)- 3 CS-alone massed tones (5 s ITI) were presented for fear retrieval in context B, after which the rats were taken back to the home cage for 10 min and then extinction training session consisting of 50 CS-alone massed presentations (5-s ITI), was performed in the same context. Our procedure was similar to Monfils et al. (2009) procedures except that we used 3 CSs, rather than 1, for retrieval; (2) Extinction before retrieval (Ext-Ret)–Behavioral procedures and retrieval sessions were same as Ret-Ext except that retrieval was given 10 min after the extinction session; (3) Normal Extinction (Normal Ext)–No retrieval was given. The extinction training session consisted of 53 massed CS-alone presentations (5-s ITI) in Context B. Note that the total number of CSs presented was equal to the number in the retrieval groups; (4) Immediate extinction (Immediate Ext) rats were trained in Context A and underwent extinction in Context B 10 min after training using the same parameters of the Normal-Ext group. Our procedure was similar to Maren and Chang (2006) and Myers et al. (2006) in which immediate extinction has been shown to elicit memory attenuation effects under some conditions; and (5) No extinction (No Ext) – Fear conditioned rats were exposed to the B context but no retrieval or extinction session was given. However, during the testing stage, they received tone test like all other groups. This group served as a fear memory retention control. When animals received the retrieval or extinction, the BL period was always 2 min (i.e., 2 min after placing the animals in context B, they received tones).

We used a 10 min interval between retrieval/reminder and extinction sessions based on previous studies (Myers et al., 2006; Monfils et al., 2009). We used three CSs to reactivate the memory instead of one. This is because the first CS typically elicited only about 40–50% freezing behavior in our rats during a typical extinction session. However, subsequent 2nd and 3rd retrieval CSs gave rise to higher freezing behavior (∼70–90%). Based on this observation and in order to fully activate all aspects of a fear memory, we decided to use a total of 3 CSs instead of 1 CS for our reconsolidation or retrieval experiments. In the first experiment, we tested the extinction memory 24 h after the extinction training procedure in Context B. Other than using a different context and omitting the US, all the test sessions were conducted the same as fear conditioning. A second test was given 21 days after the extinction session to measure the spontaneous recovery of fear in context B.

In the second experiment, we used the same training parameters as described for experiment 1, however, for half of the rats, testing was done 3 h and 15–19 min after the various extinction procedures or about 4 h after retrieval, so that they were all tested within the typical 6 h consolidation/reconsolidation window (Nader et al., 2000). For the other half of the rats, testing was done 24 h after the various extinction procedures.

#### Dependent Measure

For all experiments, freezing was the index of fear memory. We used a commercially available near-infra

data acquisition system and software (Med Associates Video Freeze) that had been calibrated to very experienced human observers. Freezing is defined as the absence of all visible movement except that required for respiration.

# Statistical Analyses

Data were statistically analyzed using between-subjects analysis of variance (ANOVAs) and repeated measures (RM) ANOVAs where appropriate. Fear acquisition and extinction data were analyzed using RM (trial, bin) ANOVAs. BL freezing and average freezing during the tone test were analyzed separately by one-way ANOVAs. Post hoc comparisons were performed following significant findings using a Tukey's multiple comparisons for two-way RM ANOVA and for one-way ANOVA. The level of significance used for all analyses was P < 0.05.

## RESULTS

# Long-Term Extinction Memory After Ret-Ext, Ext-Ret and Immediate Ext Procedures

In experiment 1, fear conditioned rats received various extinction procedures and were tested at 1 and 21 days after for spontaneous recovery of fear (schema of experiment in **Figure 1**).

**Figure 2A** shows fear acquisition data from different groups of rats. All rats developed significant tone fear during acquisition (F(3,192) = 267.79, P < 0.0001 n = 69) in context A. The main effect of group (F(464) = 0.39, P = 0.8115) or interaction (F(12,192) = 0.44, P = 0.9451) was not significant. Animals were then divided into four groups: (Ret-Ext (n = 13); Ext-Ret (n = 15); Normal Ext (n = 12); and No Ext (n = 15)) on the basis of their freezing levels to last CS during fear conditioning to ensure that groups were balanced (see the ''Materials and Methods'' Section for more details). Since rats in Immediate Ext (n = 14) group underwent fear extinction 10 min after fear conditioning, we were unable to balance in advance. However, Immediate Ext group acquired fear that was similar to all other groups (**Figure 2A**). Freezing during retrieval (3 CSs) and extinction sessions (50 or 53 CSs) are shown in **Figure 2B** as one graph but the sessions were conducted 10 min apart for the groups Ret-Ext and Ext-Ret. Extinction and subsequent tests were done in context B. Each data point for extinction trials in **Figure 2B** represent average of 3 CSs except the 17th data point that represents average of 2 CSs totaling 53 CSs. BL fear did not differ among the groups (one way-ANOVA F(4,64) = 2.212, P = 0.0775). All rats acquired significant fear extinction reduction across extinction/retrieval sessions (F(17,1088) = 64.61, P < 0.0001). The main effect of group (F(4,64) = 5.07, P = 0.0013) and interaction (F(68,1088) = 3.84, P < 0.0001) were significant. Note that rats belonging to No Ext group were simply exposed to context B but were not presented with any tone (**Figure 2A**) on the extinction day. Initial freezing in No Ext group was fear that generalized from the conditioning context and extinguished over time in context B. All extinction groups showed significant fear during initial stages of extinction session when compared to No Ext group. There were no significant differences between extinction groups and No Ext group during final stages of extinction session (for details, see **Table 1**).

On days-5 and 25 (1 and 21 day(s) after the various extinction procedures), extinction memory tests were done in context B using the same protocol used for fear conditioning (minus the shocks; **Figures 2C,D**). Freezing for 4 CSs was averaged. On Day 5 test, BL fear did not differ (one way-ANOVA F(4,64) = 1.296, P = 0.2813) among the groups (average percent Freezing—Ret-Ext = 15.51 ± 6.246, Ext-Ret = 2.740 ± 1.308, Normal Ext = 12.42 ± 5.1, Immediate Ext = 8.778 ± 4.0 and No Ext = −10.21 ± 3.639). On day 5 test, average percent freezing of No Ext group was larger than 80, which was similar to initial freezing level of all other groups during the extinction session on day 4 (**Figure 2C**). These data indicate that there is no fear expression impairments on Day-5. Day-5 test revealed a significant extinction memory (one way-ANOVA F(4,64) = 9.241, P < 0.0001) in Ret-Ext (P < 0.01), Ext-Ret (P < 0.001), Normal Ext (P < 0.01) and Immediate Ext groups (P < 0.001) when compared to No Ext group (**Figure 2C**).

On Day 25 test, BL fear did not differ (one way-ANOVA F(4,64) = 2.323, P = 0.0663) among the groups (Ret-Ext = 25.11 ± 3.681, Ext-Ret = 11.71 ± 2.182, Normal Ext = 15.60 ± 5.304, Immediate Ext = 15.47 ± 4.093 and No Ext = −23.50 ± 3.968). On day-25 test, average percent freezing

FIGURE 2 | Attenuation of fear memory after various extinction procedures. Graphs show fear and extinction learning curves (A,B) and extinction memory during a memory test carried out 1 and 21 day(s) after the end of each extinction procedure in context B (C,D). Mean ± SEM freezing during Baseline (BL) and tone for all rats were measured. (A) Fear acquisition. Rats were fear conditioned with 4 tone-shock pairings on Day-1 in context A. All animals acquired cue fear. (B) Fear retrieval and extinction. Rats received retrieval and extinction procedures in context B on Day-4. Immediate extinction group was fear conditioned in Context A with four tone-shock pairings and 10 min later they received normal fear extinction session in context B. Each data point for extinction trials represent the average of 3 CSs except the 17th data point that represents average of 2 CSs totaling 53 CSs. Data point after the line break represent the average of 3 retrieval CSs for Ret-Ext and Ext-Ret groups. All rats showed significant within session extinction learning on Day-4. (C) Fear extinction memory test. As a measure of extinction memory, rats received 4 tone presentations in context B on Day-5. (D) Spontaneous recovery test. Rats received four tone presentations in context B on Day-26. Normal Ext group showed freezing similar to No Ext group suggesting significant spontaneous recovery of fear. Ret-Ext, Ext-Ret and Immediate Ext groups showed no spontaneous recovery of fear. One-Way ANOVA with Tukey's multiple comparison test was used for Days-5 and 26. (ns, not significant; ∗∗P < 0.01; ∗∗∗P < 0.001 vs. No Extinction (C,D). <sup>+</sup>P < 0.05 vs. Normal Extinction (D). Groups: No Ext, no extinction; Normal Ext, normal extinction; Ret Ext, retrieval first + extinction; Ext—Ret, extinction + retrieval later; Immediate Ext, Fear conditioning first + immediate extinction. Rats were tested repeatedly on Day-5 and Day-26.

of No Ext group was similar to freezing on day-4 suggesting no fear memory retrieval/expression issues on Day-25 (**Figure 2D**) from our No Ext group. As expected, the Day-25 test revealed a significant spontaneous recovery of fear (one way-ANOVA F(4,64) = 9.099, P < 0.0001), in the Normal Ext group and percent freezing was not different from the No Ext group (P > 0.05, **Figure 2D**). However, Ret-Ext, Ext-Ret and Immediate Ext groups showed very little freezing (all three groups P < 0.001) when compared to No Ext group (**Figure 2D**) Interestingly, Ext-Ret (P < 0.05) and Immediate Ext (P < 0.05) groups showed lower fear than the Normal Ext group on Day-25 test. The Ret-Ext group also showed low fear compared to the Normal Ext group, however the difference fell short of statistical reliability (P = 0.0733). In general, these results are consistent with a retrieval model, in that extinction retention was facilitated by each of the procedures that differed from the standard extinction method.

# Short-Term Fear Memory After Successful Within Session Extinction in Ret-Ext Group

Both Myers et al. (2006) and Monfils et al. (2009) timed their extinction sessions to coincide with a period where cellular memory consolidation processes are assumed to be ongoing. This is based on classic consolidation studies suggesting that there is a period that starts after encoding and persisting for up to about 6 h during which memory is vulnerable to manipulations such as electroconvulsive shock or protein synthesis inhibitors (e.g., Agranoff et al., 1965; McGaugh, 1966). One characteristic of studies that disrupt consolidation and reconsolidation is that the loss of memory does not happen immediately but rather appears

#### TABLE 1 | Fear extinction data analysis for experiment 1.


Extinction data for experiment 1 were analyzed using RM (trial, bin) ANOVAs (Figure 2B). All rats showed significant within session extinction learning (Day-4). Further analysis of fear extinction data was done using corrected Tukey's multiple comparison test. Percentage freezing for CS1, CS9 and CS18 were presented in this table instead of presenting all the CSs. Each data point for extinction trials represent average of 3 CSs except the 17th data point that represents average of 2 CSs totaling 53 CSs.

during a long-term memory test 24 h or more hours later (Nader et al., 2000; Schafe and LeDoux, 2000). When memory was tested shortly after anisomycin delivery, auditory fear conditioning was intact (Nader et al., 2000; Schafe and LeDoux, 2000). Therefore, if immediate extinction and reminder-extinction treatments affect consolidation they too should leave short-term fear performance intact and deficits should emerge only at long-term test points. Therefore to diagnose this pattern we conducted a short-term test of extinction memory 3 h and 25 min after extinction (**Figure 3**). This also ensured that the interval between the retrieval treatment in the Ret-Ext group also fell within the 4 h window used by Schafe and LeDoux (2000) and Nader et al. (2000).

As shown in **Figures 4A**, **5A**, all rats acquired significant tone fear across acquisition trials (**Figure 4A**, F(3,105) = 106.96, P < 0.0001, n = 40; **Figure 5A**, F(3,102) = 110.67, P < 0.0001, n = 39) in context A. In **Figure 4A**, the main effect of group (F(4,35) = 0.62, P = 0.6501) or interaction (F(12,105) = 0.32, P = 0.9839) was not significant. In **Figure 5A**, the main effect of group (F(4,34) = 0.34, P = 0.8496) or interaction (F(12,102) = 0.40, P = 0.9608) was also not significant. Animals were equally split into two sets of groups (Ret-Ext, Ext-Ret, Normal Ext and No Ext) on the basis of their final levels of fear, ensuring that groups were balanced before extinction. Since, rats in Immediate Ext group underwent fear extinction 10 min after fear conditioning, we were unable to balance fear levels in advance. However, as shown in **Figures 4A**, **5A**, Immediate Ext group acquired fear that was similar to all other groups. Freezing during retrieval and extinction sessions is shown in **Figures 4B**, **5B**. BL fear did not differ among the groups (**Figure 4B** one way-ANOVA F(4,36) = 1.142, P = 0.3523; **Figure 5B** one way-ANOVA F(4,35) = 1.470, P = 0.2323). All rats acquired significant fear extinction across extinction trials as shown in **Figure 4B** (F(17,595) = 50.01, P < 0.0001) and **Figure 5B** (F(17,578) = 29.50, P < 0.0001) in context B. In **Figure 4B**, the main effect of group (F(4,35) = 6.13, P = 0.0008) and interaction (F(68,595) = 5.38, P < 0.0001) were significant. In **Figure 5B**, the main effect of group (F(4,34) = 5.47, P = 0.0016) and interaction (F(68,578) = 4.80, P < 0.0001) were significant. All extinction groups showed significant fear during initial stages of extinction session when compared to No Ext group. There were no differences between extinction groups and No Ext group during final stages of extinction session (for details, see **Tables 2**, **3**). Note that rats in the No Ext group were simply exposed to context B but were not presented with any tone while, the other groups received repeated tone presentations. Initial freezing in the No Ext group was fear that generalized from the conditioning context and extinguished over time in context B.

FIGURE 4 | Rats failed to show extinction memory 3.25 h after successful retrieval—extinction session. (A) Fear acquisition. Rats were fear conditioned with four tone-shock pairings on Day-1 in context A. All animals acquired cue fear. (B) Fear retrieval and extinction. Rats received retrieval and extinction procedures in context B on Day-4. Immediate extinction group was fear conditioned in Context A with four tone-shock pairing and 10 min later they received normal fear extinction session in context B. All groups showed significant within session extinction learning on Day-4. Each data point for extinction trials represent average of 3 CSs except the 17th data point that represents average of 2 CSs totaling 53 CSs. Data point after the line break represent the average of 3 retrieval CSs for Ret-Ext and Ext-Ret groups. (C) Short-term fear extinction memory test. Rats received four tone only presentations in context B 3.25 h after the extinction procedures. Normal Ext, Ext-Ret, Immediate Ext groups showed significant low fear memory where as Ret-Ext group showed no traces of extinction memory in this test. Extinction during reconsolidation did not cause immediate memory erasure. One-Way ANOVA with Tukey's multiple comparison test was used for Day-4. (ns, not significant; ∗∗P < 0.01; ∗∗∗P < 0.001 vs. No Extinction (C). <sup>+</sup>P < 0.05; ++P < 0.01 vs. Ret-Ext. Groups: No Ext, no extinction; Normal Ext, normal extinction; Ret-Ext, retrieval first + extinction; Ext-Ret, extinction + retrieval later; Immediate Ext, Fear conditioning first + immediate extinction. To avoid any potential confounding effect of the test by itself, rats tested in short-term memory after extinction were not used on 24 h memory test.

At 3.25 h test, freezing was significantly different between the groups (one way-ANOVA F(4,39) = 9.270, P < 0.0001). Despite successful within session extinction the Ret-Ext group showed

FIGURE 5 | Good extinction memory 24 h after successful retrieval extinction session. (A) Fear acquisition. Rats were fear conditioned with four tone-shock pairings on Day-1 in context A. All animals acquired cue fear. (B) Fear retrieval and extinction. Rats received retrieval and extinction procedures in context B on Day-4. Immediate extinction group was fear conditioned in Context A with four tone-shock pairings and 10 min later they received normal fear extinction session in context B. All groups showed significant within session extinction learning on Day-4. Each data point for extinction trials represent average of 3 CSs except the 17th data point that represents average of 2 CSs totaling 53 CSs. Data point after the line break represent the average of 3 retrieval CSs for Ret-Ext and Ext-Ret groups. (C) Long-term fear extinction memory test. Rats received four tone only presentations in context B 24 h after the extinction procedures. Normal Ext, Ret-Ext, Ext-Ret, Immediate Ext groups showed significant low fear memory. One-Way ANOVA with Tukey's multiple comparison test was used for Day-4.) ( ∗∗P < 0.01, ∗∗∗P < 0.001 vs. No Ext). Groups: No Ext, no extinction; Normal Ext, normal extinction; Ret-Ext, retrieval first + extinction; Ext-Ret, extinction + retrieval later; Immediate Ext, Fear conditioning first + immediate extinction. To avoid any potential confounding effect of the test by itself, different groups of rats were tested in the short-term memory and 24 h memory tests.

a robust short-term fear memory (3.25 h) in the extinction context that was not statistically different from No Ext group (P > 0.05, **Figure 4C**). Note that freezing was almost identical in No Ext (∼78%) and Ret-Ext (∼75%) groups. At 3.25 h test, this high fear in Ret-Ext was significantly different from Ext-Ret (P < 0.01), Normal Ext (P < 0.05) and Immediate Ext (P < 0.05). However, the Normal Ext (P < 0.01), Ext-Ret (P < 0.001) and Immediate Ext (P < 0.01) groups showed a significant short-term extinction memory at 3.25 h when compared to No Ext group. Replicating Experiment 1, in Experiment 2 at 24 h test, Ret-Ext, Ext-Ret, Immediate Ext and Normal Ext groups showed significant extinction memory (one way-ANOVA F(4,39) = 6.910, P < 0.0003) when compared to No Ext group (**Figure 5C**–Normal Ext P < 0.001 and rest of the groups P < 0.01 when compared to No Ext group).

# DISCUSSION

Following auditory fear conditioning we evaluated the efficacy of several extinction protocols relative to a standard massed training extinction protocol consisting of 53 presentations of the CS spaced 5 s apart. The standard extinction protocol caused significant loss of fear when the rats were tested 1 day after extinction, confirming earlier studies using similar protocols (Cain et al., 2003). As expected there was a significant return of fear when the test occurred 21 days after extinction. However, this spontaneous recovery was not observed in our three modified extinction protocols. Similar to Myers et al. (2006), we found that extinguishing fear shortly after fear conditioning defeated spontaneous recovery (for opposite results in both rodents and humans, see Chan et al., 2010; Costanzi et al., 2011; Kindt and Soeter, 2013; Stafford et al., 2013). Similar to Monfils et al. (2009) presenting a few CSs prior to the start of the regular extinction session also prevented spontaneous recovery (for similar results in both rodents and humans, see Schiller et al., 2010; Clem and Huganir, 2010; Rao-Ruiz et al., 2011; Agren et al., 2012). According to Monfils et al.'s (2009), reconsolidation update hypothesis which was confirmed at the molecular to systems level, placement of brief CS presentations prior to extinction is critical in that they are hypothesized to open a window of memory vulnerability that allows the subsequent extinction session to erase the original fear memory. However, we found that the order of a retrieval session and an extinction session made little difference as spontaneous recovery was also reduced when the ordering of the short and longer sessions was reversed. Our results are consistent with findings of previous articles that used similar Ext- Ret approach in adolescent rats (Baker et al., 2013) and alcoholic beer memory retrieval in adult rats (Millan et al., 2013). The reconsolidation update hypothesis does not anticipate such a result. As a potential alternative that is consistent with retrieval views of memory (Bouton, 1993; Bjork, 1994), we suggest that having two different types of extinction sessions close in time makes the extinction memory more retrievable and thereby reduces spontaneous recovery of fear by making the extinction better able to interfere with retrieval of the original fear memory. However, it is possible that the Retrieval-Extinction and Extinction-retrieval procedures produce their effects via different mechanisms. The Extinction-Retrieval effect could be caused because two different extinction sessions lead to better retrieval of the extinction memory. The Retrieval-Extinction effect may be caused by a true deconsolidation of the original fear memory. This is supported by the finding that the fear memory was intact at the short-term test for the Retrieval-Extinction procedure but not the Extinction-Retrieval procedure.

# Deconsolidation

Both the Myers et al. (2006) immediate extinction and the Monfils et al. (2009) reminder-extinction accounts suggest that the original fear memory is erased when extinction occurs during a period when memory has been destabilized. For immediate extinction this vulnerability is because the fear memory has not yet consolidated. For retrieval-extinction, reminding the animal of the CS opens a period of vulnerability during which a memory must be reconsolidated where the original fear memory is replaced by a new extinction memory and thus the original memory no longer exists. These ideas are based on the finding that memory is lost when a protein synthesis inhibitor is administered during these windows. Interestingly, with the protein synthesis inhibitor fear memory is intact when the CS is tested shortly (e.g., 6 h or less) after the amnestic manipulation but memory degrades after that with amnesia being observed 24 h later (Nader et al., 2000; Schafe and LeDoux, 2000). Such a finding is a fundamental aspect of consolidation and reconsolidation theory as it provides evidence that memory stabilization rather than memory expression is affected by the amnestic treatment. Therefore, we also tested memory retrieval shortly after our extinction manipulations. If the manipulation affected memory stabilization then extinction memory should be intact and fear levels high during this test. The only procedure that showed this pattern was the retrieval-extinction protocol as short-term fear memory was abated with the three other procedures. The data suggest that the retrieval-extinction order works via a mechanism that is distinct from the other procedures and the pattern observed is quite consistent with reconsolidation theory. Based on our short-term memory (STM) results, we conclude that Ret-Ext group might update extinction memory with safety information and this mechanism could explain our results. Interestingly using contextual fear conditioning procedure, Rao-Ruiz et al. (2011) reported that brief un-reinforced recall of contextual fear memory lead to initial synaptic depression and endocytosis of GluA1, A2 and A3 containing AMPAR expression within 1–4 h. In the same experiment, they found a subsequent increase in synaptic strength and increase in GluA2 containing AMPARs in the synapse at 7 h. However, high fear memory in our STM test 3.15 h after retrieval-extinction procedure, was not in parallel to the biochemical findings of Rao-Ruiz et al. (2011) that showed the hippocampal synaptic changes immediately (1–4 h) after retrieval. Using normal extinction, extinction- retrieval and immediate extinction procedures, we found results similar to the studies using normal extinction (e.g., studies that tested the memory at short-term interval <6 h; Quirk, 2002; Berman et al., 2003; but see Archbold et al., 2013 for opposite results) suggesting normal extinction learning dependent inhibition results in expression of extinction memory.

#### TABLE 2 | Fear extinction data analysis of short-term memory (STM) groups for experiment 2.


Extinction data for experiment 2 were analyzed using RM (trial, bin) ANOVAs (Figure 4B). All rats showed significant within session extinction learning (Day-4). Further analysis of fear extinction data was done using corrected Tukey's multiple comparison test. Percentage freezing for CS1, CS9 and CS18 were presented in this table instead of presenting all the CSs. Each data point for extinction trials represent average of 3 CSs except the 17th data point that represents average of 2 CSs totaling 53 CSs.

TABLE 3 | Fear extinction data analysis of long-term memory (LTM) groups for experiment 2.


Extinction data for experiment 2 were analyzed using RM (trial, bin) ANOVAs (Figure 5B). All rats showed significant within session extinction learning (Day-4). Further analysis of fear extinction data was done using corrected Tukey's multiple comparison test. Percentage freezing for CS1, CS9 and CS18 were presented in this table instead of presenting all the CSs. Each data point for extinction trials represent average of 3 CSs except the 17th data point that represents average of 2 CSs totaling 53 CSs.

## Immediate Extinction

Like Myers et al. (2006), we found that starting extinction shortly after training produced an effective loss of fear in that there was little spontaneous recovery of fear 3 weeks after extinction. While Myers et al. (2006) suggested this was caused by a disruption of memory consolidation the fact that fear was absent at the short-term test raises the possibility that fear expression rather than consolidation was affected. Such a pattern is readily explained by the ambiguity theory of Bouton (1993). Bouton (1993) suggests that a memory for both an acquisition episode and an extinction episode are formed and fear expression is determined by which episode is recalled at the time of test. When acquisition and extinction occur at the same time the subject may concatenate the two treatments into a single episode that is dominated by the extinction memory. If this happens extinction recall should be robust regardless of when memory is tested.

In contrast to Myers et al. (2006) and the results reported here, Maren and Chang (2006) found that giving extinction immediately after fear conditioning results in very poor loss of fear (see also studies in rats (Morris et al., 2005; Archbold et al., 2010), mice (Stafford et al., 2013) and human (Alvarez et al., 2007; Norrholm et al., 2008; Schiller et al., 2008; Huff et al., 2009) for failure of immediate extinction on fear memory). This failure of an immediate extinction procedure appears to be caused by strong and persistent stress or fear that continues after the fear acquisition session (Maren, 2014). The levels of BL fear prior to immediate extinction appear to be considerably lower in our study than in Maren and Chang (2006) and Chang and Maren (2009). In those studies the rats froze about 80% prior to CS presentation, while BL freezing in ours was 40% or less (**Figures 2B**, **4B**, **5B**). This BL difference occurred despite similar levels of CS elicited freezing in both labs. We used very different contexts and having distinct acquisition and extinction contexts likely caused an overall reduction in fear and anxiety during extinction with our procedures.

The diversity of findings found with parametric manipulations of extinction such as those reported here open up more questions on long and short-term dynamic aspect of fear memory reconsolidation and retrieval. Based on our results we conclude that both retrieval and reconsolidation processes contribute to long-term extinction memories. The degree to which they contribute depends on experimental procedures. All designs except the Ret-Ext appear to primarily reflect retrieval processes. Ret-Ext seems unique in that the short-term fear memory remains intact after extinction; much as it does in classic consolidation and reconsolidation studies using amnestic agents. Since there is need to develop more effective interventions, studies exploring both short-term and long-term performance may benefit translation of pre-clinical results to clinical settings.

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

AMP and JS designed the research, analyzed the data. MM, JH, P-KO'N and DW performed the research. RP designed the research, performed the research, analyzed the data and wrote the article. MSF designed the research, analyzed the data and wrote the article. IZ Performed the research and analyzed the data.

#### ACKNOWLEDGMENTS

Supported by NIMH Grant RO1-MH62122 and the Staglin Center for Brain and Behavioral Health to MSF.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Ponnusamy, Zhuravka, Poulos, Shobe, Merjanian, Huang, Wolvek, O'Neill and Fanselow. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Increased perceived self-efficacy facilitates the extinction of fear in healthy participants

#### Armin Zlomuzica\*, Friederike Preusser, Silvia Schneider and Jürgen Margraf

*Mental Health Research and Treatment Center, Ruhr-University Bochum, Bochum, Germany*

Self-efficacy has been proposed as an important element of a successful cognitive behavioral treatment (CBT). Positive changes in perceived self-efficacy have been linked to an improved adaptive emotional and behavioral responding in the context of anxiety-provoking situations. Furthermore, a positive influence of increased self-efficacy on cognitive functions has been confirmed. The present study examined the effect of verbal persuasion on perceived self-efficacy and fear extinction. Healthy participants were subjected to a standardized differential fear conditioning paradigm. After fear acquisition, half of the participants received a verbal persuasion aimed at increasing perceived self-efficacy. The extinction of fear was assessed immediately thereafter on both the implicit and explicit level. Our results suggest that an increased perceived self-efficacy was associated with enhanced extinction, evidenced on the psychophysiological level and accompanied by more pronounced decrements in conditioned negative valence. Changes in extinction were not due to a decrease in overall emotional reactivity to conditioned stimuli (CS). In addition, debriefing participants about the false positive feedback did not affect the processing of already extinguished conditioned responses during a subsequent continued extinction phase. Our results suggest that positive changes in perceived self-efficacy can be beneficial for emotional learning. Findings are discussed with respect to strategies aimed at increasing extinction learning in the course of exposure-based treatments.

Keywords: self-efficacy, extinction, fear conditioning, exposure therapy, anxiety disorders, self-regulation, top-down control

## Introduction

The concept of self-efficacy refers to the individual's perceived belief to cope effectively with upcoming situations and problems (Bandura, 1997). A higher level of self-efficacy can increase the individual's belief that his/her behavior will more likely produce a positive outcome within a given situation (Bandura, 1997; Maddux, 1999). According to Bandura (1997) four different sources of self-efficacy information can be differentiated (i.e., as a result of mastery experience, vicarious experience, persuasion, and physiological and affective states).

The beneficial impact of increased perceived self-efficacy on behavior has been confirmed across different domains of research. For instance, a positive relationship between the level of self-efficacy and sports performance (Moritz et al., 2000), the likelihood to engage in healthy behavior (Schwarzer and Fuchs, 1995), but also the ability to cope adaptively with stressful

#### Edited by:

*Manfred Schedlowski, University of Duisburg-Essen, Germany*

#### Reviewed by:

*Seth Davin Norrholm, Emory University School of Medicine, USA Christopher Cain, Nathan S. Kline Institute for Pscyhiatric Research and NYU School of Medicine, USA*

#### \*Correspondence:

*Armin Zlomuzica, Mental Health Research and Treatment Center, Ruhr-University Bochum, Massenbergstr. 9-13, 44787 Bochum, Germany armin.zlomuzica@rub.de*

Received: *28 June 2015* Accepted: *22 September 2015* Published: *16 October 2015*

#### Citation:

*Zlomuzica A, Preusser F, Schneider S and Margraf J (2015) Increased perceived self-efficacy facilitates the extinction of fear in healthy participants. Front. Behav. Neurosci. 9:270. doi: 10.3389/fnbeh.2015.00270* experiences (McFarlane et al., 1995) has been demonstrated. Self-efficacy has also been linked to cognitive performance as demonstrated in different verbal, mathematical, and spatial tasks (Lent et al., 1997; Paunonen and Hong, 2010).

The concept of self-efficacy has received a great deal of attention in clinical research. Perceived self-efficacy or the confidence in being able to refrain from smoking predicts smoking cessation outcome and has been considered a potential mechanism underlying effective smoking abstinence (Gwaltney et al., 2005; Marlatt and Donovan, 2005; Schnoll et al., 2011). A decreased self-efficacy has been discussed as a cognitive precursor or a component of anxiety, phobia, and depression (Comunian, 1989; Williams, 1995) and is associated with a greater severity of anxiety (Richards et al., 2002; Thomasson and Psouni, 2010) and an increased tendency to use dysfunctional coping strategies when confronted with anxiety-provoking situations (Thomasson and Psouni, 2010).

A positive change in perceived self-efficacy in the course of a cognitive behavioral treatment (CBT) might constitute a critical component of a successful therapeutic outcome. Increases in self-efficacy go along with reductions in anxiety symptoms following treatment (Bouchard et al., 2007; Gaudiano and Herbert, 2007; Delsignore et al., 2008). Likewise, the amount of cognitive reappraisal self-efficacy, defined as the belief that one can effectively apply emotion regulation strategies during exposure to anxiety-provoking stimuli and situations, significantly determines anxiety symptoms reductions (both immediate and long-term) following CBT (Goldin et al., 2012). Findings from these and other studies implicate that increments in self-efficacy beliefs may constitute an important mechanism through which CBT exerts its beneficial effects on fear and avoidance (Bouchard et al., 2007; Goldin et al., 2012; Gallagher et al., 2013).

The exact role of self-efficacy in CBT remains elusive. CBT for anxiety disorders usually involves a combination of exposure and a set of cognitive strategies in order to modify the patient's negative expectations and interpretations in the context of anxiety-provoking situations. Based on the research so far, it is difficult to ascertain whether increases in self-efficacy arise from symptom relief experienced by patients during CBT or whether different levels of self-efficacy in patients determine their range of emotional and/or behavioral responding in the context of anxiety-provoking situations. Most studies so far have utilized correlational designs to examine the relationship between selfefficacy and CBT outcome (Bouchard et al., 2007; Gallagher et al., 2013). Clearly, more research with experimental designs is needed to describe the exact link between increases in self-efficacy and CBT outcome.

Exposure, a core component of CBT, can lead to an enduring symptom relief in anxiety disorders (Ruhmland and Margraf, 2001a,b,c; Vögele et al., 2010). During exposure, patients are given the opportunity to reevaluate the significance of a stimulus while extinction learning has been proposed to mediate this form of so called "corrective learning" (Craske et al., 2008, 2014; Vervliet et al., 2013). Given that low self-efficacy has been linked to an increased tendency of anxious individuals to use dysfunctional coping strategies in anxiety-related situations (Thomasson and Psouni, 2010) it is reasonable to assume that changes in self-efficacy might affect extinction learning as shown previously for other forms of learning (McDougall and Kang, 2003). This might be of special importance for understanding how self-efficacy beliefs in its interaction with deficient fear extinction and elevated fear acquisition (Briscione et al., 2014; Mosig et al., 2014) contribute to the development of anxiety and stressor-related disorders. Likewise, it offers the possibility to examine how specific interventions aimed at enhancing selfefficacy can be applied to exposure to yield more enduring and stable therapy benefits (Rothbaum and Davis, 2003; Craske et al., 2008, 2014; Norrholm and Jovanovic, 2010; Vervliet et al., 2013).

In this instance, it was recently shown that self-efficacy beliefs can indeed be systematically manipulated via persuasive verbal feedback and that this manipulation affects the memory for both aversive (Brown et al., 2012b) as well as personally relevant (Brown et al., 2012a) events. In particular, following a high or low self-efficacy induction, participants with a high-self efficacy belief recalled fewer negative intrusions and showed a reduction in attentional bias associated with remembering aversive stimuli (Brown et al., 2012b). In a similar vein, it was demonstrated that students who were led to believe they possessed high selfefficacy showed an increase in episodic memory performance and problem solving capacity (Brown et al., 2012a). These findings indicate that self-efficacy beliefs can be mediated via positive and/or negative persuasive feedback beliefs (Bandura, 1997), which might (in)directly influence learning and/or retrieval of emotionally relevant information. On the neurobiological level, such effects might be comparable to those elicited via topdown modulation of fear learning and extinction by means of intentional cognitive regulation strategies. Reappraisal, for instance, has been shown to alter fear extinction via activation of downstream pathways, i.e., frontal regions including the ventromedial prefrontal cortex, which are implicated in the inhibition of fear responses (Buhle et al., 2014; Schiller and Delgado, 2010).

The present study sought to determine whether (similar to other top-down regulation strategies) the manipulation of selfefficacy beliefs via positive verbal persuasion might also affect fear extinction learning. Hence, the major aim of this study was to extend previous findings and examine the effect of an experimentally-induced increased self-efficacy on the extinction of learned fear. Since fear extinction is becoming widely accepted as a translational tool for exposure-based treatments, our findings might provide more insights into the relation between selfefficacy levels and CBT outcome. Moreover, by investigating the effect of increased self-efficacy on subsequent fear extinction and the neuronal circuitry involved we might determine whether an increase in self-efficacy can be used to enhance exposure-based treatments (Craske et al., 2008, 2014; Vervliet et al., 2013).

## Materials and Methods

#### Participants

The sample was recruited via postings in social media networks or announcements on bulletin boards at the campus of the Ruhr-University Bochum. Participants reporting current or previous mental diseases, psychological or pharmacological treatment for mental diseases, as well as severe acute or chronic somatic diseases were not eligible for participation. A total of 57 subjects (28 males, 29 females) participated in this study. Data from nine participants were excluded because they failed to acquire conditioning (e.g., higher CS-UCS contingency and CS valence ratings for the unreinforced as compared the reinforced conditioned stimuli (CS) after fear acquisition) or reported that they did not believe the manipulated self-efficacy feedback. Hence, our final analysis comprised data from 48 participants, who were randomly assigned to either the experimental group (n = 24, 50% females) or the control group (n = 24, 62.5% females). Demographic characteristics of the groups are displayed in **Table 1**. Subjects received either 15e or 1.5 course credits as compensation for time and travel. All experimental procedures were approved by the local ethics committee (Ethical committee of the Ruhr-University of Bochum, Germany) and carried out in accordance with the Declaration of Helsinki. All participants provided written informed consent.

#### Experimental Design Fear Conditioning

Each participant underwent a differential fear conditioning procedure according to a modified procedure previously described by Blechert et al. (2007) and Michael et al. (2007). Two inkblot pictures, either black and white or yellow and red in color, served as the reinforced CS+ and unreinforced CS− in a counterbalanced manner. A 500 ms mild electrical stimulation delivered to the skin of the lower arm constituted the UCS. The entire fear conditioning procedure consisted of a habituation, acquisition, extinction, and a continued extinction phase. A break of 15 min was imposed after acquisition as well as after extinction to administer the first and second experimental manipulation (see Section Experimental Manipulations). Three trials of CS+ and CS− were presented during habituation. In the acquisition phase, the CS+ and CS− were again presented 10 times each while the CS+ co-terminated with the UCS

TABLE 1 | Demographic characteristics of the experimental and control group.


*EG: positive feedback (self-efficacy induction); CG: no feedback (no self-efficacy induction). Based on N = 48 subjects (nEG = 24, nCG = 24).*

on a 60% reinforcement schedule (to extend the time course of fear responses during the subsequent extinction phase, see Haselgrove et al., 2004). The extinction phase consisted of 20 trials (10 CS+ and 10 CS−) without any UCS. After the second experimental manipulation, extinction was continued and another set of 6 (3 CS+ and 3 CS−) trials, respectively, was presented. During all phases, the CS+ and CS− were displayed for 8 s each and presented in pseudorandom order. The duration of the randomly generated inter-trial interval was between 16 and 20 s. Performance measures included skin conductance responses (SCRs) as well as CS valence and CS−UCS contingency ratings.

#### Experimental Manipulations

#### **Experimental manipulation 1: the effect of verbal persuasion on fear extinction**

The present study employed two experimental manipulations. The experimental design is illustrated in **Figure 1**. The first experimental manipulation was used to examine whether it is possible to alter fear extinction by adding verbal persuasion (vs. no verbal persuasion) aimed at increasing self-efficacy beliefs. For this purpose, all participants first underwent a fear acquisition phase. Subsequently, half of the participants (i.e., the experimental group, EG) received a slightly modified version of the positive verbal feedback used by Brown et al. (2012b). Specifically, they were told that, based on the way in which they responded to the questionnaires and their physiological responses during the task, they had been identified as being in the top 1% of "copers" and to possess excellent abilities when dealing with stressful situations (for more details, see Brown et al., 2012b). By contrast, a verbal feedback was not administered to the control group (CG).

#### **Experimental manipulation 2: the effect of debriefing and verbal persuasion on continued extinction**

The purpose of the second experimental manipulation was to investigate whether a change in continued extinction (or a return in conditioned responses) would occur in the EG after debriefing participants about the false positive verbal feedback. Thus, after the extinction phase, half of the experimental group (i.e., EG2) were debriefed about the nature of the false feedback they had received earlier, whereas the remaining subjects (i.e., EG1) did not receive any feedback.

Finally, we investigated whether the application of the verbal persuasion (i.e., the identical false positive verbal feedback as described above) is effective in promoting extinction when applied after the extinction had already been initiated. Here, participants from CG were randomly assigned either to the "false feedback" condition (i.e., CG1, receiving the identical instruction as EG, see descriptions above) or "no feedback" condition (CG2). Hence, the CG2 subgroup received no feedback at any experimental stage (but see Experimental Design).

Importantly, the delays imposed between the respective conditioning phases (i.e., acquisition and extinction; extinction and continued extinction) were identical among groups regardless of whether a verbal feedback was delivered.

revised for half of the experimental group (EG2), whereas it was administered to half of the control group (CG1). Dependent measures included valence and contingency ratings as well as skin conductance responses (SCRs).

#### Apparatus

The experiment took place in a sound-attenuated room adjacent to a control room where the experimental apparatus was stationed. Stimuli as well as rating scales of the conditioning procedure were presented on a 19-inch computer screen (Computer GmbH&Co KG, Marl, Germany) using Presentation software, version 16.1 (Neurobehavioral Systems, Inc., USA). Ag/AgCl electrodes placed on the lower left arm delivered the electrical stimulation generated by a Constant Current Isolated Stimulator PS3 (Digitimer Ltd., Welwyn Garden City, England). SCRs were measured with 5 mm inner-diameter Ag/AgCl electrodes filled with non-hydrating electrodermal response paste and positioned at the index and middle finger of the nondominant hand. Signals were recorded and digitzed at a sampling rate of 1000 Hz in a continuous mode utilizing a 16-Bit BrainAmp ExG Amplifier and Brain Vision Recorder software, version 1.2 (Brain Products GmBH, Gilching, Germany).

#### Assessments

#### Questionnaires

Prior to the conditioning procedure, selected items from the self-report Depression Anxiety Stress Scales (DASS; Lovibond and Lovibond, 1995) were applied to measure acute symptoms of anxiety, depression and stress on a 4-point Likert scale (0 = did not apply to me at all, 3 = applied to me very much, or most of the time). Furthermore, participants' perceived ability to cope with emotions, solve problems and gain social support was assessed using the Resilience Appraisal Scale (RAS; Johnson et al., 2010), which was filled in after the first false feedback. The RAS consists of 12 items scored on a five-point Likert scale (1 = strongly disagree, 5 = strongly agree). Before as well as after each false feedback, subjects indicated their current level of distraction and excitement, their mood (positive and negative), as well as their perceived self-efficacy on five Visual Analogue Scales (VAS), with each scale ranging from 1 (minimal) to 10 (maximum).

#### CS-valence and CS-UCS Contingency Ratings

After each phase of the conditioning procedure, ratings of CS valence (how pleasant/unpleasant do you feel when you see this picture?) and CS-UCS contingency (do you think that this picture is paired with an electrical stimulation?) were obtained using VAS presented on the screen. Subjects had to mark their rating with the cursor of the mouse. The anchor labels for the valence ratings and contingency ratings ranged from 0 (very pleasant) to 100 (very unpleasant) and 0 (extremely unlikely) to 100 (extremely likely), respectively.

#### Skin Conductance Responses

SCRs for each trial were calculated by subtracting the mean skin conductance level (SCL) during the 1000 ms prior to CS onset (baseline) from the maximum SCL recorded during the 8 s after CS onset. SCRs were z-transformed to attain a normal distribution.

#### General Procedure

Each participant was welcomed by the experimenter, who was dressed in a laboratory coat to increase his credibility, and led into the experimental room. Participants were seated upright in a comfortable chair in front of a computer screen and informed about the content of the experiment. Specifically, they were told that the experiment would involve the presentation of two different pictures and that one of these pictures may be paired with an electrical stimulation. In addition, to increase plausibility for the (to-be-implemented) experimental manipulations, participants were told that their physiological responses during the task as well as their responses to the questionnaires would be analyzed continuously by the experimenter in the laboratory room. Subsequently, electrodes for the electrical stimulation and measurement of SCRs were attached to the non-dominant arm. The intensity of the electrical stimulation for the conditioning procedure was adjusted to a sensation level participants experienced as "highly unpleasant but not painful" (adapted from Blechert et al., 2007). After participants had practiced and fully understood the rating scales (i.e., CS valence and CS-UCS contingency), the experimenter left the room and all participants completed the differential fear conditioning procedure (for details see Experimental Design). After each positive false feedback, the respective groups (i.e., EG after the first experimental manipulation, and CG1 after the second experimental manipulation) were asked whether they could identify with the feedback and to mention three keywords on how they cope in stressful situations (cf. Brown et al., 2012b). This was implemented as a manipulation check in order to determine whether these groups believed the false feedback (Brown et al., 2012b).

The experimenter only re-entered the room to deliver the feedback/debriefing or to distribute the VAS during the 15-min breaks of the fear conditioning procedure. At the end of the experiment, electrodes were removed and subjects were informed about the false feedback and were fully debriefed.

#### Statistical Analyses

Statistical analyses were carried out in IBM SPSS Statistics for Windows, version 20.0 (Armonk, NY, USA: IBM Corp.). Manipulation checks, i.e., induction of self-efficacy, were examined using a series of two-way mixed ANOVAs on each VAS with Group as between-subjects factor and Time (pre vs. post induction) as within-subjects factor. For the habituation, acquisition, and extinction phases, mixed ANOVAs with CS-type (CS+ vs. CS−) as within-subjects factor and group (EG vs. CG) as between-subjects factor were employed to analyze the effects of experimental manipulation 1 on subjective CS valence, CS-UCS contingency ratings and mean SCRs scores. In addition, the within subjects-factor "block" (early vs. late; averaged across the first and last five trials of the CSs, respectively) was added for the extinction and acquisition phases.

With respect to the effects of the second experimental manipulation, the end of extinction (for SCRs only the last extinction block was considered) was compared to continued extinction, using mixed ANOVAs with CS-type and Phase. Analyses were conducted separately for the experimental group (i.e., EG1 and EG2, effects of debriefing) and control group (i.e., CG1 and CG2, effects of late feedback). The critical alpha level was set to 0.05. Post-hoc analyses were performed using simple effects analysis and/or Bonferroni-corrected pairwise comparisons.

#### Results

Participants who received the self-efficacy induction during the first experimental manipulation (EG) did not differ from control participants (CG) with respect to relevant control variables such as age, gender, stress, or their scores on the DASS and its'

# Manipulation Checks

#### Experimental Manipulation 1

Descriptive statistics are displayed in **Table 2**. With respect to the first experimental manipulation and the self-efficacy scale in particular, significant main effects were found for Time [F(1, 46) = 8.639; p = 0.005; η 2 <sup>p</sup> = 0.158] and Group [F(1, 46) = 8.083; p = 0.007; η 2 <sup>p</sup> = 0.149] as well as their interaction [F(1, 46) = 9.312; p = 0.004; η 2 <sup>p</sup> = 0.168], indicating that the first experimental manipulation was successful. In particular, the experimental group showed an increase in perceived self-efficacy after the experimental manipulation (see **Table 2**). Moreover, self-efficacy ratings obtained after the experimental manipulation were higher in the experimental group relative to the control group (cf. **Table 2**). In addition, there was a significant main effect of Group on the positive mood scale [F(1, 64) = 4.578; p = 0.038; η 2 <sup>p</sup> = 0.091] (cf. **Table 2**).

#### Experimental Manipulation 2

With respect to the effects of the second experimental manipulation (i.e., debriefing and the late positive feedback) on subjects' ratings of perceived self-efficacy, main effects for Group and Time as well as their interaction failed to attain statistical significance (all p > 0.05).

Subsequent analyses of simple effects, however, revealed that the group who received the late feedback (CG1) showed higher self-efficacy ratings and a more positive mood after the induction [simple effects of time within CG1, all Pillai's trace ≥ 0.117; F(1, 44) ≥ 5.836; p ≤ 0.02, η 2 <sup>p</sup> ≥ 0.117; cf. **Table 2**]. By contrast, the debriefing (EG2) did not cause a lowered selfefficacy expectation or a less happy mood (all p ≥ 0.06).

#### Valence Ratings

#### Experimental Manipulation 1

As expected, no significant differences in valence ratings after habituation were found between groups or the CS+ and CS− (all p > 0.05) and the CS+ was rated more negatively than the CS− after fear acquisition [main effect CS-type; F(1, 46) = 245.678; p < 0.001; η 2 <sup>p</sup> = 0.842]. After extinction, a main effect of CStype was found [F(1, 46) = 18.664; p < 0.001; η 2 <sup>p</sup> = 0.289], which was qualified by a CS-type x Group interaction [F(1, 46) = 6.601; p = 0.013; η 2 <sup>p</sup> = 0.125). Analysis of simple effects revealed that this interaction was not due to significant group differences in the absolute ratings of the CS+ and CS− (both p ≥ 0.066), but driven by the fact that the control group, but not the experimental group, continued to rate the CS+ more negatively than the CS− [Pillai's trace = 0.340; F(1, 46) = 23.231; p < 0.001; η 2 <sup>p</sup> = 0.340; cf. **Figure 2** (left)].

#### Experimental Manipulation 2 **Effects of debriefing (i.e., EG1 and EG2)**

Only the main effect of CS-type [F(1, 22) = 5.932; p = 0.023; η 2 <sup>p</sup> = 0.212] and the interaction between Group and Phase [F(1, 22) = 5.592; p = 0.027; η 2 <sup>p</sup> = 0.203], with a trend towards a significant difference between groups at the end of extinction


#### TABLE 2 | Manipulation checks.

*RAS* = *resilience appraisal scale. First experimental manipulation: early positive feedback (EG) after fear acquisition; Second experimental manipulation: late positive feedback (CG1) and debriefing (EG2) after fear extinction; EG1 (n* = *12): early feedback only; EG2 (n* = *12): debriefing (early feedback); CG 1 (n* = *12): late feedback; CG2 (n* = *12): no feedback. <sup>a</sup>administered only after the first experimental manipulation.*

*<sup>b</sup>*−*ipairwise comparisons, significant at p* ≤ *0.05.*

(p = 0.083), were significant. Additional analyses of both groups separately revealed that EG1 did not discriminate among the CSs after both phases (no main effect of CS-type, nor its interaction with Phase; all p ≥ 0.130). EG2, however, showed a significant CS+/CS− differentiation [main effect CS-type; F(1, 11) = 11.397; p = 0.006; η 2 <sup>p</sup> = 0.509], which did not depend on Phase (CStype x Phase interaction; p = 0.815). In neither group was a main effect of Phase observed (both p = 0.114).

#### **Effects of the late feedback (i.e., CG1 and CG2)**

Main effects of Phase [F(1, 22) = 21.259; p < 0.001; η 2 <sup>p</sup> = 0.491] and CS-type [F(1, 22) = 24.879; p < 0.001; η 2 <sup>p</sup> = 0.531] as well as their interaction [F(1, 22) = 5.935; p = 0.023; η 2 <sup>p</sup> = 0.212] were obtained. The significant CS-type x Group interaction [F(1, 22) = 8.490; p = 0.008; η 2 <sup>p</sup> = 0.278] showed that, averaged across both phases, CG1 and CG2 differed in their reaction towards the CS+ (p = 0.025), but not toward the CS− (p = 0.52). In addition, Zlomuzica et al. Self-efficacy facilitates extinction

CG2, but not CG1, showed significantly higher ratings for the CS+ as compared to the CS− (p < 0.001). Interestingly, when groups were analyzed separately, the CS-type × Phase interaction was significant for CG1 [F(1, 11) = 5.679; p = 0.036; η 2 <sup>p</sup> = 0.340], but not for CG2 (p = 0.268). Thus, similar to the effects obtained for experimental manipulation 1, CG1 did indeed continue to discriminate among the CSs by the end of extinction (p = 0.031), but ceased to do so after continued extinction (p = 0.253). By contrast, CG2 rated the CS+ as more aversive than the CS− during both phases (both p ≤ 0.002). Results are illustrated in **Figure 2** (right).

#### CS-UCS Contingency Ratings Experimental Manipulation 1

After habituation, no differentiation between the CS+ and CS− was evident. As depicted in **Figure 3** (left), contingency ratings were higher for the CS+ than the CS− after both the acquisition and extinction phase [main effect CS-type; both F(1, 46) ≥ 22.623; p ≤ 0.001; η 2 <sup>p</sup> ≥ 0.330]. No effects of Group or a CS-type × group interaction were obtained (all p > 0.05).

#### Experimental Manipulation 2

#### **Effects of debriefing (i.e., EG1 and EG2)**

Higher CS-UCS contingency ratings were obtained for the CS+ than the CS− [main effect CS-type; F(1, 22) = 18.318; p < 0.001; η 2 <sup>p</sup> = 0.454]. There was no effect of debriefing, with groups being comparable across both phases (all other main or interaction effects non-significant; p = 0. 094).

#### **Effects of the late feedback (i.e., CG1 and CG2)**

Higher CS-UCS-contingency ratings were obtained for the extinction phase [main effect Phase; F(1, 22) = 13.685; p = 0.001; η 2 <sup>p</sup> = 0.383] and the CS+ [main effect CS-type; F(1, 22) = 17.220; p < 0.001; η 2 <sup>p</sup> = 0.439]. Interactions between CS-type and Phase [F(1, 22) = 4.205; p = 0.052; η 2 <sup>p</sup> = 0.160] as well as CS-type and Group [main effect CS-type; F(1, 22) = 3.379; p = 0.08; η 2 <sup>p</sup> = 0.133] were significant at trend level. Analyses of the simple effect of CS-type for each of the groups separately showed that EG2 discriminated between the CSs after both phases (both p ≤ 0.014). By contrast, CG1 did only show a CS+/CS− differentiation after extinction (p = 0.048), but not after continued extinction [cf. **Figure 3** (right)], which was due to a decrease in UCS-contingency attributed to the CS+ (p = 0.006).

#### Skin Conductance Responses Experimental Manipulation 1

During habituation, subjects did not respond differently towards the CS+ and CS−. There was a significant effect for CS-type [F(1, 46) = 42.365; p < 0.001; η 2 <sup>p</sup> = 0.479] and block [F(1, 46) = 6.453; p = 0.015; η 2 <sup>p</sup> = 0.123] during fear acquisition, with higher SCRs for the CS+ and the early acquisition block. A main effect for CS-type persisted over the extinction phase [F(1, 46) = 10.030; p = 0.003; η 2 <sup>p</sup> = 0.179], yet no other main or interaction effects were significant. However, when the simple main effect of CS-type was tested within each combination of Group and Block, the experimental group did not exhibit any differences in SCRs to the CSs within both the early and late extinction block (both p > 0.112), whereas the control group demonstrated higher SCRs toward the CS+ as compared to the CS− in both blocks [both Pillai's trace ≥ 0.085; F(1, 46) ≥ 4.278; p ≤ 0.044; η 2 <sup>p</sup> ≥ 0.085); cf. **Figure 4** (left)].

#### Experimental Manipulation 2

#### **Effects of debriefing (i.e., EG1 and EG2)**

All main or interaction effects did not attain statistical significance (all p ≥ 0.115).

#### **Effects of the late feedback (i.e., CG1 and CG2)**

SCRs were higher for the CS+ [main effect CS-type; F(1, 22) = 8.978; p = 0.007; η 2 <sup>p</sup> = 0.290] and for continued extinction [main effect Phase; F(1, 22) = 4.317; p = 0.05; η 2 <sup>p</sup> = 0.164] while none of these effects were subjected to group differences (all p ≥ 0.304). Results are displayed in **Figure 4** (right).

#### Discussion

The main objective of the present study was to examine the impact of an experimental manipulation aimed at increasing self-efficacy beliefs on the subsequent extinction of conditioned fear. We herein provide first evidence that a positive verbal feedback, which increases self-efficacy beliefs, can facilitate fear extinction. Participants who received the experimental induction showed enhanced extinction, as evidenced on the level of psychophysiological responding. Accordingly, they also showed a stronger reduction of conditioned negative valence after extinction relative to participants without the self-efficacy induction. However, the self-efficacy induction had no effect on CS-UCS contingency evaluation.

The results of the present study extend previous findings by Brown et al. (2012a,b) in two ways. First, similar to Brown et al., we could demonstrate that perceived self-efficacy can be experimentally manipulated via verbal persuasion. Second, while Brown et al. showed that such a manipulation can have an impact on autobiographical memory retrieval and problem solving capabilities, our data suggest that increases in perceived selfefficacy can be beneficial for emotional learning. Taken together, these results support the propositions of social-cognitive theories on the role of perceived self-efficacy as an important mediator of cognitive, motivational and affective processes (Bandura, 1997).

The putative mechanisms by which an increased perceived self-efficacy might have affected inhibitory learning performance in our experiment, however, remain elusive. Evidence from previous similar studies (Marquez et al., 2002) suggests that a systematic manipulation via verbal feedback aimed at enhancing perceived self-efficacy can lead to decreased levels of anxiety and arousal. Thus, it is possible that a reduced emotional responding to CSs, during extinction can account for the herein observed effects. Indeed, it has been shown that "state anxiety" changes both the processing of extinguished conditioned responses and the sensitivity with which individuals react to these stimuli during extinction (Vriends et al., 2011). Although the experimental manipulation in our study slightly increased positive mood in our participants, the manipulation had no effect on excitement, or negative mood. It is self-evident that a positive verbal feedback with respect to self-efficacy can lead to concomitant increases in positive mood. However, while Vriends et al. (2011) showed that the induction of a positive emotional state (by means of film induction) leads to a decrease in SCRs to both CSs during extinction, such a response pattern was not evident in our study. In fact, a closer inspection of the SCR data during extinction revealed that the groups did not differ with respect to responding to the CS+ or the CS−. Therefore, it is unlikely that the effects of the positive self-efficacy induction on fear extinction performance (either mediated via increases in self-efficacy, positive mood, or both) are due to a decreased tendency to respond or rather a temporary suppression of emotional reactivity to either the CS+ or CS−. In line with this hypothesis, debriefing participants about the false positive feedback (second experimental manipulation) had no effect on extinguished conditioned responses and thus did not lead to a subsequent revival of extinguished conditioned responses during continued extinction.

A more plausible explanation for the herein observed effect thus might be that an increased perceived self-efficacy altered extinction learning in particular (Craske et al., 2008, 2014; Vervliet et al., 2013). In support of this notion, we found no significant difference in the mean differential SCR to the CS+ and CS− during the late phase of extinction in the group who had received the false positive verbal feedback. Conversely, however, a differential SCR to the CS+ and CS− during the late extinction phase was still existent in individuals without the self-efficacy feedback. Hence, our results indicate that individuals with an increased perceived self-efficacy showed superior extinction learning performance on implicit (i.e., skin conductance responses) as well as subjective measures (i.e., valence ratings) of fear. Interestingly, while individuals with perceived self-efficacy exhibited a more pronounced decrease in conditioned negative valence rating after extinction, no changes with respect to CS-UCS contingency ratings were observed in this group. Hence, self-efficacy enhancement via verbal persuasion might affect the participant's learning about the emotional significance of CSs (reflected on SCR and valence ratings level) while it does not affect the participant's evaluation of stimulus-outcome contingencies. The functional significance of this finding needs further clarification. However, it can be speculated that self-efficacy enhancement engages different control systems (emotional vs. informational) which further rely on distinct neuronal entities to promote fear extinction.

Interestingly, it has been shown that cognitive reappraisal, an emotion regulation strategy used to counteract negative selfbeliefs and to increase adaptive emotional reactivity (Goldin et al., 2012), relies on brain systems which are directly involved in fear extinction, including (but not limited to) the dorsolateral prefrontal cortex and dorsal anterior cingulate cortex (Schiller and Delgado, 2010). Moreover, the use of reappraisal techniques during fear conditioning can facilitate fear extinction learning by selectively increasing the inhibitory input of ventromedial prefrontal-amygdala connections (Delgado et al., 2008; Schiller and Delgado, 2010). It would be interesting to investigate whether techniques to increase self-efficacy might constitute another strategy suitable to promote extinction via top-down prefrontal cortex modulation (see also Buhle et al., 2014).

Apart from fear extinction, other studies have already confirmed a positive influence of self-efficacy on learning and memory performance in various other tasks. For instance, selfreport measures on perceived self-efficacy have been shown to predict cognitive capabilities (Bouffard-Bouchard, 1990; Paunonen and Hong, 2010) and learning performance rate in procedural tasks (Eyring et al., 1993; Mitchell et al., 1994), as well as academic (Lent et al., 1986) and workrelated performances (Stajkovic and Luthans, 1998) in healthy subjects. Moreover, it has been proposed that variations in self-efficacy might account for cognitive differences among young and older adults (Seeman et al., 1996; McDougall and Kang, 2003). Interestingly, older participants show diminished performance in diverse learning and memory tasks which might be at least partly related to age-dependent decreases in perceived learning self-efficacy (Hertzog et al., 1990; Fisk and Warr, 1996; Seeman et al., 1996; McDougall and Kang, 2003). It may be inferred that attempts to increase beliefs about memory efficacy should help older subjects to use mnestic capabilities more effectively in different contexts (McDougall, 1998; Payne et al., 2012). Of course, such a conclusion might be overly simplified as an explanation for the findings of Brown et al. (2012a,b) and our study. While the experimental manipulations aimed to increase perceived self-efficacy were not related to learning self-efficacy (Berry et al., 1989; Berry, 1999) or the specific task domains used, the global positive feedback with respect to self-efficacy might have nevertheless influenced the participant's cognitive resources or the individual's comprehension in the particular paradigm (e.g., McDougall, 1998). For example, Kalpouzos and Eriksson (2013) showed that healthy adults who differ in memory self-efficacy beliefs use different cognitive strategies when encoding episodic memory information. Most importantly, participants with high vs. low self-efficacy beliefs concerning their memory also show a different pattern of brain activation. Hence, it is possible that, similar to "high-memory believers" in the study by Kalpouzos and Eriksson, participants with an increased perceived self-efficacy rely on more efficient mnestic strategies and/or recruit different brain structures during extinction learning. Further research would be needed to test this hypothesis more specifically.

Our findings have important clinical implications. Clinical studies in different anxiety disorders have identified selfefficacy as an important mediator of successful exposure-based treatments (Bouchard et al., 2007; Gaudiano and Herbert, 2007; Delsignore et al., 2008; Gallagher et al., 2013). These studies, however, focused on changes in self-efficacy derived from mastery experiences and provided only correlational evidence on the link between self-efficacy and symptom improvement during exposure-based treatment (Bouchard et al., 2007; Gallagher et al., 2013). Given that extinction learning might be analogous to exposure, our findings implicate that differences in self-efficacy levels prior to exposure can mediate anxiety reduction during and after exposure treatment. Moreover, our results challenge the notion that verbal persuasion is less important than mastery experience in increasing perceived self-efficacy (but see Bandura, 1997; Gallagher et al., 2013). In line with the theory of positive and negative cognitions in anxiety (Casey et al., 2004), our findings rather indicate that increasing patients' perceived selfefficacy via social persuasion might constitute an underestimated yet powerful strategy to increase exposure therapy efficacy.

Several limitations of the current study should be considered. First, we did not employ a verbal feedback in the "control condition." Hence, it cannot be excluded that the verbal feedback per se (independent of its positive valence) might have had a similar effect on extinction. The rationale behind this experimental design was that we anticipated a "neutral feedback on self-efficacy" to induce a state of "uncertainty" in our participants. However, since our aim was to test whether extinction can be further enhanced through verbal persuasion and hereby provide a direct implication for exposure treatments, such a control condition might not be equivalent to exposure under standardized conditions (i.e., treatment as usual). A replication of this study with an experimental design, which includes another group of participants, who receive a "neutral feedback" with respect to self-efficacy, would be helpful.

Second, our data suggest that the experimental induction used was not sufficient to elicit differences in self-efficacy between the experimental and control group on the RAS questionnaire (but see Brown et al., 2012a,b). Here, the absence of such an effect might be related to differences in the methodological approach. In contrast to the study by Brown et al. (2012b), we did not use a low self-efficacy induction as a control condition which would probably lead to more pronounced group effects on the RAS measure.

Third, the translation of our findings to useful applications in clinical populations and the therapy setting remains to be further explored. In the present study, we included non-clinical subjects who did not report current or previous mental diseases, psychological or pharmacological treatment for mental diseases, as well as severe acute or chronic somatic diseases. While these criteria were checked prior to the experimentation phase, we did not use a psychodiagnostic interview to assess possible psychiatric diagnoses. Hence, the existence of diagnostically relevant mental health problems in our participants cannot be fully excluded An important extension for future studies would be to examine whether self-efficacy can also be enhanced in patients diagnosed with emotional disorders to counteract deficits in extinction learning (see Blechert et al., 2007; Michael et al., 2007; Briscione et al., 2014). Furthermore, it would be valuable to examine whether the efficacy of exposure-based treatments can be enhanced via modification of self-efficacy beliefs. We suggest that a false positive verbal persuasion would

#### References


probably be less appropriate to investigate the role of selfefficacy on exposure therapy outcome. However, perceived selfefficacy can be increased via different sources (Bandura, 1997; Maddux, 1999). For instance, one could investigate how positive future imaging or the instructed retrieval of positive self-efficacy experiences (i.e., episodic memories and episodic future thinking, see Zlomuzica et al., 2014) affects the patient's emotional and behavioral responding during exposure treatment.

Finally, we are aware that extinction is an oversimplified model of exposure. While we acknowledge extinction as a major candidate for explaining the effects of exposure, there are several other relevant factors (Margraf and Zlomuzica, 2015) which should not be neglected in potential future clinical studies.

In summary, the present results suggest that fear extinction can be facilitated via positive manipulation of perceived selfefficacy. To our knowledge, we herein show for the first time that perceived self-efficacy alters fear extinction learning and add new evidence on the role of self-efficacy as an important mediator of learning and memory. Our findings might not only provide novel insights into the mechanisms underlying changes in self-efficacy and symptom improvement during exposure, but also trigger new ideas on how cognitive top-down modulation strategies can be used to improve CBT efficacy (Craske et al., 2008, 2014; Vervliet et al., 2013; Margraf and Zlomuzica, 2015).

#### Acknowledgments

This work was supported by the Deutsche Forschungsgemeinschaft (DFG) with grant ZL 59/2-1 and ZL 59/2-2 to AZ, SS, and JM.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Zlomuzica, Preusser, Schneider and Margraf. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Gradual extinction reduces reinstatement

#### Youssef Shiban\*, Jasmin Wittmann, Mara Weißinger and Andreas Mühlberger

Department of Clinical Psychology and Psychotherapy, Institute of Psychology, University of Regensburg, Regensburg, Germany

The current study investigated whether gradually reducing the frequency of aversive stimuli during extinction can prevent the return of fear. Thirty-one participants of a three-stage procedure (acquisition, extinction and a reinstatement test on day 2) were randomly assigned to a standard extinction (SE) and gradual extinction (GE) procedure. The two groups differed only in the extinction procedure. While the SE group ran through a regular extinction process without any negative events, the frequency of the aversive stimuli during the extinction phase was gradually reduced for the GE group. The unconditioned stimulus (US) was an air blast (5 bar, 10 ms). A spider and a scorpion were used as conditioned stimuli (CS). The outcome variables were contingency ratings and physiological measures (skin conductance response, SCR and startle response). There were no differences found between the two groups for the acquisition and extinction phases concerning contingency ratings, SCR, or startle response. GE compared to SE significantly reduced the return of fear in the reinstatement test for the startle response but not for SCR or contingency ratings. This study was successful in translating the findings in rodent to humans. The results suggest that the GE process is suitable for increasing the efficacy of fear extinction.

Keywords: gradual extinction, virtual reality, pavlovian fear conditioning, skin conductance response, startle response, contingency ratings

# Introduction

Anxiety disorders are the most common cases of mental disorders (Merikangas et al., 2010) and can be treated with exposure therapy, which has proven to be an effective strategy for treating fear (Hofmann and Smits, 2008). Exposure therapy is presumably based on extinction (Pavlov, 1927): repeated presentation of a previously learned threat stimulus without negative consequences. In most cases, this approach leads to a temporary reduction of anxiety (Vervliet et al., 2013). The return of such extinguished anxiety is a widespread problem. For clinicians, this frequent relapse after a successful extinction is a big challenge, which is why it is so crucial to understand the mechanisms of extinction.

There are different approaches to prevent a relapse: massive extinction treatment (Denniston et al., 2003), multiple context exposure (Shiban et al., 2013), renewal testing in the presence of a retrieval cue from extinction (Brooks and Bouton, 1993) and gradual extinction (GE; Gershman et al., 2013). The last method, GE, includes a modified extinction process, during which the aversive stimulus (US) is not completely absent, but the frequency of its occurrence is gradually reduced. The recent study with rats by Gershman et al. (2013) provides promising results in

#### Edited by:

Oliver T. Wolf, Ruhr University Bochum, Germany

#### Reviewed by:

Frauke Nees, Central Institute of Mental Health, Germany Tim Klucken, Justus Liebig University Giessen, Germany

#### \*Correspondence:

Youssef Shiban, Department of Clinical Psychology and Psychotherapy, Institute of Psychology, University of Regensburg, Universitätsstraße 31, 93053 Regensburg, Germany youssef.shiban@psychologie. uni-regensburg.de

Received: 21 July 2015 Accepted: 31 August 2015 Published: 15 September 2015

#### Citation:

Shiban Y, Wittmann J, Weißinger M and Mühlberger A (2015) Gradual extinction reduces reinstatement. Front. Behav. Neurosci. 9:254. doi: 10.3389/fnbeh.2015.00254 support of the efficacy of this method. In the following study, we applied these findings to a human sample. To understand the basic assumptions of this approach, a more detailed look into extinction processes is required. Extinction learning is believed to extinguish the conditioned response by presenting the conditioned stimuli (CS) without the unconditioned stimulus (US) during a number of trials.

A traditional model of Pavlovian conditioning (Rescorla and Wagner, 1972) asserts that learning is the modification of associations between the CS and US. Therefore, fear conditioning is reinforcement and extinction is weakening of the initial association. More modern approaches have found evidence that the initial fear memory does not tend to weaken: a whole new memory inhibiting the initial CS-US association is formed (Bouton, 2004). Inhibitory learning is characterized by retained original fear memory which competes with the new model. Bouton (2004) considers the fact that animals learn context, not just CS-US associations. Contextual and temporal clues during the learning process are crucial for saving new information. It would be a mistake to assume that anxiety is just a result of the association strength between the CS and US. In fact, two memories seem to coexist after extinction: an excitatory CS-US association and an inhibitory CS- no US association. The excitatory association causes a fear reaction, while the inhibitory association prevents the reaction entirely.

An approach for enhancing the inhibitory associations, consequently making extinction learning more effective, was made by Craske et al. (2014). According to them, exposure optimization strategies include: (1) expectancy violation; (2) deepened extinction; (3) occasional reinforced extinction; (4) removal of safety signals; (5) variability; (6) retrieval cues; (7) multiple contexts; and (8) affect labeling. The first strategy, expectancy violation, is based on the assumption that a mismatch between expectancy and experience is crucial for learning (Rescorla and Wagner, 1972). Expectancy violation—the discrepancy between an anticipated outcome and a real outcome concerning the frequency or intensity of aversive stimuli during the extinction phase—should be maximized, so that the inhibitory association can be strengthened.

A similar concept, which acknowledges the creation of a new ''extinction'' memory as the reason for the return of fear, is the ''state'' concept. The extinction process is postulated to be perceived as a new state of the world (Redish et al., 2007), which results in forming a new memory. Consequently, two competing memories co-exist depending on the learning context: the conditioning state and the extinction state.

Why does this new state emerge? The absence of the aversive stimulus during a traditional extinction phase signals a change to take place; expectations are violated and learning occurs (Rescorla and Wagner, 1972). According to Redish et al. (2007), prediction errors might be misinterpreted as indicators for a new state. Therefore, a new ''no-fear'' memory—which includes the new associations—is formed and starts competing with the retained original fear memory.

If massive prediction errors function as instructive signals for a new state, a fear memory could be modified by prediction errors that are small enough not to induce the formation of a new memory, but still massive enough to drive learning (Gershman et al., 2013). So, as a hypothesis, this would lead to a more efficient extinction of fear and prevention of relapses when compared to the standard extinction (SE) process.

A recent study with rats by Gershman et al. (2013) provides strong evidence in support of this hypothesis. The investigators demonstrated in two Pavlovian fear conditioning experiments that gradually reducing the frequency of the aversive stimuli, rather than eliminating them abruptly, prevents the return of fear.

The aim of the present experiment was to apply these findings from rats to humans. Based on the classical Pavlovian conditioning paradigm, fear is learned in an acquisition phase, and is afterwards extinguished in an extinction phase, which differed for the two experimental groups. The Standard group took part in the original extinction process, whereas the Gradual group underwent GE, for which the occurrence of the US was gradually eliminated so that the prediction error was high enough to drive learning, but not high enough to cause the creation of a new memory. Thus, weakening the original fear memory should be ensured. The efficiency of GE for extinguishing fear and preventing the return of fear was measured by reinstatement on a subjective level (contingency ratings) as well as on a physiological level (startle response and skin conductance response, SCR). Moreover, the experiment was conducted in a virtual reality (VR), which has been proven an efficient tool to investigate basic processes of conditioning (Glotzbach et al., 2012) and therapy research (Shiban et al., 2013).

## Materials and Methods

#### Participants

Thirty-one volunteers were recruited through advertisements at the University of Regensburg. Recruitment took place from April to September 2014. After the participants gave their written consent, the exclusion criteria (spider phobia, age <18 and >50, current involvement in psycho- or pharmacotherapy, neurologically related diseases, a history of psychotropic drug use, color blindness and hearing disorders) were assessed with a demographic questionnaire. All 31 participants (80.6% female, age ranged between 18 and 41, M = 24.0, SD = 4.69) were students at the University of Regensburg and obtained credit points as reimbursement for their participation. Participants were pseudorandomly divided (depending on survey date) into two groups based on the respective extinction process (described in detail in the ''Procedure'' Section). The two groups did not differ significantly in the number of participants, age, gender or in their FSQ and STAI scores (see **Table 1**). The Ethics Committee of the University of Regensburg approved the study.

#### Materials

A VR was presented to participants over a V Z800 3D headmounted display (HMD; eMagin, NY, USA) and was generated with the help of Steam Source engine (Valve Corporation, Bellevue, WA, USA). ''Cybersession'' software (VTplus GmbH, Würzburg, Germany) controlled the presented VR environment.

#### TABLE 1 | Demographic variables and questionnaire data.


Means (M), Standard Deviations (SD), df-, t-, and p-Values and also quantity (N) and percentage are given. Note. df, degrees of freedom; n, number of participants; Standard Group, extinction after the standard extinction process; Gradual Group, extinction after the gradual extinction process; STAI-State1, STAI-State2, STAI-Trait, State (day 1 and 2) and Trait scale of the German version of the State-Trait Anxiety Inventory (Laux et al., 1981), FAS, of the German version of the Fear of spiders questionnaire (Szymanski and O'Donohue, 1995) <sup>a</sup>Fisher's exact test, two-tailed.

The participant's head position was monitored with a Patriot electromagnetic tracking device (Polhemus Corporation, Colchester, VT, USA), which adjusted the field of view in response to head movements. Sounds and instructions were presented over headphones (Sennheiser HD-215, Sennheiser electronic GmbH, Germany). Physiological data were monitored, digitally amplified (V-Amp 16, Brain Products GmbH, Gilching, Germany) and recorded (Brain Vision Recorder software, Version 1.20, Brain Products GmbH, Germany).

The VR environment consisted of two rooms, which differed in the textures used for floor, walls and ceiling color (see **Figures 1A,B**). Participants were able to explore these rooms by looking around, but were unable to move freely. Three stimuli were used for the experiment: one US and two CS. The US involved an air blast (5 bar, 50 ms) aimed at the participant's right anterior neck. A compressed tank of air was regulated via a magnetic valve system channeling the air through a tube, which was adjusted to the participant's torso. The CS-US contingency was set at 80% for the acquisition phase. The CS were two virtual animals (virtual spider and scorpion, see **Figures 1C,D**). They were both presented sitting on a gray platform in the middle of the virtual room during the different phases of the experiment: acquisition, extinction, and reinstatement test.

For the CS+, a virtual spider—sitting on the platform and moving its legs—was presented to the participants. The CS− was a scorpion sitting sideways on the platform and moving its tail.

#### Measures

Before the VR experiment, participants filled in a demographic questionnaire (age, gender, occupation, and exclusion criteria), and the State-Trait Anxiety Inventory (STAI; Spielberger et al., 1970; German: Laux et al., 1981), which is a commonly used measure for assessing temporary anxiety (state) as well as general anxiousness (trait). Both forms of anxiety are represented by 20 items (statements) each. Answers are given on a fourpoint Likert scale (state: from 1 = not at all to 4 = very much; trait: from 1 = almost never to 4 = almost always). As the State version targets current anxiety caused by the situation at hand, it is filled in on day 1 and 2, while the Trait version for general anxiousness is completed on day 1. For the German version of the STAI (Laux et al., 1981), objectivity concerning conductance, scoring, and interpretation is given. Internal consistency (Cronbach's Alpha) lies between 0.90 and 0.94 (state) and 0.88 and 0.94 (trait). The retestreliability coefficient for trait anxiety is between 0.68 and 0.96. Convergent and divergent validity were tested with several populations and were established (Laux et al., 1981). To assess spider phobia, the German version of the Fear of Spiders questionnaire (FSQ; Szymanski and O'Donohue, 1995; German version: FAS; Rinck et al., 2002) with 18 items (which are evaluated on a seven-point Likert scale ranged from 0 = ''I do not agree at all'' to 6 = ''I completely agree'') was used. The translated FSQ demonstrates very high internal consistency, Cronbach's Alpha = 0.97, and retest reliability, rtt = 0.95 (Rinck et al., 2002). It is a sensitive measure used to differ between phobics and non-phobics (Szymanski and O'Donohue, 1995).

To measure the emotional state of the participants upon presentation of the virtual animals during different phases, participants were requested to verbally rate the probability that a negative event would occur (contingency rating). The rating scale ranged from 0 (no probability of a negative event occurring) to 10 (100% probability of a negative event occurring), and the ratings were reported at the beginning and at the end of each phase.

Apart from the subjective ratings, two different physiological values were measured. For the startle response, the muscle activity of the Orbicularis Oculi, called the Startle Reflex, was induced by a random noise (white noise: 50 ms, 103 dB), which was presented binaurally over headphones during the presentation of the conditioned stimuli with a contingency of 80%. The reflex was measured with four electrodes (Ag/AgCl, Ø = 8 mm) affixed with electrode paste (Signa Creme, Parker Laboratories, New Jersey, USA: Parker Laboratories). Two electrodes were placed under the right eye of the participant and one behind each ear at the mastoid bone for reference and grounding. Impedance level was kept below 5 kΩ.

For the SCR, two electrodes (Ag/AgCl, Ø = 8 mm) were attached with electrode cream to the thenar muscle of the non-dominant hand (TD – 246, PAR Medizintechnik GmbH). The skin was cleaned with alcohol prior to electrode attachment.

#### Procedure

The study was conducted in two sessions, which ran on two consecutive days. An interval of at least 24 h was planned between the sessions, since that is the standard in human fear recovery experiments (Shiban, 2013). Session 1 (about 120 min) involved the acquisition and extinction phase. During Session 2 (about 30 min) the return of fear was tested by a reinstatement test (see **Figure 2**).

At the first day, after the participants filled in the declaration of consent and the questionnaires (demographic, STAI and FAS), the electrodes were adjusted for the physiological measurements, along with the tube for the air puff, the headphones and the HMD. At the beginning of the experiment, there was a short introduction of the procedure and participants were asked to relax for 2 min to assess a baseline for the physiological data. Subsequently, the startle noise was presented repeatedly for a time span of 109 s to prevent distortion of the data caused by habituation to the startle noise. ''Habituation is the decline of the acoustic startle response magnitude following repeated presentation of startling stimuli within a single test session'' (Koch, 1999). Afterwards, an acclimation phase was initiated, in which all the stimuli we use in this experiment were shortly presented and the participants were instructed to look around the room using head orientation. This was conducted in order to avoid biases in the data caused by context or stimulus novelty effects. The experiment began directly after this phase. The experiment on the first day consisted of the acquisition and extinction phases, both beginning and ending with a rating of contingency. The stimuli were presented in trials, each trial took 30 s and consisted of an eight-second stimulus presentation followed by a 22 s inter-stimulus interval, during which the participant saw a black screen. During the acquisition phase, each stimulus was presented 18 times. The virtual scorpion (CS−) and the virtual spider (CS+) were presented to the participants for 8 s each, and 6 s after the appearance of the virtual animals, a startle noise was presented with a probability of 75% for both stimuli. The CS+ was followed by an aversive air puff (5 bar, 50 ms) 2 s after the appearance of the virtual animals in 80% of all cases, except during the rating phases. After a 10 min break, the experiment continued with the extinction phase, during which the CS+ and CS− were

phase for the gradual group. Each box symbolizes the presentation of a CS+, which was paired with an US at the colored boxes.

presented 22 times each. The two experimental groups differed as follows. For the SE group, there was no presentation of the US during the extinction phase. For the GE group, the presentation of the aversive air puff following the CS+ was gradually decreased during the extinction phase (see **Figure 3**). The startle noise continued to emerge as it had before in the acquisition phase, and there was no air puff following the CS−.

At the second day, after the participants completed the STAI State questionnaire for session two, the electrodes, as well as the belt for the air blast tube, were attached at the designated places, and headphones and HMD were adjusted. The session on day 2 consisted of an acclimation phase, similar to day 1, and the reinstatement test. The contingency rating was given in the end of reinstatement test, which took place in a new room (room 2). Reinstatement test started with two presentations of the air puff without showing a stimulus, followed by five CS+ and CS− appearances without the US. The startle noise appeared with a contingency of 75% during the presentation of the CS. The experiment was completed with a final extinction phase that consisted of eight presentations of the spider in room one, without any aversive stimulus or startle noise.

#### Experimental Design

In accordance with the fear conditioning study by Acheson et al. (2013), the experiment consisted of three phases: acquisition, extinction learning, and extinction recall, which was measured by a reinstatement test. The CS+, one of the two conditioned stimuli, was paired with the US. There was no presentation of the US together with the CS−. The two experimental groups were formed through random assignment of the participants (see **Figure 4**). Both groups completed all three phases of the experiment. The SE and GE groups differed in their processes during the extinction learning phase. Subjective ratings of contingency as well as the physiological reactions (Startle, SCR) represent the dependent variables.

#### Data Reduction and Statistical Analysis

Analyses focused on the participant's physiological arousal and subjective reactions to the presentation of the CS+ and CS− in the different phases of the experiment in VR: acquisition, extinction and return of fear as tested by reinstatement.

In all three phases, the between-group factor was measured for the Extinction group. The within-group factors stimulus (CS+ vs. CS−) and time were also measured for the different phases: acquisition and extinction (beginning vs. end), and reinstatement (end of extinction vs. reinstatement).

Physiological data were preprocessed with Brain Vision Analyzer 2.0 software (Brain Products GmbH, Munich, Germany) and further analyses were performed in SPSS 22.0 (IBM Corp., Armonk, NY, USA).

For the startle response, at first, differences between the two EMG electrodes were computed (see Blumenthal et al., 2005). A 249 Hz high cut-off filter, a 28 Hz low cut-off filter and a 50 Hz notch filter were applied. The data were rectified, and a moving average (50 ms) was calculated. For each startle, a baseline correction was conducted using the mean value of the 50 ms before each startle tone as baseline. Next, peaks were marked automatically and manually controlled and corrected if necessary. Finally, T-values for the startle magnitude were calculated.

For SCR, a 1 Hz cut-off filter was applied. Data were rectified and for each SCR, a baseline correction was conducted using the mean value of the 500 ms before each presentation of the stimulus as a baseline. For peak detection, data from 3000 to 6000 ms after the presentation of the stimuli were segmented. Peaks were marked automatically and manually controlled and corrected if necessary. Finally, T-values for the SCR were calculated.

For physiological outcome variables in the acquisition and extinction phases, physiological data of the first four (beginning) and last four (end) presentations of the stimuli in each phase were used to calculate means. For the reinstatement test, means were calculated with the data following four stimuli presentations. For each outcome variable (contingency ratings, startle response, SCR) that was measured in the two rooms, means for CS+ and CS− were calculated.

For contingency ratings, startle and SCR repeated-measures ANOVAs with the within-subjects factor time (Beginning vs. End), stimulus (CS+ vs. CS−) and between-subjects factor group (SE vs. GE) were applied for each phase (acquisition and extinction and reinstatement test).

In additional analyses of significant effects of time, stimulus or group Student's t-tests were performed. Partial η 2 (η 2 p ) scores and Cohen's d were used as indices of effect size. The significance level was set at two-tailed α = 0.05.

#### Results

#### Acquisition

#### Contingency Ratings

As visible in **Figure 5**, the contingency ratings for the CS+ were higher than the ratings for the CS− before the acquisition phase, as well as afterwards, and increased in both groups over time, while the CS− ratings either increased minimally (GE) or decreased (SE). An ANOVA revealed a significant main effect of stimulus, F(1,26) = 13.8, p < 0.001, η 2 <sup>p</sup> = 0.35, as well as an interaction effect of Time × Stimulus, F(1,26) = 11.3, p < 0.002, η 2 <sup>p</sup> = 0.30. Follow-up on the significant interaction effect demonstrated that at the end of the acquisition phase the CS+ and CS- differed significantly, t(26) = 4.51, p < 0.001, d = 0.84. Means and standard deviations can be viewed at **Table 2**. These results indicate that successful acquisition took place.

#### Startle Response

The two experimental groups did not differ at the beginning or end of this phase. The CS+ caused a higher reaction than the CS− during the entire phase, and the response decreased from the beginning to the end of acquisition phase for both groups (see **Table 2**). This pattern was also reflected only in a significant main effect of time, F(1,28) = 23.4, p < 0.001, η 2 <sup>p</sup> = 0.46 and stimulus, F(1,28) = 21.7, p < 0.001, η 2 <sup>p</sup> = 0.43 in the acquisition phase.

#### Skin Conductance Response

There were similar SCR levels in the two experimental groups at the beginning and the end of the acquisition phase, as shown in **Figure 7**. For the whole samples, the CS+ triggered a higher SCR than the CS− during the acquisition phase (see **Table 2**), underlined by a significant main effect of stimulus, F(1,29) = 6.60, p = 0.016, η 2 <sup>p</sup> = 0.19. These results indicate that the electrodermal activity did not change significantly with time.

# Extinction

#### Contingency Ratings

As shown in **Table 2**, the CS+ was rated higher than the CS− by both groups at the beginning as well as at the end of the extinction phase, which was confirmed by a significant main effect of stimulus, F(1,26) = 10.6, p < 0.003, η 2 <sup>p</sup> = 0.29. There were no further effects.

#### Startle Response

As **Figure 6** demonstrates, the startle response decreased from the beginning to the end of the extinction phase in both groups (see **Table 2**), which is emphasized by a significant main effect of time, F(1,25) = 27.3, p < 0.001, η 2 <sup>p</sup> = 0.52. The main effect of stimulus, F(1,25) = 5.19, p = 0.031, η 2 <sup>p</sup> = 0.17 shows it is evident that there was a higher startle response caused by the CS+ for both groups during the extinction phase.

CS−, stimulus without negative consequences; Standard, the experimental group which participated in the Standard extinction (SE) process; Gradual, the experimental group which participated in the Gradual extinction (GE) process. Mean contingency ratings are given. Standard errors are presented as error bars.



Means (M), Standard Deviation (SD) and p-Values of the contingency ratings, startle response and skin conductance response are given. Note: Standard Group, the experimental group which participated in the Standard Extinction (SE) process; Gradual Group, the experimental group which participated in the Gradual Extinction (GE) process; n, number of participants; CS+, stimulus (spider) with aversive unconditioned stimulus (US); CS−, stimulus (scorpion) without aversive US. For the contingency ratings in all three phases were n = 15 in the SE and n = 13 in the GE. For the startle response, in the acquisition were n = 15 in both groups, in the extinction were n = 13 in the SE and n = 14 in the GE, and in the reinstatement test were n = 12 in SE and n = 11 in GE. For the skin conductance response, in the acquisition and extinction were n = 15 in SE and n = 16 in GE, and in the reinstatement test were n = 15 in SE and n = 13 in GE.

#### Skin Conductance Response

As it can be seen in **Table 2**, the response increased over time and the CS− caused a higher level of activity than the CS+ during the course of the whole period, which was reflected by a main effect of time, F(1,29) = 8.27, p = 0.007, η 2 <sup>p</sup> = 0.22, and stimulus, F(1,29) = 5.02, p = 0.033, η 2 <sup>p</sup> = 0.15. Means and standard deviations can be seen in **Table 2**. These results do not suggest there was successful extinction.

#### Return of Fear: Reinstatement Contingency Ratings

For both groups, the ratings of the CS+ are significantly higher than for the CS−, which is shown by a main effect of stimulus,

F(1,26) = 12.0, p = 0.002, η 2 <sup>p</sup> = 0.32. No other effects were significant. The ratings remained constant during the period of time when the end of the extinction phase and the reinstatement test are compared. Means and Standard deviations can be seen in **Table 2**. No return of fear was noticeable from the contingency ratings during the reinstatement test.

#### Startle Response

As shown in **Figure 6**, the startle response during the reinstatement test tended to be higher than at the end of the extinction phase for each group and stimulus. The CS+ caused a higher startle response than the CS− for both experimental groups at both times. An ANOVA confirmed a significant main effect of stimulus, F(1,21) = 6.98, p = 0.015, η 2 <sup>p</sup> = 0.25) and a significant Stimulus x Group interaction, F(1,21) = 5.39, p = 0.030, η 2 <sup>p</sup> = 0.21. In order to follow up on the grouprelated interactions, a separate ANOVA for each of the two groups was conducted. For group SE, a significant main effect of stimulus, F(1,11) = 30.5, p < 0.001, η 2 <sup>p</sup> = 0.735 was found. Follow-up t-tests for the startle response showed that the CS+ was significantly higher than the CS−, t(11) = 30.5 p < 0.001, d = 0.69) for the SE group. For the GE group, an ANOVA showed no significant effects. These results indicate that, according to the startle response, more return of fear took place for the

negative consequences; Standard, the experimental group which participated in the Standard Extinction process; Gradual, the experimental group which

participated in the Gradual Extinction process. Mean startle responses are given. Standard errors are presented as error bars.

SE than the GE group. **Table 2** shows means and standard deviations.

#### Skin Conductance Response

The SCR at the end of the extinction phase was higher than at the reinstatement test, underlined by the main effect of time, F(1,26) = 9.93, p < 0.004, η 2 <sup>p</sup> = 0.28 (see **Table 2**). Furthermore, there were no significant differences between the groups or the stimuli. There is no return of fear that can be proven by the SCR data.

## Discussion

The main goal of this study was to apply the findings from Gershman et al. (2013) to a human sample in order to support the notion that GE is a successful method for preventing the return of fear following an extinction procedure. We were able to achieve similar results for the startle variable in the reinstatement test but not for the SCR and contingency rating in our human sample.

In our study, participants showed acquisition effects as reflected in a higher response towards the CS+ compared to the CS− in the contingency and some of the physiological measures. In the startle and SCR response there was a significant difference between the two stimuli over the whole acquisition period. This was expected because the measures at the beginning of the acquisition were conducted in the first four presentations of the acquisition phase and not before the acquisition. The analysis of the contingency ratings reflected a significant difference between the two stimuli only after the acquisition but not before. This was expected as well because the contingency ratings were measured pre- and post-acquisition. Furthermore, as expected, there were no group differences in this phase.

As for the extinction phase, we could see an inhibition effect during extinction in the form of a significant reduction of the startle response. However, this was not restricted to the CS+, the CS− also showed a clear inhibitory effect. Surprisingly, the SCR showed an increase in arousal through the extinction phase (in all stimuli in both groups). The contingency data shows that the CS+ contingency scores were reduced (unlike the CS− scores) but this reduction did not reach significance, possibly due to the small sample size we used. An alternative explanation might be the fact that the US did not induce a strong fear reaction. This was an unexpected result which contrasts with the results of the startle response. Interestingly, there was no significant difference in the extinction phase between the groups even though one of the groups was partly exposed to the US. This is consistent with the results from the study by Gershman et al. (2013).

During the reinstatement phase in the startle response, there was similarly a return of fear reaction in the reinstatement test; however, this effect was evident only for the SE group, just as we expected. The SCR values decreased significantly when comparing the end of the extinction phase and the reinstatement test and there was no significant difference between the two stimuli during the whole phase. Contingency ratings were higher for the CS+ compared to the CS−, but no group differences were evident.

Overall, it was evident from the startle response data that there were less fear responses in the GE group than in the SE group. This corresponds to the results from the study by Gershman et al. (2013), who also found a reduction of the fear response after GE. It is worth mentioning that the dependent variable used in their study was freezing reaction.

Based on this result we suggest that the reduction of the return of fear caused by GE is not restricted to animals but can also be seen at least partly in humans, too. This is an important issue when considering the transfer of results from laboratory research to clinical practice. Extinction serves as the laboratory counterpart of exposure therapy (Hermans et al., 2006). Replication of this effect in further studies could have major implications for the practical treatment of fear related disorder. For example, in social phobia one method is exposure in which a patient is asked to hold a public speech in front of an audience (Anderson et al., 2005). An incorporation of gradual exposure may suggest to gradually reduce the aversive reaction from an audience during exposure and not to expose a participant to a continually friendly audience. A further strength of our paradigm is the use of VR to achieve a high level of standardization during our paradigm. We also wish to emphasize the fact that we measured fear on multiple levels by using startle, SCR and contingency ratings. However, some limitations must be taken into account. Firstly, because not all measurements showed a clear acquisition and extinction effect, we suggest the use of a stronger conditioning procedure (stronger in the sense of being more aversive) as we used an air puff of only 5 bar as US. One possibility is replacing it with electric stimulation. Another advantage of using an electrical stimulation as US is the link between the US and the spider (CS+) because the US can mimic a spider bite. It is also important to discuss the unexpected results from the contingency ratings, for we didn't find significant shifts of the CS+ or the CS− during the extinction or reinstatement phases. We asked the participants to rate the probability that a negative event would occur. An improvement would be to specify this question and to ask them directly to rate the probability that an air puff will occur. Another option would be to increase this contingency from 80–100%. This, however, might influence the extinction procedure, as partial reinforcement leads to a learning effect that is robust to extinction and though slowing the extinction effect (Atkinson et al., 1995). So increasing the expectancy of the US during acquisition might cause an increased and fast extinction effect.

We found conflicting results from the startle and the SCR measures. As Hamm and Weike (2005) pointed out, startle is sensible for measuring fear learning independently from contingency awareness, but SCR requires contingency awareness learning. As we found no extinction in the contingency data we believe this might explain at least partly the lack of effect on the SCR data. We believe that improving the paradigm so that the contingency ratings will reflect a clear extinction and reinstatement effects will probably lead to similar effects in the SCR data. Importantly, our study aimed at transferring the findings from Gershman et al. (2013), who employed ''freezing'' measures to evaluate fear reaction in mice, to a human sample. As Leaton and Borszcz (1985) discuss, startle (in humans) and freezing (in mice) are two highly related measures. This is especially important as startle in humans is also related to learning without cortical involvement. Therefore, it is plausible that we found significant results for the startle measures, too, but not for the SCR measures.

The sample size of this study must also be acknowledged as further limitation. It consisted of 31 participants altogether. More distinctive results should be expected with a larger sample size.

Furthermore, it could be argued that a longer interval between the acquisition and extinction phase might ensure that participants perceive the two phases as separate stages. Using short intervals as we did might inhibit the consolidation of the long-term memory (Myers and Davis, 2007). Future studies might investigate if there is a clearer effect in the extinction and return for fear when the extinction phase follows an extended time interval.

As represented in the introduction, there are many different theories describing the reasons why fear returns after extinction. One prominent theory states that presenting some USs during the extinction phase prevents the formation of a new ''state''. Participants do not learn to create a new CS-no-US-state (Bouton, 2004) but realize instead that there is no fearful event following the CS+. As a result, they no longer perceive the CS+ as fearful, thereby transforming the original fear memory into a no fear memory. A different approach suggests that presenting the US (prior to extinction) serves as a signal which reactivates the fear structure. This renders the memory into a labile state which enables modifications (Schiller et al., 2010). It would be

#### References


interesting to investigate whether GE can induce reactivation of the fear memory similar to what Schiller et al. (2010) suggest, thereby causing fear memory to be extinguished.

In summary, GE seems to be a better alternative to the SE process because it prevents the return of fear. Future studies are needed to replicate and extend this effect. Especially interesting would be the question whether this effect can be seen in other measures than the startle response. A long term goal would be to test the effect of GE in a clinical sample. This could lead to important improvements to the structure of exposure therapy in treating patients with anxiety disorders.

## Author Contributions

YS, study conception, data analysis, wrote the manuscript. JW, study conception, data acquisition and analysis, and contribution to the manuscript. MW, study conception, data acquisition and analysis, and contribution to the manuscript. AM, study conception, data analysis, and contribution to the manuscript. All authors have approved of the final version of the manuscript and its submission.

#### Acknowledgments

We, the authors, would like to specially thank Andreas Ruider for the valuable support in designing and programming the virtual exposure scenario. In addition, we would like to thank Prof. Paul Pauli for sending the email that sparked the idea for this study!


implications for addiction, relapse and problem gambling. Psychol. Rev. 114, 784–805. doi: 10.1037/0033-295x.114.3.784.supp


**Conflict of Interest Statement**: Andreas Mühlberger is stakeholder of a commercial company that develops virtual environment research systems. The other authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Shiban, Wittmann, Weißinger and Mühlberger. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Morphine administration during low ovarian hormone stage results in transient over expression of fear memories in females

Emily M. Perez-Torres <sup>1</sup> , Dinah L. Ramos-Ortolaza<sup>2</sup> , Roberto Morales <sup>3</sup> , Edwin Santini <sup>4</sup> , Efrain J. Rios-Ruiz 1,3 and Annelyn Torres-Reveron1,5 \*

<sup>1</sup> School of Behavioral and Brain Sciences, Ponce Health Sciences University - Ponce Research Institute, Ponce, Puerto Rico, <sup>2</sup> Department of Biology, Pontifical Catholic University of Puerto Rico, Ponce, Puerto Rico, <sup>3</sup> Department of Counseling and Psychological Services, Institute of Translational Research in Behavioral Sciences, University of Puerto Rico at Ponce, Ponce, Puerto Rico, <sup>4</sup> Department of Pharmaceutical Science, Palm Beach Atlantic University, West Palm Beach, FL, USA, <sup>5</sup> Basic Sciences Division, Physiology, Ponce Health Sciences University - Ponce Research Institute, Ponce, Puerto Rico

#### Edited by:

Oliver T. Wolf, Ruhr University Bochum, Germany

#### Reviewed by:

Rebecca M. Shansky, Northeastern University, USA Ekrem Dere, University Pierre and Marie Curie Paris 6, France

#### \*Correspondence:

Annelyn Torres-Reveron, Basic Sciences Division, Physiology, Ponce Health Sciences University - Ponce Research Institute (formerly Ponce School of Medicine), 395 Zona Ind Reparada 2, Ponce, PR 00716-2347, Puerto Rico anntorres@psm.edu

> Received: 21 March 2015 Accepted: 08 May 2015 Published: 22 May 2015

#### Citation:

Perez-Torres EM, Ramos-Ortolaza DL, Morales R, Santini E, Rios-Ruiz EJ and Torres-Reveron A (2015) Morphine administration during low ovarian hormone stage results in transient over expression of fear memories in females. Front. Behav. Neurosci. 9:129. doi: 10.3389/fnbeh.2015.00129 Acute exposure to morphine after a traumatic event reduces trauma related symptoms in humans and conditioned fear expression in male rats. We aimed to determine whether acute administration of morphine alters consolidation of fear learning and extinction. Male and female rats in proestrus and metaestrus (high and low ovarian hormones respectively) underwent fear conditioning and received saline or morphine (2.5 mg/kg s.c.). The next day they underwent extinction. Results showed increased freezing during extinction only in the morphine metaestrus group while morphine did not affect males or proestrus females. Recall of extinction was similar on all groups. On a second experiment, a subset of rats conditioned during metaestrus was administered morphine prior to extinction producing no effects. We then measured mu opioid receptor (MOR) expression in the amygdala and periaqueductal gray (PAG) at the end of extinction (day 2). In males and proestrus females, morphine caused an increase in MOR in the amygdala but no in the PAG. In metaestrus females, morphine did not change MOR expression in either structure. These data suggests that ovarian hormones may interact with MORs in the amygdala to transiently alter memory consolidation. Morphine given after trauma to females with low ovarian hormones might increase the recall of fear responses, making recovery harder.

#### Keywords: fear conditioning, fear extinction, morphine, trauma, sex-differences, estrous cycle

Studies with female and male rats regarding fear conditioning and extinction have demonstrated that ovarian hormones modulate fear acquisition and fear extinction (Milad et al., 2009). The higher fear observed in female in comparison to male rats during extinction recall (Milad et al., 2009), suggests an important role in the ovarian hormones in the higher prevalence of anxiety disorders in women (McLean et al., 2011). In the Pavlovian fear conditioning paradigm repeated pairings of a neutral stimulus such as a tone (conditioned stimulus, CS) with an aversive stimulus such as a mild foot shock (unconditioned stimulus, US) lead to conditioned fear responses such as freezing. However, repeated presentations of the CS in the absence of the US will lead to a gradual reduction in conditioned fear responses commonly known as extinction. Extinction does not erase the initial CS-US association, but is thought to form a new inhibitory memory (Pavlov, 1928; Konorski, 1967). Deficits in fear extinction are thought to contribute to trauma related disorders (Milad et al., 2006, 2009; Glover et al., 2012).

Opiates like morphine, which preferentially bind to the mu opioid receptor (MOR), are one of the first line prescriptions for severe physical traumas, mostly used to ameliorate pain. Despite the large usage of morphine, little is known about the association of mental health disorders and its prescription (Seal et al., 2012). There are several reasons to believe that pharmacotherapy plays an important role in the development of trauma related disorders (Bailey et al., 2013). Clinical studies have found that acute administration of morphine has a protective effect in Posttraumatic Stress Disorder (PTSD) patients by preventing symptoms associated to the disorder and the diagnosis (Bryant et al., 2009). Endogenous opioids may be involved in certain symptoms of trauma related disorders such as numbing, stress-induced analgesia, and dissociation (Holbrook et al., 2010). Taking into considerations these clinical studies, it is evident that not only they support a role of the opioid system in trauma related disorders, but also suggest that this system could be a therapeutic target.

The circuitry of fear learning and extinction has been well mapped. The basolateral amygdala (BLA) is involved in extinction learning by stimulating inhibitory intercalated cell mass activity to inhibit the central amygdala output neurons (McNally and Westbrook, 2003; McNally et al., 2004; Likhtik et al., 2008; Parsons et al., 2010). This intercalated cells are rich in MORs (Likhtik et al., 2008). In addition, there is evidence in humans and animal suggests that estrogens may exert their influence on fear within the amygdala (Jasnow et al., 2006). Estradiol also stimulates the release of endogenous opioid peptides in the medial amygdala (Eckersell et al., 1998). Besides the amygdala, the periaqueductal gray (PAG) matter has an important role in the expression of freezing behavior (Amorapanth et al., 1999). The PAG is rich in opioid receptors (McNally, 2009) and shows sexual dimorphism (Loyd and Murphy, 2006). Opioids within the ventrolateral periaqueductal gray (vlPAG) are necessary for extinction acquisition and blocking MORs in this region prevented acquisition of extinction (McNally et al., 2005). Opioidergic signaling in the vlPAG affects plasticity across the brain circuit responsible for the formation of extinction memory (Parsons et al., 2010).

The aim of this study is to elucidate if acute morphine administration will have a stronger effect in reducing fear conditioning in female rats as compared to male rats. We also aim to know if the behavioral differences will depend on changes in MOR expression in the amygdala and the PAG in response to an opioid agonist.

We used female and male Sprague–Dawley rats (230–300 g in weight) were paired housed under a day-night (12-h) cycle. The rats received free access to food and water throughout the experiment and during at least a one-week acclimation period prior to experimentation, where the rats were handle when performing daily vaginal smears to determine the estrous cycle stage in female rats (Turner and Bagnara, 1971). Only rats with regular, 4-day estrous cycles were included in the study. All procedures were conducted in accordance with and approved by the Ponce Health Sciences University Institutional Animal Care and Use Committee.

Experiments 1, and 2 used four identical conditioning chambers (25 cm × 29 cm × 28 cm, l × w × h; Coulbourn Instruments) located inside of a sound-attenuating box (Med-Associates). The chamber floor consists of 0.5 cm stainless steel bars through which an electric shock is delivered. The chamber is equipped with a speaker and a single overhead light. The auditory tone was a 4 kHz sine wave with duration of 30 sand an intensity of 80 dB sound pressure level (Santini et al., 2004). We assessed the animal in the same conditioning chamber, so the fear expressed will not be specific to the tone, but a combination of fear to the tone and to the context. Males and females were assessed in the same chambers; all boxes were thoroughly cleaned with 70% ethanol between animals. All testing sessions occurred between 9:00 AM and 1:00 PM. The three stages of training were as follows, with the habituation and conditioning performed on the same day: habituation; the animals received one habituation trial (tone alone) in the conditioning chamber with an average intertrial interval (ITI) of 2 min and conditioning; the animals received 3 tone-footshock pairings (0.5 s 0.45 mA) in the box/context and returned to their home cages. A group of female rats was in the proestrus stage of the estrous cycle while another group was in the metaestrus stage during conditioning. Twenty-four hours after conditioning, the animals received extinction and consisted of 12 tone alone presentations. Twenty-four hours after the conditioning a sub group of metaestrus females receive a test that consisted of 2 tone alone presentations. Animals in each experiment group were treated with morphine dissolved in saline (2.5 mg/kg) or saline (0.9%) subcutaneously immediately after conditioning or 4 h before extinction. The morphine dose was chosen as it has been shown to be effective in reducing conditioned fear (Rudy et al., 1999). This dose of morphine administered acutely does not provoke withdrawal symptoms. Studies have demonstrated that there are sex differences in rats in response to morphine antinociceptive activity and differences between men and women analgesia (Cicero et al., 1997; Sarton et al., 2000). However, today is not clear the sex-differences in the pharmacokinetics of morphine. Immediately after experiments rats were anesthetized and decapitated, brains removed and frozen. The amygdala and PAG were dissected based on the atlas of Paxinos and Watson. Equal amounts of protein (100 µg) were used to identify the MORs with a rabbit polyclonal antibody (1:1000, Immunostar). Blots were washed with TBST and incubated in goat anti-rabbit IgG (1:1000, Santa Cruz). Bands were visualized using Chemidoc XRS Imaging System and Image J software (imagej.nih.gov). To perform this, we followed Ramos-Ortolaza et al. (2010) western blot protocol. All samples were run in duplicates with saline and morphine groups within the same gel. Our goal was to look at changes caused by morphine and not absolute changes in protein, therefore data was analyzed as a percent change in morphine-treated samples against control group within a given gel. For a subgroup of gels we calculated GAPDH loading control and we found a 2.3% of variability between wells, which equally affected controls and morphine treated animals.

Freezing time per trial was averaged in blocks of two and converted to percentage. We used repeated measures ANOVA considering treatment (morphine or saline), sex (female or male) and female cycle stage (metaestrus or proestrus) as betweensubject variables. For all experiments, the significance level was set at p < 0.05. Significant interactions were examined using Tukey's post hoc comparisons. The western blot was analyzed using one sample t-test against baseline (100%) for differences against saline control group for each sex group. Differences between sexes for morphine groups were analyzed using ANOVA.

Behavioral results show that for the experiment 1, and 2 when morphine (2.5 mg/kg) or saline (0.9%) where administered subcutaneously immediately after fear conditioning (**Figures 1A–D**) or 4 h prior to extinction (**Figures 1E,F**) there were no statistical differences in the levels of conditioned freezing between groups designated to receive saline or morphine; metaestrus (F(1,27) = 0.412, p > 0.05), proestrus (F(1,23) = 0.009, p > 0.05) and males (F(1,28) = 2.12, p > 0.05; **Figure 1**). However, extinction results showed that administration of acute morphine immediately after conditioning caused an increased level of conditioned freezing in the group conditioned during metaestrus compared to controls (F(1,27) = 25.41, p < 0.01; **Figure 1B**). No significant differences were observed in the extinction session for males (F(1,28) = 1.35, p > 0.05) or in female rats conditioned during proestrus (F(1,23) = 0.74, p > 0.05; **Figures 1C,D**). To further examine our finding in the metaestrus conditioned group, we selected a separate set of animals that was tested for freezing behavior 24 h after fear extinction (Day 3). This test consisted of two tonealone presentations. No effects were observed on day 3 for female rats conditioned during metaestrus (F(1,11) = 0.63, p > 0.05) (not shown), thus they remembered the extinction from previous day. We further analyzed whether the deficit in extinction could be associated with the stage of the cycle alone during extinction (day 2). Most rats (82%) injected with morphine and conditioned during proestrus switched to estrus/metaestrus stages. On the other hand, 79% the animals that were conditioned in metaestrus and treated with morphine stayed in metaestrus and/or switched to diestrus II. Thus, when morphine and saline groups were re-analyzed considering the stage of the cycle at the extinction phase, all rats that receive morphine and were in metaestrus/diestrus II still showed increased fear on Day 2 compared to saline controls. On experiment 2, additional group of metaestrus female rats were given morphine (2.5 mg/kg s.c.) or saline (0.9%) 4 h before extinction (**Figure 1E**). They were then subjected to a two tone-alone test on Day 3. There were no differences in freezing behavior between groups that received saline or morphine; conditioning (F(1,12) = 0.011, p > 0.05), extinction (F(1,12) = 0.47, p > 0.05; **Figure 1F**) and test (F(1,12) = 0.36, p < 0.05; not shown).

To quantify how MOR expression was altered in response to morphine, we dissected the amygdala and PAG of the females and male rats from experiment 1, right after the end of the extinction session on Day 2. Western blot results showed that morphine administration to male rats increased MOR expression in the amygdala (F(1,4) = 3.323, p < 0.05; **Figure 2A**), but did not affect MOR expression in the PAG (F(1,4) = 0.640, p > 0.05; **Figure 2A**). Like in the males, morphine administration increased MOR expression in the amygdala of the proestrus female rats (F(1,4) = 5.165, p < 0.05) and did not affect MOR expression in the PAG (F(1,4) = 0.239, p > 0.05; **Figure 2B**). In contrast, morphine administration to metaestrus female rats did not affect MOR expression in the amygdala (F(1,4) = 0.335, p > 0.05) or the PAG (F(1,4) = 0.065, p > 0.05; **Figure 2C**).

The data gathered in this study show a transient over expression of fear memories in female rats fear conditioned and treated with morphine during the metaestrus stage of the estrous cycle. However, there was no significant difference in the recall of extinction memory on metaestrus females suggesting a transient effect of morphine shortly after its administration. Interestingly, when treating the metaestrus rats with morphine before extinction, no effects on fear were observed. This suggests that the effects of acute morphine in females are linked to the events that start shortly after the trauma occurs but once the memory has been consolidated, morphine will have no effect.

Animal studies suggest that gonadal hormones influence extinction of conditioned fear. Milad et al. (2009) showed that female rats during the proestrous stage of the estrus cycle exhibited better extinction memory during extinction recall test. Also, when estradiol and progesterone is administered exogenously there is a facilitated extinction recall, whereas estradiol and progesterone receptor antagonists impair it. This suggests that gonadal hormones influence the consolidation of extinction memory (Milad et al., 2009). It is important to point out, that our saline animals are consistent from Milad et al. (2009) study, since he also analyzed the cycle stage of the female rat during conditioning and did not see differences in behavior between groups.

Studies indicate that low ovarian hormones are associated with higher fear expression (Milad et al., 2009; Glover et al., 2012, 2013). In our study many rats that were in metaestrus during conditioning continued in the same cycle stage or entered in diestrus II during extinction. We found that the cycle alone cannot explain the over expression of fear because it is not reduced when only the rats that transitioned out of the metaestrus/diestrus stage during the extinction phase were analized. This suggests that the higher fear responses observed during extinction is most likely due to the interaction of morphine with the low estrogen cycle stage during conditioning and not due to the cycle stage during extinction. The fact that a single dose of morphine produced a transient over expression of fear memories in metaestrus when administered immediately after conditioning but not before extinction, supports that morphine's effect is related to memory processes that occur immediately after the trauma. This suggests that ovarian hormones and morphine interact to alter fear memory, when morphine is given close to the trauma event. However, once

morphine is no longer present, animals recover and are able to show fear responses comparable to the animals that did not receive morphine.

Animal studies have examined the effects of acute morphine on male rats, on various fear conditioning behavioral protocols. Glover and Davis (2008) study demonstrated that morphine facilitates extinction in male rats exposed to fear potentiated startle. Although one of the morphine doses they used is the one we used in our study (2.5 mg/kg), the timing of s.c injection of morphine and the differences in the behavioral protocol do not allow us to fully compare their findings to our study. Szczytkowski-Thomson et al. (2013), on the other hand used a single higher dose of morphine (15 mg/kg), which was administered immediately after a stressor to males, but it did not reduce the fear response. In the same study, repeated morphine doses (7.5 mg/kg) cause a decrease in fear (Szczytkowski-Thomson et al., 2013). We decided not to treat repeatedly with morphine due to dependence and withdrawal issues, as we wanted to maintain a clinically relevant study. Our current model was designed considering the translational potential of morphine as a treatment to lower the possibilities of trauma-related symptoms manifestation, but without creating

dependence to the drug. Furthermore, there is no electric shock after the morphine administration in our protocol and this eliminates the possibility of morphine altering pain perception. In addition, post-trial morphine administration using the passive avoidance paradigm, has been shown to produce facilitation of fear memories (Mondadori and Waser, 1979). This study used very high doses of morphine (40 or 100 mg/kg) which may produce withdrawal signs in animals. Unfoltunately, they did not test females. However, we acknowledge the possibility of a post-trial reinforcement effect (Mondadori et al., 1977; Huston and Mueller, 1978), but we are inclined to think that it is related to a morphine-estrous cycle interaction, since the same behavioral outcome was not observed in males or proestrus cycling females. This possibility requires further comparative studies.

Western blots revealed that in response to morphine and extinction, males and proestrus females showed an increase in the expression of MOR in the amygdala, but this was not observed in metaestrus females. This parallels our behavioral findings showing that males and proestrus females had no differences in freezing responses compared to saline controls, but metaestrus females showed over expression of freezing. As a site of initial acquisition of extinction, it might be expected that the BLA is also a site of extinction consolidation. Intercalated (ITC) amygdala neurons constitute the likely mediators of extinction because they receive conditioned stimulus information from the BLA and contribute inhibitory projections to the central nucleus (CEA), the main output station of the amygdala for conditioned fear responses (Likhtik et al., 2008). The majority of MORs are localized in the ITCs (Likhtik et al., 2008). Therefore, we propose that an increase in MOR activity within the ITC might decrease fear response levels to that of saline animals, as observed in this study, but the presence of high ovarian hormones in females and testosterone in males is necessary for this increase to occur. Taken together, these results suggest that MORs plays a role in the molecular events underlying fear extinction. However, given the fact that proestrus and metaestrus female rats differ in their ovarian hormone levels, it is possible that the observed behavioral responses are also influenced by these hormonal differences.

In addition to the amygdala, opioids are released in the ventrolateral PAG when the animals are exhibiting fear to the conditioned stimulus during the early phases of extinction (Parsons et al., 2010). This suggests that the opioid system plays an important role in fear conditioning and extinction. However, we did not see significant results in the expression of MOR when rats were sacrificed immediately after extinction. Two possible explanations for our results are that by the time we sacrificed the animals (after extinction) those changes have already occurred; or the fact that we dissected the whole PAG instead of ventrolateral region alone thus masking what is happening in the ventrolateral PAG per se.

In conclusion, our data suggest that females exposed to trauma during low ovarian hormone stages could be more vulnerable to over-expression of the traumatic memories. There are no data in the literature regarding which brain structures are directly involved in the estrogen-morphine interaction that

#### References


modulate fear. However, our findings are beginning to fill this gap by providing data in two brain structures that are known to have an important role in fear conditioning, the PAG and the amygdala. We suggest that future studies should address in depth the interactions of ovarian hormones and opioid receptors activity that may lead to either protect or exacerbate trauma memories. This study contributes to clarifying the physiological role of morphine in memory consolidation as one of the first providing a description at molecular level in females and highlighting the protagonic role of the amygdala.

#### Acknowledgments

This study was supported by Nova Southeastern University Chancellor's Faculty Research and Development Grant (CFRDG: 335841) to Edwin Santini Ph.D. and Dinah Ramos-Ortolaza Ph.D, the Behavioral Core Facilities (G12RR003050) and Molecular and Genomics Core Facilities (G12 0007579) at Ponce Research Institute (formerly, Ponce School of Medicine and Health Sciences) and start up funds from Ponce School of Medicine to ATR. We would like to thank the undergraduate students: Sonny Defendini and Jose Chamorro (RISE program) for their respective contributions. We would like to acknowledge the useful discussions with Dr. James Porter from Ponce Research Institute.


of postconditioning exposures to morphine in the rat. J. Exp. Psychol. Anim. Behav. Process. 29, 130–142. doi: 10.1037/0097-7403.29.2.130


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Perez-Torres, Ramos-Ortolaza, Morales, Santini, Rios-Ruiz and Torres-Reveron. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Modulation of Fear Extinction by Stress, Stress Hormones and Estradiol: A Review

#### Ursula Stockhorst\* and Martin I. Antov

Experimental Psychology II and Biological Psychology, Institute of Psychology, University of Osnabrück, Osnabrück, Germany

Fear acquisition and extinction are valid models for the etiology and treatment of anxiety, trauma- and stressor-related disorders. These disorders are assumed to involve aversive learning under acute and/or chronic stress. Importantly, fear conditioning and stress share common neuronal circuits. The stress response involves multiple changes interacting in a time-dependent manner: (a) the fast first-wave stress response [with central actions of noradrenaline, dopamine, serotonin, corticotropin-releasing hormone (CRH), plus increased sympathetic tone and peripheral catecholamine release] and (b) the second-wave stress response [with peripheral release of glucocorticoids (GCs) after activation of the hypothalamus-pituitary-adrenocortical (HPA) axis]. Control of fear during extinction is also sensitive to these stress-response mediators. In the present review, we will thus examine current animal and human data, addressing the role of stress and single stress-response mediators for successful acquisition, consolidation and recall of fear extinction. We report studies using pharmacological manipulations targeting a number of stress-related neurotransmitters and neuromodulators [monoamines, opioids, endocannabinoids (eCBs), neuropeptide Y, oxytocin, GCs] and behavioral stress induction. As anxiety, trauma- and stressor-related disorders are more common in women, recent research focuses on female sex hormones and identifies a potential role for estradiol in fear extinction. We will thus summarize animal and human data on the role of estradiol and explore possible interactions with stress or stress-response mediators in extinction. This also aims at identifying time-windows of enhanced (or reduced) sensitivity for fear extinction, and thus also for successful exposure therapy.

#### Edited by:

Oliver T. Wolf, Ruhr University Bochum, Germany

#### Reviewed by:

Raffael Kalisch, Johannes Gutenberg University Medical Center Mainz, Germany Christian J. Merz, University of Trier, Germany

#### \*Correspondence:

Ursula Stockhorst ursula.stockhorst@uniosnabrueck.de

Received: 30 September 2015 Accepted: 14 December 2015 Published: 26 January 2016

#### Citation:

Stockhorst U and Antov MI (2016) Modulation of Fear Extinction by Stress, Stress Hormones and Estradiol: A Review. Front. Behav. Neurosci. 9:359. doi: 10.3389/fnbeh.2015.00359 Keywords: fear extinction, stress, monoamines, glucocorticoids, opioids, endocannabinoids, estradiol, exposure therapy

### INTRODUCTION

Classical fear conditioning (consisting of fear acquisition and extinction) is an influential experimental model to study emotional learning and memory. In fear acquisition, an initially neutral stimulus (cue or context) is paired with an aversive unconditioned stimulus (US) capable of eliciting a fear response (e.g., Phillips and LeDoux, 1992). After several pairings in the acquisition phase, the neutral stimulus becomes a conditioned stimulus (CS), now capable of eliciting a conditioned fear response (CR). Fear extinction is induced when the CS is repeatedly presented without the aversive outcome (US), resulting in a decline of CRs. In rodents, fear responses are typically assessed via freezing behavior (e.g., Bouton and Bolles, 1980; Fanselow and Helmstetter, 1988) or the fear-potentiated startle reflex (e.g., Brown et al., 1951; Davis, 1986). In humans, learning indicators used alone or combined include: skin conductance responses (SCRs), fear-potentiated startle, functional imaging data, as well as subjective ratings of US expectancy and/or CS valence/arousal (e.g., Grillon et al., 2004; Stark et al., 2006; Blechert et al., 2007; Milad et al., 2010; Soeter and Kindt, 2012; Bentz et al., 2013). Fear conditioning is highly adaptive, as it enables the organism to effectively and rapidly learn to predict danger using context information and environmental cues. However, fear learning can become dysfunctional if the organism continues to display fear responses in the absence of danger.

Both, fear acquisition and extinction, are valid to model features of anxiety disorders as well as trauma- and stressorrelated disorders (such as posttraumatic stress disorder, PTSD). Specifically, fear acquisition can serve to model features of the etiology of these disorders. Correspondingly, simple phobias, social phobia, panic disorder, and PTSD are characterized primarily by dysregulated fear responses (Ehlers and Clark, 2000; Parsons and Ressler, 2013). Moreover, these disorders are characterized by deficits in fear extinction. This inability to inhibit fear responses is assumed to largely contribute to the maintenance of anxiety disorders (Lissek et al., 2005; Delgado et al., 2006; Mineka and Zinbarg, 2006; Mineka and Oehlberg, 2008), as well as trauma- and stressor-related disorders (e.g., Ehlers and Clark, 2000; Mineka and Oehlberg, 2008; Cover et al., 2014). PTSD is assumed to be related to and even caused by a failure to consolidate and retrieve memory for fear extinction (Quirk and Mueller, 2008). Correspondingly, patients with anxiety and especially trauma-related disorders show deficits in fear extinction learning and extinction recall (Lissek et al., 2005; Blechert et al., 2007; Michael et al., 2007; Milad et al., 2008, 2009b; Jovanovic et al., 2010; Glover et al., 2011; Norrholm et al., 2011; Inslicht et al., 2013). Moreover, extinction learning also serves as a model for exposure techniques in behavioral therapy (e.g., Milad et al., 2014).

The development of anxiety disorders and especially PTSD can be conceptualized as learning under severe stress. Stress is a state of actual or potential disruption in the individual's internal/external environment registered by the brain and caused by factors we call stressors (Joëls and Baram, 2009). Stress leads to activation of the stress response—including activation of the central and peripheral nervous system and release of neuromodulators, hormones and transmitters—the stress-response mediators—in the brain and periphery. The stress response enables the organism to deal with the challenge by increasing central arousal, mobilizing energy, increasing cardiovascular tone, inhibiting costly processes such as reproduction, feeding, and digestion, and by modifying immune responses (Sapolsky et al., 2000; Chrousos, 2009). Thus, stress and the stress response are important for survival and are adaptive in nature. However, in some circumstances stress may cause pathology, as is the case in PTSD and other trauma- and stressor-related disorders. Traumatic situations not only include specific fear-related behaviors (e.g., flight, freezing) but also a significant amount of the less specific stress response. Thus, understanding how stress, the stress response, and specific stress-response mediators contribute to pathological changes seen in PTSD—such as impaired extinction—is of special importance. Moreover, understanding what conditions allow for normal functioning despite (traumatic) stress could advance our understanding of resilience and advance prevention. Last but not least, PTSD itself is accompanied by heightened stress (Maren and Holmes, 2016) and is also associated with changes of the stress system (e.g., Lupien et al., 2009). Both facts could interfere with the success of extinction-based exposure therapy. Consequently, examining effects of stress and single stress-response mediators on fear extinction could help improve treatment efficacy or even provide new targets for pharmacological treatment.

While anxiety disorders and trauma- and stressor-related disorders have a nearly twofold life-time prevalence in women as compared to men (Kessler et al., 1995, 2005; Tolin and Foa, 2006; Kilpatrick et al., 2013; for an overview, see Cover et al., 2014), there is increasing evidence for a role of the female sex hormone 17β-estradiol (E2) in these sex differences. Interestingly, there is first evidence that the quality of fear extinction is related to estrogen levels as supported by better extinction recall under high as compared to low E2-levels.

In line with the focus of the articles assembled in this Research Topic, we will concentrate on fear extinction and the role of stress and stress-response mediators in animals and in humans thereby also referring to the role of E2.

# FEAR ACQUISITION AND FEAR EXTINCTION

# Fear Acquisition and the Neuronal Fear Circuitry

The fear system can be conceptualized as an adaptive behavioral system that allows the organism to avoid, escape or face environmental threats (Rudy, 2014). The amygdala and its connections play a major role in the regulation of innate fear responses and in fear learning.

Fear acquisition involves an interplay between the basolateral amygdala (BLA), consisting of the lateral nucleus (LA), the basolateral and basomedial nuclei (together also referred to as basal nuclei or basal amygdala, BA), the central nucleus (CE), and the intercalated cell-masses (ITC), located between the BLA and the CE (LeDoux, 2007; Pape and Paré, 2010).

The LA serves as the primary input zone of the amygdala, receiving input from the auditory, visual, olfactory, somatosensory, and nociceptive systems (LeDoux, 2007; Pape and Paré, 2010; Herry and Johansen, 2014). In fear acquisition, information about the CS and the US converges into the LA. The LA is also a necessary site of synaptic plasticity underlying fear learning (LeDoux, 2007; Pape and Paré, 2010; Herry and Johansen, 2014; Tovote et al., 2015) and a main storage site for the fear memory trace (Pape and Paré, 2010).

The LA projects to the basal nuclei, and to the ITC. The ITC inhibit neurons in the CE (Royer and Paré, 2002) and thus prevent the defensive fear responses. The basal nuclei contain two types of neurons: so-called ''fear neurons'' and ''extinction neurons'' (Herry et al., 2008). Fear neurons fire when fear is expressed, and they maintain excitatory projections to neurons in the CE and in the prelimbic cortex. Extinction neurons, on the other hand, are active when fear has been extinguished and they project to the ITC (Rudy, 2014).

The CE is one main output region of the amygdala with projections to subcortical and brainstem areas. It coordinates defensive (fear) responses including freezing and endogenous opioid-mediated analgesia (periaqueductal gray, PAG), and startle reflex potentiation (nucleus reticularis pontis caudalis; Davis, 1992; Sah et al., 2003; Fanselow and Poulos, 2005; Pape and Paré, 2010). The CE is also connected to monoamine systems in the brain, including locus coeruleus (LC; noradrenaline, NA), dorsal/ventral striatum (dopamine, DA), and raphe nuclei [serotonin (5-hydroxytryptamine, 5-HT)]. These neuromodulatory connections enable the amygdala to influence the excitability of large portions of the brain, including many areas lacking a direct connection with the amygdala (Sah et al., 2003; Pape and Paré, 2010; Duvarci and Paré, 2014). Finally, the CE also activates hypothalamic nuclei producing the classical peripheral stress response with increased sympathetic arousal, hypothalamuspituitary-adrenocortical (HPA) axis activation, and increased release of glucocorticoids (GCs) and adrenaline/NA into the bloodstream.

Thus, in the amygdala, fear expression is controlled by a fine interplay of inhibitory and excitatory microcircuits involving the BLA, CE, and ITC (Wolff et al., 2014; reviewed in: Duvarci and Paré, 2014; Dejean et al., 2015). The hippocampus is important for context fear conditioning and for encoding of contextual information in the conditioning situation and is thus assumed to contribute to the context-specificity of fear responses. Accordingly, hippocampus–amygdala interactions are regarded to be important for contextual modulation of fear (e.g., Tovote et al., 2015). Importantly, fear expression is also controlled by the medial prefrontal cortex (mPFC). Neurons in the dorsal part of the mPFC [the prelimbic cortex (PL) in rodents, and dorsal anterior cingulate cortex (dACC) in humans and other primates] promote fear expression (Likhtik and Paz, 2015) and both, BLA and PL/dACC activity, are modulated by the hippocampus (Herry and Johansen, 2014).

# Fear Extinction

While the fear system per se fulfills an adaptive function it can become maladaptive and produce pathologies such as anxiety, stressor- and trauma-related disorders (Rudy, 2014). Accordingly, conditions that improve the extinction of conditioned fear and hinder the return of fear are important in order to develop successful treatments of fear and anxiety disorders.

#### Stages of Fear Extinction Learning and Extinction Memory

In fear extinction, the CS is repeatedly presented without the US and CRs decline. As already suggested by Pavlov (1927), extinction learning does not destroy the original fear memory, but involves new learning. The organism acquires an inhibitory CS-no US association preventing the expression of fear responses (Bouton, 1994, 2002; Mueller and Cahill, 2010). This is supported by the observation that conditioned fear can easily reemerge even after successful extinction (Todd et al., 2014). This is the case when the US alone is presented after extinction (reinstatement), when the CS is presented in a context different from the extinction context (contextual renewal), or just with the passage of time (spontaneous recovery).

Extinction covers three phases (**Figure 1A**): acquisition, consolidation and retrieval of extinction (Quirk and Mueller, 2008; Mueller and Cahill, 2010). Following fear acquisition (**Figure 1B**), extinction acquisition (**Figure 1C**) manifests as the decline of CRs during the initial extinction training session (Quirk and Mueller, 2008). Extinction can be quantified by the amount and speed of this decline. During consolidation, lasting at least several hours (Quirk and Mueller, 2008), cellular signaling cascades progressively stabilize the initially labile extinction memory trace into a consolidated memory (long-term memory; Baldi and Bucherelli, 2015). Extinction retrieval (or recall) is evident when subsequent presentation of the CS triggers retrieval of the extinction memory trace and only low levels of conditioned responding occur. Thus, good extinction recall will manifest in low CRs to the original CS (**Figure 1D**). Poor extinction recall—or return of fear—is evident in high CRs despite successful extinction. This is the case with reinstatement (**Figure 1E**), renewal (**Figure 1F**), and spontaneous recovery (**Figure 1G**). These returnof-fear phenomena are also a challenge for anxiety therapy, such as exposure therapies. Moreover, if the CS is paired with the US after extinction, we usually see a much faster reemergence of the CR (rapid reacquisition, **Figure 1H**).

Of note, the timing of different phases relative to each other is also important. For example, immediate extinction (minutes to hours after fear acquisition) vs. delayed extinction (24 h or more) were shown to produce different effects on return-of-fear phenomena. While in some studies immediate extinction produced less return of fear compared to delayed extinction (Myers et al., 2006), this was not replicated in other studies (Archbold et al., 2010; Golkar and Öhman, 2012). There is even evidence that immediate extinction produces poorer long-term fear reduction and more return of fear (Maren and Chang, 2006; Norrholm et al., 2008; Huff et al., 2009), i.e., an ''immediate extinction deficit'' (for a review, see Maren, 2014).

Importantly, stress, stress-response mediators and sex hormones may influence each of the phases of extinction learning and memory formation differently. Therefore, we will try to systematically review the available literature for effects on

acquisition, consolidation, and retrieval of extinction. Where available, we will also report effects on return-of-fear phenomena and consider timing differences.

Presenting the CS without the US provokes a retrieval of the original fear memory. If the number of CS presentations is limited (1–4) this can trigger reconsolidation of the fear memory rather than acquisition of extinction. As originally described in the late 1960's (Misanin et al., 1968), retrieval temporarily destabilizes the memory trace and makes it more susceptible to disruption. The reactivated memory is then actively re-stabilized (Nader et al., 2000). The increased plasticity is assumed to start about 10 min after retrieval and to last no more than 1 h (Johnson and Casey, 2015). This opens a brief reconsolidation-window during which pharmacological or behavioral intervention may erase the labile memory and prevent a return of fear. Blocking β-adrenergic receptors was found to disrupt reconsolidation in animals (e.g., Debiec and LeDoux, 2004, 2006; Rodriguez-Romaguera et al., 2009). There is also good evidence that β-adrenergic blockade could present a pharmacological tool in humans as well (Kindt et al., 2009; Soeter and Kindt, 2010, 2012; Sevenster et al., 2013; for a review, see Otis et al., 2015), but see also Bos et al. (2014). As a behavioral procedure, extinction training in the reconsolidation window was proposed and repeatedly shown to prevent the return of fear in humans in some studies (Schiller et al., 2010; Agren et al., 2012a) but not in others (Golkar et al., 2012; Kindt and Soeter, 2013). In general, blocking reconsolidation or using the reconsolidation window to modify traumatic memories is a promising technique that might reduce fear expression in patients with anxiety disorders or PTSD (for recent discussions, see Lane et al., 2015; Sandrini et al., 2015). A detailed review of reconsolidation studies is beyond the scope of our review. In the following sections, we will focus on effects of stress, and stress-response mediators on fear extinction. However, we will describe selected results from reconsolidation studies that also include measures of extinction learning and/or memory.

#### Brain Structures and Circuits Involved in Fear Extinction

Extinction, just as fear acquisition, is distributed over a network of structures, mainly covering the BLA, the hippocampus and parts of the mPFC, including the PL and the more ventral infralimbic cortex (IL) in rodents, and the dACC and ventromedial prefrontal cortex (vmPFC) as the corresponding structures in humans. Due to the underlying memory processes (consolidation, retrieval, reconsolidation) the involved neural circuits change over time (Maren and Holmes, 2016). While the same structures are involved in fear acquisition and fear extinction, different sets of neurons are assumed to act through different molecular mechanisms during fear acquisition and extinction (Maren and Holmes, 2016).

Besides fear acquisition, the amygdala is also involved in the acquisition, consolidation and retrieval of extinction (Quirk and Mueller, 2008). The hippocampus is the relevant site to recall contextual information. This becomes relevant because extinguished responses are often renewed in new contexts, differing from the original extinction context. Moreover, hippocampus and prefrontal cortex have a principal role in the regulation of the retrieval of both, fear acquisition and extinction memories (Fitzgerald et al., 2014). Concretely, the IL (in rodents) and the vmPFC (in humans) are vital structures (Tovote et al., 2015) for extinction recall (or fear suppression, Fitzgerald et al., 2014). The IL mPFC integrates CS-information with contextual information from the hippocampus to determine extinction retrieval. Thus, in the extinction context, the IL/vmPFC inhibits amygdala output to reduce the CR. The role of the IL specifically for extinction retrieval is under recent debate. A recent mice study by Do-Monte et al. (2015) showed that optogenetic silencing of glutamatergic IL neurons during cued extinction retrieval (1 day or 1 week after extinction acquisition) did not abolish retrieval. However, silencing IL neurons already during extinction acquisition impaired extinction retrieval on the following day. This supports the conclusion that—while relevant for the formation of extinction memory—the IL is not necessary for the retrieval of cued fear extinction. The one necessary structure for retrieval of cued extinction appears to be the amygdala. Moreover, these results are one example for recent data challenging the ''canonical view'' that dorsal regions (PL/dACC) of mPFC regulate fear expression and ventral regions (IL/vmPFC) fear suppression (for current reviews, see Likhtik and Paz, 2015; Giustino and Maren, 2015).

To conclude, extinction is assumed to involve functional changes in the network of amygdala, mPFC and hippocampus so that extinction networks inhibit fear networks (for an elegant review, see Tovote et al., 2015). For extinction to occur it is critical whether the CS activates the fear neurons or the extinction neurons in the BLA, whether ITC are activated, and whether there is an activation of specific subsets of neurons within the mPFC. Subsequently, when reporting effects of stress and different stress-response mediators on extinction we will consider data on direct effects in these brain structures where available.

# THE ROLE OF STRESS AND STRESS-RESPONSE MEDIATORS IN FEAR ACQUISITION AND EXTINCTION

### Relevance

Acquisition of anxiety disorders and especially of PTSD can be conceptualized as learning under severe stress. Enhanced acquisition of trauma-relevant fear is assumed to precede the development of PTSD (Bowers and Ressler, 2015). Accordingly, stress-enhanced classical fear conditioning is used as an animal model of PTSD (e.g., Rau et al., 2005). Importantly, stress and fear responses share common neural circuits, and the neuronal structures involved in fear acquisition and extinction are also highly sensitive to stress effects (e.g., Lupien et al., 2009).

Studying the effects of stress or stress-response mediators on fear acquisition and extinction thus constitutes a relevant paradigm to examine conditions that enhance or attenuate conditioned fear and fear extinction. This approach also aims at mimicking: (a) the stressful nature of the situation prevailing during the acquisition of the disorder and (b) the stressful nature of being re-exposed to the CS afterwards, e.g., during exposure therapy. Exposing subjects explicitly to additional stress or stress-response mediators within a fear-conditioning paradigm seems to be especially important in studies with humans, since the stimuli used as US are often considerably less stressful than US used in animal experiments. Despite the potential relevance, human studies on the effects of acute stress on fear extinction are still rare (for a review, see Raio and Phelps, 2015).

# Features and Mediators of the Stress Response

Stress is a process that involves multiple changes interacting in a time-dependent manner: with respect to their onset, these changes can be subdivided into a rapid first-wave and a (delayed) second-wave stress response (Sapolsky et al., 2000; Rodrigues et al., 2009). The neurotransmitters and brain areas involved in the stress response are at least partly identical with those involved in the fear circuitry.

The first-wave stress response (**Figure 2**) occurs within seconds and involves enhanced release of (a) monoamine neurotransmitters, i.e., of NA, DA, and 5-HT (Joëls and Baram, 2009), (b) corticotropin-releasing hormone (CRH) from the hypothalamus, followed by increased release of adrenocorticotropic hormone (ACTH) from the pituitary. Moreover, (c) there is a reduction in the release of gonadotropinreleasing hormone from the hypothalamus, and subsequently, a decrease in the secretion of gonadotropins, i.e., luteinizing hormone and follicle-stimulating hormone from the pituitary. Additional changes within the 1-min time frame (Sapolsky et al., 2000) involve the secretion of endogenous opiates, prolactin, glucagon, and growth hormone, as well as arginine-vasopressin


FIGURE 2 | Features of the rapid, first wave and the delayed second wave of the stress response: onset in brain and periphery and the timing of the activation of the mainly involved stress-response mediators (neurotransmitters, neuropeptides, hormones). The concrete time specifications of the stress-induced changes are based on Sapolsky et al. (2000), Joëls and Baram (2009) and Hermans et al. (2014). For glucocorticoids (GCs) and sexual steroids, we differentiate between more rapid non-genomic, and delayed genomic actions. As to the stress-induced changes in the periphery, we only refer to the following, major changes: the rapid activation of the sympathetic nervous system and the corresponding rapid secretion of catecholamines, and to the increase in the secretion of GCs and the decrease in the secretion of sexual steroids from the adrenal cortex. The peripheral changes feedback to the brain and the single stress-response mediators also act on the fear extinction circuits and microcircuits (see "Contributions of specific stress mediators: insights from pharmacological manipulations, transmitter and neuropeptide actions" and "Insights from stress-induction studies triggering the first and second wave of the stress response" Sections). When referring to effects of single stress-response mediators on fear extinction within the text, we also take neuropeptide Y and oxytocin into account due to their role in stress resilience and their impact on fear extinction. The bodily reactions allowing adaptation to the stress challenge are not explicitly addressed in this figure.

and renin. In the periphery, the first-wave stress response includes a rapid increase in sympathetic tone and secretion of adrenaline and NA from the adrenal medulla (Sapolsky et al., 2000).

The second-wave stress response, starting after several minutes, involves (among others) increased peripheral secretion of GCs as a result of the activation of the HPA-axis, and reduced secretion of gonadal steroids (i.e., estrogens and gestagens in the ovaries, and testosterone in the testes; Sapolsky et al., 2000; Joëls and Baram, 2009). This inverse action on the secretion of peripheral GCs and gonadal steroids, and the interaction between HPA and hypothalamus—pituitary—gonadal axis under stress is of interest for the present review.

First- and second-wave changes not only have distinct temporal onsets but also differ in the onset and duration of their effects. In the brain, increased NA, DA, 5-HT, and CRH (acting on CRH-receptor 1) usually act within seconds but their effects subside quickly and rarely outlast the duration of the stressor; GC concentrations in the brain only reach peak levels about 20 min after stressor onset (Joëls and Baram, 2009; Hermans et al., 2014). Genomic actions of GCs (and other steroids) take even longer to manifest (Joëls and Baram, 2009). In fact, the slower genomic GC actions were proposed to actively reverse and normalize the rapid effects of various first-wave stress mediators (Hermans et al., 2014).

The brain also receives feedback from stress-induced changes in the periphery: peripheral catecholamines which cannot cross the blood-brain barrier, activate noradrenergic cells via binding to adrenergic receptors of the vagus nerve. The vagus nerve projects to the nucleus of the solitary tract. From there, projections reach the LC (Miyashita and Williams, 2002; Hassert et al., 2004). The amplified LC-activity increases NA levels throughout the brain, including the BLA (Rudy, 2014). Here, NA binds to adrenergic G-protein-coupled receptors and initiates the cAMP signal cascade (Rudy, 2014). Moreover, heightened central NA levels are related to an improvement in memory storage (Rudy, 2014). GCs readily cross the blood-brain barrier and exert negative feedback at hypothalamic and pituitary sites to reduce CRH and ACTH secretion (de Kloet et al., 2005). GCs also bind to higheraffinity mineralocorticoid receptors (MRs) and lower-affinity glucocorticoid receptors (GRs) throughout the brain, including the amygdala, hippocampus, mPFC, and septum (Joëls and Baram, 2009), where they can influence neural signaling and synaptic plasticity. Thus, they can also influence memory processes (Schwabe et al., 2012), such as those involved in fear extinction.

# Effects of Stress on Learning—Timing is Relevant

#### Overview

Acknowledging the fact that fear extinction covers encoding, consolidation and retrieval, models that studied the impact of stress on memory in other areas—such as declarative memory in humans and avoidance learning in rodents—should also provide predictions for fear extinction: these models reveal that the timing of the stressor relative to the learning task is important. Moreover, acknowledging the complexity of the multiple stressresponse mediators over time, we will differentiate between mediators of the first wave and those of the second wave of the stress response.

#### The Role of Timing

Models derived from studies with inhibitory avoidance learning and object recognition (in animals) and declarative memory (in humans) suggest that stress effects on memory depend on the temporal proximity between learning and stressor: encoding close to the stressor (during the first wave of the stress response) will be enhanced, but encoding and recall later in time (during the second wave) will be suppressed (e.g., Schwabe et al., 2012).

In detail. Shortly after stress synergistic actions of rapid NA and non-genomic GC (as part of the first-wave stress response) in the BLA promote the encoding of emotionally relevant information by enhancing synaptic excitability and long term potentiation (Joëls et al., 2011), involving mainly glutamatergic actions. Slower genomic GC effects (as part of the second-wave stress response) will then enhance the consolidation of the material learned under stress. If however, the stressor is placed long before learning (i.e., stressor and learning are ''out of sync''), stress should suppress the encoding of new material and impair the retrieval of previously learned material via second-wave stress mediators (such as genomic GCs).

There is evidence for the validity of these predictions from animal (Roozendaal et al., 2006b; Liebmann et al., 2009; Pu et al., 2009; Karst et al., 2010) and human studies (Henckens et al., 2010; Zoladz et al., 2011).

In animal pharmacological fear conditioning studies, reviewed by Rodrigues et al. (2009), evidence suggests that the first-wave stress mediator NA plays an important role in enhancing fear acquisition, whereas GCs as second-wave stress mediators enhance the consolidation, but have no effect on the acquisition of fear. The relevance of this model for fear conditioning in humans has to be tested.

# MODIFICATION OF FEAR EXTINCTION BY STRESS-RESPONSE MEDIATORS AND RELATED PEPTIDES

We will subsequently review the impact of stress on fear extinction (acquisition, consolidation, and retrieval). First, we will summarize data from pharmacological manipulations that use drugs that affect the brain's monoaminergic, opioid and cannabinoid system, as well as CRH and GCs thus specific mediators involved in the first-wave and second-wave stress response. We here also include two peptides that might contribute to stress resilience (see ''Further mediators: Neuropeptide Y [NPY] and Oxytocin'' Section). Second, we will address stress-induction studies that trigger the first and second wave of the stress response (see ''Insights from stressinduction studies triggering the first and second wave of the stress response'' Section).

Modulation of fear extinction is also obtained via rapid glutamatergic and GABAergic transmission as well as cholinergic activity. Due to our focus on neurotransmitters and peptides that are involved in the stress response, we do not review these effects here. These systems are (only implicitly) included when we refer to the glutamatergic or GABAergic mediation of extinction processes.

# Contributions of Specific Stress Mediators: Insights from Pharmacological Manipulations, Transmitter and Neuropeptide Actions

#### Mediators of the First-Wave Stress Response **Monoamines**

Overview. There is increasing evidence that the monoamines, i.e., NA, DA, and 5-HT, which are involved in the (firstwave) stress response (see ''Features and mediators of the stress response'' Section), also have roles in fear extinction: projections from noradrenergic, dopaminergic, and serotonergic fibers reach the main structures of the fear extinction network, the amygdala, the hippocampus, and the prefrontal cortex. Accordingly, administering agonists and/or antagonists of the respective monoamine transmitter in a fear extinction setting can elucidate the role of these monoamines. Moreover, experimental trials administering the respective drugs and testing their impact on extinction can serve as an experimental model for examining effects of the combination of drug therapy and behavioral exposure therapy (i.e., extinction learning). We will thus review animal data, preclinical data from healthy humans and clinical evidence from patient studies that address the therapeutic effects of the corresponding pharmacotherapy—also as an adjunct to cognitive behavioral therapy (CBT)—in anxiety disorders and/or PTSD (see also the excellent reviews of Fitzgerald et al., 2014; Bowers and Ressler, 2015; Singewald et al., 2015).

Noradrenaline (NA). NA was validated as a memory-enhancing transmitter in many studies on emotional memory, where βadrenergic stimulation improved memory for emotional material (e.g., Cahill et al., 1994) interacting with arousal at encoding (Cahill and Alkire, 2003). Thus, it is reasonable that NA is also a key modulator of extinction learning and memory. A number of results support this:

The LC is the main localization of the cell bodies of NAproducing cells and innervates the neural structures of the extinction network, i.e., the amygdala, the hippocampus, and the prefrontal cortex (Mueller and Cahill, 2010). Mechanistically, NA acts to increase cellular excitability, and enhances synaptic plasticity within extinction-related circuits, and fear extinction was shown to require β-adrenergic activation in the IL-mPFC (Mueller and Cahill, 2010). Conditioned stimuli evoke NA release during extinction and strengthen extinction memory via β-adrenergic signaling (Mueller and Cahill, 2010). Moreover, GC-induced improvement of memory was shown to be mediated by β-adrenergic signaling (Quirarte et al., 1997; Roozendaal et al., 2002).

Animals: animal studies provide extensive evidence for a role of central NA in the acquisition of fear extinction. Selective forebrain NA-depletion prior to conditioning impaired fear extinction learning but not fear acquisition (Mason and Fibiger, 1979), suggesting that NA availability is needed for successful extinction learning. Lesions of the LC—the primer source of forebrain NA—also impaired extinction of the nictitating membrane reflex of the rabbit and the extinction deficit was positively correlated with the amount of NA depletion (McCormick and Thompson, 1982). Further support for a role of NA in enhancing extinction learning comes from studies with administration of the α2-receptor antagonist yohimbine, which leads to an increased presynaptic NA release. Yohimbine before extinction training reduced conditioned responses and enhanced the rate of fear extinction for cue (Cain et al., 2004; Mueller et al., 2009) and context fear conditioning (Cain et al., 2004).

A number of studies also suggest that central NA may enhance the consolidation and/or the recall of extinction memory: systemic yohimbine before extinction led to a superior extinction recall for cue (Cain et al., 2004; Morris and Bouton, 2007; Hefner et al., 2008) and context conditioning (Cain et al., 2004). However, yohimbine enhancement of fear extinction is context specific and does not prevent fear renewal (Morris and Bouton, 2007). Repeated systemic administration of the β-adrenergic agonist isoproterenol facilitated the consolidation of fear extinction, when administrated after each of three context exposure sessions (Do-Monte et al., 2010). A role for NA in consolidation processes of fear extinction is also supported by studies with central NA administration: NA-infusions into the right BLA, immediately (but not 3 h) after contextual fear extinction improved extinction recall at a delayed test, but NA without extinction training had no effect (Berlau and McGaugh, 2006). Other studies (Chai et al., 2014) suggest that the time window for NA effects on extinction might be different in different brain structures: Here, infusions into the CA1 area of the dorsal hippocampus improved long term extinction recall if NA was administered either immediately or 12 h after contextual fear extinction (Chai et al., 2014).

An increase of NA levels in the brain can be also achieved via electrical vagus nerve stimulation (VNS): in four experiments Peña et al. (2013) showed that VNS during CS-exposure accelerated extinction acquisition of both recent (24 h) and remote (2 weeks) fear memories. Consequently, VNS paired with the CS yielded better extinction recall compared to sham stimulation and unpaired VNS. In a newer study (Peña et al., 2014) VNS potentiated the IL-BLA pathway, thus contributing to enhanced extinction. Consistently, surgical vagal deafferentiation impaired cued fear extinction learning and was associated with decreased levels of NA and increased GABA-levels in the ventral PFC (Klarer et al., 2014).

For now, it is not entirely clear which adrenergic receptors mediate the effects of NA on extinction learning and memory. Some studies suggest central β-adrenergic receptors: repeated (but not single) systemic administration of the β-receptor blocker propranolol impaired both, the acquisition and the consolidation of contextual fear extinction (Do-Monte et al., 2010). Blocking β-adrenergic receptors in the IL PFC, impaired cued extinction recall without affecting responding during extinction learning (Mueller et al., 2008). Propranolol only impaired extinction recall when administered 10 min before extinction training, but not when administered immediately after extinction (Mueller et al., 2008), suggesting that β-adrenergic signaling in the IL during extinction training is needed to boost extinction memory formation. The delayed (12 h) enhancement of contextual extinction recall by NA in the CA1 region of the hippocampus (Chai et al., 2014) was also dependent on β-adrenergic receptors, activating the protein kinase A/cyclic adenosine monophosphate response element-binding protein (PKA/CREB) signaling pathway and increased GluR1 membrane trafficking. This supports a role for NA in the late stages of hippocampal memory formation. Central NA infusions also increased the excitability of IL pyramidal cells, and the effect was blocked by propranolol or Rp-cAMPS (a competitive antagonist of cAMP at the PKA binding site), suggesting that NA increases IL-excitability in a β-receptor and PKA dependent manner (Mueller et al., 2008).

NA effects may also be co-mediated by α-adrenergic receptors: the α1-adrenoceptor antagonist prazosin administered systemically after each extinction session over 6 days significantly slowed the reduction of conditioned freezing (Bernardi and Lattal, 2010). There is also some evidence against a role for central β-adrenergic receptors in NA-enhancement of fear extinction: systemic propranolol before extinction reduced CRs during extinction, but the reduction was significant from the first trial of extinction and there were no group differences at a recall test (Rodriguez-Romaguera et al., 2009). This is consistent with a reduction of fear expression (but not an enhancement of fear extinction). In the same study (Rodriguez-Romaguera et al., 2009), propranolol also reduced the firing rate of neurons in the PL mPFC, a region associated with fear expression (Sotres-Bayon et al., 2012; Courtin et al., 2014). One contextual fear conditioning study reports no effect on extinction recall in rats after bilateral BLA infusions of NA immediately after training (Fiorenza et al., 2012), and an improvement after the β-adrenoceptor antagonist timolol. In the same study (Fiorenza et al., 2012), bilateral infusions of NA into the vmPFC (incl. IL) impaired, whereas timolol improved contextual extinction recall.

Humans: there are only few studies investigating NA effects on fear extinction in humans. Healthy humans: in one study (Bos et al., 2012) propranolol administration before extinction impaired extinction learning in US-expectancy ratings, but not in SCRs or fear-potentiated startle. Reboxetine (a NA reuptake inhibitor, increasing central NA levels) administered after extinction acquisition had no effect on the consolidation of extinction memory (Lonsdorf et al., 2014).

Patients: in a sample of chronic PTSD-patients and traumatized controls, propranolol had no effect on delayed extinction recall when administered after extinction training (Orr et al., 2006). Finally, yohimbine was tested as adjunct to facilitate exposure therapy for anxiety disorders. Here yohimbine (vs. placebo) significantly improved the outcome of exposure therapy when combined with in vivo exposure for claustrophobia (Powers et al., 2009) and social anxiety disorder (Smits et al., 2014). Yet, in a virtual reality exposure therapy for specific phobia (fear of flying) yohimbine had no effect (Meyerbroeker et al., 2012).

In sum: animal data strongly suggest that NA can enhance extinction learning and extinction memory processes. Although NA effects were traced down to β-adrenergic signaling in extinction-relevant structures (BLA, IL-mPFC, hippocampus), the critical timing of effects and the relevant brain and molecular mechanisms are still a matter of debate, as is the contribution of other adrenergic receptor types (e.g., α1). In humans, there is very little data showing both, positive and negative results, which precludes clear conclusions. Given the strong animal evidence, more human studies on NA effects in extinction are warranted.

Dopamine (DA). There is recent evidence that long-term consolidation of extinction memory involves dopaminergic signaling. The mPFC receives dopaminergic projections from the ventral tegmental area and these projections contribute to extinction via stimulating GABA-ergic cells in the amygdala (Fitzgerald et al., 2014). This should contribute to reduced amygdala activity and diminished expression of fear-related behavior. The role of DA (and also NA) for extinction is supported by animal data showing that cued fear extinction training in rats resulted in a concomitant increase of extracellular DA and NA in the mPFC (Hugues et al., 2007). In general, it is expected that extinction recall is improved by DA, but a number of procedural and pharmacological features are relevant.

The role of DA in extinction is typically examined by systemic administration of L-dopa (as the DA precursor), and DA agonists or antagonists, as outlined for some important animal studies and the available human studies.

Animals and humans: healthy individuals: Haaker et al. (2013) used L-dopa (vs. placebo) immediately after extinction acquisition in mice (three experiments) and humans (one experiment). To reveal DA's role on extinction consolidation, return-of-fear phenomena were assessed. Mice: in a contextconditioning paradigm (experiment 1), L-dopa (as compared with placebo) led to lower spontaneous recovery 1 day, 7 days and 30 days after extinction acquisition (''extinction short'', 12 CS exposures), suggesting that L-dopa improves extinction recall. When the extinction phase was extended (''extinction long'', 30 CS exposures, experiment 2), spontaneous recovery (1 day and 38 days after extinction) was not diminished, but reinstatement (day 39) was significantly lower after L-dopa. Using cue conditioning, L-dopa (vs. placebo) resulted in less renewal (day 8) in the original acquisition context, a higher vmPFC-activity and lower amygdala-activity. These data further indicate that context-dependent extinction became contextindependent via L-dopa. Humans: In the human study, cued fear acquisition and extinction acquisition were learned on the same day 1, followed by L-dopa (vs. placebo). One day later, L-dopa-treated subjects showed less behavioral renewal and a corresponding change in the neural fear circuitry with an increase in vmPFC- and a decrease in amygdala-activity. In a very recent human study (Haaker et al., 2015), with fear acquisition and extinction now conducted on separate days, post-extinction L-dopa only affected the neuronal indicators (downregulation of amygdala activity and upregulation of vmPFC activity) but not the behavioral level, i.e., SCR, during spontaneous recovery and reinstatement, now 8 days after extinction.

Animals: in line with the positive role of L-dopa for extinction memory, systemic administration of the NA- and DA-reuptake inhibitor methylphenidate (commonly prescribed for attention deficit hyperactivity disorder), enhanced extinction acquisition in a contextual fear paradigm when administered immediately before or after extinction training (Abraham et al., 2012). Extinction was tested for another three extinction sessions on separate days. Delivering methylphenidate 4 h after extinction training was not successful in improving extinction retention. Besides timing, substance type and dose have to be considered: Using amphetamine—which inhibits the reuptake of NA and DA non-selectively—prior to extinction, did not affect extinction acquisition of cued fear when ultra-low doses were used (Carmack et al., 2010), but was effective when given in higher doses (Mueller et al., 2009).

Considering the role of DA-receptor subtype actions, there was some initial evidence for a differential effect of D1 vs. D2-receptors: unexpectedly, antagonizing D2-receptors (via sulpiride), improved extinction memory whereas a D2 agonist (i.e., quinpirole), partially blocked extinction memory compared with the placebo controls (Ponnusamy et al., 2005). But a very well-controlled study (Mueller et al., 2010) meanwhile revealed that D2-receptor signaling is even necessary for extinction consolidation: the D2-antagonist raclopride which has a higher specificity and is more potent than sulpiride, was given systemically and locally (IL PFC) prior to extinction training. D2-antagonizing impaired not only extinction acquisition (day 2) but also extinction recall in a drug-free state (day 3). The authors even controlled for putative motor deficits provoked by D2-antagonists by circumventing its systemic action via direct administration into the IL PFC and found a reduced responsiveness of IL neurons to the CS after raclopride, assessed via single-unit recording in the IL PFC. This supports the interpretation that D2 receptors facilitate extinction by actions on IL PFC neurons that consolidate extinction.

Moreover, genetic differences in the dopamine transporter genes have to be taken into account (Agren et al., 2012b).

Interestingly, there is first evidence that estrogen is involved in modifying the efficacy of DA activity in the vmPFC, and thereby might affect extinction recall: D1-receptor activation in rats in a low estrogen status was able to rescue the extinction impairment found in these low-estrogen status rats (Rey et al., 2014). On the other hand, the initially better extinction recall in the highestrogen rats diminished under the same D1-receptor agonist. These data reveal that the natural estrogen level and DA interact in a cycle-phase dependent manner and that a sex-hormonerelated deficit can be compensated by actions of a D1-receptor agonist.

In sum: data support the idea that dopaminergic signaling does improve extinction memory, but that the drug's dose, the temporal distance between acquisition and extinction, the duration of the initial extinction acquisition, the temporal distance between DA manipulation and testing, and female estrogen status has to be taken into account. DA-agonists might be useful as an adjunct to CBT.

Serotonin (5-hydroxytryptamine, 5-HT). Amygdala, hippocampus, and mPFC contain 5-HT and 5-HT receptors. Interestingly, fear conditioning per se (CS and US presentation) was shown to induce 5-HT increase: in cued fear conditioning, the aversive US led to an increase of extracellular 5-HT (and also DA and glutamate) in the BLA (Yokoyama et al., 2005). While the glutamate increase occurred only during acquisition (day 1), 5-HT and DA also increased as a response to the CS during extinction (day 2), although to a smaller amount and for a shorter duration (Yokoyama et al., 2005). Moreover, CS-exposure in fear-conditioned rats also induced a 5-HT release in the prefrontal cortex (Yoshioka et al., 1995). In vitro electrophysiological recording under administration of escitalopram, a selective serotonin reuptake inhibitor (SSRI), increased both, firing rate and bursts of dopaminergic neurons in the mPFC (Schilström et al., 2011).

The serotonergic system covers seven receptor families (5-HT1–5-HT7) with a total of 16 receptor subtypes with complex actions (Singewald et al., 2015). Research in fear extinction mainly addresses the role of three receptor subtypes: 5-HT1a, 5-HT<sup>2</sup> and 5-HT3. 5-HT1<sup>a</sup> and 5-HT<sup>2</sup> receptors have been shown to be involved in the regulation of the excitability of the LA and the mPFC via activation of pyramidal cells and GABAergic interneurons (Singewald et al., 2015) and to contribute to an improvement of extinction memory. For 5-HT<sup>3</sup> receptors, there is some evidence, that antagonists improve extinction (Singewald et al., 2015).

Studies addressing the role of 5-HT on fear extinction typically use SSRIs, SSNRIs (i.e., selective serotonin and noradrenaline reuptake inhibitors), or amphetamine, a less selectively-acting monoamine agonist that stimulates the release of all three monoamines, NA, DA, and 5-HT. Again, some exemplary animal studies and the available studies in humans are described.

Animals: in animals, chronic administration (21 days) of the SSRI fluoxetine (vs. placebo) after fear acquisition but prior to extinction acquisition was effective to improve extinction recall and to reduce return of fear (lower spontaneous recovery; Deschaux et al., 2011). Moreover, fluoxetine given after extinction acquisition prevented reemergence of fear when rats experienced a stressor (weak intensity shock) as a re-inducer of fear (Deschaux et al., 2013). Venlafaxine, a SSNRI, administered prior to extinction acquisition also improved extinction recall and diminished return of fear (Yang et al., 2012), while an effect on extinction acquisition was not found. This suggests a special impact of venlafaxine on extinction recall, not extinction acquisition. But substancespecific effects have to be taken into account: Chronic (22 days) administration of citalopram (which increases synaptic availability by binding to the 5-HT transporter protein) prior to extinction acquisition (Burghardt et al., 2013), resulted even in an impairment of extinction acquisition, accompanied by a downregulation of a specific subunit of the NMDA (N-methyl-D-aspartate)-receptor (NR2B) in the lateral and basal nuclei of the amygdala. Subchronic (9 days) administration did not induce this impairment.

SSRIs (although not all types) could be an important adjunct for success of exposure therapy, as evident in rats in a study comparing the effect of fluoxetine alone (vs. placebo), extinction training alone and the combination of both (Karpova et al., 2011): only the combination of chronic fluoxetine treatment (2 weeks after acquisition) and extinction training led to less return of fear, i.e., less renewal and reinstatement. On the neuronal level, fluoxetine increased synaptic plasticity in the BLA and the CA1 region of the hippocampus, related to a higher local action of brain-derived neurotrophic factor (BDNF). Thus, the authors conclude that erasure of fear needs ''synergy'' of pharmacological and behavioral treatment. However, one has to keep in mind, that fluoxetine was not only delivered prior to extinction and during extinction acquisition, but also during extinction recall testing. Thus, from this study, it remains unclear whether fluoxetine protection against return-of-fear phenomena (here: renewal, reinstatement) will also appear when being off the drug during recall testing. So far, effects might also be explained by statedependency.

There is first evidence that sex and estrogen levels interact with the serotonergic system (Lebrón-Milad et al., 2013) in fear extinction: Chronic (14 days), not acute, fluoxetine following fear acquisition and preceding extinction acquisition improved extinction acquisition and extinction recall in female, not in male rats. In females, reduction of freezing was better when they were in a cycle phase with low as compared to high estrogen levels during extinction acquisition, suggesting that SSRIs might be helpful in women under cycle conditions of low estrogen levels where extinction recall is usually impaired (see ''Sex, sex hormones and fear extinction'' Section).

Humans: data in humans using fear acquisition and extinction measures is sparse so far. In healthy humans, 2 weeks of pretreatment with the very specific SSRI escitalopram (vs. placebo) prior to fear acquisition resulted in less SCR-responding to the CS+, i.e., improved extinction acquisition while fear acquisition was not modified (Bui et al., 2013).

Patients: there are several studies on the combinatory effect of serotonergic drugs and CBT, but the effects are validated by clinical ratings, not by fear conditioning (for reviews, see Bowers and Ressler, 2015; Singewald et al., 2015). So far, results are diverse, by either showing a positive effect of CBT only, of drug only, or a beneficial effect of the combination of both. But more data and especially experimental designs that really address both factors, drug and extinction (or exposure therapy) within the same study are needed.

In sum: the effects of serotonergic drugs on fear extinction vary with a number of factors: 5-HT receptor subtype, the actions of the drug on the presynaptic vs. postsynaptic binding sites (Bauer, 2015), acute vs. chronic administration, and the learning phase that is targeted (fear acquisition vs. fear extinction). Improvement of extinction (with extinction recall being more affected than extinction acquisition) was obtained mainly under chronic as compared to acute administration of 5-HT agonists given prior to extinction.

#### **Opioids**

Endogenous opioids (including endorphins, enkephalins, dynorphins and hemorphins; Singewald et al., 2015) are also involved in improving extinction learning (Quirk and Mueller, 2008). An increase of endogenous opioids is among the quick (<1 min) changes after stressor exposure (e.g., Sapolsky et al., 2000). Thus, a stress-mediated increase of opioids might participate in stress-mediated improvement of extinction, mainly via stressors affecting the first-wave stress response.

Opioid receptors (µ-, κ-, and δ receptors) are expressed in extinction-relevant brain areas, i.e., in amygdala, prefrontal cortices, and the hippocampus (for a review, see Singewald et al., 2015). Interestingly, ITC show a high expression of µ-opioid receptors (Blaesse et al., 2015), further suggesting that µ-opioid signaling might contribute to an improvement of fear extinction. The ventrolateral PAG (vlPAG) is a main site of the opioid actions during extinction learning, and vlPAG opioids are assumed necessary for extinction learning to occur. A very convincing interpretation for the involvement of opioids in extinction learning is that omission of an expected aversive US is rewarding (Quirk and Mueller, 2008).

Studies addressing the role of opioids for fear mainly rely on the administration of opioid antagonists, as illustrated by a few animal studies and studies in humans.

Animals: in rats, the administration of the µ-receptor opioid antagonist naloxone impaired within-session extinction of fear, while the impairment did not occur when naloxone was administered after extinction training (McNally and Westbrook, 2003). Facilitation of fear extinction was obtained by withinvlPAG-infusion of RB101(S), a drug that inhibits encephalin degrading enzyme, but not when infused outside the vlPAG (McNally, 2005).

Interestingly, there are interactions between morphine and female sex-hormone levels in fear extinction (Perez-Torres et al., 2015): subcutaneous morphine injections immediately after fear acquisition (day 1) led to impaired extinction acquisition (day 2) when female rats were in a cycle phase of low estradiol and low progesterone (P4) level (metestrus)—as compared to females with high levels of ovarian hormones and males. Moreover, in the metestrus group, there was no increase of µ-opioid receptor expression in the amygdala. Group differences were not found when morphine was injected only at the next day (day 2), 4 h prior to extinction. Thus, in a state of low-level ovarian hormones, morphine administered immediately after a trauma (fear acquisition) might unintendedly even increase the recall of fear.

Humans: in humans, extinction deficits were found under genetic variations of opioid-receptor expression: subjects with a single nucleotide polymorphism in the gene encoding prodynorphin (acting on the κ-opioid receptor), exhibited impaired fear extinction and reduced functional connectivity between amygdala and vmPFC (Bilkei-Gorzo et al., 2012).

Patients: Arntz et al. (1993) examined the effect of the blockade of opioid transmission via the opioid antagonist naltrexone in 48 spider-phobics that underwent a 2 h exposure session given low-dose, or high-dose naltrexone, or placebo. Behavioral avoidance measures, emotional, physiological and cognitive measures were assessed prior to, during, and one week after exposure. Opioid blockade resulted in a dose-related relapse of behavioral avoidance while the other measures did not differ. Morphine was also found to be important for the secondary prevention of PTSD (for a review, see Bowers and Ressler, 2015): children traumatized by acute burns developed less PTSD symptoms when given morphine post trauma (e.g., Stoddard et al., 2009). Moreover, the morphine dose in the 48 h after trauma was one of the predictors of PTSD severity 3 months after the trauma: patients with the later diagnosis PTSD had obtained less morphine in the aftermath of the traumatic injury (Bryant et al., 2009). Based on the data from Perez-Torres et al. (2015), estradiol levels should be taken into account and be validated for PTSD immediately after trauma exposure.

In sum: data converge to reveal an impairment of extinction via opioid antagonists and evidence for an improvement of symptoms in trauma-related disorders when morphine is given after the trauma. But the exact timing of opiate administration and/or of opioid action as well as the cycle phase in females has to be taken into account. With regard to its pain-inducing effects, the cold pressor test (CPT) used as a means to induce acute stress in fear conditioning studies in humans (see ''Insights from stressinduction studies triggering the first and second wave of the stress response'' Section), is a very interesting candidate because it does not only provoke a rapid increase of NA, but should also affect the opioid system.

#### **Cannabinoids**

Cannabinoid-type 1 (CB1) receptors have a high density in amygdala and hippocampus. The endogenous production of cannabinoids via the endocannabinoid system is regarded to be relevant for emotional and cognitive processing of threatening stimuli (Bitencourt et al., 2008). As recently reviewed (Quirk and Mueller, 2008; Fitzgerald et al., 2015; Papini et al., 2015), there is evidence for a facilitating role of endocannabinoid signaling in extinction learning via the CB1-receptor. A role of endocannabinoids (eCBs) for improving fear extinction is also supported by increased levels of eCBs in the BLA after extinction training (Marsicano et al., 2002). Moreover, eCBs and CB1 receptors were shown to induce depression of GABA-mediated inhibitory currents (Marsicano et al., 2002).

Importantly, stress and GCs influence eCBs levels (reviewed in Maren and Holmes, 2016): stress and GCs increase eCBs in the BLA, and eCBs act on brain GCs and on the HPA axis (Hill et al., 2010). eCBs were also shown to contribute to an NA-induced improvement of extinction memory by the following GC-related actions: An increase of GCs (induced in an emotionally aversive situation) at the brain GRs was shown to activate pathways that induce eCB synthesis. eCBs then inhibit GABAergic neurons, and thus disinhibit NA release (Atsak et al., 2012)—or in other words, increase NA recruitment. Thus, eCBs might contribute to an NA-mediated improvement of extinction memory after preceding GC increase.

The role of eCBs in fear extinction is typically examined using transgenic mice lacking the CB1-receptors, or by administering CB1 agonists and antagonists or inhibitors of enzymes that are involved in the reuptake or breakdown of eCBs (for a review, see Papini et al., 2015).

Animals: CB1-knockout mice show normal acquisition of fear conditioning (freezing behavior), but impairment of extinction acquisition and extinction retention suggesting a role of CB1 for extinction, not acquisition of fear (Marsicano et al., 2002). Accordingly, blockade of the CB1 receptor was shown to impair extinction learning, while administration of CB1 agonists and CB reuptake inhibitors improved extinction in animals in a number of studies as specified below:

As to CB1-antagonists, SR141716A (Marsicano et al., 2002) impaired extinction acquisition and recall, but not fear acquisition. Similarly, the CB1-antagonist rimonabant dosedependently reduced extinction learning of fear-potentiated startle (Chhatwal et al., 2005). Correspondingly, CB1 agonists improved extinction recall: the CB1-agonist phytocannabinoid cannabidiol (Bitencourt et al., 2008), and AM404, an eCB reuptake inhibitor injected prior to extinction acquisition, facilitated extinction recall (Chhatwal et al., 2005; Bitencourt et al., 2008) and reduced reinstatement (Chhatwal et al., 2005). AM-3506 (preventing the degradation of anandamide), given either systemically or infused locally into the amygdala prior to extinction, also resulted in better extinction recall, while it had no effect in the absence of extinction training (Gunduz-Cinar et al., 2013).

Evidence for the local action of eCB in fear extinction circuitry is further supported by results that pre-extinction infusions of CB antagonists into fear-relevant brain areas impaired extinction, while pre-extinction infusion of agonists improved extinction recall, but results are still mixed (Papini et al., 2015).

Humans: Klumpers et al. (2012) compared effects of ∆-9-tetrahydro-cannabinol (THC) vs. placebo administered prior to extinction acquisition in healthy humans: they found an improvement of extinction acquisition (lower number of CRs) in SCRs, but not in fear-potentiated startle; the improvement did no longer remain in the retention test. But effects on extinction recall were found in other studies (Das et al., 2013; Rabinak et al., 2013, 2014): when administering cannabidiol, a CB1-receptor agonist and non-psychoactive component of cannabis (vs. placebo), either prior to or following extinction acquisition (Das et al., 2013), extinction acquisition was not affected, but extinction recall improved (manifesting in expectancy ratings) when cannabidiol had been administered after extinction acquisition, thus suggesting an effect on consolidation of extinction memory. Less reinstatement (trend level in the SCRs) was found after both time-points of eCB-administration, prior and after extinction training. Longer-lasting effects on extinction recall (day 3) were also observed after a single oral dose of the synthetic form of ∆-9-THC, dronabinol (vs. placebo) administered prior to (not after) the extinction acquisition (day 2; Rabinak et al., 2013). In a similar study, another synthetic THC, marinol, now did not affect SCR during extinction recall (day 3) but led to an increase of vmPFC and hippocampus activation to a previously extinguished CS+ in a differential conditioning paradigm (Rabinak et al., 2014).

Probably, genetic markers might help to reconcile the partly inconsistent results: healthy humans with reduced CB1-receptor expression showed impaired fear extinction in a virtual reality paradigm (Heitland et al., 2012).

Patients: the contribution of impaired eCB-signaling in patients with PTSD is also a matter of debate and is elegantly reviewed by Papini et al. (2015). In a recent study using positron emission tomography (PET), patients with PTSD were shown to have a ∼50% reduction of the peripheral eCB anandamide and a ∼20% higher availability (upregulation) of CB1 receptors in amygdala, hippocampus, and the cortico-striatal network than healthy controls. This pattern was more pronounced in women (Neumeister et al., 2013). Thus, a lower anandamide-tone might contribute to symptoms of PTSD and might lead to impaired extinction.

In sum: there is evidence for a positive role of eCBs for enhancing extinction recall and—correspondingly—for an impairment of extinction recall when eCB signaling is impaired. Moreover, acute, not chronic drug administration appears to be helpful to improve extinction recall. Further, more studies have to be conducted to find out how low eCB levels and upregulation of brain CB1-receptors might contribute to PTSD. The interaction between eCBs and stress is very interesting in the context of this review.

#### **Corticotropin releasing hormone (CRH)**

CRH neurons and CRH receptors are not only available in stressresponsive areas, but also in areas of the fear circuitry, i.e., in the BLA and the CE of the amygdala (Gafford and Ressler, 2015).

Animals: animal data show that intra-BLA infusion of CRH and of CRH binding protein (leading to an increase of endogenous CRH levels) prior to fear extinction impaired extinction recall while not affecting extinction acquisition. Correspondingly, a CRH-receptor antagonist improved extinction recall (Abiri et al., 2014). Similarly, in a transgenic mouse model missing the GABA(A)α1 subunit of the CRH neuron, CRH messenger RNA (mRNA) was elevated in the amygdala, the BNST and the hypothalamic paraventricular nucleus. The transgenic mice showed impaired extinction of conditioned fear and higher plasma cortisol levels. Deficits were successfully treated by systemic as well as local (BNST) infusion of a CRH-antagonist (Gafford et al., 2012).

Humans: in patients with PTSD, a hyperarousal of CRH pathways is described (Gafford and Ressler, 2015). Currently, a multicenter randomized controlled trial is in progress testing the effects of a CRH-type 1 receptor antagonist (GSK561679) in women with PTSD (Dunlop et al., 2014). Women will receive either 6 weeks drug treatment or placebo, and among a large number of dependent variables, fear acquisition and extinction will be assessed.

#### Further Mediators: Neuropeptide Y (NPY) and Oxytocin

Recent data also provide a role for NPY and oxytocin in extinction memory. Both peptides are known for being involved in stress resilience.

Neuropeptide Y. High concentrations of NPY are found in the cortex, amygdala, the hippocampus and the PAG, and in hypothalamic regions. NPY acts through six receptor subtypes, (Y1 to Y6), with Y1, Y2, Y4 and Y5 as the functional subtypes in the human brain (Bowers et al., 2012). NPY is co-localized with GABA within the BLA (Bowers et al., 2012). Since NPY also contributes to resilience after stress (Feder et al., 2009; McGuire et al., 2011) it might have an interesting role in regulating fear acquisition and extinction under stressful conditions.

Animals: intracerebroventricular NPY resulted in an improvement of extinction acquisition in fear-potentiated startle (Gutman et al., 2008), while mice lacking both, Y1 and Y2 receptors, showed accelerated acquisition of conditioned fear, excessive recall of conditioned fear and impaired fear extinction (acquisition and recall; Verma et al., 2012). Overexpression of the selective Y2-agonist NPY3–36 in the CE of the amygdala resulted in improved extinction acquisition in cued (not contextual) fear conditioning and minor return of fear (less spontaneous recovery and reinstatement), while local destruction of Y2-receptors impaired extinction memory (Verma et al., 2015).

Humans: we are not aware of data addressing NPY in fear extinction studies with healthy humans. Indirect evidence comes from data revealing that subjects with genotypes that predispose to have low NPY levels showed a higher responsiveness to aversive stimuli (negative words) in the mPFC and rostral ACC than those with intermediate or high expression (Mickey et al., 2011). Further, patients with PTSD (Rasmusson et al., 2000) were shown to have lower NPY baseline levels than healthy controls.

In sum: first animal data converge that NPY might help to improve fear extinction acquisition and recall. The results in humans only come from quasi-experimental and correlative studies in patients, but provide a first hint that high NPY levels might also be relevant for improving fear extinction in humans.

Oxytocin. Among other locations, oxytocin receptors are also located in the amygdala and the dorsal and ventral hippocampus. Moreover, due to its regulatory effects on the HPA-axis activation, oxytocin is an interesting neuropeptide for fear extinction under stress.

Animals: oxytocin was shown to inhibit excitatory output to brain-stem fear-expression centers when injected into the CE of the amygdala, here interacting with vasopressin (Huber et al., 2005). It also resulted in higher heart rate variability (as an indicator of a more vagal tone of heart rate modulation) during extinction acquisition and reduced expression of fear (Viviani et al., 2011). Thus, one should expect oxytocin to improve extinction recall, but the time-point of administration appears to be very important.

Intracerebroventricular injection of synthetic oxytocin prior to fear acquisition did not affect fear acquisition but facilitated extinction acquisition and retrieval, an effect that was abolished by an oxytocin-receptor antagonist. Interestingly, oxytocin given prior to extinction acquisition instead impaired extinction learning; this effect was again abolished by blockade of the oxytocin receptor (Toth et al., 2012). This suggests, that oxytocin needs to be already administered prior to fear acquisition, and that caution is necessary when considering the use of oxytocin after the acquisition of a trauma. Recently, oxytocin was shown to reduce reconsolidation of fear memories in rats after a single systemic injection after the reactivation of fear (Hou et al., 2015).

Humans: there is some evidence for facilitating effects of pre-extinction oxytocin on subsequent extinction acquisition (Eckstein et al., 2015) and fear extinction recall (Acheson et al., 2013) while a pilot clinical study using oxytocin prior to an exposure session in spider phobia revealed even debilitating effects (Acheson et al., 2015).

In a fear-potentiated startle paradigm (Acheson et al., 2013), subjects received either intranasal oxytocin or placebo after fear acquisition, followed by extinction acquisition (45 min after substance administration) on day 1, and extinction recall on day 2 (24 h later). While the oxytocin group showed higher startle magnitudes in the early extinction trials, both groups manifested reduced responding by the end of extinction acquisition on day 1. On day 2, extinction recall was better after previous oxytocin treatment, suggesting an oxytocin-induced enhancement of extinction consolidation. Eckstein et al. (2015) only addressed fear acquisition and extinction acquisition on a single day in a differential conditioning paradigm. They measured SCR and blood oxygen dependent (BOLD) signal from functional magnetic resonance imaging (fMRI). Intranasal oxytocin (vs. placebo) was again given after fear acquisition. In the early trials of extinction acquisition, oxytocin (vs. placebo) even induced a higher SCR, but also a higher activity in the right PFC. By the end of extinction, oxytocin resulted in lower SCRs. Moreover, oxytocin led to an unspecific inhibition (affecting CS+ and CS−) in amygdalar responses in early and late extinction acquisition. Extinction recall was not assessed here.

Patients: in a recent study, intranasal oxytocin (vs. placebo) was delivered as a pretreatment immediately prior to a single-session exposure treatment for spider phobia (Acheson et al., 2015). Dependent measures covered self-report of spider-phobia symptoms, phobic avoidance behavior, and trust into treatment (treatment creditability and therapeutic alliance). Follow-up was located 1 week and 1 month after exposure. Oxytocin- as compared to placebo-treated subjects manifested even higher clinical ratings of fear symptoms, higher avoidance behavior (as rated by clinicians) and less confidence into the treatment and a trend for less therapeutic alliance.

In sum: data suggest that some caution is necessary when delivering oxytocin prior to extinction (or prior to exposure therapy). There is evidence for both, extinction-improving and extinction-debilitating effects. More data are needed to find out whether oxytocin might especially affect extinction consolidation (and not extinction acquisition) and how oxytocin acts on extinction neurocircuits.

#### Second-Wave Mediators

#### **GCs, extinction learning and the role of circadian variation/rhythm**

There is a substantial amount of evidence suggesting that GCs have a positive influence on fear extinction learning and memory. Systemic (Cai et al., 2006; Yang et al., 2006, 2007; Brinks et al., 2009; Blundell et al., 2011) and intraamygdala administration (Yang et al., 2006) of GC agonists (e.g., corticosterone, dexamethasone) facilitates fear extinction memory when given prior to or directly after extinction training. In contrast, GR antagonists or the GC-synthesis inhibitor metyrapone impair extinction memory when administered systemically (Barrett and Gonzalez-Lima, 2004; Yang et al., 2006, 2007; Blundell et al., 2011) or into the amygdala (Yang et al., 2006). Importantly, the facilitating effect of GR-agonist dexamethasone was blocked by intra-amygdala infusion of the GR-antagonist mifepristone (Yang et al., 2006), suggesting that GRs in the amygdala are mediating the enhancement of extinction. Comparing post-reactivation administration of GC or a β-blocker (Abrari et al., 2008) suggest that GCs affect extinction memory per se, without compromising reconsolidation. Yet, GCs may have opposite effects on fear extinction mediated by different actions on GRs and MRs. Systemic administration of a low-dose GR agonist (dexamethasone) or administration of the MR-antagonist spironolactone both enhanced contextual fear extinction (Ninomiya et al., 2010), whereas MR-agonist fludrocortisone impaired extinction. High-dose dexamethasone or the GR-antagonist mifepristone had no effect (Ninomiya et al., 2010).

Importantly, data also suggest, that GRs can alleviate stress-induced extinction impairments. In the immobilization stress model of PTSD stress exposure impairs fear extinction a week later (Sawamura et al., 2015). However, systemic GR-agonist dexamethasone 4 h before extinction rescued extinction deficits in previously stressed rats and improved extinction acquisition and recall (Sawamura et al., 2015). Conversely, in the single prolonged stress (SPS) model, GC-synthesis inhibitor metyrapone exacerbated SPSinduced extinction recall deficits (Keller et al., 2015b). GR-signaling in the hippocampus has also been recently implicated to be one of the pathways mediating the enhancement of contextual fear extinction by novelty (Liu et al., 2015).

In sum, animal studies show that endogenous GCs play an important part in successful fear extinction and extinction memory, and GC administration can improve extinction. These effects are probably mediated through central GRs, especially in the amygdala and hippocampus.

Humans: studies addressing the effects of GC administration on fear extinction in humans are scarce and results are mixed. While there are reports that hydrocortisone (30 mg) administration impaired extinction learning in men (SCR and BOLD; Merz et al., 2014b), other studies find an enhancement in (BOLD-signal) differentiation during extinction in women using hormonal contraceptives (Tabbert et al., 2010). Here however (Tabbert et al., 2010), hydrocortisone increased responses to the CS− (never paired with the US) vs. CS+ (previously paired with the US) which is hard to interpret. Another study found no effect of the same dose of hydrocortisone (30 mg) in men and free-cycling women (Merz et al., 2012a). On the other hand, clinical studies have shown enhanced outcome of exposure therapy, when GCs were administered prior to exposure training for specific phobias (Soravia et al., 2006, 2014; de Quervain et al., 2011). For PTSD, combining exposure techniques with GCs led to lower avoidance/numbing symptoms compared to placebo 1 week after intervention, but the effects did not last at a 1 month follow-up (Surís et al., 2010). Lowdose cortisol alone has also shown some promising results in a pilot study with 3 PTSD-patients (Aerni et al., 2004). Endogenous levels of GCs show a marked diurnal variation with a peak in the early morning after awakening. This peak has been linked to enhanced extinction/extinction recall (Pace-Schott et al., 2013) and a better outcome of exposure therapy in humans in the morning (Lass-Hennemann and Michael, 2014). However, only one study shows a circadian effect with a direct positive association between endogenous GC levels and exposure therapy outcome (Meuret et al., 2015).

To summarize: in line with animal studies, in humans, GC administration combined with exposure therapy has shown some promise to yield a better therapy outcome. Similar conclusions can be drawn from diurnal effects in humans. Both lines of evidence indirectly point to a possible enhancement of fear extinction by GCs in humans, as repeatedly shown in rodents. However, the few studies with GC administration directly examining fear extinction in healthy volunteers cannot back up clinical trials yet.

# Insights from Stress-Induction Studies Triggering the First and Second Wave of the Stress Response

So far, we reported effects of single mediators of the stress response from pharmacological manipulations. This is important, but no substitute for experimentally induced stress. As stress responses are typically defined by the interplay of multiple mediators (Joëls and Baram, 2009), and as GC effects in the amygdala were shown to depend on arousal-induced NA-signaling (Roozendaal et al., 2006a), single pharmacological manipulations might be insufficient. Despite the potential relevance for clinical settings, especially human studies on the effects of acute stress on fear extinction are still rare (for a review, see Raio and Phelps, 2015). Below, we review animal and human stress-induction studies on fear extinction focusing broadly on acute stress effects. Effects of chronic stress on fear extinction and effects of stress during pre- and postnatal development on extinction are beyond the scope of this review.

Animals: the field has focused on designs where extinction follows one to several days after stress, thus reflecting longer lasting effects of second-wave mediators. A number of procedures have been suggested to model features of PTSD in laboratory animals (e.g., Rau et al., 2005). The SPS model is especially well studied with fear extinction. SPS comprises 2 h restraint stress, 20 min of forced swim stress, and exposure to ether until general anesthesia (Yamamoto et al., 2008). When applied 7 days before fear acquisition and extinction the SPS was repeatedly shown to impair extinction learning and especially extinction recall while leaving fear acquisition intact (Yamamoto et al., 2008, 2009; Knox et al., 2012a,b; Ganon-Elazar and Akirav, 2013; Matsumoto et al., 2013). The delayed effects of SPS on fear extinction were linked to enhanced GR-expression in the hippocampus and PFC (Knox et al., 2012b) and to upregulation of NMDA-receptor mRNA in the hippocampus (Yamamoto et al., 2008; Matsumoto et al., 2013). However, no impairment of extinction was found after a shorter time interval between stress and learning (Knox et al., 2012a), or after reducing stressor intensity by omitting either one of the SPS components (Knox et al., 2012b). In a different model, placing repeated forced swim stress (10 min/day for 3 days) 24 h before cued fear acquisition and 48 h before extinction also impaired extinction learning (but not fear acquisition) and produced dendritic retraction in IL neurons in the PFC (Izquierdo et al., 2006).

Behavior during/after extinction can be seen as the result of a competition between the original fear memory and the new inhibitory extinction memory. Therefore, placing stress before fear acquisition makes precise inferences about stress effects on extinction difficult, as stress could have modified the original fear memory. However, there is also evidence that stress impairs fear extinction even when animals are stressed after fear acquisition: 30-min of elevated platform stress 24 h before extinction impaired extinction learning and was associated with changes in BLA morphology, including dendritic retraction and debranching but also increased spine density (Akirav et al., 2009; Maroun et al., 2013).

In some cases, stress exposure can also facilitate fear extinction. A single exposure to 3 h of immobilization stress 14 days (but not 2 days) before contextual fear acquisition and extinction significantly improved extinction recall (Kirby et al., 2013). The stress effects were linked to increased neurogenesis in the dorsal hippocampus and seem to be mediated through stressinduced GCs activating basic fibroblast growth factor (FGF2; Kirby et al., 2013).

In sum: the combined evidence suggests that stress impairs the consolidation/recall of fear extinction. The impairment requires time (>24 h) and is associated with structural changes in extinction-relevant structures including dendritic morphology, and receptor density. Stress-induced extinction impairments seem more likely with severe, prolonged and high-intensity stressors.

Humans: here, we group studies by (a) the type of stressor and its validity for triggering the first- vs. second-wave stress response and (b) by considering the timing of the stressor relative to stages of extinction memory formation (encoding, consolidation, recall). Stress placed already before fear acquisition will give a better comparison to the animal studies above. Stress placed after fear acquisition but before extinction gives insights into effects on encoding. Stress after extinction learning informs us about consolidation effects. Finally, stress before extinction recall captures influences on recall independent from learning and consolidation.

Although considerably fewer in number, human studies also suggest that stress placed before fear acquisition (and thus also extinction) may impair fear extinction. We have recently shown (Antov et al., 2013), that inducing the first-wave stress response immediately before fear acquisition impaired extinction even without a second-wave GC-increase. We used the CPT, where participants immerse their hand into ice-cold water for a max of 3 min and found increased extinction resistance compared to control, probably due to a strengthening of the original fear memory. Yet, a psychosocial stressor inducing significant GCincreases had no effects on conditioning (Antov et al., 2013). In contrast, a psychosocial stressor 70 min before fear acquisition impaired extinction (Jackson et al., 2006), and uncontrollable shocks 7 days before fear acquisition reduced extinction recall (Hartley et al., 2014).

Placing first-wave stress (CPT) after acquisition but immediately before extinction training facilitated extinction acquisition and extinction recall without second-wave GCincreases (Antov et al., 2015). Bentz et al. (2013) also placed the CPT prior to extinction training: they found that CPT impaired recall of the original fear memory (measured in subjective US expectancy) in men but not in women. As discussed by the authors (Bentz et al., 2013), this study has some important limitations precluding inferences about stress effects on extinction: (1) there was no evidence for fear learning (or extinction) in physiological conditioning measures (SCR, heart rate), (2) there was also no extinction learning in any measure. A variant of the CPT including social evaluation (SECPT, Schwabe et al., 2008), also inducing GC-increases, placed after fear extinction impaired extinction recall 24 h later, especially when tested in the acquisition context (Hamacher-Dang et al., 2015). Finally, two studies suggest that stress also affects the retrieval of fear extinction: CPT associated with GC-increases 15 min before delayed extinction recall (Raio et al., 2014) significantly impaired recall performance. Conversely, the SECPT placed 20 min prior to extinction recall and 24 h after extinction learning, reduced contextual fear renewal (Merz et al., 2014a), probably indicative of enhanced extinction recall.

In sum: the few available human studies yield mixed results, but there is some evidence that first-wave stress before extinction learning improves extinction acquisition and recall whereas second-wave stress before recall testing impairs extinction recall.

# SEX, SEX HORMONES AND FEAR EXTINCTION UNDER STRESS

#### Sex, Sex Hormones and Fear Extinction

Compared to men, the prevalence of anxiety (Kessler et al., 2005; Somers et al., 2006; Eaton et al., 2012) and trauma- and stressor-related disorders (Kessler et al., 1995; Tolin and Foa, 2006; Kilpatrick et al., 2013; Zoladz and Diamond, 2013) is up to two times higher in women.

Levels of circulating gonadal hormones (mainly estrogens) are regarded to contribute to these sex differences in psychopathology (Lebron-Milad and Milad, 2012; Cover et al., 2014).

Estrogens are a class of steroidal sex hormones that include estrone (E1), estradiol (E2), estriol (E3), and estetrol (E4). E2 is produced by the ovaries and (to a smaller amount) by the adrenal cortex, and by the testes in men by conversion from testosterone via the enzyme aromatase or by conversion of androstenedione to E1 and conversion of E1 to E2. E1 is the dominant estrogen during menopause and E3 and E4 dominate during pregnancy. E2 is the predominating estrogen in non-pregnant women in their reproductive years when E2-levels are considerably higher in women than in men. In humans, an idealized menstrual cycle lasts 28 days and includes the following phases of different concentrations of E2 and P4 (Becker et al., 2005): E2 and P4 are low during the early follicular phase (approx. cycle days 1–8). During the late follicular phase, P4 remains low, while E2 rises to reach its peak immediately before ovulation (midcycle, approx. days 13–14). Peak P4 levels are only reached during the midluteal phase, where there is also a second less prominent peak in E2. In rodents, the estrous cycle lasts 4–5 days and covers four phases: metestrus (low E2 and low P4), diestrus (moderate P4 and low E2), proestrus (high E2 and high P4), and estrus (low E2 and low P4; Maeng and Milad, 2015).

E2 exerts its effects via two types of steroidal estrogen receptors located in the cell nucleus or the cytoplasm: estrogen receptor alpha (ERα) and beta (ERβ) which are both widely distributed across the brain (Gillies and McArthur, 2010). ERα are broadly distributed across cortical and subcortical structures including hippocampus, amygdala, hypothalamus, and brain stem, whereas ERβ distribution is less wide-spread with medium to high ERβ density throughout the cortex including the vmPFC, high density in the hippocampus, and in some hypothalamic nuclei (Gillies and McArthur, 2010; Cover et al., 2014). Importantly, ERs are well expressed in brain regions critical for fear acquisition and extinction including amygdala subnuclei, hippocampus and vmPFC (Shughrue et al., 1997; Österlund et al., 1998, 2000; Zhang et al., 2002; Weiser et al., 2008).

Neuroimaging studies reveal that circulating levels of E2 in women affect the reactivity of brain structures involved in both, stress and fear acquisition and extinction. Specifically, women scanned during the midcycle phase (peak E2, but low P4) exhibited significantly less responses to high arousing, aversive visual stimuli (International Affective Picture System, IAPS; Lang et al., 2008) in the amygdala, ACC, orbitofrontal cortex, mPFC, hippocampus, PAG, and several hypothalamic nuclei compared to both, men and women tested during the early follicular phase (low E2 and low P4; Goldstein et al., 2005, 2010).

Estrogens were shown to be especially important in fear extinction: in an elegant series of experiments in rats, Chang et al. (2009) found faster extinction of contextual fear conditioning in female than male rats, especially when in the high E2 phase of the estrous cycle (proestrus). The authors also observed enhanced extinction after systemic and central E2 administration to ovariectomized animals and traced the effects back to hippocampal ERβ by using selective ERα/ERβ agonists.

There is accumulating evidence for the role of E2 especially for extinction recall in both laboratory animals and healthy humans: low E2-levels were shown to impair and high E2 levels to enhance extinction recall, typically tested 24 h after extinction training (Milad et al., 2006, 2009a, 2010; Zeidan et al., 2011; Graham and Milad, 2013). Studies also include data on low-level estrogen under contraceptives (Graham and Milad, 2013). Moreover, in mice both, high endogenous E2 and activation of ERβ, enhanced glutamatergic transmission and synaptic plasticity in the IL mPFC (Galvin and Ninan, 2014), a structure associated with consolidation and recall of extinction. Interestingly, inhibition of estradiol synthesis by an aromatase inhibitor (fadrozole) in male rats, significantly impaired their extinction recall (Graham and Milad, 2014). Adequate controls (such as delivering a single dose of E2 after previous aromatase inhibitor) gave first evidence that fear extinction in males may also depend on acute effects of estrogen synthesized de novo and acting presumably via non-genomic mechanisms. In contrast to the majority of studies supporting an enhancement of extinction by high E2-levels, there is recent evidence for the opposite effect, i.e., enhanced fear memory and impaired extinction memory under high E2-levels (McDermott et al., 2015). But this study did not assess female rats under natural E2 conditions but studied E2 replacement in female ovariectomized mice (vs. mice after sham surgery) instead. Chronic high-dose, not low-dose, E2 administration in pre-puberty (4 weeks) and in adult age (10 weeks), impaired extinction but only in contextual, not cued, fear conditioning. This suggests an adverse effect of chronic high E2-levels for hippocampus-mediated fear inhibition. Anyway, we have to take into account, that chronic high E2 actions after ovariectomy do not necessarily reflect ''normal'' E2 actions within the brain and periphery.

Addressing brain correlates during fear extinction and considering different E2-levels in humans revealed that women using oral contraceptives (having suppressed E2-levels) showed higher differential BOLD responses to the CS+ (previously paired with the US) during fear extinction compared to men and women in the luteal phase of the menstrual cycle (high P4, medium-high E2) in the amygdala, ACC, and vmPFC (Merz et al., 2012a). This is in line with impaired extinction when E2-levels are low. The study also included administering cortisol vs. placebo but this did not affect the results (see also ''Second-wave mediators'' Section). Accordingly, a recent study examining women during the early follicular and luteal phases of the menstrual cycle (Wegerer et al., 2014) reported that low E2 but not P4 was associated with poorer extinction and with higher intrusive memories.

Patients: with PTSD exhibit marked fear extinction deficits as compared to healthy controls (Milad et al., 2009b; Inslicht et al., 2013) or trauma-exposed and non-trauma-exposed healthy controls (Milad et al., 2009b). Interestingly, fear inhibition/extinction deficits have been linked to low levels of E2 in both female PTSD-patients (Glover et al., 2012) as well as in healthy and traumatized women (Glover et al., 2013). A recent study by Shvil et al. (2014), addressing sex differences in trauma-exposed healthy controls vs. PTSDpatients, found impaired extinction recall (higher SCRs to the extinguished CS+) and higher activity in the left rostral dACC in men as compared to women within the PTSD group, but not in trauma-exposed healthy controls. The result also suggests that interventions improving fear extinction are highly important in male patients with PTSD.

In sum: the available data suggest an important role of E2 for fear extinction with high E2-levels enhancing extinction acquisition and especially extinction recall (with most of the studies comparing different E2-levels within females). The assessment of E2-levels should be thus included in fear extinction studies. Comparing women in different cycle phases and under different natural E2-levels is only a quasi-experimental approach. Thus, experimental studies would be important with experimental variation of sex-hormone levels.

# Interaction of Stress and Sex Hormones in Fear Extinction

Although trauma- and stressor-related disorders are associated with deficits in fear extinction, we still do not know much about the effects of acute stress on fear acquisition and extinction in healthy humans (Raio and Phelps, 2015). Importantly, animal data suggests that stress effects on fear conditioning are sex specific (Dalla and Shors, 2009). For example, in ovariectomized female rats an injection of E2 (45 µg/kg) was able to alleviate conditioned fear responses after SPS (Mirshekar et al., 2013), suggesting an E2 × stress interaction. In intact females (as compared to males), SPS did not impair extinction learning or recall, regardless of estrous cycle phase (Keller et al., 2015a).

In humans, Zorawski et al. (2005) found that the level of postacquisition GCs was positively correlated with fear acquisition performance (SCR) in men, but not in women. This was later replicated (Zorawski et al., 2006) with an added stressor after acquisition to achieve higher GC-levels. Again, post-acquisition GCs were only correlated with acquisition in men, but not in women. This suggested a sex-specific relationship between GCs and fear acquisition. Similarly, Merz et al. (2013a) report a positive correlation between basal cortisol levels and amygdala activity during fear acquisition only in men and in women taking oral contraceptives, but not in women in their luteal cycle phase. In a first experimental study (Jackson et al., 2006) reported that psychosocial stress (inducing cortisol increases) enhanced differential SCRs during both acquisition and immediate extinction in men, but had no effect in women. However, subsequent neuroimaging studies report the opposite effect after a psychosocial stressor: reduced SCRs and BOLD responses in men during acquisition and enhanced BOLD responses after stress in women (Merz et al., 2013b). Similar results, with higher conditioned BOLD responses in women, and impaired responses in men were reported from the same group (Stark et al., 2006; Merz et al., 2010) for fear acquisition after administration of cortisol (oral, 30 mg hydrocortisone) vs. placebo.

Cortisol effects on BOLD responses during fear acquisition in women were shown to depend on gonadal hormone availability: women using oral contraceptives showed enhanced conditioned BOLD responses after cortisol, whereas cortisol impaired responding in men and free-cycling women in their early follicular (low E2, low P4) and mid-luteal (low to mid E2, high P4) cycle phases (Merz et al., 2012b). But in another study of that group, comparing men and women in their luteal phase and women taking contraceptives, the additional experimental factor of administering cortisol (vs. placebo) had no effect on extinction acquisition (Merz et al., 2012a). In a recent study of our group (Antov and Stockhorst, 2014), we tested the effect of a psychosocial stressor that preceded fear acquisition. Fear acquisition started in the maximum of the peripheral cortisol peak. We examined men and free-cycling women either in the early follicular phase (low E2, low P4) or in the mid-cycle phase (high E2, low P4) and tested fear acquisition, extinction acquisition (day 1) and extinction recall (day 2). We found an interaction between stress exposure and natural E2-status within women: in mid-cycle women, extinction recall was better when fear acquisition had been preceded by stress, whereas the inverse was true in early follicular women. Thus, extinction recall of conditioned fear acquired after stress depends on estrogen status in women. Consequently, we suggest that for extinction-based exposure therapy in women, cycle phase and/or E2-level should be taken into account. We assume that the mid-cycle phase (with high E2- and low P4-levels) provides an adequate phase to reexpose women to the aversive CSs during exposure therapy. In healthy volunteers, exposure to aversive stimuli during this high E2-cycle phase was also associated with less intrusive memories (Wegerer et al., 2014).

Considering the higher prevalence of anxiety, trauma- and stressor-related disorders in women, and extinction deficits under low as compared to high E2-levels (within women), one might wonder why low levels of E2 are disadvantageous for women, but (often) not for men who also have low levels of circulating E2. Most importantly, one has to keep in mind that testosterone is rapidly converted to estrogen in the brain via aromatase. Thus, men do take advantage of estradiol actions within the brain, although they have low levels of circulating E2. This is supported by data showing that the inhibition of estradiol synthesis by an aromatase inhibitor in male rats significantly impaired extinction (Graham and Milad, 2014). When it comes to the interaction between female sex hormones and stress in fear conditioning studies with humans, there is some initial evidence that there might be a higher resemblance in the response pattern of men and naturally cycling women in a high estrogen status than with women in a low estrogen status (Antov and Stockhorst, 2014), or than with women using contraceptives (Merz et al., 2012b). Here—in addition to the above argument—one might also consider differences in binding capacities of MRs and GRs for corticosterone, or cortisol, respectively. These binding capacities were shown to differ between males and females, and—within females—also between different cycle phases (Ter Horst et al., 2012). But, most importantly, much more studies are needed to assess the interaction between stress and sex, or sex hormones, especially with regard to extinction.

# A Role of Male Sex Hormones and of Hormonal Transitions in Fear Extinction

While fear extinction studies measuring female sex hormone levels are already sparse, there are hardly any studies addressing the role of male sex hormones for fear extinction. Moreover, these remaining data are mainly based on studies using castration, not exogenous administration of androgens.

In an early study (Anagnostaras et al., 1998), adult male rats (age 12–17 weeks) were either castrated or sham-operated or remained intact. Groups did not differ in their extinction acquisition of contextual fear, or in hippocampal long-term potentiation. Different results were obtained by McDermott et al. (2012): castration during prepuberty (4 weeks old) as well as thereafter (10-weeks old) resulted in an impairment of extinction in a contextual conditioning paradigm, not in cued conditioning, suggesting that testosterone might act to improve fear extinction in hippocampal-dependent, not in amygdalamediated fear memory. In a recent fear conditioning study with healthy men (Pace-Schott et al., 2013), testosterone levels were also measured. Interestingly, a higher testosterone/cortisol (T/C) ratio predicted better extinction acquisition, but only in the morning hours (i.e., when endogenous cortisol levels are high). The T/C ratio (not the single hormones) was negatively correlated with the remaining CR (SCRs). Although these data are only correlative, they encourage examining the role of testosterone on fear extinction under stressful conditions. Experimental studies are of interest where testosterone is manipulated under stressful vs. control conditions. In the meantime, it would be helpful to measure testosterone levels in fear conditioning studies.

Another approach to study the role of sex hormones for fear-related memory and extinction memory is to examine developmental phases of larger hormonal changes. Again, data are sparse. There is some evidence (recently reviewed by Baker et al., 2014) for impaired fear extinction in adolescents as compared to both, younger individuals as well as adults. One explanation refers to functional changes in the connectivity between prefrontal cortex and amygdala during this developmental stage. Moreover, adolescence is a stage of high vulnerability to stress (Lupien et al., 2009).

For women, pregnancy and perimenopausal stages are additional phases of larger hormonal changes. For example, there is a heightened risk for depression when E2 production is sharply reduced after menopause (for an overview, see Cover et al., 2014) and there is evidence, that estradiol replacement can improve depressive symptoms in women with a history of perimenopausal depression (e.g., Schmidt et al., 2015). More studies specifically addressing fear extinction during developmental stages and hormonal transitions are necessary. This might also include conditions where estrogen receptors are blocked. This is the case in the pharmacological adjunct therapy in estrogen-receptor positive breast cancer patients receiving the estrogen-receptor blocker tamoxifen.

# CONCLUSION

Our review aimed at elucidating how single mediators of the stress response (hormones, neurotransmitters, and neuropeptides) and the entire natural stress response (covering the sequence of the first and the second wave of the stress response) contribute to fear extinction. Our stress approach is guided by the idea that classical stress-response mediators are also agents within the fear macro- and microcircuits. Thus, explicitly investigating stress or stress-response mediators in fear acquisition and fear extinction might mimic the natural situation of acquiring anxiety, trauma- and stressor-related disorders and of being re-exposed to the fear reminders during extinction-based exposure therapy.

Concerning the mediators of the first-wave stress response (monoamines, opioids, eCBs), animal data strongly suggest that rapidly acting NA enhances extinction learning and extinction memory processes; for humans, data are less conclusive and more human studies on NA effects in extinction are warranted. Dopaminergic as well as serotonergic signaling improves extinction memory in animals and humans, while a number of modifying conditions (dose, temporal spacing, targeted receptorsubtypes, chronic vs. acute administration, gene polymorphisms) have to be considered. An improvement of extinction also occurs under opioid signaling and eCBs, with first evidence also in humans. Endogenous GCs play an important part in successful fear extinction and extinction memory. In animal studies, data converge that GC administration can improve extinction when the learning task is in ''sync'' with the cortisol peak thus improving consolidation of extinction memory. In line with animal studies, in humans GC administration combined with exposure therapy has shown some promise to yield a better therapy outcome. Similar conclusions can be drawn from diurnal effects in humans. However, the few studies with GCs administration directly examining fear extinction in healthy volunteers cannot back up clinical trials yet.

Considering the natural stress response, animal studies suggest that stress impairs the consolidation/recall of fear extinction. The impairment requires time (>24 h) and is associated with structural changes in extinction-relevant structures including dendritic morphology, and receptor density. While results are mixed in humans, there is some evidence that extinction acquisition during the first-wave stress response (e.g., after the CPT) improves extinction recall. On the other hand, second-wave stress before testing impairs extinction recall.

While the prevalence of anxiety, trauma- and stressor-related disorders is up to two times higher in women than in men, studies addressing the level of E2 reveal better fear extinction, especially fear extinction recall, and higher vmPFC-activity, in cycle phases of a high as compared to low E2-level in women. On the other hand, in PTSD-patients, behavioral fear-extinction deficits as well as reduced vmPFC activity have been linked to low E2-levels. There is first evidence for an interaction between stress exposure and natural E2-status within women: in midcycle women, extinction recall was better when fear acquisition had been preceded by stress, whereas the inverse was true in early follicular women. Thus, extinction recall of conditioned fear acquired after stress depends on estrogen status in women. Consequently, we suggest that for extinction-based exposure therapy in women, cycle phase and/or E2-level should be taken into account. In case of low E2-levels, drugs increasing dopamine and serotonin levels might improve the success of extinction, or exposure therapy.

### REFERENCES


We predict that behavioral and pharmacological interventions using the mediators of the first-wave stress response applied prior to extinction acquisition should enhance extinction memory. For GC-related interventions (being related to the second wave of the stress response), we should consider that the consolidation phase of the conditioning task (i.e., extinction acquisition) is in ''sync'' with the GC peak in order to take advantage of extinction-improving GC effects whereas high GC levels at retrieval are expected to impair extinction memory retrieval.

Since exposure therapy can be described as a form of extinction learning, the presented data thus might also allow predicting which of the above pharmacological interventions and behavioral types of stress induction could constitute adjuncts to exposure therapy. Concretely, DA- and 5-HT-reuptake inhibitors, endorphins, eCBs, and GCs prior to extinction (in ''sync'' with the consolidation phase) as well as stressors strongly affecting the first-wave stress response (such as the CPT increasing the noradrenergic tone) could be successful. Moreover, data presented here also encourage taking cycle phase and/or contraceptive use into account when scheduling exposure sessions for women.

# AUTHOR CONTRIBUTIONS

Both authors, US and MIA, fulfill the four criteria for authorship and contributed equally to the present manuscript.

### ACKNOWLEDGMENTS

We acknowledge support by Deutsche Forschungsgemeinschaft (DFG) and Open Access Publishing Fund of Osnabrück University.


in healthy humans. Neuroscience 168, 652–658. doi: 10.1016/j.neuroscience. 2010.04.030


Morris, R. W., and Bouton, M. E. (2007). The effect of yohimbine on the extinction of conditioned fear: a role for context. Behav. Neurosci. 121, 501–514. doi: 10. 1037/0735-7044.121.3.501


within the female rat brain. Brain Res. Mol. Brain Res. 54, 175–180. doi: 10. 1016/s0169-328x(97)00351-3


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Stockhorst and Antov. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Differential Effects of Controllable Stress Exposure on Subsequent Extinction Learning in Adult Rats

Osnat Hadad-Ophir 1,2† , Noa Brande-Eilat 3† and Gal Richter-Levin1,2,3 \*

1 "Sagol" Department of Neurobiology, University of Haifa, Haifa, Israel, <sup>2</sup> The Institute for the Study of Affective Neuroscience (ISAN), University of Haifa, Haifa, Israel, <sup>3</sup> Department of Psychology, University of Haifa, Haifa, Israel

Deficits in fear extinction are thought to be related to various anxiety disorders. While failure to extinguish conditioned fear may result in pathological anxiety levels, the ability to quickly and efficiently attenuate learned fear through extinction processes can be extremely beneficial for the individual. One of the factors that may affect the efficiency of the extinction process is prior experience of stressful situations. In the current study, we examined whether exposure to controllable stress, which is suggested to induce stress resilience, can affect subsequent fear extinction. Here, following prolonged twoway shuttle (TWS) avoidance training and a validation of acquired stress controllability, adult rats underwent either cued or contextual fear-conditioning (FC), followed by an extinction session. We further evaluated long lasting alterations of GABAergic targets in the medial pre-frontal cortex (mPFC), as these were implicated in FC and extinction and stress controllability. In cued, but not in contextual fear extinction, within-session extinction was enhanced following controllable stress compared to a control group. Interestingly, impaired extinction recall was detected in both extinction types following the stress procedure. Additionally, stress controllability-dependent alterations in GABAergic markers expression in infralimbic (IL), but not prelimbic (PL) cortex, were detected. These alterations are proposed to be related to the within-session effect, but not the recall impairment. The results emphasize the contribution of prior experience on coping with subsequent stressful experiences. Moreover, the results emphasize that exposure to controllable stress does not generally facilitate future stress coping as previously claimed, but its effects are dependent on specific features of the events taking place.

Keywords: stress controllability, cued fear conditioning, contextual fear conditioning, extinction, resilience, infralimbic, interneuron, neuropeptides

### INTRODUCTION

Fear conditioning (FC) and extinction are extensively studied in the context of stress related behaviors, and specifically in anxiety disorders. Impaired fear extinction is perceived as a central symptom of disorders caused by emotional trauma (Graham and Milad, 2014). Fear extinction is an expression of an active learning process (reviewed by Myers and Davis, 2002), in which a new, safe association is formed. The new ''CS-no shock'' association competes with the original acquired association but does not erase it (Bouton, 2002; Eisenberg et al., 2003). It was previously demonstrated that the extinction level can be affected by different factors. For example, it is attenuated by cocaine treatment (Burke et al., 2006) and sleep deprivation (Silvestri, 2005).

#### Edited by:

Denise Manahan-Vaughan, Ruhr University Bochum, Germany

#### Reviewed by:

Alicia Izquierdo, University of California, Los Angeles, USA Volker Korz, Medical University, Austria Christine A. Rabinak, Wayne State University, USA

#### \*Correspondence: Gal Richter-Levin

galrichterlevin@gmail.com †These authors have contributed

equally to this work.

Received: 14 October 2015 Accepted: 21 December 2015 Published: 12 January 2016

#### Citation:

Hadad-Ophir O, Brande-Eilat N and Richter-Levin G (2016) Differential Effects of Controllable Stress Exposure on Subsequent Extinction Learning in Adult Rats. Front. Behav. Neurosci. 9:366. doi: 10.3389/fnbeh.2015.00366 Moreover, exposure to a stressor extrinsic to the context of FC was found to impair the extinction of fear (reviewed by Akirav and Maroun, 2007). Interestingly, it was demonstrated that exposure to escapable tail-shock results in facilitated extinction while inescapable shock damages it (Baratta et al., 2007). Understanding the factors that affect extinction acquisition is important not only because extinction of aversive memories is implicated in anxiety disorders and vulnerability to extreme stress (e.g., Lissek et al., 2005; Rauch et al., 2006; Hofmann, 2008), but also because of its involvement in the resilience to them. Stress resilience is more likely to be developed in individuals who display facilitated extinction (Haglund et al., 2007).

The actual, or apparent, control over a stressor is defined as the ability to alter the onset, duration, intensity or pattern of an aversive experience (Overmier and Seligman, 1967). The degree of behavioral control an organism exerts over a stressor critically determines the behavioral consequences of the stressful experience (Maier and Watkins, 2005). Various physiological alterations are evident under different controllability levels. For example, exposure to an uncontrollable stressor led to increased secretion of corticosterone (CORT; Weiss, 1971; Prince and Anisman, 1990; Akirav et al., 2001; Ilin and Richter-Levin, 2009) and increased the severity of gastric lesions (Weiss, 1968). Distinctively, the sense of control over a stressor has the ability to protect from the deleterious behavioral effects of stress and thus to potentiate a trait of stress resilience (Amat et al., 2006; Ilin and Richter-Levin, 2009; Lucas et al., 2014).

Stress controllability was found to induce alterations in several related brain regions. Elevation in noradrenaline release in the amygdala and hypothalamus was observed a few days after uncontrollable training, in comparison to rats exposed to controllable stress (Tanaka, 1999). In addition, 6 days training of two-way shuttle (TWS) avoidance task lead to pERK activation of the basolateral amygdala (BLA) after uncontrollable but not controllable stress (Ilin and Richter-Levin, 2009). Furthermore, controllability-dependent alterations in medial pre-frontal cortex (mPFC) and dorsal raphe nucleus were also reported (Amat et al., 2005; Rozeske et al., 2011). It was demonstrated that acquisition of stress controllability involved structural changes in the mPFC, as its outputs to the dorsal raphe nucleus, a stress-responsive brainstem nucleus, were enhanced (Maier and Watkins, 2005). It was proposed by Amat et al. (2005), that this output activation is responsible for behavioral changes and protective effects of behavioral control on stress-induced brainstem activity. Furthermore, an initial experience of controllable stress appears to alter the mPFC in such way that a subsequent uncontrollable stressor, which normally does not activate mPFC output, will now do so. Hence, gaining controllability in prior experience will result in a protective effect against the neurochemical and behavioral impacts of an uncontrollable stressor (Amat et al., 2006).

Notably, there exists a profound overlap in neuro-circuitry underlying both learning types, controllability over stress and extinction learning. Both contextual and cued FC and extinction were also repeatedly shown to involve the BLA, hippocampus and the mPFC (Maren, 2001; Milad et al., 2007; Maren et al., 2013). Moreover, the inhibitory effect of infralimbic (IL) projections to the central amygdala through the intercalated cells (ITC; Vertes, 2006) is crucial for extinction acquisition (McDonald et al., 1996; Smith et al., 2000; Berretta et al., 2005; Sierra-Mercado et al., 2011).

Interestingly, Izquierdo et al. (2006) have demonstrated that brief uncontrollable stress causes morphological alterations specifically in the IL, and not prelimbic (PL) cortex of the mPFC, and attenuated the cued fear extinction rate relative to nonstressed controls. This raises the possibility that prior experience of stress controllability may result in alteration of IL output to the amygdala, which could potentially facilitate the acquisition of FC extinction. The impact on IL output can rise from alteration in its excitation-inhibition balance, through changes in GABAergic interneuron transmission. The latter is known to be related to stress and anxiety states in the relevant circuitry (Kim et al., 2005; Bergado-Acosta et al., 2008; Jacobson-Pick et al., 2008; Yarom et al., 2008; Jacobson-Pick and Richter-Levin, 2010; Heldt et al., 2012). Specifically, GABAergic marker alterations also appear in the BLA after acute exposure to controllable and uncontrollable stress. A decreased expression of specific targets of interest (i.e., glutamate decarboxylase, GAD65, GAD67) was detected under controllable conditions, while uncontrollable conditions led to elevation in those genes (Hadad-Ophir et al., 2014). In addition, null mutation of GAD65 in mice resulted in increased anxiety and resistance to conditioned fear extinction along with hyperactivation of the amygdala and the hippocampus (Stork et al., 2000, 2003; Bergado-Acosta et al., 2008; Müller et al., 2014, 2015). Interneurons also use neuropeptides, such as cholecystokinin (CCK) and neuropeptide Y (NPY), as co-transmitters that exert profound effects on fear, anxiety, learned helplessness behavior and stress response (Ishida et al., 2007; Sherrin et al., 2009; Lach and de Lima, 2013; Serova et al., 2014), and stress was found to affect their expression as well (Hadad-Ophir et al., 2014).

We previously developed a behavioral task based on prolong exposure to the TWS avoidance task, which resulted in gained controllability (Ilin and Richter-Levin, 2009; Lucas et al., 2014). In the current study, we employed this model to assess the long-term impact of prolonged controllable stress exposure on subsequent fear extinction. We first verified the behavioral differences between the controllable stress group and a control group. Next, cued or contextual FC was conducted, followed by an extinction training in order to further examine if the beneficial effects of prolonged controllable training will be also evident in fear extinction acquisition. In addition, in order to assess stress controllability-induced alterations preceding the FC and extinction training, we evaluated messenger RNA (mRNA) expression levels of selected GABA transmission related targets in the mPFC, 2 weeks after completion of the controllable training.

### MATERIALS AND METHODS

#### Animals

Male Sprague–Dawley rats were obtained at an age of 60 PND (weight 275–300 g) from Harlan Laboratory (Jerusalem, Israel). Animals were maintained in groups of 4 on a 12 h light: 12 h dark cycle (lights on 07.00 am) with food and water ad libitum. After 5 days of acclimation rats were assigned to behavioral training. All experiments were carried out during the light phase (9.00 am–5.00 pm), in accordance with the NIH guidelines for the care and use of laboratory animals and were approved by the University of Haifa ethical committee (Ethical No. 230/11).

#### Behavioral Protocol

#### Experimental Design

After acclimation, rats were randomly assigned to two different groups. ''Controllable'' group (n = 18) went through active avoidance training in a TWS avoidance task. ''Unexposed'' group (n = 27) were placed in the TWS box for an equivalent amount of time of free exploration as the controllable group. This group was not exposed to any tones or shocks during the training period. Two weeks after the end of TWS training animals' behavior were assesses by TWS re-exposure and elevated plus maze (EPM) tests. Two days after, animals went through cued or contextual FC and extinction training (**Figure 1**).

#### TWS Apparatus

The TWS avoidance box was a rectangular chamber (60 × 26 × 28 cm), divided by an opaque partition with a passage (10 × 8 cm) into two equal size compartments, within a dimly-lit, ventilated, sound-attenuated cupboard (Panlab, Harvard Apparatus, Barcelona, Spain).

#### TWS Training

TWS avoidance training (adapted from Tsoory and Richter-Levin, 2006) was composed of 6 days with 50 trials per day. Rats were given 10 min of free exploration period in the first day and 1 min of exploration in the next 5 days. Shuttling number between the chambers served as a measure for exploration level. After exploration period in each training day, training session started with the delivery of the conditioned stimulus (CS; 3000 Hz tone, 75 db, 10 s), immediately followed by the unconditioned stimulus (US; electrical foot-shock, 0.8 mA, 10 s maximum) with an inter trial interval (ITI) of 30 s Responses of the rats during each trial were divided into three types: avoidance (shuttling during the tone and thus avoid the shock), escape (Esc; shuttling during the shock), and Esc failure (animals do not perform shuttling either during the tone or shock). Rats' location was tracked automatically via the weight-sensitive metal grid floors in both compartments and was collected for offline-analysis via the ShuttAvoid Software (Panlab, Harvard Apparatus, Barcelona, Spain). The criterion for successful avoidance learning was set as reaching an

after the end of TWS training animals' behavior was assessed by TWS re-exposure and elevated plus maze (EPM) tests. Two days later, animals underwent cued or

contextual fear-conditioning (FC) and extinction training.

avoidance rate of more them 50% during the training. Rats that haven't reached the criterion were excluded from the analysis.

#### TWS Re-Exposure Test

Two weeks after completing TWS training, rats' behavior was assessed in the TWS box. After 3 min of free exploration in the TWS, rats were presented with 10 presentations of the CS (3000 Hz tone, 10 s maximum) separated by an ITI of 30 s.

#### EPM

Immediately after the end of TWS re-exposure test, all rats were tested in the EPM, a cross-shaped maze 70 cm above the floor, consisting of two opposing open arms and two opposing closed arms (with 30 cm high walls and no roof; total length of arms 112 cm, 8 cm wide). Following 5 min of habituation to the room in a standard cage, each animal was placed in the center of the maze, facing an open arm. Animal was allowed to explore the arena freely for 5 min while its behavior was recorded via the Etho-Vision video tracking system (Noldus Information Technology, Wageningen, Netherlands). Time spent, distance traveled and frequent of entries in the closed and open arms were collected, analyzed, and served as measures of anxiety-related behavior.

#### FC and Extinction Training

After 2 days in the home cage animals went through either cued (unexposed, n = 9; controllable, n = 6) or contextual (unexposed, n = 8; controllable, n = 6) FC and extinction. FC boxes consisted of a square chamber (24 × 26 × 27 cm. Panlab, Harvard Apparatus, Barcelona, Spain). Both FC protocols were conducted in ''context A'' (grid-floor, black walls and full lighted chamber), rats were placed in the FC box and were allowed to explore for 120 s Then, rats were exposed to three CS (10 slight), followed immediately by an US (1 s 0.6 mA shock). It is important to note that we used light instead of tone as a CS in the cued FC and extinction, in order to avoid generalization with respect to the TWS training tone (reviewed by Myers and Davis, 2007). Rats that went through contextual FC were put in ''context A'' for equivalent amount of time as in cue FC, and received three shocks, separated by equal ITIs as in the cued FC protocol.

In the following 3 days rats were subjected to an extinction protocol. During cued fear extinction protocol 10 CSs were presented (every two CSs were later averaged and referred as ''Blocks''), separated by a 120 s interval in ''context B'' (white walls surrounded by a round transparent Plexiglas, metal plain served as the floor, Plexiglas door and dim light. Walls and floor were cleaned with 30% ethanol). Extinction of contextual FC took place in ''context A'', in which the rats were put for an equivalent amount of time as in the cued extinction, with no cue presentation.

Freezing levels during FC and extinction were measured automatically via the weight-sensitive floor and were collected for offline-analysis via the Freezing Software (Panlab, Harvard Apparatus, Barcelona, Spain). Analysis of FC and extinction evaluated freezing levels during CSs presentation in the cued paradigm and at equivalent time periods at the contextual paradigm. Two days prior to FC procedure, rats were habituated for 10 min to ''context B''.

# Brain Preparation

Six hours after TWS and EPM behavioral tests, a subset of animals (unexposed, n = 10; controllable: n = 6) were decapitated, their brains were removed and immediately snap-frozen on powdered dry ice and stored at –80◦C. Brains were mounted on the cerebellum in the cryostat apparatus (chamber temperature −20◦C). The brain was sliced until the mPFC was reached (3.2 mm from Bregma; Paxinos and Watson, 1998). With stainless steel puncher tissue punches of IL and PL sub regions were taken for molecular analysis of alterations in GABA-related gene expression.

# RNA Isolation and Quantitative Real-Time PCR

Sample lysis and subsequent isolation of total RNA via a spin column system was conducted with the RNA Purification Kit, (NORGEN, Thorold, ON, Canada) according to manufacturer's instructions, including steps for removal of genomic DNA. RNA samples were stored at −80◦C until further processing. First-strand synthesis of cDNA was performed with the OuantiTech Reverse Transcription kit (QIAGEN, Hilden, Germany), following an additional step for removal genomic DNA, in the presence of Ouantiscript RT buffer 5× as well as RT primer mix (oligo-dT and random primers) and Ouantiscript reverse transcriptase (Omniscript and Sensiscript Reverse Transcriptase with RNase inhibitor) at 42◦C for 20 min, followed by enzyme inactivation at 95◦C for 3 min. A 1:5 dilution of cDNA samples was used for determination of expression levels of selected target genes by quantitative PCR using the ABI Prism Step One real time PCR apparatus (Life Technologies, Darmstadt, Germany) and TaqMan<sup>r</sup> reagents with predesigned assays for GAD65 (Gad2; assay ID Rn00561244\_m1), GAD67 (Gad1; assay ID Rn00566593\_m1), NPY (assay ID Rn00561681\_m1), CCK (assay ID Rn00563215\_m1) and the housekeeping gene glycerinaldehyd-3-phosphat-dehydrogenase (GAPDH; endogenous control, Life Technologies, Darmstadt, Germany). Target and housekeeping genes were labeled with different fluorescent dyes, allowing for quantitative multiplex PCR. Samples were run in triplicate assays, consisting of 50 cycles of 15 s at 95◦C and 1 min at 60◦C, preceded by a 2 min decontamination step at 50◦C with Uracil-N-Glycosidase and initial denaturation at 95◦C for 10 min.

For data analysis, the mean cycle threshold (CT) was determined for each triplicate assay and relative quantification (RQ) of each target gene was conducted with the ddCT method (Livak and Schmittgen, 2001), normalizing each sample to the overall content of cDNA using GAPDH as an internal control (dCT; dCT = (CT (target gene)) − (CT (GAPDH))). Normalization of all ddCT values was done relative to unexposed group with ddCT = dCT (sample) − mean dCT (unexposed group). Transformation to RQ values for a specific target gene and area was done according to RQ = 2−ddCT .

### Statistical Analysis

Paired or independent samples t-tests and one-way or mixed model repeated-measures ANOVA were conducted on normal distributed data sets (assessed by Shapiro-Wilk's test). Degrees of freedom were corrected when necessary in t-test or repeated measures ANOVA, according to Leven's test or Mauchly's test, respectively. When found a significant interaction effect at mixed-model repeated measures ANOVA, follow up analysis was conducted using one-way repeated measures ANOVA when asking to asses trends in separate groups. Additionally, independent samples t-tests were conducted when asking to assess between-groups simple effects. Variables with distribution deviating from normality were tested using nonparametric tests (specifically, Mann-Whitney U test).

# RESULTS

# Prolonged Exposure to Controllable Stress Within the TWS

On the first day, 10 min pre-training exploration in the TWS was equal in both groups. Controllable and unexposed animals shuttled freely in the TWS arena (Mann-Whitney U test; U = 194.00, Z = −0.41, n.s; **Figure 2A**). In addition, in the following training days a session × group effect was found for the exploration rate (Repeated measures test; F(4,172) = 3.05, p < 0.05; **Figure 2B**), whereas exposure to TWS training significantly decreased shuttling during the first 1 min of exploration in each day (t-test; D2: t(43) = 4.08, p < 0.001; D3: t(43) = 9.30, p < 0.001; D4: t(43) = 9.07, p < 0.001; D5: t(43) = 8.20, p < 0.001; D6: t(43) = 5.36, p < 0.001). When animals freely explored the TWS arena 2 weeks after the end of the TWS training, no difference was detected between groups (t-test; t(43) = −1.36, n.s; **Figure 2C**).

The learning curve of the controllable group was improved during training in the TWS as successful avoidance responses gradually increased (One way repeated-measures ANOVA; F(3,73) = 128.31, p < 0.001), while Esc responses decreased (One way repeated-measures ANOVA; F(3,48) = 52.73, p < 0.001; **Figure 3A**). The acquired responses persisted: when examined 2 weeks later, controllable animals significantly exhibited higher levels of avoidance responses compared to unexposed animals (Mann-Whitney U test; U = 133.50, Z = −2.56, p < 0.05; **Figure 3B**). The number of shuttles during the ITIs was also increased during training (Repeated measures test for days in the TWS; F(5,85) = 26.90, p < 0.001; **Figure 4A**) and 2 weeks later both groups exhibited similar shuttling rates during the ITIs (t-test; t(43) = −0.65, n.s; **Figure 4B**).

#### EPM Test

The controllable group exhibited lower levels of anxiety as reflected by EPM behavioral test in parameters of distance covered (Mann-Whitney test; U = 71.00, Z = −3.99 p < 0.001), time spent (Mann-Whitney test; U = 67.00, Z = −4.08, p < 0.001) and frequency of entries (t-test; t(43) = −4.80, p < 0.001) in open vs. closed arms in comparison to unexposed group (**Figure 5**).

# FC and Extinction

A significant block × group effect was observed for cued FC freezing levels (Mixed model repeated-measures ANOVA; F(2,26) = 7.78, p < 0.01). Accordingly, follow up analysis was performed in order to examine the changes in freezing during the course of training in each of the groups separately. A significant increase during the cue presentations was found in both groups (One-way repeated measures ANOVA; unexposed: F(1,9) = 370, p < 0.001, controllable: F(2,10) = 18.39, p < 0.001). In order to examine differences between the groups in each block, post hoc analysis was performed. However, no significant difference was observed at any of the groups (t-tests; FC: Block 1: t(5) = −2.39, n.s; Block 2: t(6) = −1.60, n.s; Block 3: t(5) = 1.32, n.s).

Taken together, and considering that both groups reached high percentage levels of freezing (100% with no standard deviation and 95.37 ± 8.57 for the unexposed and controllable groups, respectively), it can be concluded that both groups properly acquired FC learning to the same level.

A group × block interaction effect was evident in each of the extinction days (Mixed-model repeated measures ANOVA; Day 1: F(2,32) = 13.71, p < 0.001; Day 2: F(4,52) = 3.25, p < 0.05; Day 3: F(5,52) = 4.13, p < 0.01). Thus, in each of the extinction days, there was a difference in the trend of freezing levels reduction between the groups. Follow-up analysis showed that indeed both groups displayed a reduction in within-session freezing levels at all days (One-way repeated measures ANOVA; Day 1: unexposed: F(4,32) = 7.53, p < 0.001, controllable: F(4,20) = 17.56, p < 0.001; Day 2: unexposed: F(4,32) = 11.09, p < 0.001, controllable: F(4,20) = 22.58, p < 0.001; Day 3: unexposed: F(4,32) = 11.13, p < 0.001, controllable: F(4,20) = 17.45, p < 0.001). Post hoc comparisons of between-groups differences in each block showed that the interaction effect stemmed from a steeper reduction rate in the controllable group compared to the unexposed group. While at the first block of every extinction day there was no difference between the groups (t-test; Day 1: t(13) = −0.09, n.s; Day 2: t(6.20) = 1.08, n.s; t(13) = −0.49, n.s), towards the end of the session (at either the 4th, 5th blocks, or both) the freezing levels of controllable group were significantly lower compared to the unexposed group (t-test; Day 1, 4th block: t(13) = 3.30, p < 0.01; 5th block: t(13) = 8.77, p < 0.001; Day 2: 4th block: t(13) = 3.92, p < 0.01; Day 3: 5th block: t(13) = 2.76, p < 0.05). Hence, we can conclude that while both groups showed reduction in freezing levels, the controllable group displayed faster and greater within-session extinction level.

In addition, when examining ''recall'' of extinction learning between days, a significant difference between second and first day, was revealed (Mixed-model repeated measures ANOVA; group × block interaction effect: F(1,13) = 52.82, p < 0.001). Follow up analysis of the recall effect in each group separately, revealed that freezing levels did not change significantly between first and second day of extinction for the unexposed group

each day of training was decreased in the controllable group along training while unexposed animals maintained the same exploration rate. (C) Two weeks after the end of TWS training no difference in exploration rate was detected between groups. Values presented as mean ± SEM. ∗∗∗significant difference between groups with p < 0.001.

the rate of Esc response was decreased. (B) Two weeks after the end of TWS training controllable animals exhibited more avoidance responses in comparison to the unexposed group. Values presented as mean ± SEM. <sup>∗</sup> significant difference between groups with p < 0.05.

(Related-samples sign test; Z = 1.06, n.s), while freezing levels of the controllable group were significantly elevated between days (Related-samples sign test; Z = 2.04, #p < 0.05, # significant difference between blocks for controllable group). When analyzing the recall effect between second and third day, there was not a significant group × block effect (Mixed-model repeated measures ANOVA; F(1,13) = 4.09, p = 0.064), however, when examining the difference between the groups the same effect was found. The unexposed group showed no difference between second and third day while controllable group did (unexposed: n.s; controllable: #p < 0.05, # significant difference between blocks for controllable group; **Figure 6A**).

FIGURE 4 | Number of shuttles during ITIs within TWS exposure. Number of shuttles during ITIs of controllable group was measured during exposure to TWS training and for both groups in TWS re-exposure test. (A) In the controllable group the number of shuttles during training increased with time. (B) Two weeks after the end of TWS training both groups exhibited similar shuttling rates during ITIs. Values presented as mean ± SEM.

Analysis of contextual FC revealed that both groups reached high percentage levels of freezing by the end of the session (unexposed: 97.30 ± 3.74, controllable: 96.37 ± 4.02), however a significant block × group effect was found (Mixed model repeated-measures ANOVA; F(2,24) = 4.44, p < 0.05). Followup analysis was performed in order to examine the change in freezing during the course of the training in each of the groups separately. A significant increase was found in both groups (Oneway repeated measures ANOVA; unexposed: F(2,14) = 57.48, p < 0.001, controllable: F(1,5) = 29.05, p < 0.01). Taken together, we concluded that both groups properly acquired FC. In order to examine differences between the groups at each FC block, post hoc analysis was performed. A significant difference was observed only at the second block (t-tests; Block 1: t(12) = −1.047, n.s; Block 2: t(12) = −3.44, p < 0.01; Block 3: t(12) = 0.45, n.s). Despite that and due to high levels of freezing at the end of FC training it is safe to assume that contextual FC was achieved. At the first extinction day, no significant group × block interaction effect was found (Mixed-model repeated measures ANOVA; F(4,48) = 0.966, n.s). However, there was a significant main

# significant difference between measures of controllable group with p < 0.05.

effect for the training block (Mixed-model repeated measures ANOVA; F(4,48) = 6.06, p < 0.001). In addition, no significant group main effect was found (Mixed-model repeated measures ANOVA; F(1,12) = 0.48, n.s). Thus, we can conclude that both groups showed reduction of freezing levels within the first extinction session to the same extent. Distinctively, at the second and 3 days of the extinction training, a significant group × block interaction effect was found (Mixed-model repeated measures ANOVA; Day 2: F(4,48) = 12.71, p < 0.001; Day 3: F(2,27) = 5.09, p < 0.05). Thus, in the last 2 days of the extinction training, there was a difference in the trend of freezing level reduction between the groups. Follow-up analysis showed that in the second extinction day both groups displayed a reduction in within-session freezing levels (One-way repeated measures ANOVA; unexposed: F(4,28) = 4.65, p < 0.01, controllable: F(4,20) = 15.81, p < 0.001). Post hoc comparisons of betweengroups differences in each of the blocks showed that the interaction effect stemmed from a steeper reduction rate in the controllable group, compared with the unexposed group. While at the first two blocks of the extinction day, freezing levels of the controllable group were elevated compared to the unexposed group (t-test; Block 1: t(12) = −2.75, p < 0.05; Block 2: t(12) = −4.95, p < 0.01), at the subsequent blocks no significant differences were observed (p > 0.05 for blocks 3–5).

Follow-up analysis of the interaction effect at the third day of contextual fear extinction showed that the controllable group displayed a reduction in freezing levels, while the unexposed group did not (One-way repeated measures ANOVA; unexposed: F < 1, n.s, controllable: F(4,20) = 5.77, p < 0.01). Post hoc comparisons of between-groups differences in each block showed that, similarly to the second day, at the first block of the extinction day freezing levels of the controllable group were elevated compared to the unexposed group (t-test; t(12) = −2.60, p < 0.05), and at the subsequent blocks there was no significant difference (p > 0.05 for blocks 2–5).

In addition, when examining ''recall'' of extinction learning between days, mixed-model repeated measures ANOVA revealed a significant group × block interaction effect between the first and the second day (F(1,12) = 8.25, p < 0.05), and also between the second and third day (F(1,12) = 11.94, p < 0.01). Follow-up analysis of recall interaction effect in each group separately, revealed a dissociation between the groups. Between day 1 and day 2 and between day 2 and day 3, freezing levels of the unexposed group did not change significantly, while freezing levels of the controllable group increased (#p < 0.05, significant difference between blocks for controllable group; **Figure 6B**).

## Gene Expression in the mPFC

Expression of GAD65, GAD67, CCK and NPY were assessed in the PL and IL sub-regions of the mPFC. In the IL, statistical analysis revealed that the controllable group exhibited lower expression levels of GAD65, GAD67 and CCK, but no difference for NPY mRNA levels (t-test; t(14) = 3.05, p < 0.01; t(14) = 2.98, p < 0.05; t(14) = 1.95, p < 0.05; t(14) = 0.77, n.s; respectively. **Figure 7A**). In the PL no significant differences were observed between groups for all genes (t-test; GAD65: t(14) = −0.68, n.s; GAD67: t(14) = −0.34, n.s; CCK: t(14) = −0.32, n.s; NPY: t(14) = −1.36, n.s; **Figure 7B**).

## DISCUSSION

The present study examined the effect of stress controllability on later fear extinction and alterations in GABAergic transmission in mPFC that may mediate this effect. Adult rats underwent prolonged controllable stress training, followed by extinction of either cued or contextual FC. Acquisition of long-term emotional controllability was verified by TWS re-exposure and EPM tests, in which resilient behavior was observed. In addition, controllable stress led to enhanced within-session extinction of cued, but not contextual FC. However, impaired extinction recall was detected in both extinction types following controllable stress. Moreover, exposure to controllable stress led to alterations in GABAergic marker expression in the IL but not in the PL.

Many studies examining controllable vs. uncontrollable experiences employed single day exposure protocols (Drugan et al., 1985; Heinsbroek et al., 1991; Tanaka, 1999; Brennan et al., 2003; Bland et al., 2006; Rozeske et al., 2009). A previous study conducted in our lab demonstrated that after a single day of exposure to controllable training, rats gained operational controllability (avoided the shock when presenting the cue).

relative quantification (RQ) to unexposed group and mean ± SEM per group. ∗ significant difference between groups with p < 0.05.

However, despite the avoidance response acquisition, rats still exhibited high levels of freezing to the context, indicating that they have not yet gained emotional controllability. Similar high freezing levels were observed in rats that were exposed to uncontrollable stress (Ilin and Richter-Levin, 2009). Thus, in the present study, in order to test the subsequent effects of not only operational but also emotional controllability, we employed the previously established 6 days controllable TWS training (Ilin and Richter-Levin, 2009). In the present study, performance improved along days of TWS training, and reached sufficient learning levels after the second day of training (indicated by reaching performance of more than 50% avoidance). It implies that animals gained operational controllability after 2 days of training. The exploration behavior during the TWS test 14 days after training validated that not only operational but also emotional controllability was acquired. Here, exploration levels of the controllable group were comparable to those of the unexposed group, implying reduced anxiety levels in the previously aversive training context. Two weeks after completion of TWS training, rats were also tested in the EPM in order to evaluate anxiety-related behavior. Whereas unexposed animals explored equally the open and closed arms, the controllable group tended to explore more the open arms (evident in several independent measures). Taken together, these results serve as a validation of a long lasting behavioral phenotype difference between the groups (both in and outside of the TWS context), induced by the training protocol. Moreover, they are in line with prior findings, of controllable stress' beneficial effects (Lucas et al., 2014), implying the evolvement of resilience after exposure to a stressful and challenging background.

We next sought out to examine whether the beneficial emotional impact of controllable stress would be also expressed in subsequent extinction learning, despite the aversive experience component of the TWS training. The within-session decrease in freezing levels is a component by which extinction level can be evaluated. A steeper decrease in freezing levels, within each session, is considered to reflect better withinsession extinction. Our results imply a differential impact of the initial TWS training on within-session extinction, dependent upon the FC paradigm type. Contextual fear withinsession extinction of the controllable group was comparable to that of the unexposed group in each day. Distinctively, within-session of cued FC in the controllable group was facilitated, compared to unexposed animals. Overall, while previous experience to controllable stress had a beneficial impact on subsequent cued FC within-session extinction, it did not lead to such an advantage in contextual FC extinction.

The differential effect in within-session extinction may reflect a difference in the way each of the two extinction types correspond to the common prior learning experience in the TWS. It is possible that the extinction of cued FC is facilitated by the TWS training due to the resemblance in the learning processes. Extinction level is the behavioral outcome of two conflicting learning processes: the ''excitatory'' CS-US pairing trace acquired during the FC session, and the ''inhibitory'' CS-noUS trace which is attained during extinction (Eisenberg et al., 2003). Similarly, attainment of active avoidance requires two consecutive and opposing learning processes. The first and essential phase is conditioned reaction to the CS. Then, a suppression of this conditioned response is required in order to allow acquisition of instrumental avoidance contingency (Solomon and Wynne, 1954; Moscarello and LeDoux, 2013). Thus, gaining control over the US involves an inhibition of fear responses that can later lead to reduced anxiety in stressful situations (LeDoux, 2012). It is important to note that the facilitation of extinction we observed is probably not merely due to a sensory generalization process, because the two procedures are dependent upon different modalities. Alternatively, the observed facilitated within-session extinction of cued FC is suggested to stem from the prior experience, which involves a similar learning process in the TWS. In addition, the differential effect in within-session fear extinction cannot be explained by difference in FC acquisition level. In both FC paradigms, controllable and unexposed groups successfully acquired FC learning. This is in agreement with the lack of differences in PL gene expression, a region which is known to play a central role in FC acquisition (Corcoran and Quirk, 2007; Sierra-Mercado et al., 2011).

In addition to the controllable stress training impact on within-session fear extinction, another interesting effect was detected. In the extinction of both FC paradigms an impairment of long-term memory of the successful extinction of the previous day was observed in the controllable group, indicated by impaired between-session extinction recall. The controllable group displayed high levels of freezing in the beginning of each day in comparison to the low levels of fear memory that were established the day before. The fact that this phenomenon was manifested in the controllable group in both extinction paradigms, suggests that the prior TWS training experience served as a crucial factor leading to it. Importantly, impaired extinction recall is known to be a symptom in anxiety disorders and in animal models of stress (Graham and Milad, 2014).

In order to further understand the molecular background of controllability on our FC and extinction results we evaluated mRNA expression levels of GABAergic related markers in mPFC sub-regions. We performed an examination of interneuronassociated neuropeptides, due to their central role in neuronal activity modulation (Baraban and Tallent, 2004). We have previously demonstrated that the expression of GABAergic markers and neuropeptides is modulated in sub-regions of the hippocampus and the BLA after learning and emotional controllability (Hadad-Ophir et al., 2014). In this study, we extended the evaluation of GABAergic interneuron marker expression within mPFC sub regions, due to their well-known role in FC and extinction paradigms. The molecular results revealed alterations in GABAergic marker expression in the mPFC, in a sub-region dependent manner. While no changes in gene expression were observed in the PL, we detected significant alterations within the IL region. Expression of GAD65, GAD67 and the neuropeptide CCK were reduced after controllable stress, while NPY expression remained unaffected. Interestingly, the combination of the behavioral and molecular findings echoes and complements previous findings by Izquierdo et al. (2006). In their work, brief uncontrollable stress led to morphological changes in IL, but not PL. In addition, the uncontrollable stress had no effect on cued FC acquisition; however, within-session extinction was attenuated, in comparison to unstressed controls. The neural dissociation was observed in the current study as well, and it similarly corresponded to the behavioral dissociation. This detected dissociation revealed a beneficial effect of controllable stress with regards to within-session extinction of cued fear, in contrast to the negative effect (Izquierdo et al., 2006) found.

The general trend of a reduced steady-state expression of markers for inhibitory transmission in the controllable group in comparison to the unexposed group may imply a shift in excitation-inhibition balance towards elevated transmission of the IL to downstream regions. One major target is the ITC that inhibits central amygdala neurons. In conjunction with the extinction facilitation that was observed in the current study, the results are in line with numerous studies that relate enhanced activation of IL neurons to reduced freezing during extinction training (e.g., Santini et al., 2004; Sierra-Mercado et al., 2011). However, elevation in IL transmission was also previously associated with intact or facilitated retrieval of extinction memory, a result that is seemingly contradictory to the current findings. In a recent study, Do-Monte et al. (2015) have challenged this view. The authors show that IL transmission is not necessary during the retrieval itself, but is crucial for the storage of extinction memory in target structures. It was suggested that intact IL activity during extinction leads to potentiation of BLA projections to the ITC, which mediates the reduction in freezing levels at the retrieval session. Thus, it is plausible that despite the proposed elevation in IL transmission following controllable stress, the plasticity in this downstream pathway is deficient in these animals, resulting in poor extinction retrieval. Further investigation of this issue will contribute to elucidate these effects.

In conclusion, it appears that controllable stress carries a protective effect on within-session extinction performance. However, it seems that prolonged controllable exposure does not completely abolish the harmful effects of the stressful experience, as controllable animals exhibit impaired fear extinction recall. We propose that stress controllability induces changes in the circuitry that controls extinction, and thereby is likely to underlie the observed facilitation of the withinsession extinction. Resilience induced by controllable stress was

### REFERENCES


previously proposed to evolve from a general resistance to later stressors, and was proposed to be a consequence of inhibitory control exerted by increased activity of the mPFC (Maier and Watkins, 2010). Our findings concur with this proposition, but also suggest a more refined prediction of controllable stress impact on coping with subsequent stressors. It suggests that the resilience induced by stress controllability does not lead to a generalized immunity against later aversive events, as previously proposed, but that the beneficial effect will be dependent upon the features of the controllable stress. Stressors that will resemble the initial experience will enable better coping as revealed by the dissociation between cued and contextual fear extinction following controllable stress training. The results of the current study serve as an example of a complex picture, in which prior stress sets the background for the outcome of subsequent stressful experiences. It demonstrates that the same experience may have a different impact, as a function of the environment in which the later experiences takes place, and the degree to which it shares common features with past learning. Moreover, the results suggest that a prior adverse experience, when controllable, can induce resilience in some aspects, as others remain impaired. Such complexity in considering the effects of stress on later coping is also suggested by the mismatch hypothesis (Schmidt, 2011). This point of view can be beneficial when trying to understand the considerable individual differences observed in anxiety-related pathologies, which may require more complex behavioral interpretations based on the personal history of stress.

# AUTHOR CONTRIBUTIONS

OH-O, NB-E, and GR-L conceived and designed the experiments. OH-O and NB-E performed the experiments. OH-O, NB-E and GR-L analyzed and discussed the data. OH-O, NB-E and GR-L wrote the manuscript.

# ACKNOWLEDGMENTS

This research was funded by The German Israeli Project Cooperation (DIP) RI 1922/1-1 HE 1128/16-1, to GR-L and by a USAMRMC award (10071009) to GR-L.


neurogenesis and fibroblast growth factor-2. Neuroreport 17, 593–597. doi: 10. 1097/00001756-200604240-00008


receptors in the dorsal raphe nucleus. J. Neurosci. 31, 14107–14115. doi: 10. 1523/JNEUROSCI.3095-11.2011


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Hadad-Ophir, Brande-Eilat and Richter-Levin. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Could Stress Contribute to Pain-Related Fear in Chronic Pain?

Sigrid Elsenbruch<sup>1</sup> \* † and Oliver T. Wolf 2†

1 Institute of Medical Psychology and Behavioral Immunobiology, University Hospital Essen, University of Duisburg-Essen, Essen, Germany, <sup>2</sup> Department of Cognitive Psychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Bochum, Germany

Learning to predict pain based on internal or external cues constitutes a fundamental and highly adaptive process aimed at self-protection. Pain-related fear is an essential component of this response, which is formed by associative and instrumental learning processes. In chronic pain, pain-related fear may become maladaptive, drive avoidance behaviors and contribute to symptom chronicity. Pavlovian fear conditioning has proven fruitful to elucidate associative learning and extinction involving aversive stimuli, including pain, but studies in chronic pain remain scarce. Stress demonstrably exerts differential effects on emotional learning and memory processes, but this has not been transferred to pain-related fear. Within this perspective, we propose that stress could contribute to impaired pain-related associative learning and extinction processes and call for interdisciplinary research. Specifically, we suggest to test the hypotheses that: (1) extinction-related phenomena inducing a re-activation of maladaptive pain-related fear (e.g., reinstatement, renewal) likely occur in everyday life of chronic pain patients and may alter pain processing, impair perceptual discrimination and favor overgeneralization; (2) acute stress prior to or during acquisition of pain-related fear may facilitate the formation and/or consolidation of pain-related fear memories; (3) stress during or after extinction may impair extinction efficacy resulting in greater reinstatement or contextdependent renewal of pain-related fear; and (4) these effects could be amplified by chronic stress due to early adversity and/or psychiatric comorbidity such depression or anxiety in patients with chronic pain.

#### Edited by:

Tobias Kalenscher, Heinrich-Heine-Universität Düsseldorf, Germany

#### Reviewed by:

Ales Stuchlik, Academy of Sciences of the Czech Republic, Czech Republic Etienne Vachon-Presseau, Northwestern University, USA

#### \*Correspondence: Sigrid Elsenbruch

sigrid.elsenbruch@uk-essen.de

†These authors have contributed equally to this work.

Received: 27 August 2015 Accepted: 23 November 2015 Published: 17 December 2015

#### Citation:

Elsenbruch S and Wolf OT (2015) Could Stress Contribute to Pain-Related Fear in Chronic Pain? Front. Behav. Neurosci. 9:340. doi: 10.3389/fnbeh.2015.00340 Keywords: chronic pain, pain-related fear, Pavlovian conditioning, extinction, memory, stress

# INTRODUCTION AND SCOPE

Pain is a ubiquitous and uniquely aversive experience with strong emotional components. As such, unavoidable pain is universally feared and literally ''hard to forget''. Indeed, virtually every one of us can readily recall previous painful episodes, even if they occurred years or decades ago, typically motivating strong avoidance behavior driven by pain-related fear. Importantly, pain-related memories are not limited to sensory-discriminative information such type and duration of pain but also include emotional responses as well as information about the entire context surrounding the painful episode. This is due to the fact that acute pain is more than merely a sensory experience. It rather evokes a range of reactions encompassing complex cognitive, emotional, motivational and motor components that are ultimately aimed at self-protection (Lumley et al., 2011). This set of responses is centrally coordinated within the brain, and involves multiple interconnected bodily systems including afferent sensory pathways and efferent Elsenbruch and Wolf Stress and Pain-Related Fear

effector systems including the hypothalamus-pituitary-adrenal (HPA) axis and the sympathetic nervous system (SNS). As such, from an evolutionary perspective, the complex response to acute pain constitutes a highly adaptive, fundamental response that is preserved across species. Fear of pain is an essential component of this adaptive response, which however may come to be maladaptive in chronic pain (Vlaeyen, 2015). Pain-related learning and memory processes envoked by recurrent painful experiences induce complex emotional and behavioral changes which likely contribute to the pathophysiology of chronic pain (Flor, 2012). From a learning perspective, associative as well as instrumental learning and memory processes play a crucial role (Flor, 2012; Gatzounis et al., 2012; Vlaeyen, 2015) and are probably intricately intertwined involving both interoceptive and exteroceptive conditioning. We are only beginning to understand how fear of pain is acquired and extinguished. Within this perspective, we attempt to integrate evidence on associative learning and memory processes from the fields of stress and transfer it to pain. We propose that stress may contribute to impaired pain-related learning and extinction processes and thereby play a role in the transition from acute to chronic pain and/or the maintenance of chronic symptoms. In the case of extinction learning and extinction retrieval, the fascinating question arises whether stress influences the original acquisition memory trace and/or the later developed inhibitory extinction memory trace. Specifically, we propose that: (1) extinctionrelated phenomena inducing a re-activation of maladaptive painrelated fear (e.g., reinstatement, renewal) likely occur in everyday life of chronic pain patients and may alter pain processing, impair perceptual discrimination and favor overgeneralization; (2) acute stress prior to or during acquisition of pain-related fear may facilitate the formation and/or consolidation of painrelated fear memories; (3) stress during or after extinction may impair extinction efficacy resulting in greater reinstatement or context-dependent renewal of pain-related fear; and (4) these effects could be amplified by chronic stress due to early adversity and/or psychiatric comorbidity such as depression or anxiety in patients with chronic pain. Based on these considerations, ideas for much-needed interdisciplinary research are generated that could bridge the cognitive neurosciences with the fields of stress and chronic pain. Note that this perspective focusses specifically on associative learning processes. Nevertheless, instrumental (or operant) learning may be equally relevant in the pathophysiology and treatment of chronic pain (Flor, 2012; Gatzounis et al., 2012; Vlaeyen, 2015), but are beyond the scope.

# CHRONIC PAIN AND STRESS

Chronic pain is a major and unresolved healthcare problem with significant individual as well as societal implications (Breivik et al., 2006). The broad relevance of psychosocial stress in the context of bio-psycho-social disease models of chronic pain is well-established (Lumley et al., 2011; Jennings et al., 2014). Chronic stress and psychiatric comorbidity constitute risk factors for the development and persistence of different types of chronic pain, including visceral pain such as in the irritable bowel syndrome or functional dyspepsia (Elsenbruch, 2011; van Oudenhove and Aziz, 2013), fibromyalgia syndrome (Schmidt-Wilcke and Clauw, 2011), chronic musculoskeletal pain (Finestone et al., 2008; Diatchenko et al., 2013) and migraine (Borsook et al., 2012). For many chronic pain conditions, nonpharmacological treatment approaches incorporate cognitivebehavioral techniques aiming to reduce stress, improve coping and/or ameliorate affective disturbances. In addition, central and/or peripheral stress mechanisms have been proposed as novel targets for therapeutics in the treatment of pain (McEwen and Kalia, 2010; Johnson and Greenwood-Van Meerveld, 2014; Nekovarova et al., 2014). In spite of this converging clinical evidence supporting the crucial importance of stress and altered stress systems in the etiology and pathophysiology of chronic pain (Borsook et al., 2012; Vachon-Presseau et al., 2013), the central mechanisms underlying interactions between stress (or stress mediators) and altered pain-related learning and memory processes remain unclear. Meanwhile, functional and structural brain alterations involved in the pathophysiology of chronic pain are increasingly well-characterized (Ossipov et al., 2010), and overlap with brain circuits involved in emotion regulation and stress (Baliki and Apkarian, 2015) as well as with regions mediating fear expression and recovery (Dejean et al., 2015).

# PAIN-RELATED FEAR

Learning to predict pain based on predictive internal or external cues can be considered a fundamental and highly adaptive process which allows the organism to evoke the above described range of complex responses aimed at self-protection. Pavlovian fear conditioning as a translational model in the neurosciences has proven highly fruitful to elucidate associative learning and extinction processes involving aversive stimuli (Milad and Quirk, 2012), including pain. Conceputally, as a result of contingent pairing of pain-predictive conditioned cues (CS+) with pain as unconditioned stimuli (US), differential conditioned responses (CR) in anticipation of pain can be evoked by presentation of the pain-predictive CS<sup>+</sup> when compared to another cue that remains unpaired (CS−). These CR occur during pain anticipation and have been termed pain-related fear (or fear of pain), a concept that increasingly gains attention in the pain field (De Peuter et al., 2011; Vlaeyen, 2015; Zaman et al., 2015). In addition to pain-related fear as the most prominent response, painpredictive CS demonstrably evoke a range of reactions, including increased arousal and selective attention, in line with the complex responses to acute pain described above. At the same time, cues signaling the absence of impeding pain (i.e., CS<sup>−</sup> that remain unpaired with the US) appear to aquire a separate set of responses, in line with their role as safety signals. The relevance of a safety learning process as part conditioning with aversive US is not only supported by a recent brain imaging meta-analysis of human fear conditioing studies (Fullana et al., 2015), but also by experimental data in patients with chronic pain (Volders et al., 2012; Meulders et al., 2014; Icenhour et al., 2015b). While the role of deficient safety learning in the pathophysiology and treatment of chronic pain remains to be clarified, it is conceivable that these signals may further reinforce safety-seeking behavior as a key component of avoidance. Hence, it is likely that the interplay of conditioned pain-related danger and safety signals drives maladaptive avoidance behavior in chronic pain.

Whereas the relevance of classically-conditioned fear is wellestablished in the context of anxiety disorders (Milad and Quirk, 2012; Tovote et al., 2015), it is only beginning to be elucidated in the context of chronic pain. Fear conditioning studies have demonstrated altered fear learning in various patient groups with chronic pain, including fibromyalgia, chronic back pain, chronic tension-type headaches and irritable bowel syndrome as reviewed herein (Vlaeyen, 2015). In light of this converging evidence across diverse chronic pain conditions, it appears that altered acquisition of pain-related fear is clearly characteristic for chronic pain. Meanwhile, the specific contribution of conditioned pain-related fear to the pathophysiology of chronic pain remains an issue of ongoing research. Conceptually, painrelated fear has been embedded in fear avoidance models of chronic pain (Leeuw et al., 2007; den Hollander et al., 2010; De Peuter et al., 2011; Crombez et al., 2012). These models assume that a vicious circle of exaggerated pain-related fear and dysfunctional avoidance is maintained by emotional factors like increased anxiety as well as hypervigilance and pain catastrophizing. It has also been proposed that conditioning may lower pain thresholds (Williams and Rhudy, 2007) or promote sensitization (Overmier, 2002; Jensen et al., 2015) and thus contribute to hyperalgesia, impair perceptual discrimination acuity (Zaman et al., 2015), enhance fear generalization (Meulders et al., 2015) or interfere with normal habituation processes (Lowén et al., 2015). As part of a surgence in new research studies coming from within the pain field, innovative experimental paradigms have been introduced which implement different types of clinically-relevant painful stimuli as US and/or CS, including movement-related (e.g., Meulders and Vlaeyen, 2012) or visceral stimuli (e.g., Yágüez et al., 2005; Kattoor et al., 2013; Icenhour et al., 2015a) aiming to address pain-related fear in the context of different chronic pain conditions characterized by specific types of pain. Meanwhile, brain imaging studies addressing neural mechanisms in patients with chronic pain remain scarce (Labus et al., 2013; Icenhour et al., 2015b), and virtually nothing is known about the possible roles of affective comorbidity and stress in shaping disturbed acquisition and/or impaired extinction of pain-related fear.

# EXTINCTION OF PAIN-RELATED FEAR

Pain-related extinction processes and their underlying neural circuitry remain uncharted research territory, despite first evidence suggesting the efficacy of exposure-based interventions for chronic pain (Vlaeyen et al., 2002; Linton et al., 2008; Woods and Asmundson, 2008; Craske et al., 2011; Ljotsson et al., 2014) and behavioral data supporting impaired extinction in chronic low back pain (Schneider et al., 2004). Extinction—as a form of new inhibitory learning—can be studied using retrieval techniques including reinstatement and renewal paradigms, which have been applied in the context of explaining relapse and return of fear in anxiety disorders (Milad and Quirk, 2012; Vervliet et al., 2013a,b). The renewal effect describes the return of CR to the CS due to a change of the extinction context, while reinstatement is defined as the retrieval of an extinguished memory after unexpected and unpaired exposure to the US. Both techniques are considered important tools into the mechanisms of memory consolidation and reconsolidation, but have rarely been studied in humans with brain imaging techniques. It is readily conceivable, however, how both reinstatement and renewal phenomena could occur in everyday life of chronic pain patients, with possibly detrimental effects: For instance, an unexpected pain experience (i.e., reinstatement) or context change (i.e., renewal) may lead to the retrieval of previously conditioned pain-related fear, resulting in a resurgence of maladaptive avoidance behaviors. In our own experimental work in the field of visceral pain, we tested for reinstatement by presenting unpaired painful stimuli subsequent to an extinction phase. After two feasibility studies in healthy individuals (Kattoor et al., 2013; Gramsch et al., 2014), in a first study in patients with IBS we could show enhanced reactivation of previously extinguished conditioned fear as evidence by differential neural activation (Icenhour et al., 2015b). This calls for more work in patients with chronic pain in order to complement and extend fear conditioning with contextual manipulations, reviewed in Maren et al. (2013), especially using conditioning with clinicallyrelevant painful stimuli (Icenhour et al., 2015a).

# PERIPHERAL AND CENTRAL MECHANISMS OF STRESS

Stress shapes many types of adaptive behaviors by interacting with emotional and cognitive central processes in order to facilitate adaptation. Given the well-known overlap between stress, affective disturbances, deficits in emotion regulation and chronic pain, integrating stress into future research into extinction learning appears important and in fact necessary.

The first, rapid response to acute stress is orchestrated by activation of the SNS resulting in a rapid release of adrenalin and noradrenalin. These hormones cannot easily pass the blood brain barrier, but stimulate the vagus nerve, which causes an increased noradrenergic tone in the brain by its action on regions in the brain stem (Roozendaal et al., 2009). These regions in turn influence several brain areas including the amygdala and prefrontal cortex, which are both crucially involved in the regulation of learned fear (Dejean et al., 2015; Maren and Holmes, 2015) as well as central pain processing (Ossipov et al., 2010; Baliki and Apkarian, 2015). A second, slower response is orchestrated by the HPA axis. Corticotrophin releasing hormone (CRH) is released from the paraventricular nucleus of the hypothalamus into the portal blood system. On reaching the pituitary, CRH stimulates adrenocorticotrophin (ACTH) release into the peripheral blood stream, which initiates the secretion of glucocorticoids (GCs; corticosterone in most laboratory animals, cortisol in humans) from the adrenal cortex (Joels and Baram, 2009). In contrast to the catecholamines, naturally occurring GCs (like all other steroid hormones) can pass the blood brain barrier. In the brain, GCs can act via two different intracellular receptors (sometimes referred to as type I or mineralocorticoid (MR) and type II or glucocorticoid (GR) receptor), which differ in their distribution and affinity (Joels et al., 2008). Moreover GCs can exert rapid non-genomic effects, via membrane bound MRs and GRs or via interaction with other neurotransmitter receptors (Joels et al., 2008). GCs can influence neuronal excitability, neuronal plasticity, dendritic remodeling and neurogenesis (Roozendaal et al., 2009; Maren and Holmes, 2015). Besides, multiple neurotransmitter systems like the cholinergic, noradrenergic, serotonergic and dopaminergic system are influenced by GCs (Joels et al., 2008). In sum, catecholamines and GCs can have rapid as well as delayed effects on the function and structure of the brain, and thereby affect emotion regulation, including the acquisition and extinction of learned fear. Of note, existing experimental data in humans primarily address effects of acute stress (or stress mediators, particularly GC) rather than chronic stress, as explained below.

# EFFECTS OF ACUTE STRESS ON ACQUISITION AND CONSOLIDATION

Effects of acute stress on learning and memory processes are demonstrably (learning) phase-dependent, requiring a careful separation of the processes underlying acquisition, consolidation, reconsolidation and retrieval (for an illustration, see **Figure 1**). In addition to aspects of timing, consequences of stress exposure and/or application of stress hormones appear to vary with the type of learning with possible differences between for example rather neutral declarative versus emotional learning tasks. Briefly, as reviewed in Raio and Phelps (2015) animal research on cued fear supports that stress exposure facilitates the acquisition and consolidation of cued fear. Similar findings exist in humans. It has been observed that pre-learning GC treatment (Buchanan and Lovallo, 2001; Kuhlmann and Wolf, 2006) or immediate post learning stress (Cahill et al., 2003; Preuß and Wolf, 2009) enhanced memory consolidation resulting in enhanced retrieval days to weeks later. In several studies, this effect was more pronounced for arousing material (Kuhlmann and Wolf, 2006; Smeets et al., 2008). The number of human studies explicitly addressing effects of stress on conditioned fear remains small with partially inconsistent findings, which may in part be due to sex differences (Merz et al., 2010, 2013) and divergent effects depending on the timing of stress with respect to the beginning of the acquisition (Hermans et al., 2014). Effects of acute stress on the acquisition of pain-related fear have not been studied thus far. We speculate that acute stress prior to or during acquisition of pain-related fear may facilitate memory formation and/or consolidation, which would result in greater retrieval of pain-related fear at later time points (see **Figure 1A**). This would be in line with clinical evidence that stress can lead to symptom reoccurrence in a wide range of disorders including stress-related and anxiety disorders (Wolf, 2008; Maren and Holmes, 2015).

# EFFECTS OF ACUTE STRESS ON EXTINCTION

In the case of extinction learning and extinction retrieval, the fascinating question arises whether stress influences the original acquisition memory trace or the later developed inhibitory extinction memory trace. The influence of stress and the potential role of GCs on extinction have been investigated in rodents already in the 70's of the last century (Bohus et al., 1970; Kovács et al., 1977). From these studies, the notion emerged that GCs facilitate extinction, an interpretation supported by more recent reports (Yang et al., 2006; Brinks et al., 2009). For example, Yang et al. (2006) demonstrated that intra amygdala infusion of a GR receptor agonist facilitated extinction, while blockage of GR production with metyrapone impaired extinction. Most of these previous studies did not test the long-term consequences (extinction retrieval) of the extinction manipulation (for a review, see Rodrigues et al., 2009).

In a series of studies involving healthy human volunteers, we have recently tested the impact of stress on extinction retrieval using a renewal paradigm (Hamacher-Dang et al., 2013; Merz et al., 2014). Results revealed that stress impaired extinction retrieval in a predictive learning task but impaired the retrieval of the original fear memory trace in a fear-conditioning task. These results are in line with the hypothesis that the more emotional memory trace is more heavily influenced by stress. With respect to extinction consolidation it could be demonstrated that post-extinction stress led to a more contextdependent extinction memory, which was associated with a more pronounced renewal effect (Hamacher-Dang et al., 2013, 2015). Taken together, the findings demonstrate that stress has a phasedependent effect on extinction learning (see **Figure 1**) which is further modulated by the emotionality of the learning material and by the context in which the learning took place. Given the emotional components of pain and pain-related fear as an essentially emotional construct, this has interesting implications for the design of mechanistic studies aiming to test differential effects of stress on the retrieval of pain-related fear, as detailed below. The most important prediction that can be made is that acute stress during extinction learning may improve extinction learning resulting in greater retrieval of the extinction memory and thus reduced reinstatement or renewal (**Figure 1B**). On the other hand, stress just before or during extinction retrieval may result in reduced retrieval of the extinction memory trace and thus greater reinstatement or context-dependent renewal (see **Figure 1C**). These predictions could be tested using psychosocial stress models or administration of appropriate agonists of the HPA axis and/or the SNS.

# REACTIVATION, RECONSOLIDATION AND ITS MODIFICATION THROUGH STRESS HORMONES

Recently interest in the phenomenon of reconsolidation has surged. Building on findings from the sixties (Misanin et al., 1968), Nader and colleagues were able to show in rodents that established fear memories become labile after reactivation (exposure to the CS) and have to reconsolidate again. Post reactivation protein synthesis inhibition completely erased the fear memory (for a review, see Nader and Hardt, 2009). For fear conditioning, similar results (i.e., impaired reconsolidation) could be obtained using a beta receptor blocker in rodents as well as humans (Kindt et al., 2009). Recent

research from our group in humans has revealed that the stress hormone cortisol enhances fear reconsolidation (Meir Drexler et al., 2015). The enhancing effects of stress mediators on memory reconsolidation may in part explain the long lasting memories of aversive events. Each stressful reactivation will further strengthen the memory trace. Applying these findings to chronic pain, one could postulate that stress results in a reactivation of the pain-related memory trace and/or facilitates its reconsolidation, ultimately making the pain-related fear memory more permanent. This process may contribute to the maintenance of pain-related fear and maladaptive avoidance behavior as part of a vicious circle maintained by stress and fear. Furthermore, research into interactions between affective comorbidity, acute stress and memory processes may contribute to elucidating individual risk and vulnerability factors and neuropharmacological treatment options for chronic pain (Nekovarova et al., 2014).

# CONCLUSIONS AND FUTURE DIRECTIONS

We propose that stress may be linked to impaired extinction and enhanced retrieval of pain-related fear in patients with chronic pain—hypotheses that are yet to be tested. In this much needed work, it is important to consider that the modulation of memory processes may differ depending on the type and duration of stress. A difference between acute and chronic stress is supported by data outside of the pain field: Early adversity as well as chronic stress has been linked to structural alterations in the brain causing a hyperactive amygdala and impaired prefrontal inhibition (Roozendaal et al., 2009). These alterations could underlie the extinction impairments observed in several mental disorders (Maren and Holmes, 2015), which is interesting in the context of chronic pain given the high comorbidity between chronic pain and affective disorders. Finally, effects of acute stress on pain-related memory retrieval may be fundamentally different in normals and patients with chronic pain. While stressinduced effects in normals are adaptive, they may be altered and in fact maladaptive and in patients with chronic pain. For example in patients with PTSD, cortisol enhanced rather than impaired memory retrieval [for review, see Wingenfeld and Wolf (2015)]. Another future research area is to address if and to what extent stress may affect overgeneralization, as recently shown in patients with fibromyalgia (Meulders et al., 2015) and/or perceptual discrimination (Zaman et al., 2015). Clearly, more patient-oriented experimental work is needed to disentangle the complex interactions between acute and chronic stress and different pain-related memory processes encompassing extinction, extinction retrieval and memory consolidation and reconsolidation. Ultimately, this work could be the basis for an improvment of existing treatment approaches for patients with chronic pain, who benefit from exposure-based interventions. Studies in anxiety patients have observed that cortisol enhances the success of extinction-based therapies in patients with fear of heights (de Quervain et al., 2011) as well as in spider phobics (Soravia et al., 2014), presumably by boosting extinction

### REFERENCES


consolidation. Although effects of acute stress or acute GR administration on extinction of pain-related fear have not been tested humans, this could be a promising endeavor for patients with chronic pain.

### FUNDING

Both authors are members of the research unit ''Extinction Learning: Behavioural, Neural and Clinical Mechanisms'' funded by the German Research Foundation (DFG, FOR 1581).


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Elsenbruch and Wolf. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Contingency Awareness Shapes Acquisition and Extinction of Emotional Responses in a Conditioning Model of Pain-Related Fear

#### Franziska Labrenz † , Adriane Icenhour † , Sven Benson and Sigrid Elsenbruch\*

Institute of Medical Psychology and Behavioral Immunobiology, University Hospital Essen, University of Duisburg-Essen, Essen, Germany

As a fundamental learning process, fear conditioning promotes the formation of associations between predictive cues and biologically significant signals. In its application to pain, conditioning may provide important insight into mechanisms underlying pain-related fear, although knowledge especially in interoceptive pain paradigms remains scarce. Furthermore, while the influence of contingency awareness on excitatory learning is subject of ongoing debate, its role in pain-related acquisition is poorly understood and essentially unknown regarding extinction as inhibitory learning. Therefore, we addressed the impact of contingency awareness on learned emotional responses to pain- and safety-predictive cues in a combined dataset of two painrelated conditioning studies. In total, 75 healthy participants underwent differential fear acquisition, during which rectal distensions as interoceptive unconditioned stimuli (US) were repeatedly paired with a predictive visual cue (conditioned stimulus; CS<sup>+</sup>) while another cue (CS<sup>−</sup>) was presented unpaired. During extinction, both CS were presented without US. CS valence, indicating learned emotional responses, and CS-US contingencies were assessed on visual analog scales (VAS). Based on an integrative measure of contingency accuracy, a median-split was performed to compare groups with low vs. high contingency accuracy regarding learned emotional responses. To investigate predictive value of contingency accuracy, regression analyses were conducted. Highly accurate individuals revealed more pronounced negative emotional responses to CS<sup>+</sup> and increased positive responses to CS<sup>−</sup> when compared to participants with low contingency accuracy. Following extinction, highly accurate individuals had fully extinguished pain-predictive cue properties, while exhibiting persistent positive emotional responses to safety signals. In contrast, individuals with low accuracy revealed equally positive emotional responses to both, CS<sup>+</sup> and CS<sup>−</sup>. Contingency accuracy predicted variance in the formation of positive responses to safety cues while no predictive value was found for danger cues following acquisition and for neither cue following extinction. Our findings underscore specific roles of learned danger and safety in pain-related acquisition and extinction. Contingency accuracy appears to distinctly impact learned emotional responses to safety and danger cues, supporting aversive learning to occur independently from CS-US awareness.

#### Edited by:

Denise Manahan-Vaughan, Ruhr University Bochum, Germany

#### Reviewed by:

Christa McIntyre, University of Texas, USA Christine A. Rabinak, Wayne State University, USA

\*Correspondence: Sigrid Elsenbruch sigrid.elsenbruch@uk-essen.de

†These authors have contributed equally to this work.

Received: 30 September 2015 Accepted: 08 November 2015 Published: 27 November 2015

#### Citation:

Labrenz F, Icenhour A, Benson S and Elsenbruch S (2015) Contingency Awareness Shapes Acquisition and Extinction of Emotional Responses in a Conditioning Model of Pain-Related Fear. Front. Behav. Neurosci. 9:318. doi: 10.3389/fnbeh.2015.00318 The interplay of cognitive and emotional factors in shaping excitatory and inhibitory pain-related learning may contribute to altered pain processing, underscoring its clinical relevance in chronic pain.

Keywords: contingency awareness, pain-related fear, safety learning, fear conditioning, extinction, visceral pain, chronic pain

# INTRODUCTION

As a translational model in the neurosciences, fear conditioning is increasingly implemented in the field of pain research. One important argument in support of applying human fear conditioning in the context of pain is the high comorbidity of chronic pain with anxiety disorders (Breivik et al., 2006), suggesting shared mechanisms underlying both, pathological fear and pain. Indeed, altered fear learning has been demonstrated in various patient groups with chronic pain, including fibromyalgia, chronic back pain, chronic tension-type headaches and irritable bowel syndrome (IBS; Vlaeyen, 2015). These converging findings support that conditioned pain-related fear may play a role in the transition from acute to chronic pain as well as in the maintenance of chronic pain, as postulated by fear-avoidance models (Vlaeyen, 2015). Moreover, extinction-based treatment approaches have successfully been translated into the development and application of exposure therapy for chronic pain (den Hollander et al., 2010), specifically targeting pain-related fear and maladaptive avoidance behaviors (Vlaeyen, 2015). Meanwhile, the mechanisms underlying the formation and especially the extinction of pain-related fear remain incompletely understood even in healthy individuals, calling for more experimental work.

In recent years, a number of research groups has introduced innovative conditioning paradigms with clinically relevant pain stimuli as unconditioned stimulus (US) or conditioned stimulus (CS) to capture different aspects of pain-related learning and extinction processes in healthy volunteers (Meulders et al., 2011, 2013; Pappens et al., 2012, 2015; Kattoor et al., 2013; Benson et al., 2014; Gramsch et al., 2014; Icenhour et al., 2015a) and patients with chronic pain (Meulders et al., 2014, 2015; Icenhour et al., 2015b). One aspect that has not been specifically addressed in these promising experimental approaches is the role of conscious awareness of contingencies between predictive cues (i.e., CS) and pain. In general, the acquisition of emotional memories has long been assumed to rely on the awareness of relationships between cue and outcome, operationalized as the ability to verbalize this relation (Lovibond and Shanks, 2002). Contingency awareness was accordingly conceptualized as a mediator between associative learning and the display of conditioned responses (Lovibond, 2003, 2004). In support of this notion, several human studies reliably observed conditioned responses in perceived CS valence and in physiological changes of skin conductance responses in aware subjects only, suggesting that contingency awareness is necessary for successful fear conditioning (Lovibond and Shanks, 2002; Tabbert et al., 2006, 2011; Klucken et al., 2009; Mitchell et al., 2009; Lovibond et al., 2011; Weidemann and Antees, 2012; Weidemann et al., 2013). However, others have questioned this assumption given evidence that autonomic fear responses also occur without explicit knowledge regarding contingencies (Wiens and Öhman, 2002; Smith et al., 2005; Knight et al., 2006, 2009; Schultz and Helmstetter, 2010; Raio et al., 2012).

In light of this debate and first data supporting altered contingency learning and extinction in chronic pain patients (Jenewein et al., 2013; Meulders et al., 2014; Icenhour et al., 2015b), our goal was to address the putative role of contingency awareness in shaping acquisition and extinction of conditioned emotional responses to predictive cues in a visceral painrelated conditioning model. To do so, we analyzed behavioral data in a large, pooled sample of healthy volunteers with a focus on classically-conditioned changes in perceived valence of predictive cues that were either consistently paired with visceral pain as US (i.e., pain-predictive CS+) or cues that were never paired with pain (i.e., CS−). We conducted separate analyses for conditioned emotional responses to CS<sup>+</sup> and CS<sup>−</sup> rather than relying solely on differential measures. The rationale was that recent evidence from the broader fear conditioning field supports that safety learning processes, induced by CS<sup>−</sup> as safety cues, engage distinct brain regions (Fullana et al., 2015), suggesting a separate process which may indeed play a unique role in the context of pain (Volders et al., 2012; Jenewein et al., 2013; Meulders et al., 2014; Icenhour et al., 2015a,b; Labrenz et al., 2015). To address the putative role of contingency awareness in shaping distinct negative and positive emotional learning and extinction regarding pain and safety, we implemented a new integrative measure of contingency accuracy, and tested the hypothesis that individuals with high contingency accuracy would show more pronounced negative as well as positive emotional responses following acquisition. In addition, we examined whether higher contingency accuracy would result in impaired extinction of emotional responses, characterized by persisting negative and positive valence of formerly pain-predictive as well as safety cues after extinction in individuals with high contingency accuracy. Finally, we explored associations between contingency accuracy and valence changes and tested accuracy as a predictor of variance in pain-related negative and positive emotional learning and extinction.

#### MATERIALS AND METHODS

#### Participants

For this analysis, behavioral data from two brain imaging studies implementing identical differential conditioning paradigms Labrenz et al. Contingency Awareness and Pain-Related Conditioning

using painful rectal distensions as US were pooled (Icenhour et al., 2015a; Labrenz et al., unpublished data) resulting in a sample of N = 75 healthy individuals (38 women, 37 men; mean age 28.87 ± 9.6 years) for the acquisition phase. Since only one of these studies (Icenhour et al., 2015a) contained an extinction phase, the sample size was N = 48 (24 women, 24 men; mean age 29.87 ± 10.84 years) for extinction. Exclusion criteria for both studies were age <18 or >60 years, any known medical and psychiatric conditions or chronic medication use (except hormonal contraceptives or occasional use of over-the-counter allergy or pain medications) based on self-report. All but N = 7 women were on oral contraceptives. A standardized in-house questionnaire was used to exclude any symptoms suggestive of functional or gastrointestinal conditions (Lacourt et al., 2014) and all participants were tested for perianal tissue damage (i.e., painful hemorrhoids) potentially interfering with balloon placement. Pregnancy was ruled out with a commercially available urinary test on the day of the study. Any previous participation in a conditioning study was also exclusionary. Screening for current anxiety or depression symptoms was accomplished with the German version of the Hospital Anxiety and Depression Inventory using published cut-off values, i.e., ≥8 (HADS; Herrmann-Lingen et al., 2005). The study protocols were approved by the local ethics committee (University of Duisburg-Essen, Germany) and followed the Declaration of Helsinki. All participants gave informed written consent and were paid for their participation.

# Experimental Protocol

The differential conditioning protocol with visceral pain as US has previously been applied in healthy volunteers (Kattoor et al., 2013; Gramsch et al., 2014) and patients with IBS (Icenhour et al., 2015b). In brief, moderately painful rectal distensions, accomplished with a pressure-controlled barostat system (modified ISOBAR 3 device, G & J Electronics, ON, Canada), served as clinically relevant and effective visceral US, representing a valid and reliable experimental model for the investigation of visceral pain processing (Mayer et al., 2008; Keszthelyi et al., 2012). The stimulus intensity (i.e., distension pressure) for conditioning was initially determined based on individual rectal pain thresholds in order to ensure comparably painful US in all individuals. To do so, individualized distension pressures corresponding to perceived pain intensities between 60 and 70 on a 0–100 mm visual analog scale (VAS) were chosen for US application during conditioning. The protocol consisted of an acquisition and an extinction phase, which were each followed by VAS ratings of CS valence and contingencies (see below). Initially, participants were instructed that during the experiment, they would see visual signals and experience repeated rectal distensions but received no information regarding experimental phases or cue-outcome contingencies. During acquisition, one geometric visual symbol (CS+) was consistently paired with a painful rectal distension (US) while a second visual cue (CS−) was never followed by the US (differential delay conditioning). Overall, 32 CS were presented (16 CS<sup>+</sup> and 16 CS−) in pseudo-randomized order with a 75% reinforcement schedule to induce uncertainty and ensure more robust conditioned responses (Kalisch et al., 2006; Sehlmeyer et al., 2009). US onset varied randomly between 8 and 12 s after CS<sup>+</sup> onset and both stimuli co-terminated. Intertrial intervals (ITI) were 20 s. During extinction, 24 CS were presented (12 CS<sup>+</sup> and 12 CS−) in pseudo-randomized order without any US presentations. Note that extinction was conducted with a subtle context change consisting of a change in CS background color in half of the participants. However, given no context-related effects on behavioral measures or skin conductance responses (Icenhour et al., 2015a), data were pooled herein.

# Valence Ratings of CS

Conditioned changes in perceived valence of previously neutral predictive cues constitute an established behavioral marker in fear conditioning, capturing learned emotional responses which are demonstrably associated with neural correlates of pain-related fear and safety learning (Kattoor et al., 2013). To quantify emotional responses to pain-predictive cues (CS+) and safety cues (CS−), participants rated CS valence using a hand-held fiber optic response system (LUMItouchTM, Photon Control Inc., Burnaby, BC, Canada). Specifically, participants were prompted to indicate the unpleasantness of each cue separately on a digitized +100 to −100 VAS with end points labeled as ''very unpleasant'' and ''very pleasant'', while ''neutral'' (= 0) was indicated in the middle of the scale. These ratings were accomplished prior to acquisition (baseline) and immediately following acquisition and extinction.

# Contingency Awareness and Accuracy

Awareness of CS-US contingencies was assessed with digitized VAS at the conclusion of acquisition and extinction phases. Participants were prompted to indicate how often each of the cues was followed by pain on a 0–100 mm scale with end points labeled ''never'' and ''always''. In addition, we computed a novel and integrative measure to adequately quantify the accuracy of contingency awareness. The rationale was that in differential learning paradigms, a differentiation between CS+- and CS−-related contingencies is often reported as a marker of successful contingency learning (e.g., Tabbert et al., 2011). However, the mere differentiation does not provide sufficient information on distinct influences of pain- and safetyrelated behavioral responses. Moreover, contingency ratings do not resemble a direct and explicit measure of accurate contingency awareness, especially during partial reinforcement schedules where an overestimation of CS+-US pairings would be falsely interpreted as high awareness. Therefore, VAS ratings for CS+- and CS−-US contingencies were transformed into contingency accuracy scores in percent, assigning 100% accuracy to CS+-US ratings of 75 on the VAS (representing the correct contingency in this study) and 100% to CS−-US contingency ratings of 0 (which was also the correct contingency herein), thereby providing separate but comparable measures for CS+- and CS−-related awareness and deviations from full accuracies. Hence, for an individual CS+-US rating of 75 mm, contingency accuracy was considered 100%, whereas ratings of either 65 mm (i.e., underestimation of real contingency) or 85 mm (overestimation of real contingency) would result in accuracy scores of 100–13.3% = 86.7%. This measure was used for comparisons of groups with high and low contingency accuracy as well as for regression analyses (see below).

#### Statistical Analyses

Statistical analyses were carried out with the Statistical Package for the Social Sciences (SPSS, IBM Corp., IBM SPSS Statistics for Windows, Version 22.0. IBM Corp., Armonk, NY, USA). To initially confirm differential changes in CS valence and CS-US contingencies, analyses of variance (ANOVA) were conducted, reporting results with Greenhouse-Geisser correction for significant interactions. Post hoc testing was accomplished with Bonferroni correction to control for inflation of alpha values set at p < 0.05 due to multiple comparisons. In order to compare participants with high vs. low contingency accuracy, a median-split was conducted based on the mean accuracy score in percent and groups were compared using ANOVA followed by post hoc two sample t-tests. Correlational analyses were accomplished using Pearson's r, followed by multiple regression analyses predicting valence changes after acquisition and extinction, respectively. All data are given as mean ± standard error of the mean (SEM), unless indicated otherwise.

### RESULTS

# Changes in CS Valence after Acquisition and Extinction

We initially confirmed differential changes in CS valence irrespective of contingency awareness for acquisition and extinction in the whole sample. For acquisition, ANOVA revealed a significant phase × CS-type interaction (F(1,74) = 109.333, p < 0.001) as well as significant main effects (both p < 0.001). Post hoc t-tests confirmed that while CS<sup>+</sup> and CS<sup>−</sup> were rated as equally neutral at baseline, CS<sup>+</sup> was perceived as significantly more unpleasant compared to CS<sup>−</sup> following acquisition (t(74) = 11.388, p < 0.001; **Figure 1**). Importantly, while CS<sup>+</sup> unpleasantness significantly increased from baseline to after acquisition (t(74) = 11.249, p < 0.001) indicating learned aversion, CS<sup>−</sup> was rated as significantly more pleasant following acquisition when compared to baseline (t(74) = 3.938, p < 0.001), suggestive of safetyrelated learning during differential conditioning (**Figure 1**). For extinction, a significant phase × CS-type interaction (F(2,46) = 61.778, p < 0.001) as well as significant main effects (both p < 0.001) were found. Post hoc testing yielded diminished CS+-CS<sup>−</sup> differentiation following extinction as well as significant changes of both, CS<sup>+</sup> (t(47) = 10.716, p < 0.001) and CS<sup>−</sup> valence (t(47) = 3.356, p = 0.002) when compared to acquisition levels, indicating successful extinction (**Figure 1**).

#### Contingency Awareness and Accuracy

Contingency ratings regarding CS+-US and CS−-US pairings were assessed to ensure differential contingency awareness following acquisition and awareness of changed contingencies following extinction. After acquisition, perceived CS+-US contingency (M = 71.84 ± 2.53 mm) was significantly different from perceived CS−-US contingency (M = 21.11 ± 2.98 mm; t(74) = 11.741, p < 0.001), confirming the formation of differential contingency awareness. After extinction, participants were aware of changed contingencies, as evidenced by comparable ratings (CS+-US: 11.96 ± 3.10 mm; CS−-US: 7.29 ± 2.10 mm).

Analysis of contingency accuracy scores revealed comparable accuracies for both the CS<sup>+</sup> and the CS<sup>−</sup> after acquisition (CS+- US: 79.89 ± 2.49%; CS−-US: 78.89 ± 2.98%) as well as after extinction (CS+-US: 88.04 ± 3.10%; CS−-US: 92.71 ± 2.10%). Although accuracies for both CS were comparable, they neither reached 100% after acquisition nor after extinction, indicating deviations from perfect contingency accuracies during both acquisition as excitatory and extinction as inhibitory learning. Interestingly, while acquisition CS<sup>+</sup> and CS<sup>−</sup> accuracy scores were not inter-correlated (r = 0.116; p = 0.321), a significant inter-correlation was found for extinction (r = 0.570; p < 0.001). This supports distinct and independent contributions of both, CS<sup>+</sup> and CS<sup>−</sup> processing to the formation but not the extinction of perceived CS-US contingencies.

# Analyses in Subgroups with High vs. Low Contingency Accuracy

In order to compare participants with high vs. low contingency accuracy with respect to valence changes (see below), a mediansplit was conducted based on the mean accuracy score in percent (83.33%), resulting in a group with high mean accuracy (N = 41; M = 91.54 ± 0.80%) and a low accuracy group (N = 34; M = 64.74 ± 2.81%). Note that these groups did not differ with respect to distribution of sex or age (data not shown). Independent sample t-tests confirmed significant group differences for both, CS+-US (t(74) = 4.008, p < 0.001) as well as CS−-US contingency accuracies (t(74) = 7.047, p < 0.001; **Figure 2**).

In order to compare the groups with respect to valence changes, repeated measures ANOVA followed by t-tests were conducted. For acquisition, results indicated a significant group × CS-type interaction (F(1,73) = 26.750; p < 0.001). While both groups showed significant CS+-CS<sup>−</sup> differentiation in CS valence (high accuracy: t(40) = 14.903; p < 0.001; low accuracy: t(33) = 4.537; p < 0.001), differential emotional responses were more pronounced in highly accurate individuals (t(73) = 5.172; p < 0.001; **Figure 3A**). Specifically, highly accurate participants perceived the CS<sup>+</sup> as more unpleasant (t(73) = 3.348; p = 0.001) while the CS<sup>−</sup> was rated as more pleasant (t(73) = 4.902; p < 0.001), supporting enhanced emotional learning of both, danger and safety cue properties (**Figure 3A**). Following

extinction, ANOVA of valence ratings revealed a trend towards CS-type × group interaction (F(1,46) = 3.819; p = 0.057). T-tests yielded a significant between-group difference in CS<sup>+</sup> valence (t(46) = 2.850; p = 0.007), resulting from higher pleasantness in the low accuracy group (**Figure 3B**). Furthermore, there was a significant difference between CS<sup>+</sup> and CS<sup>−</sup> valence in the highly accurate group only (t(24) = 2.124; p = 0.044, **Figure 3B**), which was attributable to persistently higher pleasantness of CS<sup>−</sup> in relation to CS+, indicating incomplete extinction particularly of learned safety cue properties.

#### Correlations and Regression Analyses

To address associations between valence ratings and contingency accuracy for the CS<sup>+</sup> and CS−, correlation analyses were conducted (**Table 1**). For acquisition, analyses revealed significant correlations between CS<sup>+</sup> and CS<sup>−</sup> valence.

Furthermore, CS<sup>−</sup> contingency accuracy was significantly associated with both CS<sup>+</sup> and CS<sup>−</sup> valence, whereas no associations were found for CS+-related accuracy. For extinction, CS<sup>+</sup> and CS<sup>−</sup> contingency accuracy scores and valence ratings were significantly inter-correlated. Besides, significant associations between CS<sup>+</sup> valence following acquisition and extinction were observed (**Table 1**).

In a final step, multiple regression analyses were conducted to test if contingency accuracy constitutes a significant predictor for CS valence after acquisition or extinction. As shown in **Table 2**, the model for valence of CS<sup>−</sup> after acquisition was predicted by CS−-US accuracy, along with CS<sup>+</sup> valence, supporting a role of contingency accuracy in the acquisition of positive emotional responses to conditioned safety cues. The other models revealed no evidence for contingency accuracy as a significant predictor (**Table 2**). Together, the results suggest that explicit knowledge about CS-US relations, particularly regarding safety, is a predictor for learned positive emotional responses, while not predicting extinction of learned cue properties.

# DISCUSSION

The importance of pain-related fear in the pathophysiology and treatment of chronic pain is increasingly recognized, inspiring experimental work to elucidate the mechanisms underlying pain-related learning and memory processes specifically in the context of pain (Vlaeyen, 2015). Human fear conditioning studies with highly aversive or painful US support distinct emotional learning processes characterized not only by negative emotions in response to predictive danger cues (i.e., CS+) but also by positive emotions in response to cues signaling safety (i.e., CS−). At the behavioral level, these processes are reflected by changes in perceived valence of previously neutral predictive cues that turn into unpleasant or pleasant signals, respectively, depending on cue-outcome contingencies.

Importantly, these learned emotional responses may be mediated by distinct neural networks (Fullana et al., 2015), and appear to be altered in patients with chronic pain (Jenewein et al., 2013; Icenhour et al., 2015b). One important, yet unsolved question concerns the putative role of conscious awareness of cue-outcome relations in shaping the acquisition and extinction of emotional responses to predictive cues. Therefore, we aimed to address the role of contingency awareness, operationalized as a novel and integrative measure of contingency accuracy, in a large, pooled sample of behavioral data from healthy volunteers undergoing visceral pain-related conditioning (Icenhour et al., 2015a; Labrenz et al., unpublished data). We hypothesized that individuals with high contingency accuracy would show more pronounced emotional responses, reflected by CS valence ratings, after conditioning. In line with this assumption, results revealed significantly more pronounced negative emotions in response to the CS<sup>+</sup> as well as greater positive emotions in response to the CS<sup>−</sup> in individuals with high compared to individuals with low contingency accuracy.

The findings support a role of contingency awareness in shaping distinct emotional responses to conditioned danger and safety cues. These results in healthy volunteers are in accordance with earlier evidence from our group that patients with IBS showed higher contingency awareness specifically of safety cues along with more pronounced positive emotions to the same cues (Icenhour et al., 2015b), calling for future research addressing cognitive factors in safety learning in chronic pain. To further substantiate the group differences in the present dataset, we tested if


Significant results are indicated in bold. ∗∗p < 0.01; ∗∗∗p < 0.001.


TABLE 2 | Results of multiple regression analyses with CS valence during acquisition and extinction as outcome variables and accuracy of CS-US contingencies and CS valence as predictors.

Significant results are indicated in bold. ∗∗p < 0.01; ∗∗∗p < 0.001.

contingency accuracy was a predictor of conditioninginduced valence changes after acquisition. We found that contingency accuracy significantly predicted conditioned positive emotional responses to safety cues, suggesting a role of cognitive aspects in safety learning. On the other hand, variance in conditioned negative emotional responses to danger cues was not predicted by contingency accuracy. Hence, the acquisition of negative emotional responses to pain-related danger signals does not appear to require accurate cognitive awareness of the associative strength between cue and painful outcome. Our data therefore extend evidence from fear conditioning studies actively manipulating contingency awareness, which question the assumption of explicit knowledge about cue-outcome contingencies as a prerequisite in human aversive learning (Knight et al., 2006; Schultz and Helmstetter, 2010; Raio et al., 2012). Unlike approaches actively manipulating contingency awareness to create groups with full vs. no contingency awareness through masking (Knight et al., 2006, 2009; Weidemann et al., 2013), explicit instructions and/or distraction (Klucken et al., 2009; Schultz and Helmstetter, 2010; Tabbert et al., 2011) we herein addressed contingency accuracy developed ''naturally'' over the course of differential learning. While our findings may not generalize to results in individuals fully unaware of CS-US contingencies, varying manifestations of differentially acquired contingency accuracy appear closer to clinical reality in chronic pain patients. Our findings indicate that inaccurate contingencies, including over- or underestimation of associations between predictors and an expected pain-related outcome, affect learned emotional responses in a distinct manner.

Conditioning processes involving pain-related emotional learning may well play a role in the pathophysiology and/or maintenance of chronic pain (Vlaeyen, 2015). This also holds true for extinction, which although less well-studied, appears to be impaired in patients with chronic pain (Labus et al., 2013; Icenhour et al., 2015a). At the same time, extinction processes provide a framework for cognitive-behavioral treatment approaches involving exposure therapy which have been successfully tailored to the treatment of chronic pain (den Hollander et al., 2010). Therefore, understanding a putative role of cognitive factors in emotional inhibitory learning is highly relevant. We herein explored whether contingency accuracy affects the extinction of learned emotional responses, expecting persistent negative as well as positive valence of formerly predictive cues as a function of high contingency accuracy. Results of group comparisons revealed that individuals with high contingency accuracy after acquisition demonstrated persistent differential emotional responses after extinction. Interestingly, this effect was solely driven by maintenance of positive emotional responses to former safety cues, while negative emotional responses to former danger signals were fully extinguished, indicating reduced extinction particularly of positive emotional responses to cues predicting safety during acquisition. Additionally, individuals with low contingency accuracy demonstrated an unexpected reversal of previously learned emotional responses to cues formerly signaling danger, resulting in equally positive valence of both, former danger and safety signals after extinction. While these results suggest distinct processes underlying extinction of emotional responses to former danger and safety signals as well as a direct impact of contingency accuracy, regression analyses revealed no independent contribution of contingency accuracy to extinction, unlike hypothesized. Our findings rather suggest that other, possibly more complex interactions between cognitions and emotions may be involved in pain-related inhibitory learning. Nevertheless, it is conceivable that low contingency awareness and decreased cue differentiation may favor spreading of emotional attributes to the safe context of extinction, characterized by the absence of any aversive event, as a possible mechanism underlying overgeneralization with detrimental long-term effects in anxiety disorders (Lissek et al., 2010) as well as chronic pain (Jenewein et al., 2013; Meulders et al., 2014). Furthermore, persistent safety cue properties following extinction irrespective of contingency accuracy suggest a resistance especially of learned safety to inhibitory learning, thereby potentially interfering with extinction-based treatment approaches (Volders et al., 2012).

Together, our findings clearly favor a separate consideration of conditioned responses to danger and safety cues. The analysis of differential measures alone, as common practice in fear conditioning studies, may disguise dissociable influences of conditioned danger and safety signals, thereby disregarding distinct neural correlates (Fullana et al., 2015) which appear to be uniquely involved in pain-related emotional learning and memory processes (Benson et al., 2014; Gramsch et al., 2014; Icenhour et al., 2015a,b; Labrenz et al., 2015). Likewise, the ongoing debate regarding the putative impact of contingency awareness on different outcome measures of fear conditioning has widely neglected safety learning (e.g., Klucken et al., 2009; Schultz and Helmstetter, 2010; Lovibond et al., 2011; Tabbert et al., 2011). Our behavioral findings strongly support previous conclusions that safetyrelated learning processes deserve more attention in the context of pain (Vlaeyen, 2015) and in the broader field of fear conditioning (Fullana et al., 2015) and particularly encourage that contingency awareness does not only shape danger but also safety learning. Future research is clearly needed to extend these behavioral findings, for example by testing

#### REFERENCES


distinct effects of contingency accuracy on approach/avoidance behaviors. Additionally, broader methodological approaches including psychophysiological measures and functional brain imaging appear essential in light of previous reports supporting independent effects of contingency awareness on neural, psychophysiological and evaluative responses during classic fear conditioning in instructed aware and unaware subjects (Klucken et al., 2009; Tabbert et al., 2011). Finally, given evidence that contingency learning is altered in chronic pain (Jenewein et al., 2013; Meulders et al., 2014; Icenhour et al., 2015b), the results reported certainly call for further investigation to clarify the putative role of contingency awareness in impaired extinction in chronic pain.

# AUTHOR CONTRIBUTIONS

AI, FL, SE designed the research studies. AI, FL performed the research. AI, FL, SB analyzed the data. AI, FL drafted the manuscript. All authors contributed to the interpretation of data, revised the manuscript and approved the final version of the manuscript.

## FUNDING

This re-analysis was based on two studies supported by a grant from the German Research Foundation (DFG, FOR1581 ''Extinction Learning'').

## ACKNOWLEDGMENTS

We gratefully acknowledge the assistance of Dr. Sarah Hampel and Armgard Boekstegers in conducting the experiments.


hypersensitivity in irritable bowel syndrome. Am. J. Physiol. Gastrointest. Liver Physiol. 303, G141–G154. doi: 10.1152/ajpgi.00060.2012


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Labrenz, Icenhour, Benson and Elsenbruch. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Contextual Change After Fear Acquisition Affects Conditioned Responding and the Time Course of Extinction Learning—Implications for Renewal Research

Rachel Sjouwerman, Johanna Niehaus and Tina B. Lonsdorf\*

Department of Systems Neuroscience, University Hospital Hamburg-Eppendorf, Hamburg, Germany

Context plays a central role in retrieving (fear) memories. Accordingly, context manipulations are inherent to most return of fear (ROF) paradigms (in particular renewal), involving contextual changes after fear extinction. Context changes are, however, also often embedded during earlier stages of ROF experiments such as context changes between fear acquisition and extinction (e.g., in ABC and ABA renewal). Previous studies using these paradigms have however focused exclusively on the context switch after extinction (i.e., renewal). Thus, the possibility of a general effect of context switch on conditioned responding that may not be conditional to preceding extinction learning remains unstudied. Hence, the current study investigated the impact of a context switch between fear acquisition and extinction on immediate conditioned responding and on the time-course of extinction learning by using a multimodal approach. A group that underwent contextual change after fear conditioning (AB; n = 36) was compared with a group without a contextual change from acquisition to extinction (AA; n = 149), while measuring physiological (skin conductance and fear potentiated startle) measures and subjective fear ratings. Contextual change between fear acquisition and extinction had a pronounced effect on both immediate conditioned responding and on the time course of extinction learning in skin conductance responses and subjective fear ratings. This may have important implications for the mechanisms underlying and the interpretation of the renewal effect (i.e., contextual switch after extinction). Consequently, future studies should incorporate designs and statistical tests that disentangle general effects of contextual change from genuine ROF effects.

Keywords: context, skin conductance response (SCR), subjective fear ratings, fear potentiated startle, fear conditioning, fear extinction

# INTRODUCTION

In our daily lives events are usually embedded in a broader set of circumstances (i.e., context). These contexts do not only frame the perception and interpretation of an event but also guide what is later remembered. In addition, contexts function as retrieval cues and thus ultimately gate behavioral responses. The definition however, of what constitutes a context is complex. Generally,

#### Edited by:

Oliver T. Wolf, Ruhr University Bochum, Germany

#### Reviewed by:

Andreas Mühlberger, University of Regensburg, Germany Silke Lissek, Ruhr University Bochum, Germany

> \*Correspondence: Tina B. Lonsdorf t.lonsdorf@uke.de

Received: 28 September 2015 Accepted: 20 November 2015 Published: 08 December 2015

#### Citation:

Sjouwerman R, Niehaus J and Lonsdorf TB (2015) Contextual Change After Fear Acquisition Affects Conditioned Responding and the Time Course of Extinction Learning—Implications for Renewal Research. Front. Behav. Neurosci. 9:337. doi: 10.3389/fnbeh.2015.00337 the physical environment (i.e., the combination of internal and external states) is considered to constitute the context (Maren et al., 2013). Importantly, patients suffering from anxiety and stress-related disorders often fail to respond appropriately in clearly innocuous situations, which can be distinguished from dangerous situations through contextual signals. For instance, patients suffering from spider phobia might drop their plate of hot food even when seeing a spider on television or posttraumatic stress disorder patients might jump to the floor looking for shelter after hearing a smashing door.

The acquisition of such fearful behavior can be modeled in the laboratory in classical conditioning paradigms where a neutral cue (conditioned stimulus, CS+), such as a geometric figure, acquires the capacity to predict an aversive event (unconditioned stimulus, US), such as an electrotactile stimulus. After conditioning, the CS+ elicits a conditioned response (CR), which can be measured through physiological responding (e.g., skin conductance response, fear potentiated startle) while a neutral cue that is not predictive of the US (CS−) generally does not. A waning of the conditioned response (i.e., extinction) can be achieved through presentations of the CS+ without being followed by the US. Thereby, extinction does not lead to erasure of the initial CS-US memory but induces new inhibitory (safety) learning (for a review see Milad and Quirk, 2012). This is made evident from return of fear (ROF) phenomena (Bouton, 2002) such as ROF after the mere passage of time (spontaneous recovery, SR), un-signaled US presentations (reinstatement, RI), and contextual changes (renewal, RN) after successful extinction. Critically, these ROF phenomena are context dependent, involving changes in the temporal (SR) or physical context (RN) as well as involving context conditioning (RI).

Importantly, the context in which extinction, or in clinical terms, treatment of anxiety disorders, takes place (context B) is nearly always different from the context in which fear was originally acquired (context A). This is of critical importance, as the context gates which memory type (CS-US vs. CS-noUS) is eventually expressed when confronted with ambiguous cues (Bouton, 1993; Maren et al., 2013). Presenting acquisition and extinction in different contexts in the laboratory is thought to disambiguate the CS-US association from the CS-noUS association through learning that under certain contextual circumstances the CS-US association is not valid (occasion setting). Furthermore, different contexts during acquisition and extinction allow for the investigation of extinction without the confounding effects of the fear–inducing acquisition context, which boosts fearful responding also in the absence of the CS+.

Consequently, a context change from acquisition (in context A) to extinction (in context B) is common in experimental designs. Critically, the most frequently used paradigms in renewal research (ABA and ABC renewal) involve a context switch after both acquisition and extinction (Vervliet et al., 2013). In rodents, AAB renewal with conditioning and extinction taking place in the same context (A) has also been described, but it is not as robust as ABA and ABC renewal (Bouton and King, 1983). Per definition, the main focus of renewal studies is the context change after extinction (i.e., renewal), which is common to all three paradigms (ABA, ABC, AAB). In contrast, the possible impact of a context change after acquisition (i.e., in ABA or ABC but not in AAB renewal) has not received much attention to date. However, if a context switch following acquisition affects conditioned responding already during extinction, this might have important implications for the possible mechanisms underlying renewal effects induced by a context change following extinction. More precisely, if contextual change exerts a general effect on conditioned responding that is not pertinent to already occurred extinction learning, this may challenge the interpretation of the mechanisms thought to underlie renewal.

Indeed preliminary evidence for an effect of contextual change following acquisition on early extinction performance can be derived from the renewal literature. However, firm conclusions are precluded due to the selective focus on renewal in both study design (i.e., ABA/ABC paradigms without AAA/AAC control groups) and statistical analyses. Three studies report longer response times for US expectancy ratings to both CSs (Neumann and Kitlertsirivatana, 2010; Bandarian Balooch and Neumann, 2011) and a decrease in CS-discrimination (Effting and Kindt, 2007) on the first extinction trial after a context change following acquisition (ABA) as compared to no context change (AAA) while a forth study did not observe such an effect in US expectancy ratings (Dibbets et al., 2008). Furthermore, when reconciling a study (ABA/ABB) by Milad et al. (2005), larger SCRs for both CSs were evident on the first trial following a context switch after acquisition. As no AAA control group was included, the effect of context switch following acquisition could however not be determined statistically.

In addition, there is suggestive evidence for a different course of extinction learning after a context switch. Vansteenwegen et al. (2005) reported faster and incomplete extinction (as assessed by SCRs) following a context switch (ABA vs. AAA) on a descriptive level while Effting and Kindt (2007) do not find a modulating effect of context in SCRs on either a statistical or descriptive level.

Besides the renewal literature, studies investigating the effect of exposure to multiple contexts during extinction learning on ROF (Shiban et al., 2013; Dunsmoor et al., 2014) may be informative with respect to the effects of contextual change from acquisition to extinction. However, also these studies did not statistically test possible effects of contextual change on extinction learning and conditioned responding during extinction.

Taken together, the preliminary findings from the (renewal) literature concerning the impact of a context switch between acquisition and extinction on extinction learning are difficult to interpret and incomplete. Further complicating matters, different dependent measures may differentially reflect specific aspects of the context switch phenomenon, which calls for a multi-modal approach and a systematic investigation in future studies.

To fill this gap, the current study aimed to test the effects of context change on conditioned responding and extinction learning by comparing a group with and without a context change after acquisition (AA vs. AB) with regard to multiple fear responses (skin conductance, subjective ratings and fear potentiated startle) in a fear-conditioning and extinction paradigm. Thereby, we specifically focus on immediate shifts

in and the time-course of conditioned responding during extinction. Thereby, the different dependent measures were employed to capture effects of contextual change on different levels of responding such as the affective level (FPS), general arousal (SCRs), and a more cognitive level (self-reports).

# MATERIALS AND METHODS

## Participants

The study sample included 216 right-handed (as assessed by self-report) healthy individuals. Three participants aborted the experiment, 28 had to be excluded due to either technical problems during data acquisition or electrode misplacement, leaving 185 participants for final analyses. Participants were randomly assigned to one of four experimental groups, one undergoing context change (AB; n = 36, 26 females) and the others undergoing no contextual change between conditioning and extinction (AA; n = 149, 109 females) (see experimental design for details). The AA group consisted of three experimental groups differing in a post-extinction manipulation, which will be reported elsewhere. The two experimental groups did not differ in age and sex distribution (see **Table 1**). None of the participants reported a history of psychiatric disorders. Written informed consent in accordance with the Declaration of Helsinki was obtained from each participant, and the Ethical Review Board of the German Psychological Association (DGPS) approved the study. Participants were payed for their participation.

## Questionnaires

State anxiety, personality traits, and internal and external locus of control were examined by using German versions of the STAI (Spielberger et al., 1983), NEO-FFI (Borkenau and Ostendorf, 1993), and IPC (Levenson, 1974; Krampen, 1985) questionnaires, respectively. The STAI was always completed right before the experiment. Other questionnaires were also completed before the experiment but could be finished after the experiment if required by time management.

# Material—Electrotactile Stimulus

A train of three 2 ms electrotactile square-waves (ISI: 50 ms) was administered to the dorsal part of the right hand and served as the US, generated by a DS7A electrical stimulator (Digitimer, Welwyn Garden City, UK) and delivered through a platinum pin


<sup>a</sup>Missing data of two participants.

surface electrode (Specialty Developments, Bexley, UK). Prior to the experiment, US intensity was individually adjusted to a level that was considered as being unpleasant but tolerable (range 0.4–40 mA). A standardized protocol was used to calibrate shock intensity. First, the pain threshold was determined, defined as the value that was clearly sensible but not painful. Next, the US intensity was determined by increasing intensities with on average steps of 0.5 mA and asking participants to rate each electrotactile stimulus on a 10 point scale, with 10 being painful and not tolerable. It was aimed at to achieve a rating of the electrotactile stimulus that had a value between 7 and 8. Experimental groups did not differ in final intensities (see **Table 1**) and there were no significant differences between intensities calibrated by the three experimenters, p > 0.23.

# Visual Material

Black geometrical shapes served as conditioned stimuli (CS; i.e., an ellipse and a rectangle) which were presented on a background color (blue, purple, green or yellow) that served as context for 6 s (Maren et al., 2013; Lonsdorf et al., 2015). One of these shapes (CS+) co-terminated with the US (100% reinforcement ratio during conditioning), whereas the other shape did not (CS−). The context color remained constant for each participant and experimental phase (see also experimental design).

Allocation of the shapes to the CS+ and CS− and background colors was counterbalanced between individuals, as well as the order in which the CS+/CS− appeared. CS presentations were interleaved with a variable inter trial interval (ITI) of 11.5 ± 1.5 s, consisting of a white cross on a black background. Presentation of all stimuli was controlled using Presentation Software (NeuroBehavioral Systems, Albany California, USA).

# Auditory Material

A burst of 95dB(A) white noise ("startle probe") was presented binaurally via headphones (Sennheiser, Wedemark, Germany) 4 or 5 s post CS-onset in half of the habituation trials, 2/3 of the fear conditioning and extinction trials and 5 or 7 s after ITI onset in 1/3 of all ITIs. The last CS presentations during conditioning as well as the first CS presentations during extinction were always startled.

## Experimental Design

The experiment consisted of seven experimental phases (see **Figure 1**): US intensity calibration, startle habituation (five startle probes were presented on a black screen to achieve a stable baseline for reactivity), CS habituation (two trials per CS-type, explicitly US-free), conditioning (in context A, nine presentations per CS type), immediate extinction, reinstatement and reinstatement-test. Thereby, extinction took place either in the same context A (AA-group) or a different context B (AB-group) as during conditioning. Reinstatement and reinstatement test differed between participants in contextual allocation and results of this manipulation will be reported elsewhere. Three different female experimenters conducted the experiment.

# Subjective Ratings and CS-US Awareness

Participants had 7 s to indicate their level of fear, anxiety and distress ("How much stress, fear or anxiety did you experience the last time you saw symbol X?," with the "X" referring to one of the CS-types at a time) toward both CS-types intermittently throughout the experiment on a visual analog scale (VAS) ranging from 0 (none) to 100 (maximum). The ratings were distributed in a way that they did not indicate the different phases of the paradigm; one was presented at the end of habituation, three during conditioning, three during extinction, and two during reinstatement test. The last rating in the conditioning phase occurred either after conditioning trial 7 or 8 and the first rating in extinction occurred after either extinction trial 1 or 2.

After the experiment participants filled in a post-experimental awareness questionnaire (estimations on the total number of received electrotactile stimuli, questions about CS-US contingencies during the experiment) which were orally confirmed with the experimenter. Based on this, participants were classified as aware, unaware, or uncertain of CS-US contingencies, with the latter in case participants reported a tendency toward the correct contingencies but also unsureness. The number of participants is reduced for rating analyses because some participants failed or were too slow to log their rating during the complete experiment (n = 8).

# Physiological Parameters—SCR

Physiological data were recorded using a BIOPAC MP100 amplifier (BIOPAC Systems Inc., Goleta, California, USA) and AcqKnowledge 3.9.2 software. Data preprocessing was conducted in Matlab version 2014b (Mathworks, Natick, MA, USA).

For skin conductance responses (SCR), hands were precleaned with warm water, and consecutively two with hydrogel and Ag/AgCl-sensor recording electrodes (Ø 55 mm) were attached to the surface of the left inner hand, i.e., on the distal and proximal hypothenar eminence. SCR data were recorded continuously at 1000 Hz with a gain of 5 µ. Data were offline down sampled to 10 Hz and scored semi-automatically using a custom-made program as foot-to-peak (0.9–4.0 s post CS/US onset) according to published guidelines (Boucsein et al., 2012). The absence of a response within this window, or an increase smaller than 0.02 µS, was scored as a zeroresponse. SCR measurements that showed recording artifacts or excessive baseline activity were discarded and scored as missing values. Raw SCR amplitudes were normalized by using a log transformation, and range corrected (SCR/SCRCS\_or\_US\_max) to control for inter individual variability (Lykken and Venables, 1971). Furthermore, data were smoothed within each phase of the experiment by using a local regression function that used weighted linear least squares and a second-degree polynomial model using Matlab. Participants showing more than 2/3 missing SCRs responses toward US presentations (excluding non-reactions) were classified as non-responders and excluded from all SCR analyses (n = 4).

# Physiological Parameters—FPS

Fear potentiated startle was measured underneath the right eye by using two AG/AgCl electromyogram (EMG) electrodes placed over the orbicularis oculi muscle and one placed on the participants' forehead as a reference. Startle data were sampled with a gain of 5000 at 1000 Hz and band-pass filtered (28–500 Hz) online, rectified and integrated (averaged over 20 samples). Data were scored semi-automatically as foot-to-peak (20–150 ms post startle probe onset) using the same program as for SCRs according to published guidelines (Blumenthal et al., 2005). Blinks up to 50 ms before the startle probe, recording artifacts or excessive baseline activity were scored as missing values. Raw data were T-transformed. Participants showing more than 1/3 zero-responses or missings were excluded from FPS analyses (n = 6).

# Statistical Analyses

First, a repeated measures ANOVA [mean of CS-type: CS+/CS−] with group (AA, AB) as between subject variable was performed to confirm that participants were successfully conditioned in both groups. Thereby the first trial of each CS-type was excluded from mean calculation, as no conditioning could have possibly taken place.

To test for the immediate impact of contextual change from acquisition to extinction on subjective and physiogical responding a 2 [CS-type: CS+/CS−] × 2 [time: last acquisition/first extinction trial] repeated measures ANOVA with group (AA, AB) as between subject variable was performed on the SCR, FPS, and rating data. Some participants had to be excluded from immediate analyses due to either missing data points on the last acquisition trial or on the first extinction trial (SCR: none; FPS:76; ratings: 32).

Furthermore, to investigate progression of extinction learning two separate repeated measures ANOVAs [CS-type: CS+/CS−] with group (AA, AB) as between subject variable were performed for early and late extinction on physiological and subjective rating data. For physiological measures, early and late extinction was defined as the first half and last half of the trials respectively [SCR: 4 vs. 5, FPS: 3 vs. 3]. Because only three subjective fear ratings were collected during extinction, the first and last rating during the extinction phase defined early and late extinction respectively.

Sex and CS-US awareness were included as covariates of no interest in all analyses, whereas CS-discrimination (difference between CS+ and CS−) was only included as a covariate in time course analyses. A p-value of <0.05 was considered as significant and Greenhouse-Geisser corrected degrees of freedom are reported when appropriate. Partial Eta<sup>2</sup> (pη 2 ) is reported as a measure of effect size. Effects of interest were further tested with additional ANOVA's or univariate analyses. Statistical analyses were performed using IBM SPSS Statistics for Windows, version 22 (Armonk, NY: IBM Corp.). For covariates, only significant or trend-wise main and interaction effects are reported.

# RESULTS

# Manipulation Check: Successful Fear Conditioning

Successful fear conditioning was confirmed by a significant main effect of CS-type in SCRs (see **Figure 2**), ratings (Supplementary Figure 1) and FPS (Supplementary Figure 2), which reflected stronger responses to the CS+ than to the CS−, see **Table 2**. In addition a main effect of context group was observed for startle responses, with mean responding in the AA group being higher than in the AB group. This indicates pre-existing differences between both groups prior to the experimental manipulation of context change. No main effects of context group were observed for ratings and SCRs (**Table 2**). In addition, no CS-type<sup>∗</sup> context group interactions were observed for any dependent measure.

#### Effects of Covariates (Covariates of No Interest) **Awareness**

For SCRs and subjective fear ratings, trend-wise main effects of awareness [SCR: F(1, 177) = 2.80, p = 0.096, <sup>p</sup>η <sup>2</sup> = 0.02; ratings: F(1, 168) = 3.64, p = 0.058, <sup>p</sup>η <sup>2</sup> = 0.02] as well as trend-wise and significant CS-type<sup>∗</sup> awareness interactions were observed [SCRs: F(1, 177) = 2.98, p = 0.086, <sup>p</sup>η <sup>2</sup> = 0.02; ratings: F(1, 168) = 20.49, p < 0.001, <sup>p</sup>η <sup>2</sup> = 0.11], indicating an expected impact of cognitive contingency awareness on conditioned responding.

FIGURE 2 | SCRs to the CS+ (red) and CS− (blue) during conditioning and extinction in (A) a group with both conditioning and extinction in context A (AA) and (B) a group with conditioning in context A and extinction in a new context B (AB). Error bars represent the standard error of the mean. Asterisks indicate significant effects with \* indicating p < 0.05. Dotted lines separate early extinction from late extinction trials.



# Immediate Effect of Context Switch Following Acquisition on Conditioned Responding SCR

An immediate effect of context switch following acquisition (see **Figures 2A,B**) was evident from a significant CStype<sup>∗</sup> time<sup>∗</sup> context interaction in SCRs [F(1, 177) = 6.08, p = 0.015, <sup>p</sup>η <sup>2</sup> = 0.03] in absence of any main effects (both F's <3, p's > 0.10) or two-way interactions (all F's < 1, p's > 0.35).

When testing both CS-types separately, a time<sup>∗</sup> context interaction was observed for the CS+ only [F(1, 177) = 6.52, p = 0.011, <sup>p</sup>η <sup>2</sup> = 0.04] reflecting increased SCR responding from the last acquisition to the first extinction trial in the AB group (10.11) as compared to the AA group (1-0.01) (see **Figures 2A,B**). No significant main effect of or interactions with context were observed for the CS− (all F's < 1, p's > 0.32).

#### Subjective Fear Ratings

In contrast to SCRs, subjective fear ratings revealed a time<sup>∗</sup> context [F(1, 144) = 8.91, p = 0.003, <sup>p</sup>η <sup>2</sup> = 0.06] interaction in absence of a significant CS-type<sup>∗</sup> time<sup>∗</sup> context interaction (which was significant in SCRs) or CS-type<sup>∗</sup> context, and CStype<sup>∗</sup> time interactions (all F's < 2.23, all p's > 0.13). Exploring the time<sup>∗</sup> context interaction in more detail revealed that the CS-type independent decrease in fear ratings was conditional to contextual change (AB: 1-4.77, AA: 10.88) (see **Figure 3**). In addition, significant main effects of CS-type [F(1, 144) = 15.25, p < 0.001, <sup>p</sup>η <sup>2</sup> = 0.10; CS+ >CS−] and time [F(1, 144) = 4.49, p = 0.036, <sup>p</sup>η <sup>2</sup> = 0.03, conditioning>extinction] were observed in subjective fear ratings.

In sum, both SCR and subjective fear ratings indicate pronounced effects of contextual change on conditioned responding immediately after a context switch. Thereby, the context switch seems to primarily affect CS+ specific responses—at least for SCRs. The absence of a significant effect on subjective ratings might be explained by different times of acquisition for both dependent measures. While SCRs are acquired on a trial-by-trial base, ratings are only acquired intermittently (i.e., after the 1st or 2nd extinction trial). In addition, anticipatory SCR reactions are scored prior to the experience of non-reinforcement during extinction (i.e., CS onset) while ratings are always acquired after the experience of non-reinforcement. As a consequence, rating data might reflect an already partly extinguished phenomenon and thus reflect reduced subjective fear after a context switch while SCRs reflect stronger responding immediately (1st trial and thus prior to extinction learning) after a context switch.

#### FPS

For FPS (Supplementary Figure 2), in contrast to SCRs and ratings, no main effects or interactions including the factor context, or any other interactions were observed (all F's < 1.50, p's > 0.23) while a trend-wise main effect for CS-type was observed [F(1, 95) = 3.12, p = 0.081, <sup>p</sup>η <sup>2</sup> = 0.03; Mcs<sup>+</sup> = 52.96 ± 7.30, Mcs<sup>−</sup> = 49.28 ± 5.76] in absence of a significant main effect of time (F < 1, p > 0.81).

#### Effects of Covariates (Covariates of No Interest) **Awareness**

A main effect of awareness was observed in subjective ratings [F(1,144) = 7.03, p = 0.009, <sup>p</sup>η <sup>2</sup> = 0.05; uncertain>aware>unaware] while a trend-wise or significant CS-type<sup>∗</sup> awareness interaction was observed for SCRs [F(1, 177) = 3.38, p = 0.067, <sup>p</sup>η <sup>2</sup> = 0.019] and subjective ratings respectively [F(1, 141) = 14.80, p < 0.001, <sup>p</sup>η <sup>2</sup> = 0.10]. As expected, aware individuals showed stronger CS discrimination than unaware or uncertain individuals during the end of acquisition and at the beginning of extinction in SCRs and subjective fear ratings.

#### **Sex**

In addition, a trend-wise significant time<sup>∗</sup> sex interaction was observed for FPS, F(1, 95) = 2.85, p = 0.095, <sup>p</sup>η <sup>2</sup> = 0.03.

# Effects of Context Switch Following Acquisition on Extinction Learning Early Extinction—SCR

During early extinction, a significant CS-type<sup>∗</sup> context interaction [F(1, 176) = 3.90, p < 0.050, <sup>p</sup>η <sup>2</sup> = 0.02] and a trendwise significant main effect of CS-type [F(1, 176) = 2.99, p = 0.086, <sup>p</sup>η <sup>2</sup> = 0.02] were observed in SCRs. Irrespective of contextual change, mean responses tended to be higher for the CS+ (M = 0.17 ± 0.17) than for the CS− (M = 0.13 ± 0.14). Subsequent univariate analyses showed that mean responses for the CS− were significantly lower in the AB group (M = 0.08 ± 0.10) than in the AA group (M = 0.15 ± 0.15), but did not differ for the CS+ between context conditions as indicated by a main effect of context for the CS− [F(1, 181) = 5.23, p = 0.023, <sup>p</sup>η <sup>2</sup> = 0.03] but not for the CS+ [F < 1, p > 0.83].

#### Late Extinction—SCR

During late extinction however, this CS-type<sup>∗</sup> context interaction had vanished [F < 1, p > 0.47] while a main effect of context was observed, [F(1, 176) = 7.05, p = 0.009, <sup>p</sup>η <sup>2</sup> = 0.04], reflecting generally (i.e., CS-unspecific) lower SCR responses in the AB group (M = 0.05 ± 0.07) than in the AA group (M = 0.11 ± 0.12).

Of note, while a context switch following conditioning lead to immediate (i.e., first trial) CS+ specific increases in conditioned SCR responding, effects on the course of extinction learning were CS− specific and reflect a decrease in responding in early extinction and a generally (i.e., CS independent) reduction in SCR responding by the end of extinction.

#### Early and Late Extinction—Fear Ratings and FPS

In contrast, rating and startle data did not reveal any interactions of or main effects with the factor context and an effect of CS-type was absent in both early and late extinction [both F's < 2.74, both p's > 0.10].

#### Effects of Covariates (Covariates of No Interest) **Awareness**

For SCRs, a trend-wise CS-type<sup>∗</sup> awareness interaction was observed during early [F(1, 176) = 3.41, p = 0.066, <sup>p</sup>η 2 = 0.02] but not during late extinction (F < 1, p > 0.87), while for subjective ratings, a main effect of awareness was observed during early [F(1, 143) = 4.47, p = 0.036, <sup>p</sup>η <sup>2</sup> = 0.03, uncertain>aware>unaware] but not late extinction (F = 1.78, p = 0.184) that was further qualified by a trend-wise CStype<sup>∗</sup> awareness interaction during late extinction only [early: F = 0.75, p = 0.388; late: F(1, 143) = 3.84, p < 0.052, pη <sup>2</sup> = 0.06]. This implies that cognitive contingency awareness had an impact on conditioning responding, especially during early extinction.

**Sex** Furthermore, during both early and late extinction, a main effect of sex was observed in SCRs [early: F(1, 176) = 4.13, p = 0.044, pη <sup>2</sup> = 0.02; late: F(1, 176) = 9.33, p = 0.003, <sup>p</sup>η <sup>2</sup> = 0.05], indicating generally lower SCRs in women than in men. This effect, has been observed previously by our group (Lonsdorf et al., 2015) but an in-depth discussion is beyond the scope of this manuscript. We refer the interested reader to other sources on this topic (Cover et al., 2014; Lonsdorf et al., 2015).

#### **CS-discrimination during conditioning**

In addition, SCR responding during extinction was significantly affected by the level of CS-discrimination on the last conditioning trial during both early extinction [F(1, 176) = 9.22, p = 0.003, pη <sup>2</sup> = 0.05] and late extinction [F(1, 176) = 6.57, p = 0.011, pη <sup>2</sup> = 0.04].

In addition, a significant interaction between CSdiscrimination on the last conditioning trial and CS-type was observed for subjective rating data during both extinction phases: F(1, 143) = 26.87, p < 0.001, <sup>p</sup>η <sup>2</sup> = 0.16; late: F(1, 143) = 18.24, p < 0.001, <sup>p</sup>η <sup>2</sup> = 0.11], indicating that CS discrimination during extinction strongly depends on the discrimination at the end of conditioning.

### DISCUSSION

#### Summary

The aim of the current study was to investigate the impact of a contextual change between fear acquisition and extinction on conditioned responding and on the time-course of extinction learning by using a multimodal approach. Generally, our data demonstrate pronounced effects of such contextual change on both immediate conditioned responding and on the time course of extinction learning, which may have important implications for the interpretation of the renewal effect (i.e., effects of contextual switch after successful extinction). We report three major findings.

First, immediately after a context switch (i.e., first extinction trial) as compared to no context switch, increased SCRs were observed specifically to the CS+, likely reflecting immediate and intensified conditioned responding. In contrast, subjective fear ratings to both CSs were attenuated after contextual change. This apparent discrepancy may be explained by different times of acquisition of SCR and rating data. While ratings were collected after one or two extinction trials, SCRs were recorded to each CS onset. Hence, the first SCR response following context change is recorded prior to the experience of non-reinforcement (i.e., prior to the possibility of extinction learning), while ratings are provided only after non-reinforcement has been experienced at least once, allowing for extinction to occur. As such, ratings may in fact reflect an already partly extinguished phenomenon while SCR data for the first extinction trial do not.

Second, SCRs to the CS− were attenuated during early extinction (i.e., first half of extinction trials) following a context change. As the effect of early extinction in SCRs, in contrast to the immediate effect, allows extinction to occur, these results line up with lower (albeit in a CS− unspecific way) subjective ratings immediately following a context switch (see above). Consequently, a context change indeed seems to facilitate extinction learning speed for explicit fear ratings to both CS types and specifically to the CS− for the SCRs.

Third, during late extinction (i.e., second half of extinction trials) both CS+ and CS− elicited less SCR responses after a context change following conditioning than without contextual change, indicating not only faster extinction but possibly also more successful extinction (in SCRs) following this context change. However, no difference between both groups were observed in subjective evaluation of the CSs which may reflect a floor effect as subjective ratings had extinguished to floor-level by the end of extinction in both groups.

#### Implications

These three major findings highlight that a change in context has a profound differential effect on conditioned responding both immediately following a context change and on the time course of subsequent extinction learning. The direction of this effect however (i.e., facilitated vs. attenuated conditioned responding) is strongly dependent on time (i.e., immediately after contextual change, early and late extinction). While a context switch induced CS+ specific facilitation of conditioned responding in SCRs before extinction learning may take place (i.e., SCRs to the first CS onset), CS− specific facilitated extinction learning in SCRs was observed in subsequent trials. The latter was supported by facilitated extinction learning speed to both CSs in subjective ratings.

Furthermore, our data highlight that single trial analyses in contrast to blocks of averaged trials may reveal divergent findings. These main findings therefore emphasize that the measurement unit (single trial vs. blocked) used for statistical analyses exerts a strong impact on the results which has recently been discussed as an important methodological challenge for ROF research in humans (Haaker et al., 2014). In light of the present data, this may be particularly relevant for ROF studies as both renewal and reinstatement involve contextual changes to varying degrees, and are typically very transient and restricted to one or a few single trials (Haaker et al., 2014).

As studies on extinction and ROF often routinely employ a contextual switch from conditioning to extinction (i.e., AB design), our findings may suggest that the effect of subsequent ROF manipulations may differ from studies employing no contextual change between acquisition and extinction (i.e., AA). First, extinction learning speed was affected by a contextual change from acquisition to extinction, possibly leading to different levels of end-point extinction between AB and AA designs, as shown by our results. Importantly, this end-point extinction responding serves as a baseline to which conditioned responding following ROF is compared to statistically in ROF studies (Haaker et al., 2014). Hence differences in end-point extinction responding are likely to affect the outcome of ROF manipulations.

Second, and perhaps most important, after successful extinction, an inhibitory fear memory trace (extinction memory) is thought to co-exist with the original fear memory trace. Return of fear manipulations are thought to promote recall and expression of this fear memory trace over the extinction memory trace through contextual change (i.e., renewal), which manifests as enhanced (possibly CS+ specific) conditioned responding in the first trial following the contextual change. Our data however demonstrate such a CS+ specific response enhancement following contextual change in absence of the existence of the second inhibitory memory trace. More specifically, CS+ specific response enhancement was induced by contextual change occurring prior to extinction learning and thus prior to the generation of an inhibitory memory trace. As such, our data suggest that a context switch may exert a general effect on conditioned responding that may -at least partly- also contribute to renewal (and reinstatement) effects.

## Limitations and Future Directions

Remarkably, in contrast to SCRs and subjective ratings, no effects of contextual change were observed in FPS conditioned responding either immediately after contextual change or during the time-course of extinction. One explanation might be that FPS is a measure of the central nervous system activity (Blumenthal et al., 2005) and indicates fear, whereas electrodermal activity is generally taken as an indication of general arousal (Hamm and Weike, 2005). It might therefore be possible that arousal (SCR) is more sensitive than fear (FPS) to the effects of contextual change. Another possible explanation for the FPS null-finding might be reduced power for FPS data due to numerous missing data points on either the last conditioning or the first extinction trial.

Second, we employed a single-day paradigm with all experimental phases following immediately upon each other. As it has been shown that timing of extinction after conditioning (immediate vs. delayed extinction) affects the course and strength of conditioned responding during extinction ("immediate extinction deficit") (Maren, 2013), future studies need to investigate whether allowing for memory consolidation in between these phases (acquisition-extinction) in a multiple-day paradigm may result in different findings.

Third, following conditioning, context change and CSs were presented simultaneously and might be perceived a one single compound stimulus. Future studies may thus profit from implementing the context as a constant variable to allow for more clear-cut interpretations or from employing virtual reality techniques (Baas et al., 2004). The latter would contribute to a broader concept/operationalization of context as suggested by Maren et al. (2013) and enhance translational value, as contextual manipulations in rodent work are usually affecting multiple sensory channels.

# CONCLUSION

In sum, our results demonstrate that a context change between fear conditioning and extinction has a pronounced impact on conditioned responding and on the time-course of extinction learning. As we have demonstrated that the effect of a contextual change on conditioned responding is not exclusively conditional to completed extinction learning (i.e., renewal), our results may challenge the interpretation of mechanisms underlying return of fear induced by renewal. Hence, it is urgent to systematically investigate the role of ROF specific and non-specific effects of contextual change.

# AUTHOR CONTRIBUTIONS

All authors contributed extensively to the work presented in this paper. TL conceived and designed the study, RS and JN acquired, all authors analyzed, and TL and RS interpreted the data. TL and RS jointly drafted the manuscript and JN critically revised the manuscript. All authors give final approval of the version to be published and agree to be accountable for all aspects of the work.

### REFERENCES


#### ACKNOWLEDGMENTS

This work was supported by a grant from the German Research Foundation (DFG TL1980/1-1) to TL. The authors thank Anna Heinrich for help with data collection.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnbeh. 2015.00337


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Sjouwerman, Niehaus and Lonsdorf. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Impaired Contextual Fear Extinction Learning is Associated with Aberrant Regulation of CHD-Type Chromatin Remodeling Factors

Alexandra Wille1‡, Verena Maurer 2‡, Paolo Piatti 1 †, Nigel Whittle2 † , Dietmar Rieder <sup>3</sup> , Nicolas Singewald<sup>2</sup> \* and Alexandra Lusser <sup>1</sup> \*

*<sup>1</sup> Division of Molecular Biology, Biocenter, Medical University of Innsbruck, Innsbruck, Austria, <sup>2</sup> Department of Pharmacology and Toxicology, Centre for Molecular Biosciences, Institute of Chemistry and Pharmacy, Leopold-Franzens University of Innsbruck, Innsbruck, Austria, <sup>3</sup> Division of Bioinformatics, Biocenter, Medical University of Innsbruck, Innsbruck, Austria*

#### Edited by:

*Onur Gunturkun, Ruhr-University Bochum, Germany*

#### Reviewed by:

*Iva B. Zovkic, University of Toronto Mississauga, Canada Robert Kumsta, Ruhr-University Bochum, Germany*

#### \*Correspondence:

*Nicolas Singewald nicolas.singewald@uibk.ac.at; Alexandra Lusser alexandra.lusser@i-med.ac.at*

#### †Present Address:

*Paolo Piatti, Zymo Research GmbH, Freiburg, Germany; Nigel Whittle, Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland*

*‡ These authors have contributed equally to this work.*

Received: *19 August 2015* Accepted: *03 November 2015* Published: *18 November 2015*

#### Citation:

*Wille A, Maurer V, Piatti P, Whittle N, Rieder D, Singewald N and Lusser A (2015) Impaired Contextual Fear Extinction Learning is Associated with Aberrant Regulation of CHD-Type Chromatin Remodeling Factors. Front. Behav. Neurosci. 9:313. doi: 10.3389/fnbeh.2015.00313* Successful attenuation of fearful memories is a cognitive process requiring initiation of highly coordinated transcription programs. Chromatin-modulating mechanisms such as DNA methylation and histone modifications, including acetylation, are key regulators of these processes. However, knowledge concerning the role of ATP-dependent chromatin remodeling factors (ChRFs) being required for successful fear extinction is lacking. Underscoring the potential importance of these factors that alter histone-DNA contacts within nucleosomes are recent genome-wide association studies linking several ChRFs to various human cognitive and psychiatric disorders. To better understand the role of ChRFs in the brain, and since to date little is known about ChRF expression in the brain, we performed a comprehensive survey of expression levels of 24 ATP-dependent remodelers across different brain areas, and we identified several distinct high molecular weight complexes by chromatographic methods. We next aimed to gain novel insight into the potential regulation of ChRFs in different brain regions in association with normal and impaired fear extinction learning. To this end, we established the 129S1/SvImJ (S1) laboratory mouse strain as a model for compromised contextual fear extinction learning that can be rescued by dietary zinc restriction (ZnR). Using this model along with genetically related but fear extinction-competent 129S6/SvEv (S6) mice as controls, we found that impaired fear extinction in S1 was associated with enhanced ventral hippocampal expression of CHD1 and reduced expression of CHD5 that was normalized following successful rescue of impaired fear extinction. Moreover, a select reduction in CHD3 expression was observed in the ventral hippocampus (vHC) following successful rescue of fear extinction in S1 mice. Taken together, these data provide novel insight into the regulation of specific ChRFs following an impaired cognitive process and its rescue, and they suggest that imbalance of CHD-type remodeler levels, which consequently may lead to changes of transcriptional programs, may be an underlying mechanism involved in impaired fear extinction learning and its therapeutic rescue.

Keywords: epigenetics, nucleosome remodeling, amygdala, anxiety behavior, gene regulation

# INTRODUCTION

Anxiety and trauma-related disorders are the most prevalent mental disorders in Western societies, with current estimates suggesting that 30% of the population may be afflicted at least once during their lifetime (Kessler et al., 2005; Wittchen et al., 2011). These disorders including phobias, panic, and posttraumatic stress disorder have an important learning component and are often associated with impaired extinction learning, the central mechanism for successful exposure-based therapies (Bouton et al., 2001; Mineka and Zinbarg, 2006). In recent years, it has become increasingly clear that mechanisms that alter the structure and properties of chromatin, sometimes broadly summarized by the term epigenetics, are key players in the regulation of cognitive and emotional processes and thus also of different aspects of fear acquisition, memory, and extinction (reviewed e.g., in Jakovcevski and Akbarian, 2012; Dias et al., 2013; Zovkic et al., 2013; Fischer, 2014; Rudenko and Tsai, 2014; Whittle and Singewald, 2014).

Chromatin remodeling factors (ChRFs) are energy-dependent molecular motor proteins that belong to the SNF2 protein family and can be classified into 23 subgroups according to sequence differences in their ATPase domains and the presence of additional sequence motifs. In mammals, the best studied ChRF subfamilies are the SWI/SNF (switch/sucrose non-fermenting), ISWI (imitation switch), CHD (chromo helicase DNA binding), and the INO80 (inositol auxotroph 80) subfamilies (Lusser and Kadonaga, 2003; Marfella and Imbalzano, 2007; Clapier and Cairns, 2009; Hargreaves and Crabtree, 2011; Piatti et al., 2011). ChRFs use the energy derived from ATP hydrolysis to disrupt and reform histone-DNA contacts. This activity can result in diverse outcomes ranging from the repositioning of nucleosomes along the DNA (sliding), to ejection and assembly of nucleosomes or replacement of canonical with variant histones (Clapier and Cairns, 2009). As a consequence, access to the DNA for transcription factors and the transcription machinery is enhanced or suppressed leading to activation or repression of gene activity.

In contrast to other chromatin-regulatory mechanisms, such as histone modifications or DNA methylation, ChRFs have been given very little attention in brain research. Only recently, several studies have uncovered genetic association of some ChRFs with various intellectual and behavioral disorders (reviewed in Ronan et al., 2013; Krumm et al., 2014; Vogel-Ciernia and Wood, 2014). Given that ChRFs are major regulators of chromatin and transcriptional dynamics and therefore are likely to occupy a central position in the regulation of transcriptional plasticity required for all phases of learning and memory, a better understanding of their role in brain function is highly desirable. Thus, we conducted a broad survey of ChRF expression and investigated their regulation following normal and impaired fear extinction learning. Fear extinction dampens fear expression in response to a conditioned stimulus (CS) or context that no longer predicts aversive events. It is characterized by new learning of a negative relationship between the CS or context and the aversive event while the original fear memory is still in place (reviewed in Johnson and Casey, 2015; Singewald et al., 2015). Investigations into the molecular mechanisms underlying impaired extinction and its therapeutic normalization are important for the development of novel treatment strategies for patients suffering from anxiety and trauma-related conditions since deficient fear extinction can lead to prolonged anxiety and result in stress and anxiety-related disorders. The laboratory mouse strain 129S1/SvImJ (S1) constitutes a convenient model for fear extinction studies as it exhibits compromised fear extinction learning upon cued fear conditioning (Hefner et al., 2008) that can be rescued by dietary zinc (Zn)-restriction (Whittle et al., 2010). Since it was recently shown that the ATPdependent chromatin remodeling complex nBAF is involved in contextual but not cued fear learning (Vogel-Ciernia et al., 2013), we investigated here potential behavior-associated alterations in ChRF expression levels in contextual fear extinction using S1 mice as well as the genetically related strain 129S6/SvEvTac (S6).

# MATERIALS AND METHODS

# Animals and Husbandry

Subjects were male 3-month-old 129S1/SvImJ (S1), 129S6/SvEvTac (S6) mice and C57BL/6 mice (obtained from Charles River and Taconic, Germany) that were housed (4–5 per cage) in a temperature- (22 ± 2 ◦C) and humidity- (50–60%) controlled vivarium under a 12 h light/dark cycle (lights on at 7:00 a.m.). All experimental procedures were approved by the Austrian Animal Experimentation Ethics Board.

# Dietary Zinc Restriction (ZnR)

Animals were fed food pellets (ssniff Spezialdiäten) containing low Zn (12.3 mg/kg or 40% of the recommended daily intake requirement; Reeves et al., 1993) or standard food pellets containing normal quantities of Zn (65 mg/kg) as previously described (Whittle et al., 2010). Mice were fear conditioned on standard diet before being placed on ZnR diet for 2 weeks followed by fear expression or extinction training sessions.

# General Procedures for Contextual Fear Conditioning

An automated fear-conditioning system (TSE Systems, Bad Homburg, Germany) was used for contextual fear conditioning. Mice were conditioned in a 25 × 25 × 35 cm chamber with transparent walls and a metal rod floor, cleaned with water and illuminated to 300 lux ("context A"). After a 120 s acclimatization period, mice received 2 s scrambled foot shock unconditioned stimulus (US) (0.6 mA) for three times with a 120 s intertrial interval. After the final US there was a 120 s no-stimulus consolidation period before mice were returned to the home cage. Fear expression or extinction training was performed 14 days later by re-exposing the mice to the conditioning context A for 4 or 16 min, respectively. Freezing was measured as an index of fear (Blanchard and Blanchard, 1969), manually scored based on DVD recordings, defined as no visible movement except that required for respiration, and converted to a percentage [(duration of freezing within the context exposure/total time of the context exposure) × 100] by a trained observer blind to the experimental group.

# Statistical Analysis of Behavior Experiments

The percentage of freezing is presented as mean ± standard error of the mean (SEM). Freezing levels during fear conditioning, expression and extinction training were analyzed using multiplefactor ANOVA with repeated-measures for trial, followed by a Fisher LSD post-hoc analysis in case of significant interaction effects. Level of statistical significance was set to P < 0.05.

# Brain Dissections

Mice were sacrificed 2 h after fear expression or fear extinction training and brains were removed. Amygdala, medial prefrontal cortex (mPFC), dorsal (dHC), and ventral hippocampus (vHC) of both hemispheres were dissected, weighed and snap frozen. Where necessary, dissected regions from two to three animals were pooled. Frozen tissue was stored at −80◦C.

# RNA Isolation and qRT-PCR

Total RNA was isolated from different brain areas using Trireagent (Sigma Aldrich) followed by DNaseI digestion and spincolumn clean-up (Zymo Research). Up to 5µg of RNA were reverse-transcribed using the GoScript Reverse Transcription System (Promega) according to the manufacturer's instructions. Real time PCR was performed in triplicate using POWER SYBR Green PCR mastermix (Applied Biosystems) with 25 ng cDNA and 0.4µM of target-specific primers. Primer sequences are available upon request. Note that no amplification was obtained for ERCC6, RAD54b, RAD54, and RAD54L2. Data were normalized against Gapdh, 1C<sup>T</sup> values were centered at the median and subjected to hierarchical clustering analysis using Genesis software (Sturn et al., 2002).

# Nuclear Extract Preparation

Frozen tissues were pulverized using the Cryoprep system (Covaris) and resuspended in five volumes (v/w) homogenization buffer (10 mM Tris-HCl pH 7.9, 5 mM MgCl2, 10 mM KCl, 0.34 M sucrose, 1× protease inhibitor cocktail (Roche), 0.1 mM PMSF, 1 mM DTT). The homogenate was centrifuged for 10 min at 4◦C and 2000×g. The nuclear pellet was carefully resuspended in two volumes (v/w) extraction buffer (15 mM Tris-HCl pH 7.9, 0.25 mM EDTA, 0.43 M NaCl, 10% glycerol, 1× protease inhibitor cocktail) and incubated on ice for 30 min with gentle mixing. Nuclear extract was obtained by centrifugation at 10,000×g for 30 min at 4◦C.

# Chromatography Procedures

Nuclear extract of six brains from 4-week-old male C57BL/6 mice was dialyzed against buffer CB (50 mM Tris-HCl pH 7.9, 100 mM NaCl, 5 mM MgCl2, 1 mM EDTA, 10% glycerol, 0.1 mM PMSF, 1 mM DTT) and loaded onto a 1 ml Source15Q anion exchange column (GE Healthcare) on an Äkta Explorer FPLC system (GE Healthcare). After washing with 10 column volumes (CV) buffer CB, proteins were eluted with a 15 CV linear gradient from 100 to 500 mM NaCl in buffer CB. 0.3 ml fractions were collected and subjected to immunoblotting using antibodies against different ChRFs and HDACs. Source15Q fractions containing peak amounts of the analyzed proteins (200– 280 mM NaCl) were pooled, applied to a 100 ml Superose 6 size exclusion column (GE Healthcare) and eluted with buffer CB. Two milliliters fractions were collected and proteins were precipitated by addition of 20% (final) trichloroacetic acid (TCA) and incubation for 20 min on ice. Precipitates were collected by centrifugation at 17000×g for 15 min, washed twice with acetone, dried on ice and dissolved in 1× SDS loading buffer (75 mM Tris-HCl, pH 6.8, 0.6% SDS, 15% glycerol, and 1.075 M β-mercaptoethanol) for subsequent SDS-gel electrophoresis and western blotting.

# Immunoblotting

Proteins were separated by SDS-PAGE, transferred to nitrocellulose membrane and incubated with antibodies against CHD1 (Proteintech 20576-1-AP; 1:1000), CHD2 (Cell Signaling 4170S; 1:500), CHD3 (Cell Signaling 4241S; 1:500; Novus Biologicals NBP1-51593; 1:1000), CHD4 (Cell Signaling 4245S; 1:500), CHD5 (gift of Michael J. Pazin, HD5A-A Day 77; Potts et al., 2011; 1: 15000), CHD7 (Santa Cruz Biotechnology sc-79207; 1:1000), ATRX (Novus Biologicals NBP1-32851; 1:1000), and Snf2H (Abcam AB3749; 1:500), HDAC1 (Zymed-Invitrogen, 34–8300), HDAC2 (Zymed-Invitrogen, No 34–6400), HDAC3 (Zymed-Invitrogen, 34-7700), and TBP (Millipore 05-1531; 1:250).

# Quantification and Statistical Analysis

For relative quantification of protein amounts Image Studio Lite software (LI-COR Biosciences) was used. Intensity values were normalized against signals of TBP, which was used as a loading control. For statistical evaluation GraphPad Prism 6.0 software (GraphPad Software, San Diego, CA, USA) was used and Two-way ANOVA with Bonferroni's post-hoc test was applied.

# Immunofluorescence Microscopy

Twelve weeks old male mice fed with standard food pellets containing normal quantities of Zn (65 mg/kg), were perfused with 4% formaldehyde as described previously (Muigg et al., 2009). Brains were quickly removed and postfixed at 4 ◦C overnight in 4% paraformaldehyde in phosphate buffer and sectioned. Coronal free-floating brain sections of 40µm thickness were incubated for 30 min in TBS (0.1 M Tris-HCl pH 7.4, 0.9% NaCl) with 1% H2O2, followed by three washings with TBS. After incubation in 50% formamide/2xSSC (300 mM NaCl, 30 mM sodium citrate tribasic, pH 7) for 2 h at 65◦C, sections were washed twice in 2xSSC, treated with 2 M HCl for 30 min at 37◦C, washed in 0.1 M borate buffer (pH 8.5) for 10 min followed by three washes in TBS. Samples were blocked for 90 min using 1% bovine serum albumin (BSA) in TBS/0.1% Triton X-100 (TBST/1%BSA). Primary antibody incubations were performed in TBST/1%BSA for 48 h at 4◦C with gentle shaking. The following antibodies were used: CHD3 (Novus Biologicals NBP1-51593; 1:1000), Satb2 (Abcam AB92446; 1:800), GABA (Sigma-Aldrich A2052; 1:7000). Following three extensive washing steps with TBST/1%BSA, secondary antibody (anti-mouse ALEXA647, Jackson Immuno Wille et al. Chromatin Remodeling Factors in Fear Extinction

Research 715-605-150; 1:500; anit-rabbit CY2, Jackson Immuno Research 711-225-152; 1:500) in TBST/1%BSA was added for 2 h. Sections were again washed 3 times, mounted on microscope slides and dried overnight. ProLong Gold Antifade Mountant with DAPI (Life Technologies P-36931) was applied and the slides were cover-slipped. Microscopy was performed with an Olympus BX51 fluorescence microscope equipped with UPlan Apo 10×/0.40 and PlanApo 60×/1.42 oil immersion objectives. Images were processed using cellSense dimension 1.5 software (Olympus) and Adobe Photoshop CS3.

# RESULTS

# Characterization of ChRF Expression Patterns and Complex Formation in the Mouse Brain

To gain an initial overview of the expression of ChRFs in the brain, we performed reverse-transcription qPCR (RT-qPCR) analysis of 24 SNF2-type ATPases belonging to all known mammalian subfamilies (Flaus et al., 2006) from brain stem, cerebellum, midbrain, hypothalamus, hippocampus/thalamus/septum, cortex, and olfactory bulb regions (**Figure 1A**). Because it has been shown previously that many ChRFs exhibit high expression in mouse embryonic stem cells (ESCs; Efroni et al., 2008), RNA isolated from ESCs was analyzed for comparison. Cluster analysis of median-centered 1C<sup>T</sup> values revealed two large expression groups (**Figure 1B**). Group I shows overall higher expression in the brain than in ESCs and comprises 13 ChRFs. CHD3, Brm, and CHD5 of this group displayed the most pronounced enrichment in the brain compared to ESCs. The expression levels of group II factors are generally lower than those of group I in the brain but are similar to the corresponding levels in ESCs. In addition, some factors display specific expression patterns within the brain: for instance, CHD7 is specifically overrepresented in the cerebellum, while CHD5 is depleted from the cerebellum but slightly enriched in the hypothalamus and the cortex; CHD6 is relatively depleted in the hippocampus/thalamus/septum region and BTAF1 is relatively enriched in the olfactory bulb (**Figure 1B**). With respect to the spatial pattern of ChRF expression the data show that transcript levels deviate most often in the cerebellum from those of other brain regions (**Figure 1B**).

Next we sought to biochemically analyze ChRF complexes to determine, if different ChRFs form distinct protein complexes in the brain as has been reported for other tissues (Lusser and Kadonaga, 2003; Clapier and Cairns, 2009; Becker and Workman, 2013). In these experiments, we focused on factors of the CHD-subfamily of ChRF (CHD1, CHD3, CHD5) and on ATRX for the following reasons: (i) Antibodies against these factors were commercially available and successfully detected the corresponding proteins in brain protein extracts (Note: we also tested antibodies against CHD2, CHD4, and CHD7 but obtained either no or very faint signals or signals that did not correspond to the calculated size of the protein). (ii) CHD3 and 5 show relatively high expression on the transcript level enabling detection with limited tissue amount. (iii) All these factors have been linked to brain development and/or brain function before (Gaspar-Maia et al., 2009; Bérubé, 2011; Nogami et al., 2011; Potts et al., 2011; Piatti et al., 2015). High salt nuclear extracts were prepared from whole brains; fractionated by Source 15Q anion exchange chromatography and subjected to western blotting. CHD3 and CHD5 segregated clearly from each other and the other two ChRFs in the salt gradient elution, while CHD1 and ATRX coeluted at ∼270 mM NaCl (Supplementary Figure 1). To determine if the ChRFs were contained in multisubunit complexes, peak fractions from Source 15Q were pooled and applied to Superose 6 size exclusion chromatography (SEC). CHD3, CHD5, and ATRX eluted with peaks larger than the 660 kDa marker protein thyroglobulin indicating that indeed high-molecular weight complexes are present (**Figure 1C**). CHD1 signals were too faint after SEC (despite concentration by TCA precipitation) to reliably obtain any size information (data not shown). Since CHD3 and CHD5 are known to form complexes with HDACs in various tissues (Tong et al., 1998; Wade et al., 1998; Xue et al., 1998; Zhang et al., 1998; Potts et al., 2011), we also tested the elution patterns of HDACs1, 2, and 3 on SEC. Signals corresponding to these histone modifying proteins were detected in ChRF-containing fractions. The elution profile of HDAC3 was relatively focused and overlapped well with that of ATRX, CHD5, and CHD3. The profiles of HDAC1 and HDAC2 were broader but still overlapping (**Figure 1C**). Hence, the elution behavior of CHD3, 5, and ATRX in both anion exchange and SEC suggest that they form distinct high molecular weight complexes in the mouse brain that likely contain HDACs as previously reported for other tissues.

# S6 Mice Display Normal Fear Extinction Learning Compared to Extinction-deficient S1 Mice

Since the building of fear and fear extinction memories requires considerable changes in the transcriptional program of specific brain areas, such as the amygdala, the hippocampus, and the prefrontal cortex (reviewed e.g., in Orsini and Maren, 2012), and it is well known that ChRFs are heavily involved in regulating gene expression at the transcriptional level (Marfella et al., 2006; Hargreaves and Crabtree, 2011; Becker and Workman, 2013), we hypothesized that ChRFs might be critical players in these memory processes and that such a role might be reflected by changes in the amounts of certain ChRFs during different stages of extinction learning. To address this idea, we used two different laboratory mouse strains: The 129S1/SvImJ (hereafter termed S1) mouse strain has previously been shown to exhibit a severe fear extinction deficit when subjected to cued fear conditioning (Hefner et al., 2008). The second strain is the 129S6/SvEvTac (hereafter termed S6) strain, which is closely related to the S1 strain and therefore suitable for comparisons on a molecular level.

The first objectives of this experiment were to examine (i) whether S6 mice show normal fear acquisition and extinction behavior and (ii) whether S1 mice fail to attenuate contextdependent fear expression similar to what has been shown before

for cued conditioned fear (Hefner et al., 2008). The experimental set-up is depicted in **Figure 2A**. During conditioning, all experimental groups (n = 6/group) showed a significant increase in freezing across conditioning trials [time (freezing to the context) effect: F(3, 60) = 100.53, P < 0.001], which did not differ between the strains [strain (S6 vs. S1) effect: F(1, 20) = 2.51, P > 0.05] or groups [group (expression vs. extinction) effect: F(1, 20) = 0.00019, P > 0.05]. During fear expression, S6 and S1 mice displayed similar levels of freezing to the context [time × strain effect: F(1, 10) = 0.41, P > 0.05; n = 6/group; **Figure 2B**]. During fear extinction training, there was a significant time × strain interaction for freezing [F(7, 63) = 8.29, P < 0.001; n = 5−6/group; **Figure 2B**]. Post-hoctests revealed that freezing was significantly lower in S6 than in S1 mice starting after 8 min until the end of the experiment.

It has previously been shown that dietary zinc restriction (ZnR) can successfully induce extinction learning in extinctionimpaired S1 mice following cued fear conditioning (Whittle et al., 2010). Therefore, we assessed whether ZnR can also rescue impaired extinction learning in S1 mice in a contextual fear

freezing levels during fear expression training (*n* = 6/group; *P* > 0.05). During fear extinction training, S1 mice displayed significantly higher freezing levels over time

than S6 mice (*n* = 5.6 per group; *P* < 0.001). (C) S1 mice on ZnR diet show a significant reduction of freezing compared to Ctrl-diet S1 mice (*n* = 5/group; *P* < 0.001), while Zn restriction had no effect on fear expression. (D) Zinc restriction does not affect freezing of S6 during conditioning, fear expression and extinction learning (*P* > 0.05; *n* = 6/group).

conditioning paradigm (**Figure 2C**). Freezing levels increased upon three US presentations regardless of group assignment [time (freezing) effect: F(3, 60) = 100.78, P < 0.001; diet (Ctrl fed vs. ZnR fed) effect: F(1, 20) = 0.04, P > 0.05; group (expression vs. extinction) effect: F(1, 20) = 0.23, P > 0.05; n = 6/group] (**Figure 2C**). There was no difference in freezing between Ctrl-fed and ZnR S1 mice during fear expression [time × diet effect: F(1, 10) = 0.22, P > 0.05; n = 6/group], and fear extinction training led to a significant decline in freezing of ZnR S1 mice starting after 8 min until the end of the experiment [time × diet effect: F(7, 56) = 15.20, P < 0.001; n = 5/group] indicating successful fear extinction, while high freezing levels persisted in Ctrl-fed S1 mice (**Figure 2C**).

## Dietary ZnR does not Affect Normal Fear Extinction Learning in S6 Mice

We also examined the effects of ZnR on fear expression and extinction in extinction-competent S6 mice following contextual fear conditioning (**Figure 2D**). All groups (n = 6/group) showed a similar increase in freezing to the context across US presentations. This was ascertained from ANOVA results that revealed a significant effect on freezing during conditioning of conditioning trials [time (freezing to the context) effect: F(3, 60) = 116.72, P < 0.001], but not of diet [diet (Ctrl fed vs. ZnR fed) effect: F(1, 20) = 1.90, P > 0.05] or group [group (expression vs. extinction) effect: F(1, 20) = 0.10, P > 0.05]. During fear expression, freezing did not differ between Ctrl and ZnR S6 mice [time × diet effect: F(1, 10) = 0.49, P > 0.05; n = 6/group]. Moreover, ZnR did not further promote extinction learning in S6 mice during extinction training, as ANOVA revealed a significant effect of time [F(7, 70) = 38.90, P < 0.001], but no significant time–diet interaction [F(7, 70) = 0.79, P > 0.05].

Taken together these results demonstrate that S6 mice display intact contextual fear expression and extinction behavior. By contrast, S1 mice have a deficit in extinguishing context-dependent fear and this deficit can be rescued by dietary Zn restriction.

### Fear Extinction Involves Specific Changes in CHD1, CHD3, and CHD5 Protein Levels

We used S6, S1, and S1 animals rescued by ZnR diet to dissect brain tissues 2 h after the end of the fear expression or extinction training (**Figure 2A**) for nuclear protein extract preparation and western blotting with antibodies against the ChRFs CHD1, CHD3, CHD5, CHD7, ATRX, and SNF2h. Specifically, we examined the following brain areas due to their importance for fear learning and memory processes: the amygdala, dorsal hippocampus (dHC), vHC, and the mPFC (Fanselow and Dong, 2010; Marek et al., 2013; Wang et al., 2013; Johnson and Casey, 2015; Singewald et al., 2015; Tovote et al., 2015). While most tested ChRFs did not exhibit significant changes in protein levels between fear expression and fear extinction in the different brain regions and mouse strains (**Table 1** and data not shown), we observed significant behavior-dependent regulation of protein levels of CHD1, CHD3, and CHD5 specifically in the vHC. CHD1, which is mostly known as a ChRF regulating active transcription, was upregulated in extinctionimpaired S1 mice after unsuccessful extinction training (i.e., prolonged CS exposure). By contrast, when extinction training was successful, such as in S6 and S1 ZnR mice, CHD1 amounts did not vary between short (fear expression) and long (fear extinction) CS exposure (**Figure 3A** and **Table 1**). Conversely, the transcriptional co-repressor CHD3 remained unchanged in non-extinguishing S1 mice, but was downregulated after extinction training by 61% in behaviorally rescued S1 ZnR mice. Likewise, a trend toward lower CHD3 amount (−39%, P = 0.31) was observed in the vHC of extinction-competent S6 animals following extinction training (**Figure 3B**, **Table 1**). Finally, our analyses revealed that CHD5, which is closely related to CHD3 and has predominantly been found as a repressor of transcription, was downregulated by ∼30% (P = 0.014) in the

TABLE 1 | ChRF expression changes during fear extinction training in different brain areas of extinction-competent S6 and S1 ZnR mice and extinction-deficient S1 mice.


*vHC, ventral hippocampus; dHC, dorsal hippocampus; Amy, amygdala; mPFC, medial prefrontal cortex. Negative values indicate a decrease.* \**P* < *0.05,* \*\*\**P* < *0.005.*

vHC of S1 mice after non-successful extinction training, while protein levels did not change in extinction-competent S6 and S1 ZnR mice (**Figure 3C**). Thus, the CHD-family remodelers CHD1, CHD3, and CHD5 show marked deregulation in the extinction-deficient S1 mice that is rescued along with the behavioral defect by dietary restriction of zinc.

# CHD3 Localizes to Excitatory and Inhibitory Neurons in the Hippocampus

Since we have found fear behavior-related differences of CHD1, CHD3, and CHD5 only in the vHC and not in other examined areas, we next asked if these factors show specific subcellular localization in the hippocampus. In a

previous study, CHD5 was found to localize to neurons of the hippocampus predominantly in the CA1-CA3 regions but not to glia cells (Bergs et al., 2014). However, CHD1 and CHD3 localization in the hippocampus have not been shown to date. Therefore, we performed immunofluorescence stainings of hippocampal sections with antibodies against CHD3, CHD1, and the excitatory neuron marker Satb2 (Huang et al., 2013) as well as the inhibitory neuron marker GABA. Unfortunately, the CHD1 antibodies were not suitable for tissue stainings regardless of the protocol used. By contrast, robust signals were obtained with antibodies against CHD3 in all nuclei of both the dorsal and vHC including those of Satb2<sup>+</sup> excitatory and GABA<sup>+</sup> inhibitory neurons (**Figures 4A–D**).

# DISCUSSION

ChRFs of the SNF2 family of ATPases are known to be involved in the regulation of diverse mechanisms of DNA metabolism, such as transcription, replication, nuclear architecture, or DNA damage repair (Clapier and Cairns, 2009). While several ChRFs have been studied in the context of brain development, research regarding the role of ChRFs in cognitive and behavioral functions is extremely sparse. The recent discovery of genetic associations between several ChRFs (e.g., BAF complex, CHD8) and intellectual and psychiatric disorders (Ronan et al., 2013; Krumm et al., 2014; Vogel-Ciernia and Wood, 2014), however, underscores the importance of understanding their specific roles in the CNS. Therefore, we performed the first comprehensive analysis of ChRF expression in the brain. The expression levels of 24 members of SNF2-family remodelers revealed distinct patterns for several factors in different brain areas. CHD3 and BRM were the most highly expressed ChRFs in all brain regions, and they exhibited clearly higher levels than in ESCs, which are considered to be particularly enriched for ChRFs (Efroni et al., 2008). BRM is one of the ATPase subunits of nBAF, a neuron-specific chromatin remodeling complex that has recently been implicated in learning and long-term memory formation (Vogel-Ciernia et al., 2013). Its high expression throughout the brain suggests possible functions in general brain physiology. CHD3 is commonly found as part of a multiprotein complex termed NuRD (Nucleosome Remodeling and Deacetylation) which also contains the histone deacetylases HDAC1 and/or HDAC2 (Denslow and Wade, 2007). Although it is currently not known if CHD3 is part of a NuRD-like complex in the brain, we now show that CHD3 is contained in a megadalton complex in the brain that coelutes with HDAC1 and HDAC2 indicative of a NuRD complex. Aside from the widespread expression of CHD3 mRNA in the brain, we detected the protein in Satb2<sup>+</sup> excitatory and GABA<sup>+</sup> inhibitory neurons.

High expression levels throughout the brain were also detected for CHD5. This protein is closely related to CHD3 and it was shown to be specifically expressed in mouse brain and testes (Bergs et al., 2014). CHD5 is required for neuronal differentiation during development (Egan et al., 2013) and acts as a tumor suppressor in various cancers (Stanley et al., 2013). Moreover, it was found to form a NuRD-like complex in the brain predominantly containing HDAC2 over HDAC1 (Potts et al., 2011). Our results are consistent with these previous findings, since we also detected CHD5 in a megadalton complex coeluting with HDAC2 in SEC as well as in anion exchange chromatography. We further show that ATRX, another factor that is highly expressed throughout the brain, also forms a high molecular weight complex in brain nuclear extracts. ATRX is recognized as a ChRF that localizes to heterochromatic regions and interacts with a number of transcriptional co-repressors (Ratnakumar and Bernstein, 2013). Mutations in the ATRX gene were found to cause impaired contextual fear memory in mice (Nogami et al., 2011) and α-Thalassaemia/mental Retardation X-linked syndrome in humans (Gibbons et al., 1995). Although in SEC, ATRX perfectly coelutes with HDAC3, the separation profile on anion exchange chromatography argues against a direct association between the two proteins. Taken together, we have characterized the expression levels of most known ChRFs in the brain and we have identified the existence of various highmolecular weight ChRF complexes, including two NuRD-like complexes containing either CHD3 or CHD5.

# Aberrant ChRF Protein Levels Associated with Impaired Contextual Fear Extinction

Our studies of ChRFs in the course of fear extinction learning identified three CHD-type remodelers, CHD1, CHD3, and CHD5, to exhibit aberrant protein levels in the extinctioncompromised S1 mouse model. Moreover, we found that these changes are restricted to the vHC suggesting a particularly critical role for CHD-type remodelers in this region in the contextual fear extinction process. The role of the vHC in contextual fear conditioning is not entirely clear. Unlike the dHC, the vHC region is directly connected to the amygdala (Pitkänen et al., 2000) and the mPFC (Laroche et al., 2000; Cenquizca and Swanson, 2007; Tovote et al., 2015). Based on evidence from several different studies, it has been suggested that the dorsal hippocampal area is mainly responsible for spatial processing, while the vHC mediates the expression of fear and extinction via projections to the amygdala and the mPFC (Moser and Moser, 1998; Bannerman et al., 2004; Fanselow and Dong, 2010). Importantly, inactivation of the vHC results in impaired extinction of fear (Sierra-Mercado et al., 2011), and it typically interferes with both cued and contextual fear conditioning (Fanselow and Dong, 2010). The S1 mouse model used in this study exhibits fully competent fear learning and expression in response to either a tone stimulus (Hefner et al., 2008) or a context stimulus (**Figure 2**), but is severely compromised in fear extinction learning for both conditioned stimuli (Hefner et al., 2008; **Figure 2**). The observed chromatin regulator changes between expression and extinction in the vHC but in none of the other tested areas (dHC, amygdala, mPFC) may suggest that epigenetically balanced regulation of transcriptional programs specifically in the vHC is particularly important for contextual fear extinction learning.

We found that successful but not unsuccessful fear extinction resulted in vHC-specific downregulation of CHD3. These data are consistent with an earlier study in which downregulation of CHD3 mRNA in the HC was found in extinction-competent C57Bl/6 mice using a different contextual fear extinction training protocol (Agis-Balboa et al., 2011). Furthermore, successful fear extinction was found to be associated with decreased levels of HDAC2, which is a partner of CHD3 in the NuRD complex (Wei et al., 2012). NuRD is most often linked to transcriptional repression. However, several studies demonstrating its localization at large numbers of active genes support the possibility that it might have activating as well as repressing roles (Reynolds et al., 2012; Zhang et al., 2012). The targeting of the NuRD complex to specific genes involves interactions with specific transcription factors, such as Ikaros in lymphoid cells or Cdk2ap1 in embryonic stem cells (Kim et al., 1999; Deshpande et al., 2009). It is possible that behaviorally successful fear extinction learning requires regulation of a subset of CHD3-responsive genes in the vHC, which might

be compromised in extinction-deficient S1 mice that show no change of CHD3 levels in the course of unsuccessful extinction training.

In contrast to CHD3, we found that unsuccessful extinction training of S1 mice was associated with upregulation of CHD1, which was not observed in extinction-competent S6 and behaviorally rescued S1 mice. CHD1 is generally considered to promote gene activation because it is mostly found at transcriptionally active genes and interacts, for instance, with elongation and splicing factors or the mediator complex (Lin et al., 2011; Park et al., 2014; Siggens et al., 2015). On the other hand, knock-down experiments in ESCs have shown that more genes were upregulated in the absence of CHD1 (including genes involved in neurogenesis) supporting repressive roles for CHD1 (Gaspar-Maia et al., 2009). We have shown previously that a CHD1 N-terminal mutant protein causes defects in ESC differentiation leading to predominant neuronal differentiation (Piatti et al., 2015). Although we show here that the overall expression level of CHD1 in adult mouse brain is rather low, the pronounced deregulation during unsuccessful extinction training in the vHC of S1 suggests that locally and quantitatively restricted expression of this remodeling factor may be important to enable successful extinction learning.

While unsuccessful extinction training in S1 mice led to increased CHD1 levels, the opposite was true for CHD5. Although CHD5 has been shown to form a brain-specific NuRDlike complex, and NuRD complexes are generally regarded as transcriptional repressors, roughly equal numbers of genes were up- and downregulated upon knock-down of CHD5 in primary rat neurons (Potts et al., 2011). Interestingly, this study found that upon CHD5 knock-down, genes classified under the GO term "Behavioral fear response" failed to be upregulated to the same extent over time in culture as observed in control cells. Furthermore, Baf53b and other components of the neuronal nBAF chromatin remodeling complex were upregulated in CHD5 knock-down cells. Baf53b was recently shown to be required for long-term memory of contextual fear (Vogel-Ciernia et al., 2013). In light of these and our data, it is likely that CHD5 is directly and/or indirectly via the nBAF complex involved in the molecular regulation of fear behavior. Thus, it is tempting to speculate that downregulation of CHD5 following impaired

#### REFERENCES


fear extinction may result in unbalanced nBAF expression and substantiation of fear memory.

Collectively, our data suggest a possible scenario for the molecular functions of CHD1, CHD3, and CHD5 in compromised fear extinction learning in S1 mice in that a subset of genes that is regulated by CHD3 is not upregulated because CHD3 levels do not decrease following impaired fear extinction. Instead, different subsets of genes that are positively controlled by CHD1 and/or negatively controlled by CHD5 might be activated (e.g., nBAF) thus preventing fear extinction learning and memory. Future gene-expression profiling studies employing the mouse models used in this study combined with brain area specific knock-down of the remodeling factors will be necessary to test these hypotheses. We provide here a first overview of ChRF expression in mouse brain, and we show that CHD-type remodelers are deregulated in a behavior-related manner during contextual fear extinction in the extinction-deficient S1 mouse model.

#### AUTHOR CONTRIBUTIONS

AW, VM, PP, NW, NS, and AL conceived the study and designed the experiments; AW, VM, PP, NW, DR performed the experiments; AW, VM, PP, NW, DR, NS, AL analyzed the data and wrote the manuscript.

#### ACKNOWLEDGMENTS

We thank Drs M. Pazin and F. Ferragutti for the generous gift of antibodies and C. Murphy and C. Schmuckermair for technical help and Drs Hubert Hackl, Markus Reindl and Nadia Stefanova for advice and discussions. Research on this project was funded by the Austrian Science Fund (F4408-B19 to AL and F4410 to NS).

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnbeh. 2015.00313


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Wille, Maurer, Piatti, Whittle, Rieder, Singewald and Lusser. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Low-Cost Avoidance Behaviors are Resistant to Fear Extinction in Humans

#### Bram Vervliet 1, 2 \* and Ellen Indekeu<sup>3</sup>

*<sup>1</sup> Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA, <sup>2</sup> Center for Excellence on Generalization in Health and Psychopathology, University of Leuven—KU Leuven, Leuven, Belgium, <sup>3</sup> Department of Psychology, University of Leuven—KU Leuven, Leuven, Belgium*

Elevated levels of fear and avoidance are core symptoms across the anxiety disorders. It has long been known that fear serves to motivate avoidance. Consequently, fear extinction has been the primary focus in pre-clinical anxiety research for decades, under the implicit assumption that removing the motivator of avoidance (fear) would automatically mitigate the avoidance behaviors as well. Although this assumption has intuitive appeal, it has received little scientific scrutiny. The scarce evidence from animal studies is mixed, while the assumption remains untested in humans. The current study applied an avoidance conditioning protocol in humans to investigate the effects of fear extinction on the persistence of low-cost avoidance. Online danger-safety ratings and skin conductance responses documented the dynamics of conditioned fear across avoidance and extinction phases. Anxiety- and avoidance-related questionnaires explored individual differences in rates of avoidance. Participants first learned to click a button during a predictive danger signal, in order to cancel an upcoming aversive electrical shock (avoidance conditioning). Next, fear extinction was induced by presenting the signal in the absence of shocks while button-clicks were prevented (by removing the button in Experiment 1, or by instructing not to click the button in Experiment 2). Most importantly, post-extinction availability of the button caused a significant return of avoidant button-clicks. In addition, trait-anxiety levels correlated positively with rates of avoidance during a predictive safety signal, and with the rate of pre- to post-extinction decrease during this signal. Fear measures gradually decreased during avoidance conditioning, as participants learned that button-clicks effectively canceled the shock. Preventing button-clicks elicited a sharp increase in fear, which subsequently extinguished. Fear remained low during avoidance testing, but danger-safety ratings increased again when button-clicks were subsequently prevented. Together, these results show that low-cost avoidance behaviors can persist following fear extinction and induce increased threat appraisal. On the other hand, fear extinction did reduce augmented rates of unnecessary avoidance during safety in trait-anxious individuals, and instruction-based response prevention was more effective than removal of response cues. More research is needed to characterize the conditions under which fear extinction might mitigate avoidance.

Keywords: fear, extinction, avoidance, response prevention, exposure

#### Edited by:

*Denise Manahan-Vaughan, Ruhr University Bochum, Germany*

#### Reviewed by:

*Sigrid Elsenbruch, University Hospital Essen, Germany Gemma Cameron, Swansea University, UK*

#### \*Correspondence:

*Bram Vervliet bram.vervliet@ppw.kuleuven.be*

Received: *29 September 2015* Accepted: *03 December 2015* Published: *24 December 2015*

#### Citation:

*Vervliet B and Indekeu E (2015) Low-Cost Avoidance Behaviors are Resistant to Fear Extinction in Humans. Front. Behav. Neurosci. 9:351. doi: 10.3389/fnbeh.2015.00351*

# INTRODUCTION

Most studies on fear extinction focus on passive emotional reactions, like freezing in the rat or skin conductance reactivity in humans. Pavlovian fear conditioning first installs these reactions, by pairing a neutral stimulus (the conditioned stimulus, CS) repeatedly with an aversive stimulus (the unconditioned stimulus, US). Over CS-US pairings, the CS starts eliciting de novo fear reactions, in anticipation of the US. Once these reactions are firmly established, they can be removed again by repeatedly presenting the CS in the absence of its US, which leads to a gradual decline of the CS-elicited fear reactions (i.e., fear extinction). Pavlovian fear conditioning and extinction serve as widely used translational models to study the psychobiological mechanisms of the development and treatment of clinical anxiety (Milad and Quirk, 2012). According to the Pavlovian conditioning model, irrational fears stem from erroneous associations between intrinsically safe situations (CS) and imagined dangerous consequences (US). Therefore, exposure-based treatments use the fear extinction principle to counter these erroneous associations and decrease the irrational fears, by exposing the patient to the objects/situations of fear over and over again (Vervliet et al., 2013). Meta-analyses of clinical studies have confirmed the overall efficacy of this technique for reducing fear, although relatively high rates of both nonresponding and relapse pose continuous, serious challenges (Craske and Mystkowski, 2006).

Anxiety disorders are characterized by elevated fears of safe situations, as well as excessive avoidance of those situations (American Psychiatric Association, 2013). Avoidance is a form of self-protective action that serves to minimize confrontations with a feared danger (for a review, see Krypotos et al., 2015). Although avoidance is often adaptive in the face of real danger, it is superfluous when the fears are irrational and the danger absent. Moreover, it maintains these irrational fears by precluding learning opportunities that could otherwise show the actual absence of danger and produce fear extinction (Lovibond et al., 2009; see also Krypotos et al., 2015). Persistent avoidance is therefore not only a cardinal symptom across the anxiety disorders, but also a major reason why irrational fears do not extinguish spontaneously in the anxiety patient. An important part of exposure-based treatments is to identify and neutralize avoidance behaviors prior to conducting exposures to the feared situations, in order to optimize the extinction learning process (termed "response prevention with extinction," RPE). The success of exposure-based treatment is determined by reductions in fear as well as avoidance. Some even claim that reducing avoidance is the only relevant outcome measure of anxiety treatments (Hayes et al., 2006). In contrast, contemporary fear extinction research focuses almost exclusively on removing passive fear reactions, with no inclusion of avoidance in the fear conditioning history or during the extinction test phase. Hence, little is known about the effects of fear extinction on avoidance extinction. It remains unclear, e.g., to what extent behavioral and/or pharmacological enhancers of fear extinction might also mitigate avoidance.

Mitigating rates of avoidance was a major focus of preclinical animal research in the 1960s–1970s, and RPE was the most investigated treatment at the time (also termed "flooding"). In a seminal study on avoidance learning with high intense shocks in dogs, Solomon et al. (1953) found that preventing the avoidance response (jumping over a hurdle avoided the shock, a glass-barrier prevented the jumping) led to avoidance extinction only in 7 out of 9 dogs when the glass-barrier was removed. Later studies with less intense shocks and rats as subjects showed that RPE does speed up later avoidance extinction compared to rats that received extinction without response prevention or no treatment (reviewed by Mineka, 1979). Unfortunately, these early studies only reported the number of trials-to-criterion of extinction, but did not report initial rates of avoidance responding during test. Nevertheless, the fact that avoidance extinction was never immediate suggests that the avoidance response initially returned when the prevention was lifted, before entering into extinction. This was recently confirmed by a behavioral conflict study in which hungry rats had to chose between pressing a lever for food under threat of shock vs. jumping on a platform that protected against the shock but with no food available (Bravo-Rivera et al., 2015). The safe platform constituted a costly avoidance response as it implied the loss of food (high-cost avoidance). Removal of the platform (and the shocks) initially increased fear-related freezing that subsequently extinguished (response prevention with extinction). Despite complete fear extinction, returning the platform to the cage triggered significant return of shockavoidance responses (even in the absence of actual shocks). Moreover, the amount of return correlated with c-Fos measured neural activity in the prelimbic prefrontal cortex and the ventral striatum (brain regions closely linked to anxiety and avoidance), but not in the infralimbic prefrontal cortex or the basolateral amygdala (brain regions closely linked to fear extinction). These results show that avoidance behaviors in the rat can persist irrespective of fear extinction.

A seminal study on shock-avoidance conditioning in humans confirmed that preventing an established avoidance response triggers a return of conditioned fear responses (Lovibond et al., 2009), but subsequent fear extinction and its effect on avoidance were not examined. To date, this issue remains untested in humans, despite its clinical relevance. For that purpose, we merged components of the avoidance protocol of Lovibond et al. (2009) with components of a widely used fear extinction protocol (Milad et al., 2005). Of note, the (Lovibond et al., 2009) protocol involves a low-cost avoidance response (merely clicking a button with no associated costs), which differs from the costly avoidance response in the Bravo-Rivera et al. (2015) study. Arguably, clinical avoidance comprises both high- and low-cost avoidance behaviors that prevent extinction and maintain anxiety in the long run. Subtle safety behaviors like carrying anxiety pills are an example of low-cost avoidance that can go unnoticed and are sometimes difficult to treat. Moreover, because of the low cost, these avoidance behaviors may be especially persistent and unaffected by fear extinction. This, in turn, may pose a continuous vulnerability for relapse of the fear and avoidance symptomatology. For these reasons, we focused on low-cost avoidance to investigate the effects of response prevention and extinction.

Two different colorings of a lamp in a room picture signaled the imminence of an aversive electrical shock, or nothing. Skin conductance responses and danger-safety ratings tracked the development of anticipatory arousal and threat appraisal to the lamp colorings. Mouse-clicking a button on the room pictures served to avert the shock. We operationalized response prevention by removing the button (Experiment 1) or by instructing participants that the button was no longer available (Experiment 2), while shocks were no longer delivered (extinction). We counterbalanced two different lenghts of extinction in each experiment, in order to minimize the chances that spontaneous fluctuations of fear would contribute to persistent avoidance. For the critical test of this study, we assessed persistent avoidance by re-introducing the (availability of the) avoidance button and by recording the number of buttonclicks accordingly. Based on the Bravo-Rivera et al. findings, we predicted a return of avoidance following fear extinction. For exploratory purposes, we next removed the (availability of the) avoidance button again to test the persistence of fear extinction. Finally, we explored relationships between anxietyand avoidance-related personnality traits (measured by validated questionnaires) and rates of avoidance responses before and after fear extinction (cf. Lommen et al., 2010; van Meurs et al., 2014).

# EXPERIMENT 1

# MATERIALS AND METHODS

# Participants

Twenty individuals (age 17–21, average = 18.9, 18 females), mostly from first grade psychology, participated to earn course credis or financial compensation (8 EUR). Given the administration of electrocuaneous shocks in the experiment, participants were screened and excluded for the following conditions: pregnancy, cardivascular, pneuomological, neurological or other serious medical conditions, psychiatric conditions, chronic pain near the wrists, electronic implants, or having received medical instructions to avoid stressful situations. Participants were randomly assigned to Group Long-Ext and Group Short-Ext. The study was approved by the Social and Societal Ethical Committee and the Medical Ethical Committee of the University of Leuven–KU Leuven. All participants gave informed consent and were informed that they could decline further participation at any time during the experiment.

# Materials

The conditional stimuli were pictures of an office room with an desk top lamp that could color yellow or blue (taken from Milad et al., 2005), presented on a computer screen located on eye-level in front of the participant at approximately 500 mm. The avoidance stimulus was a picture of a red button that could appear over the room pictures (top left). Danger-safety ratings were measured on a trial-by-trial basis during each room picture presentation. A vertical scale was presented on the left of the screen with three options from low to high: "Safe, Uncertain, Danger" (translated from Dutch). Participants could move over the scale by using the computer mouse, and completed their rating by clicking on the left mouse button. A 2 ms electrocutaneous shock delivered to the forearm of the left hand served as unconditional stimulus (US). It was administered by a Digitimer DS7A constant current stimulator (Hertfordshire, UK) via a pair of 11-mm Fukuda Standard AG/AGCl electrodes, filled with K-Y Jelly. The intensity of the shock was individually selected to a level where it was "uncomfortable but not painful." Participants were seated in an armchair in a sound attenuated room, adjacent to the experimenter's room.

Electrodermal activity was recorded using a skin conductance coupler manufactured by Colbourn Instruments (model V71- 23, Allentown, PA). During skin conductance measurement, the coupler applied a constant voltage of 0.5 V across a pair of sintered-pellet silver chloride electrodes (8 mm), attached to the hypothenar palm of the left hand. The inter-electrode distance was approximately 10 mm. The electrodes were filled with K-Y Jelly. The resulting conductance signal was submitted through a Labmaster DMA 12-bit analog-to-digital converter (Scientific Solutions, Solon, Ohio) and digitized at 10 Hz from 2 s prior to CS onset until 6 s after CS offset.

#### Trait Portion of the State-Trait Anxiety Inventory (STAI-T)

The STAI measures trait anxiety (STAI-T) via 20 questions with scores ranging from 20 to 80, with higher scores indicating higher levels of anxiety (Spielberger et al., 1970). The Dutch version by van der Ploeg (2000) was used, which has good reliability and validity.

#### Cognitive-Behavioral Avoidance Scale (CBAS)

This 31 item questionnaire measures four dimensions of avoidance: Cognitive-Social, Cognitive-Nonsocial, Behavioral-Social, and Behavioral-Nonsocial (Ottenbreit and Dobson, 2004). The total CBAS score correlates highly with depression and anxiety inventories (e.g., STAI). The Dutch version by Vandromme et al. (2007) was used, which shows good reliability and validity.

#### Intolerance of Uncertainty Scale (IU)

This 27 time questionnaire measures emotional, cognitive and behavioral reactions to ambiguous situations, implications of being uncertain, and attempts to control the future (Freeston et al., 1994). The Dutch version by de Bruin et al. (2006) was used, which shows good reliability and validity.

# Procedure

Following general instructions (about the use of pictures and electrical shocks in the experiment, and the measurement of skin conductance) and completion of the informed consent, participants were fitted with electrodes and were led through the work-up procedure to select a "definitely uncomfort- able, but not painful" shock level. Next, participants received explicit instructions that the blue lamp would signal the electrical shock, and that the yellow would signal the absence of the electrical shock (this was done to ensure a fast development of fear reactions to the CSP with a minimal number of actual CS-US conditioning trials). The danger-safety ratings scale and the red button were explained to them (move the pointer over the desired location and mouse-click).

The room pictures were always presented for 12 s. Three seconds after room picture onset, the lamp colored yellow or blue for the remaining 9 s. One second after lamp coloring onset, the red button appeared for 2 s (on trials that contained the red button). Two seconds later, the rating scale appeared until the participant clicked on it or until the picture disappeared from the screen (during Pavlovian conditioning, the rating scale appeared 2 s earlier). The electrical shock was delivered at 500 ms before picture offset (on CSP trials during some phases). Inter-trial intervals varied between 13 and 17 s, with a mean of 15 s. The experiment consisted of five phases: Fear conditioning, avoidance conditioning, fear extinction with response prevention, avoidance test, and reextinction test (see **Figure 1**). All phases occurred consecutively without interruptions. The fear conditioning phase consisted of two presentations of the yellow and the blue light, where the blue light (CSP) was always followed by the US, while the yellow light was not (CSM). During the subsequent avoidance conditioning phase, the red button appeared during all eight CSP and CSM presentations. Next, both the button and the USs were removed during the CSP and CSM presentations of the fear extinction and response prevention phase (eight presentations of each CS in group Extinction Short and 12 presentations of each CS in group Extinction Long). In order to measure the recovery of avoidance behavior, the red button returned during the four CSP and CSM presentations of the avoidance test phase (no shock administrations). In order to measure residual skin conductance and shock-expectancy, the button was removed again during the four CSP and CSM presentations of the reextinction test phase. The numbers of trials for each phase were chosen based on standard fear conditioning, extinction and avoidance studies that include consideration of requirements for skin conductance measurements (e.g., Lovibond et al., 2013; Vervliet and Geens, 2014).

### RESULTS

Skin conductance responses (SCRs) were calculated by subtracting trial-by-trial baseline levels (average skin

(four trials each). (B): Timeline of an avoidance conditioning trial. The room picture (context) is presented for 3 s, before the desktop lamp colors blue (or yellow) for 9 s. One second after lamp coloring, a red button appears for 2 s (during which the participant can choose to click the button using the computer mouse). Two seconds after removal of the button, a mouse-controlled vertical rating scale appears on the left of the screen comprising three levels of increasing threat: Safe (lowest)—Uncertain (middle)—Danger (highest). Finally, the aversive shock is administered at picture offset (unless the participant clicked the button earlier).

conductance level during 2 s window prior to context presentation) from peak levels (maximal skin conductance level during 4 s window prior to CS offset). Negative responses were replaced by zero (cf. Soeter and Kindt, 2010). Prior to statistical analyses, SCRs were Z-transformed per participant across all phases. Avoidance reponses were scored as 1 (vs. 0) and averaged per participant, per CS and per phase prior to statistical analyses. The vertical rating scale comprised three categories denoting increasing threat value: safe (lowest) uncertain (middle)—danger (highest), which we considered to be an interval scale allowing parametric testing (analysis of variance, ANOVA). Nevertheless, to account for the possibility that the scale may only be ordinal, we additionally performed non-parametric tests (Wilcoxon signed rank test) for crucial comparisons (effects of adding/removing the button).

# Avoidance Responses

**Figure 2A** suggests a higher proportion of avoided CSP trials compared to CSM trials, which remains during Avoidance test, despite a general decrease. This was confirmed by a 2 (Group) × 2 (CS) × 2 (Phase) RM-ANOVA, revealing a main effect of CS, F(1, 18) = 33.04, p < 0.001, η <sup>2</sup> = 0.65, a main effect of Phase, F(1, 18) = 4.60, p < 0.05, with no CS × Phase interaction, F(1, 18) = 1.01, p = 0.33.

# Danger-Safety Ratings

#### Fear Conditioning

**Figure 2B** suggests robust differential danger-safety ratings with no difference between the groups, which was confirmed by a 2 (Group) × 2 (CS) × 2 (Trial) RM-ANOVA, revealing a main effect of CS, F(1, 17) = 104.46, p < 0.001, η <sup>2</sup> = 0.86, with no CS × Group interaction, F(1, 17) = 1.94, p = 0.18.

#### Avoidance Conditioning

**Figure 2B** suggests gradual decrease of danger-safety ratings over avoidance trials, which was confirmed by a 2 (Group) × 2 (CS) × 8 (Trial) RM-ANOVA, revealing a main effect of CS, F(1, 14) = 25.17, p < 0.001, η <sup>2</sup> = 0.64, and a significant CS × Trial interaction, F(7, 98) = 8.48, p < 0.001, η <sup>2</sup> = 0.38, with significant linear and quadratic trends, F(1, 14) = 22.24, p < 0.001, η 2 = 0.61, and F(1, 14) = 8.53, p < 0.05, η <sup>2</sup> = 0.38, respectively. The Group × CS × Trial interaction was not significant, F(7, 98) = 1.69, p = 0.12, suggesting similar decrease of danger-safety rating across the two groups.

#### Transition from Avoidance Conditioning to Response Prevention

**Figure 2B** suggests a strong return of differential danger-safety ratings upon removal of the avoidance button, which was confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last avoidance trial, first extinction trial) RM-ANOVA that revealed a CS × Trial interaction, F(1, 16) = 79.34, p < 0.001, η <sup>2</sup> = 0.83, which was unexpectedly qualified by a marginally significant Group × CS × Trial interaction, F(1, 16) = 3.82, p = 0.07, η <sup>2</sup> = 0.19. A 2 (Group) × 2 (CS) × 2 (Trial: last Pavlovian, first extinction) RM-ANOVA further revealed that the level of differential danger-safety ratings was statistically indistinguishable from the

Pavlovian conditioning phase, as suggested by the absence of a CS × Trial interaction, F(1, 16) = 0.44, p = 0.52. Again, the Group × CS × Trial was unexpectedly signficant, F(1, 16) = 8.17, p < 0.05, η <sup>2</sup> = 0.34, suggesting stronger return in Group LongExt vs. Group ShortExt although the two groups had received identical treatment up to this point. A Wilcoxon signed rank test comparing the last CSP avoidance trial with the first CSP extinction trial confirmed that ratings shifted to higher scalecategories upon removal of the avoidance button, Z = −3.88, p < 0.001.

#### Response Prevention and Extinction

Focusing on the first eight trials of extinction, **Figure 2B** suggests gradual decrease of danger-safety ratings. This was confirmed by a 2 (Group) × 2 (CS) × 8 (Trial) RM-ANOVA, revealing a main effect of CS, F(1, 15) = 147.67, p < 0.001, η <sup>2</sup> = 0.91, and a significant CS × Trial interaction, F(7, 105) = 7.68, p < 0.001, η <sup>2</sup> = 0.34. Unexpectedly, the Group × CS interaction was also significant, F(1, 15) = 14.16, p < 0.01, η <sup>2</sup> = 0.49, suggesting more differential danger-safety ratings in Group LongExt (see **Figure 1**), but the Group × CS × Trial interaction was not significant, F(7, 105) = 0.45, p = 0.87, confirming similar extinction curves across the two groups. In order to compare the end-points of extinction between the groups, we compared the last 4 extinction trials in a 2 (Group) × 2 (CS) × 4 (Trial: ext5-8 in Group ShortExt, ext9-12 in Group LongExt), revealing a main effect of CS, F(1, 18) = 10.00, p < 0.01, η <sup>2</sup> = 0.36, with no Group × CS interaction, F(1, 18) = 0.25, p = 0.62. This suggests that extinction was not complete, equally so in both groups.

#### Avoidance Test

**Figure 2B** suggests that the return of the avoidance button had no detectable impact on the level of differential dangersafety ratings. We calculated the average danger-safety ratings for each CS during the four avoidance test trials and compared this with the averaged last four extinction trials (ext5-8 in Group ShortExt, ext9-12 in Group LongExt). The resulting 2 (Group) × 2 (CS) × 2 (Phase: last extinction trials vs. avoidance test trials) revealed a main effect of CS, F(1, 18) = 11.51, p < 0.01, η <sup>2</sup> = 0.39, and a marginally significant CS × Phase interaction, F(1, 18) = 4.35, p = 0.052, η <sup>2</sup> = 0.20, suggesting a further decrease in differential dangersafety ratings when the avoidance operant was made available again.

#### Reextinction Test

**Figure 2B** suggests an increase of differential danger-safety ratings upon removal of the avoidance button, which was confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last avoidance test trial, first reextinction test trial) RM-ANOVA, revealing a significant main effect of CS, F(1, 15) = 23.77, p < 0.001, η <sup>2</sup> = 0.61, and most importantly, a significant CS × Trial interaction, F(1, 15) = 16.64, p < 0.01, η <sup>2</sup> = 0.53, that was similar across the two groups, Group × CS × Trial, F(1, 15) = 0.06, p = 0.81. Moreover, differential danger-safety ratings also increased against the last extinction trial prior to avoidance test, as confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last extinction and first post-avoidance test) RM-ANOVA, which revealed a main effect of CS, F(1, 15) = 14.54, p < 0.01, η <sup>2</sup> = 0.49, a main effect of Trial, F(1, 15) = 9.93, p < 0.01, η <sup>2</sup> = 0.40, and most importantly, a significant CS × Trial interaction, F(1, 15) = 8.53, p < 0.05, η <sup>2</sup> = 0.36, that was not qualified by Group, F(1, 15) = 1.71, p = 0.21. Post hoc comparisons revealed that the CS × Trial interaction was driven by a significant increase of danger-safety ratings to the CSP, F(1, 15) = 9.94, p < 0.01, η <sup>2</sup> = 0.40, while danger-safety ratings to CSM did not change, F(1, 15) = 1.13, p = 0.30. A Wilcoxon signed rank test comparing the last CSP extinction trial with the first CSP reextinction trial confirmed that ratings shifted to higher scale-categories following the avoidance test, Z = −2.46, p < 0.05.

### Skin Conductance Fear Conditioning

**Figure 2C** suggests successful conditioning of differential SCR in both groups, which is confirmed by a 2 (Group) × 2 (CS, averaged over the two trials) RM-ANOVA, revealing a main effect of CS, F(1, 18) = 7.74, p = 0.01, η <sup>2</sup> = 0.30.

#### Avoidance Conditioning

**Figure 2C** suggests differential SCR with a general decrease over trials, which is confirmed by a 2 (Group) × 2 (CS) × 8 (Trial) RM-ANOVA, revealing a main effect of CS, F(1, 18) = 12.12, p < 0.01, η <sup>2</sup> = 0.40, and a main effect of Trial, F(7, 126) = 2.59, p < 0.05, η <sup>2</sup> = 0.13, with no CS × Trial interaction, F(7, 126) = 0.79, p = 0.59.

#### Transition from Avoidance Conditioning to Response Prevention

**Figure 2C** suggests an increase of differential SCR upon removal of the avoidance button, which is confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last avoidance trial, first extinction trial) RM-ANOVA, revealing a main effect of CS, F(1, 18) = 17.92, p < 0.01, η <sup>2</sup> = 0.50, a main effect of Trial, F(1, 18) = 12.30, p < 0.01, η <sup>2</sup> = 0.41, and most importantly, a significant CS × Trial interaction, F(1, 18) = 11.59, p < 0.01, η <sup>2</sup> = 0.39.

#### Response Prevention and Extinction

Focusing on the first eight trials, **Figure 2C** suggests a gradual extinction of differential SCR, which is confirmed by a 2 (Group) × 2 (CS) × 8 (Trial) RM-ANOVA, revealing a main effect of CS, F(1, 18) = 38.94, p < 0.001, η <sup>2</sup> = 0.68, a main effect of Trial, F(7, 126) = 4.31, p < 0.01, η <sup>2</sup> = 0.19, and most importantly a significant CS × Trial interaction, F(7, 126) = 3.46, p < 0.01, η <sup>2</sup> = 0.16. In order to compare the end-points of extinction between the groups, we compared the last 4 extinction trials in a 2 (Group) × 2 (CS) × 4 (Trial: ext5-8 in Group ShortExt, ext9-12 in Group LongExt), revealing a main effect of CS, F(1, 18) = 4.93, p < 0.05, η <sup>2</sup> = 0.22, with no Group × CS interaction, F(1, 18) = 0.85, p = 0.37. This suggests that extinction was not complete, equally so in both groups.

#### Avoidance Test

**Figure 2C** suggests that the return of the avoidance button had no detectable impact on the level of differential SCR, but produced a general decrease of SCR. We calculated the average SCR for each CS during the four avoidance test trials and compared this with the averaged last four extinction trials (ext5-8 in Group ShortExt, ext9-12 in Group LongExt). The resulting 2 (Group) × 2 (CS) × 2 (Phase: last extinction trials vs. avoidance test trials) revealed a marginally significant main effect of CS, F(1, 18) = 3.43, p = 0.08, η <sup>2</sup> = 0.16, a significant main effect of Phase, F(1, 18) = 12.78, p < 0.01, η <sup>2</sup> = 0.42, but no CS × Phase interaction, F(1, 18) = 0.22, p = 0.64, suggesting an overall decrease in SCR.

#### Reextinction Test

**Figure 2C** suggests no return of differential SCR, but a further decrease in overall SCR. This was confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last extinction trial vs. first reextinction test trial), revealing only a significant effect of Trial, F(1,16) = 6.10, p < 0.05, η <sup>2</sup> = 0.28, and by a 2 (Group) × 2 (CS) × 2 (Phase: mean last four extinction trials vs. mean post-avoidance test) RM-ANOVA, revealing only a significant main effect of Trial, F(1,16) = 9.25, p < 0.01, η <sup>2</sup> = 0.37. Both the 2 (Group) × 2 (CS) × 2 (Trial: last avoidance test vs. first post-avoidance test) and the 2 (Group) × 2 (CS) × 2 (Phase: avoidance test vs. post-avoidance test) RM-ANOVAs revealed no significant effects.

# DISCUSSION

Experiment 1 was set up to validate a response prevention and extinction (RPE) protocol in human avoidance conditioning, and to assess the effects of a return of avoidance availability on avoidance frequency and conditioned fear responding. Following differential fear conditioning (CSP/CSM), participants learned to produce the avoidance response primarily to the danger cue CSP and less so to the safety cue CSM. Shockexpectancy and SCR gradually decreased over avoidance trials, but sudden removal of the avoidance availability (response prevention) elicited a strong return of shock-expectancy and SCR that gradually decreased over the extinction trials. These results are in line with typical observations in exposure treatment (initial increase of anxiety, followed by extinction) as well as in animal RPE research. Moreover, the results revealed that a return of avoidance availability triggered a return of avoidance responding to the danger cue CSP. Subsequent removal of the avoidance availability produced an increase in shock-expectancy relatively to the end of RPE. Together, these results suggest that RPE effects are difficult to generalize to the original situation without response prevention, as evidenced by a return of avoidance and shockexpectancy.

# EXPERIMENT 2

The addition/removal of the avoidance button constituted a salient visual event in Experiment 1 that may have hindered generalization of RPE effects across the different phases (Nakajima, 2014). During exposure treatment, on the other hand, response prevention is often accomplished by instructing patients not to engage in avoidance activities, rather than physically removing their availability altogether (e.g., instructing to sit far away from the exit during agoraphobic exposure exercises in a theater). Hence, we decided to use verbal instructions to indicate the (un)availability of the avoidance button in Experiment 2, while the button featured during all phases of the experiment (except for the initial Pavlovian conditioning phase). Otherwise, Experiment 2 was exactly identical to Experiment 1.

# MATERIALS AND METHODS

# Participants

Twenty individuals (age 18–23, average age = 19.2, 14 females) participated in the experiment. Enrollment, screening and exclusion criteria were exactly identical as Experiment 1.

# Apparatus

Identical to Experiment 1.

# Procedure

The procedure was identical to Experiment 1, except for the fact that the red button was also present during the Extinction with Response Prevention phase and the Reextinction Test phase, while these phases were preceded by written instructions on the screen: "Please don't click the red button from now on." Participants maintained control over the mouse and could, in principle, still click the button. The Avoidance Test phase was preceded by the following instructions: "You are free to click the red button from now on."

# RESULTS

# Avoidance Responses

**Figure 3A** suggests a higher proportion of avoided CSP trials compared to CSM trials, which remains during Avoidance test despite a general decrease in responding. This was confirmed by a 2 (Group) × 2 (CS) × 2 (Phase) RM-ANOVA, revealing a main effect of CS, F(1,18) = 68.03, p < 0.001, η <sup>2</sup> = 0.79, a main effect of Phase, F(1,18) = 24.31, p < 0.001, with no CS × Phase interaction, F(1,18) = 2.04, p = 0.17. The data show the absence of button-clicks during the Response Prevention and Extionction Phase or during the Reextinction Test phase.

# Danger-Safety Ratings Fear Conditioning

**Figure 3B** suggests robust differential danger-safety ratings, which was confirmed by a 2 (ShortExt, LongExt) × 2 (CS+, CS−) × 2 (Trials) repeated measures ANOVA, revealing a main effect of CS, F(1,17) = 262.01, p < 0.001, η <sup>2</sup> = 0.94, with no interaction with Group, F(1,17) = 1.77, p = 0.20.

#### Avoidance Conditioning

**Figure 3B** suggests gradual decrease of differential danger-safety ratings over avoidance conditioning trials. This was confirmed by a 2 (Group) × 2 (CS) × 8 (Trial) RM-ANOVA, revealing a significant main effect of CS, F(1,16) = 13.97, p < 0.01, η <sup>2</sup> = 0.47, as well as a significant CS × Trial interaction, F(7, 112) = 3.55, p < 0.01, η <sup>2</sup> = 0.18. Unexpectedly, this interaction was qualified by Group, F(7, 112) = 2.19, p < 0.05, η <sup>2</sup> = 0.12.

#### Transition from Avoidance Conditioning to Response Prevention

**Figure 3B** suggests a strong return of differential danger-safety ratings upon removal of the avoidance button, which was confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last avoidance trial, first extinction trial) RM-ANOVA that revealed a CS × Trial

interaction, F(1,18) = 119.12, p < 0.001, η <sup>2</sup> = 0.87, with no Group × CS × Trial interaction, F(1,18) = 0.53, p = 0.47. A 2 (Group) × 2 (CS) × 2 (Trial: last fear conditioning trial, first extinction trial) RM-ANOVA further revealed that this return of differential danger-safety ratings was not complete, evidenced by the signficant CS × Trial interaction, F(1,18) = 6.79, p = 0.05, with no Group × CS × Trial interaction, F(1,18) = 0.14, p = 0.71. A Wilcoxon signed rank test comparing the last CSP avoidance trial with the first CSP extinction trial confirmed that ratings shifted to higher scale-categories upon removal of the avoidance button, Z = −4.10, p < 0.001.

#### Response Prevention and Extinction

Focusing on the first eight trials of extinction, **Figure 3B** suggests gradual decrease of danger-safety ratings. This was confirmed by a 2 (Group) × 2 (CS) × 8 (Trial) RM-ANOVA, revealing a main effect of CS, F(1, 16) = 36.57, p < 0.001, η <sup>2</sup> = 0.70, and a significant CS × Trial interaction, F(7, 112) = 11.93, p < 0.001, η <sup>2</sup> = 0.43, with no Group × CS × Trial interaction, F(7, 112) = 0.23, p = 0.98. In order to compare the end-points of extinction between the groups, we compared the last four extinction trials in a 2 (Group) × 2 (CS) × 4 (Trial: ext5-8 in Group ShortExt, ext9- 12 in Group LongExt), revealing a significant main effect of CS, F(1, 17) = 12.01, p < 0.01, η <sup>2</sup> = 0.41, with no Group interaction, F(1, 17) = 0.25, p = 0.63. This suggest incomplete extinction, equally so across the two groups.

#### Avoidance Test

**Figure 3B** suggests that the return of the avoidance button had no detectable impact on the level of differential danger-safety ratings. We calculated the average danger-safety ratings for each CS during the four avoidance test trials and compared this with the averaged last four extinction trials (ext5-8 in Group ShortExt, ext9-12 in Group LongExt). The resulting 2 (Group) × 2 (CS) × 2 (Phase: last extinction trials vs. avoidance test trials) revealed a significant main effect of CS, F(1, 18) = 22.05, p < 0.001, η <sup>2</sup> = 0.55, with no CS × Phase interaction, F(1, 18) = 1.18, p = 0.29.

#### Reextinction Test

**Figure 3B** suggests an increase of differential danger-safety ratings upon removal of the avoidance button, which was confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last avoidance test, first reextinction test) RM-ANOVA, revealing a significant main effect of CS, F(1,17) = 22.74, p < 0.001, η <sup>2</sup> = 0.57, and most importantly, a significant CS × Trial interaction, F(1,17) = 15.26, p < 0.001, η <sup>2</sup> = 0.47, that was qualified by Group, F(1,17) = 6.78, p = 0.02, η <sup>2</sup> = 0.29. Separate CS × Trial RM-ANOVAs per group confirmed a significant return of differential dangersafety ratings in Group ShortExt, F(1, 8) = 12.00, p < 0.01, η <sup>2</sup> = 0.60, but not in Group LongExt, F(1, 9) = 2.25, p = 0.17. Moreover, differential danger-safety ratings also increased against the last extinction trial prior to avoidance test, as evidence by a 2 (Group) × 2 (CS) × 2 (Trial: last extinction, first post avoidance test) RM-ANOVA, revealing a main effect of CS, F(1,18) = 17.04, p < 0.01, η <sup>2</sup> = 0.49, and a CS × Trial interaction, F(1,18) = 5.19, p < 0.05, η <sup>2</sup> = 0.22, which was not qualified by Group, F(1,18) = 0.11, p = 0.75. Post-hoc comparisons revealed that the CS × Trial interaction was driven by a significant increase of danger-safety ratings to CSP, F(1,18) = 5.33, p < 0.05, η <sup>2</sup> = 0.23, while dangersafety ratings did not change for CSM, F(1,18) = 1.00, p = 0.33. A Wilcoxon signed rank test comparing the last CSP extinction trial with the first CSP reextinction trial confirmed that ratings shifted to higher scale-categories following the avoidance test, Z = − 2.13, p < 0.05.

# Skin Conductance

#### Fear Conditioning

**Figure 3C** suggests successful differential SCR, which is confirmed by a 2 (Group) × 2 (CS, averaged over the two trials) RM-ANOVA, revealing a main effect of CS, F(1,18) = 16.76, p < 0.01, η <sup>2</sup> = 0.48.

#### Avoidance Conditioning

**Figure 3C** suggests differential SCR that decreases over trials, which is confirmed by a 2 (Group) × 2 (CS) × 8 (Trial) RM-ANOVA, revealing a main effect of CS, F(1,17) = 9.86, p < 0.01, η <sup>2</sup> = 0.37, and a main effect of Trial, F(7, 119) = 3.59, p < 0.01, η <sup>2</sup> = 0.17, as well as a CS × Trial interaction, F(7, 126) = 2.25, p < 0.05, η <sup>2</sup> = 0.12.

#### Transition from Avoidance Conditioning to Response Prevention

**Figure 3C** suggests an increase of differential SCR upon removal of the avoidance button, which is confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last avoidance, first extinction) RM-ANOVA, revealing a main effect of CS, F(1, 18) = 27.17, p < 0.001, η <sup>2</sup> = 0.60, a main effect of Trial, F(1, 18) = 22.60, p < 0.001, η 2 = 0.56, and most importantly, a significant CS × Trial interaction, F(1, 18) = 13.11, p < 0.01, η <sup>2</sup> = 0.42.

#### Response Prevention and Extinction

Focusing on the first eight trials of extinction, **Figure 2C** suggests gradual extinction of differential SCR, which is confirmed by a 2 (Group) × 2 (CS) × 8 (Trial) RM-ANOVA, revealing a main effect of CS, F(1, 18) = 20.71, p < 0.001, η <sup>2</sup> = 0.54, a main effect of Trial, F(7, 126) = 7.55, p < 0.001, η <sup>2</sup> = 0.30, and most importantly a significant CS × Trial interaction, F(7, 126) = 4.65, p < 0.001, η <sup>2</sup> = 0.21. In order to compare the endpoints of extinction between the groups, we compared the last four extinction trials in a 2 (Group) × 2 (CS) × 4 (Trial: ext5- 8 in Group ShortExt, ext9-12 in Group LongExt), revealing a significant main effect of CS, F(1, 18) = 4.91, p < 0.05, η <sup>2</sup> = 0.22, with no significant Group × CS interaction, F(1, 18) = 0.00, p = 0.96. This suggests incomplete extinction, equally across groups.

#### Avoidance Test

**Figure 3C** suggests that the return of the avoidance button had no detectable impact on the level of differential SCR. We calculated the average SCR for each CS during the four avoidance test trials and compared this with the averaged last four extinction trials (ext5-8 in Group ShortExt, ext9-12 in Group LongExt). The resulting 2 (Group) × 2 (CS) × 2 (Phase: last extinction trials vs. avoidance test trials) revealed a significant main effect of CS, F(1, 18) = 12.76, p < 0.01, η <sup>2</sup> = 0.42, with no CS × Phase interaction, F(1, 18) = 1.52, p = 0.23.

#### Reextinction Test

**Figure 3C** suggests a general increase in SCR upon removal of the avoidance button, which is confirmed by a 2 (Group) × 2 (CS) × 2 (Trial: last avoidance test trial, first post-avoidance test trial) RM-ANOVA, revealing a main effect of CS, F(1, 18) = 10.12, p < 0.001, η <sup>2</sup> = 0.36, a main effect of Trial, F(1, 18) = 6.73, p < 0.05, η <sup>2</sup> = 0.27, but no significant CS × Trial interaction, F(1, 18) = 0.19, p = 0.67. A 2 (Group) × 2 (CS) × 2 (Trial: last extinction trial, first post-avoidance test trial) RM-ANOVA revaeled a main effect of CS, F(1, 18) = 10.18, p < 0.01, η <sup>2</sup> = 0.36, but no CS × Trial interaction, F(1, 18) = 0.75, p = 0.40. A 2 (Group) × 2 (CS) × 2 (Phase: averaged last four extinction trials, averaged post-avoidance test trials) RM-ANOVA, yielded similar results, a main effect of CS, F(1, 18) = 8.02, p < 0.05, η <sup>2</sup> = 0.31, but no CS × Phase interaction, F(1, 18) = 0.44, p = 0.52.

# Correlations with Questionnaire Scores: Experiments 1 and 2 Combined

Experiment 1 and 2 are exactly identical up until the avoidance conditioning phase. In order to explore effects of anxietyrelated personality, we collapsed the two data from the two experiments and calculated correlations between the rates of button-clicking during avoidance conditioning and individual questionnaire scores (see **Table 1** for a summary of descriptive statistics). None of the questionnaire scores correlated with the proportion avoided CSP trials, STAI-T: r = 0.11, p = 0.51, IU: r = 0.26, p = 0.11, CBAS: r = −0.05, p = 0.77 (uncorrected p-values). However, STAI-T did correlate with the proportion avoided CSM trials, STAI-T: r = 0.35, p < 0.05, IU: r = 0.13, p = 0.41, CBAS: r = 0.22, p = 0.17 (uncorrected pvalues). Interestingly, this correlation was no longer significant following RPE treatment, when avoidance was available again, r = −0.02, p = 0.90, and STAI-T correlated significantly with the decrease in proportion avoided CSM trials between the avoidance conditioning phase and the avoidance test phase, r = 0.46, p < 0.05 (Bonferroni-corrected across eight correlation tests), while there was no such correlation with CSP trials, r = −0.12, p = 0.46. These results suggest that, although it was not able to wipe out avoidance altogether, RPE treatment did attenuate rates of unnecessary avoidance in higher anxious participants.

# Comparing the Effects of Fear Extinction Across Experiments 1 and 2

The only difference between Experiments 1 and 2 is the operationalization of response prevention (removal of the button in Experiment 1, instructions in Experiment 2).

This allowed us to examine differences in efficacy of these two response prevention treatments on persistence of avoidance. **Figures 2A,3A** suggest that instruction-based



response prevention may have been generally more effective than actual removal of the button. This was confirmed by a 2 (Experiment) × 2 (Phase: avoidance conditioning, avoidance test) × 2 (CS, averaged per phase) ANOVA that revealed a significant Experiment × Phase interaction, F(1, 38) = 7.13, p < 0.05, η <sup>2</sup> = 0.16, with no triple interaction, F(1, 38) = 0.18.

# DISCUSSION

The results of Experiment 2 are strikingly similar to those of Experiment 1: Differential danger-safety ratings and SCR returned sharply when participants were suddenly told that the avoidance button was unavailable, followed by gradual extinction (RPE). Subsequent instructions of renewed availability of the avoidance button led to a return of avoidance rates, indicating limited effects of PRE on avoidance when the response prevention is lifted. Finally, instructions of renewed unavailability of the avoidance button triggered an increase of extinguished differential danger-safety ratings in the Short Extinction group. This effect was not observed in the Long Extinction group, which may suggest that longer extinction could prevent this avoidance-induced increase of threat appraisal. Together, these results suggest that RPE effects are not only disturbed by visual changes (button present/absent, Experiment 1), but also by changes in instructed beliefs about avoidance availability. Exposure treatments also rely on therapist-patient instructions to exclude avoidance behaviors during exposures (RPE). Hence, the current results could imply that pure RPE treatments have limited effects on avoidance rates in everyday contexts where the avoidance options are typically available.

Over the two experiments combined, individual trait anxiety (STAI-T) correlated with the proportion of avoided CSM trials during avoidance conditioning, while there was no significant correlation with CSP trials. This finding adds to the diagnostic validity of the current procedure, and calls for studying avoidance responding during safety cues, rather than danger cues, in preclinical research on anxiety (see also Lommen et al., 2010; van Meurs et al., 2014). Interestingly, RPE treatment did decrease the proportion avoided CSM trials from avoidance conditioning to avoidance testing, and the size of the decrease correlated with STAI-T. This positive outcome may indicate that although RPE treatment failed to substantially reduce avoidance during a conditioned danger cue, it has the power to reduce the maladaptive avoidance during safety cues that characterizes high anxious individuals. Finally, comparing the two experiments directly revealed that instruction-based response prevention was generally more effective in reducing persistent avoidance than removing the response button. This may suggest that the learning not to avoid may be more effective in the presence vs. absence of avoidance cues.

# GENERAL DISCUSSION

The current study was set up to investigate response prevention and extinction (RPE) in a human avoidance conditioning protocol, and to assess its effects on the rate of avoidance when the response prevention was subsequently lifted. In line with a recent rodent study (Bravo-Rivera et al., 2015), we found persistent avoidance following fear extinction. Avoidance consisted of mouse clicking a button on the computer screen that appeared during both a conditioned danger (CSP) and safety cue (CSM). During avoidance conditioning, (1) participants learned to perform the avoid action more during CSP compared to CSM trials, (2) levels of danger-safety ratings and skin conductance reactivity (SCR) decreased as participants learned to avoid effectively, and (3) individual levels of trait anxiety correlated positively with unnecessary avoidance actions during the conditioned safety cue (CSM). Subsequent response prevention by removal (Experiment 1) or instructed unavailability (Experiment 2) of the avoidance button triggered a sharp increase in danger ratings and SCR to the CSP, followed by gradual reduction (extinction). Reintroduction of avoidance availability triggered a strong return of differential avoidance responding (CSP vs. CSM), but less so in Experiment 2 where response prevention had been induced through instructions while the avoidance button was always present. Finally, differential danger-safety ratings and SCR remained low during avoidance testing, but danger-safety ratings increased again when the avoidance availability was subsequently removed. Together, these results show that RPE effects can be studied in a human avoidance protocol and that lifting the response prevention can renew avoidance behaviors and lead to renewed expectancy of harm. The current study sets the stage for more research on avoidance extinction in humans and on developing/screening techniques to enhance transfer of RPE effects across contexts of avoidance (un)availability.

Several mechanisms may have contributed to the return of avoidance following RPE. Since fear is a motivator of avoidance (Krypotos et al., 2015), the return of avoidance may stem from a recovery of extinguished fear. Indeed, fear extinction is known to be specific to the spatio-temporal context in which extinction learning occurred (Bouton, 2002); the current RPE results suggest that it may also remain specific to the "context" of avoidance unavailability. The return of avoidance availability, through reappearance of the button or through instructions, may have functioned as a context change that triggered a recovery of fear and therefore avoidance as well. We found some support for this hypothesis during the subsequent removal of avoidance availability, which triggered a significant return of differential danger-safety ratings. Although this test was formally in a context identical to RPE, the preceding contextual changes may have disturbed that extinction context. An alternative possibility is that avoidance availability signals both safety and threat, as the avoidance action can become associated with the feared event (threat) that it effectively prevents (safety). Support for the safety-signaling hypothesis comes from recent studies on the predictive effects of avoidance actions (Lovibond et al., 2008, 2013). Support for the threat-signaling hypothesis comes from a recent study where performing an avoidance action during a conditioned safety cue elicited increased threat appraisal (Engelhard et al., 2015). Hence, the mere return of avoidance availability may have increased threat appraisal and fear, and thereby triggered the return of avoidance.

To the extent that the current RPE protocol relates to exposure treatment, the results would imply that conducting exposures in an avoidance-free therapy context enhances fear extinction within that context, but may compromise its generalizability to everyday contexts that have routine avoidance availability. Indeed, the presence of avoidance cues during fear extinction decreased the persistence of avoidance in Experiment 2. Incorporating the immediateness of avoidance availability in treatment may enhance extinction generalization (1) by augmenting the similarity with everyday contexts, and (2) by targeting the threat-signaling properties of avoidance cues. The judicious use of so-called safety behaviors in treatment is an ongoing question in clinical exposure research, with mixed evidence and diverging opinions (Rachman et al., 2008). The current analysis adds to this literature by pointing to the potential influence of avoidance (un)availability on the generalization of fear extinction and avoidance extinction.

The most important limitation of the current study is that fear extinction was not complete in the two experiments, even after as much as twelve extinction trials (compared to two Pavlovian conditioning trials). The level of differential danger-safety ratings and SCR did decrease significantly over extinction trials, but remained significant over the last four trials of extinction. This delayed extinction effect could result from the instructed fear procedure (participants were informed beforehand which CS would be followed by shock), if instructionbased learning is more difficult to correct by experiencedbased learning. Alternatively, the avoidance conditioning trials may have strengthened the underlying CS-US association, leading to slower extinction. The latter hypothesis could be investigated by dropping the contingency instructions and by manipulating the amount of avoidance conditioning trials prior to extinction. Irrespective of the exact mechanism leading to slower extinction, it is possible that the residual fear levels were responsible for the subsequent return of avoidance. This would show that, at least in the case of low-cost avoidance behavior, minimal levels of fear are sufficient to trigger a return of avoidance behavior when the opportunity arrives. In that case, preventing return of low-cost avoidance behaviors would require a complete elimination of fear reactions during treatment, as well as a complete prevention of return of fear.

### REFERENCES


A second limitation is that the avoidance response carried no cost. This is different from clinical avoidance behaviors, which are often very costly as they prevent the patient from engaging in other desired activities. Future research should investigate the influence of response costs on the return of avoidance after RPE. Nevertheless, as we suggested in the Introduction section, a patient may use many different avoidance behaviors, some of which are costly and some of which are not. The costly avoidance behaviors will be salient and probably be part of the primary complaints of the patient and an explicit target of treatment. Low-cost avoidance behaviors, on the other hand, may be more difficult to detect and therefore more difficult to treat. And, as the current study suggest, they may be especially prone to persist after fear extinction.

To conclude, fear extinction is generally viewed as an experimental model of exposure treatment, but the critical component of response prevention is often lacking. The current study established a human avoidance protocol in order to study fear extinction following a history of avoidance, and examined the effects of a return of avoidance availability on the return of avoidance responding. The results suggest that avoidance responding returns easily following fear extinction. This calls for more extinction research focusing on rates of avoidance behaviors in addition to levels of fear reactions. Arguably, incorporating avoidance into the learning history as well as in the extinction test situation may enhance the external validity of the fear extinction model and improve its translational value to the clinical setting.

# AUTHOR CONTRIBUTIONS

BV developed the research questions. BV and EI designed the experimental procedures and analyzed the data. BV wrote the manuscript, commented by EI, BV. EI approved of the final version and agreed to be accountable for all aspects of the work.

# FUNDING

This research was supported by a Marie Curie International Outgoing Fellowship within the 7th European Community Framework Programme (PIOF-GA-2013-627743) and by a Center for Excellence grant from the University of Leuven—KU Leuven (PF/10/005).

eds M. G. Craske, D. Hermans, and D. Vansteenwegen (Washington, DC: APA Books), 217–233.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Vervliet and Indekeu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.