Noradrenergic stimulation modulates activation of extinction-related brain regions and enhances contextual extinction learning without affecting renewal

Renewal in extinction learning describes the recovery of an extinguished response if the extinction context differs from the context present during acquisition and recall. Attention may have a role in contextual modulation of behavior and contribute to the renewal effect, while noradrenaline (NA) is involved in attentional processing. In this functional magnetic resonance imaging (fMRI) study we investigated the role of the noradrenergic system for behavioral and brain activation correlates of contextual extinction and renewal, with a particular focus upon hippocampus and ventromedial prefrontal cortex (PFC), which have crucial roles in processing of renewal. Healthy human volunteers received a single dose of the NA reuptake inhibitor atomoxetine prior to extinction learning. During extinction of previously acquired cue-outcome associations, cues were presented in a novel context (ABA) or in the acquisition context (AAA). In recall, all cues were again presented in the acquisition context. Atomoxetine participants (ATO) showed significantly faster extinction compared to placebo (PLAC). However, atomoxetine did not affect renewal. Hippocampal activation was higher in ATO during extinction and recall, as was ventromedial PFC activation, except for ABA recall. Moreover, ATO showed stronger recruitment of insula, anterior cingulate, and dorsolateral/orbitofrontal PFC. Across groups, cingulate, hippocampus and vmPFC activity during ABA extinction correlated with recall performance, suggesting high relevance of these regions for processing the renewal effect. In summary, the noradrenergic system appears to be involved in the modification of established associations during extinction learning and thus has a role in behavioral flexibility. The assignment of an association to a context and the subsequent decision on an adequate response, however, presumably operate largely independently of noradrenergic mechanisms.


INTRODUCTION
Renewal in extinction learning occurs when a response acquired in a particular context and extinguished in a different, novel context, reappears during extinction recall in the context present during acquisition (Bouton and Bolles, 1979). A prototypical renewal experiment therefore consists of three phases: acquisition refers to learning of an association between a cue and a consequence/response in context A. In the following phase, extinction learning, the cue is presented in context B and no longer followed by its original consequence, which leads to extinction of the conditioned response. In the final test phase termed extinction recall, the cue is again presented in context A, renewing the response learned during acquisition. The renewal effect of extinction evoked by this so-called ABA design has been demonstrated in a wide variety of tasks, ranging from fear extinction learning (Bouton and King, 1983), taste aversion learning (Rosas and Bouton, 1997) and appetitive conditioning (Bouton and Peck, 1989) in rats to fear conditioning (Vansteenwegen et al., 2005(Vansteenwegen et al., , 2006 and predictive learning Lachnit, 2006, 2008;Lachnit et al., 2008;Lissek et al., 2013) in humans. The renewal effect in humans was recently shown to be mediated by vmPFC and hippocampus in concert (Lissek et al., 2013), with hippocampus encoding the relation between context and cue-outcome during extinction learning and vmPFC active during extinction recall to retrieve this association. The renewal effect impressively underlines the context-dependency of extinction learning. It has been suggested that this sensitivity to context is caused by the unexpected change in the cueoutcome relation occurring during extinction learning, which in turn triggers enhanced processing of environmental stimuli that correlate with this unexpected event, such as the context (Bouton, 2004;Rosas and Callejas-Aguilera, 2006). Attention is assumed to have a central role in processing the extinction context and thus in evoking a renewal effect (Darby and Pearce, 1995;Rosas and Bouton, 1997;Rosas and Callejas-Aguilera, 2006;Uengoer and Lachnit, 2012). Recent studies of human extinction learning showed that extinction learning aroused attention to the context (Nelson et al., 2013), in particular if context is relevant (Lucke et al., 2013;Rosas et al., 2013). These results are in line with the finding that attention towards stimuli with high informational value is stronger than towards those with low informational value, i.e., irrelevant stimuli (George and Pearce, 2012). A brain region crucial for attentional processing as well as for extinction learning and retrieval is the medial prefrontal cortex (PFC). Medial PFC is involved in attention (Bussey et al., 1997;Kahn et al., 2012), particularly in mediating shifts of attention (Owen et al., 1991(Owen et al., , 1993Birrell and Brown, 2000). In addition, ventromedial PFC has a prominent function in contextual extinction learning (Maren et al., 2013) and the renewal effect (Lissek et al., 2013). Consistent with its role for attention, the medial PFC is a target region for the noradrenergic system. Noradrenaline (NA) in the forebrain is provided by the locus coeruleus, which projects to prefrontal regions, in particular infralimbic medial PFC, to the hippocampus and the amygdala (Jones et al., 1977;Loughlin et al., 1986) Therefore, manipulating NA in this region may modulate contextual extinction learning which requires attentional processing.
Noradrenaline, in general, is involved in processing and control of attention (Selden et al., 1990) by exerting alerting and arousing effects. The noradrenergic system appears to have a role in functionally integrating the brain regions involved in attention, as demonstrated by a study that showed the influence of an alpha2-adrenoceptor-agonist upon the interaction of these regions (Coull et al., 1999). Data from animal studies suggest a role for NA in directing attention towards relevant, salient information (Berridge and Waterhouse, 2003). Consistent with this, NA participates in cognitive flexibility, such as in attentional set-shifting (Kehagia et al., 2010). A study investigating the role of NA in humans found that it modulated attention in a centrally mediated manner, with impairment by the non-selective beta blocker propanolol independent of target valence (De Martino et al., 2008). In addition, modulation of extinction by NA has been reported in various studies on (contextual) extinction learning. Fear extinction in mice and rats was facilitated in a context different from the acquisition context by systemic administration, prior to extinction training, of the alpha-2-receptor antagonist yohimbine, which increases NA release (Cain et al., 2004;Morris and Bouton, 2007). In contrast, systemic administration of the beta-receptor antagonist propanolol impaired retrieval of contextual conditioned fear (Ouyang and Thomas, 2005). Extinction learning of an appetitive response was impaired after NA depletion (Mason andIversen, 1975, 1978;Mason, 1979), while extinction learning of an instrumental response to a compound of extinguished stimuli was found enhanced by systemic administration of the NA reuptake inhibitor atomoxetine in rats (Janak and Corbit, 2010). In addition, long-term extinction was increased by atomoxetine in rats (Janak et al., 2012).
The role of NA in prefrontal regions for extinction learning has been researched in only a few animal studies. NA release was increased in mPFC of rats exposed to an appetitive extinction paradigm (Mingote et al., 2004). Another study measuring NA release in rat medial PFC during reversal/extinction learning found that it remained constant over conditions (van der Meulen et al., 2007), pointing towards a generally arousing role for NA. Infusions of propanolol into infralimbic medial PFC impaired fear extinction but not consolidation if administered after extinction training (Mueller et al., 2008), suggesting that the crucial role of infralimbic NA is restricted to the extinction learning process proper. Hippocampal NA activation is also a factor in extinction learning. Local infusion of NA into dorsal hippocampus after extinction training enhanced the long term memory for fear extinction (Chai et al., 2014) and extinction retrieval (Rosa et al., 2014), while NA depletion in hippocampal/cortical regions was found correlated with the size of the extinction deficit of a classically conditioned response (McCormick and Thompson, 1982).
Atomoxetine as a selective NA reuptake inhibitor has been shown to modulate attentional processes in various tasks in addition to its effects upon extinction learning. In both mice and rats, atomoxetine increased extracellular levels of NA in PFC (Bymaster et al., 2002;Koda et al., 2010) and in hippocampus (Swanson et al., 2006). With regard to behavioral effects in healthy human subjects, a single dose of atomoxetine has been shown to increase inhibitory control (Chamberlain et al., 2009) as well as heighten phasic alertness, with enhanced neuronal sensitivity for errors that may be due to a more salient representation of the task (Graf et al., 2011). In rats, atomoxetine was able to reverse attentional deficits caused by noradrenergic deafferentiation of the mPFC, while at the same time causing performance deficits in set-shifting in non-lesioned animals (Newman et al., 2008). Thus, atomoxetine appears to be suitable for effectively manipulating prefrontal and hippocampal activation as well as behavior.
In the present study, we aimed to investigate the effects of an NA reuptake inhibitor, in humans upon extinction learning and the renewal effect in an associative learning task. In this task, participants were required to learn relations between cues and outcomes presented in particular contexts, which are reversed during the extinction learning phase. This predictive learning task (Üngör and Lachnit, 2006), which we already used in a previous study (Lissek et al., 2013) features an ABA design previously shown to reliably evoke a renewal effect.
We hypothesized that the enhanced activation of the attentional system in the experimental group will lead to superior extinction learning performance compared to a placebo group. In consequence, we expect a more prominent renewal effect in the experimental group. Moreover, we assume that the brain areas that are active both in extinction and in attentional processing, such as medial PFC, are presumably core regions for exhibiting NA-related effects in extinction learning and the renewal effect. Enhanced recruitment of attentional resources should reflect in increased brain activation in regions processing extinction learning and retrieval decisions.

PARTICIPANTS
Forty healthy right-handed volunteers (20 females, 20 males), mean age 24.89 years +/− 3.16 years st.dev., range 19-31 years, Frontiers in Behavioral Neuroscience www.frontiersin.org February 2015 | Volume 9 | Article 34 | 2 FIGURE 1 | Predictive learning task. (A) Example of a trial during acquisition of the task. Participants learned to predict whether certain kinds of food, eaten in a certain restaurant, would cause a stomach ache or not. After an intertrial interval of 5-9 s the stimulus was presented in its context for 3 s, then a question was superimposed on the screen "Will the patient get a stomach ache?" for maximum 4 s response time.
Feedback was shown for 2 s, providing the correct answer, e.g., "The patient does not have a stomach ache" . (B) Design of the predictive learning task. In condition AAA, extinction occurs in the same context as acquisition. In condition ABA, extinction occurs in a context different from that during acquisition. In both conditions, the final test for the renewal effect is performed in the context of acquisition. (C) Food images used as stimuli. Reprinted from Lissek et al. (2013) with permission from Elsevier.
without a history of neurological disorders participated in this study. The participants received a monetary compensation for their participation (in the amount of e 60). Participants were randomly assigned to the experimental atomoxetine (ATO) and placebo control (PLAC) groups. Mean age within the groups was 24.89 years +/− 3.23 st.dev., range 19-30 years in ATO and 24.88 years +/− 3.20 st.dev., range 20-31 years in PLAC. Within these groups, participants were further assigned to either a renewal (REN) group or a no renewal (NOREN) group depending on whether or not they showed a renewal effect during the test phase of the predictive learning task (the procedure is described in detail in Section "Behavioral Data Analysis)".

ETHICS STATEMENT
All subjects participated in this study after giving written informed consent. The protocol was approved by the local Ethics Committee of the Ruhr-University Bochum. The study conforms to the Code of Ethics of the World Medical Association (Declaration of Helsinki). Prior to the experiments, participants received handouts informing them about the functional magnetic resonance imaging (fMRI) procedures and the NA reuptake inhibitor atomoxetine.

PREDICTIVE LEARNING TASK
The predictive learning task that we used in this study was originally conceived by Üngör and Lachnit (2006) to explore and further illustrate the context-dependency of extinction learning.
Its efficiency in evoking a renewal effect was demonstrated in several behavioral studies using this specific design Lachnit, 2006, 2008;Rosas and Callejas-Aguilera, 2006;Nelson and Callejas-Aguilera, 2007;Lucke et al., 2013). We adapted this task for use in an fMRI setting and have used it in a previous fMRI study (Lissek et al., 2013). In the predictive learning task, participants were asked to put themselves in the position of a physician and predict whether various articles of food served in different restaurants would lead to the aversive consequence of a stomach ache in their patient. The learning process consisted of the three successive phases of (a) acquisition of associations; (b) extinction learning; and (c) recall phase (see Figure 1). During the acquisition phase (80 trials), participants learned to associate an article of food with a specific consequence. In each trial, one of eight stimuli (vegetables or fruits) was presented to the participant in one of two different contexts (indicated by the restaurant names "Zum Krug" (The Mug) and "Altes Stiftshaus" (The Dome) and a frame in either red or blue color). The stimulus in its context was first presented alone for 3 s, then a question asking whether the patient will develop a stomach ache was superimposed, with the response options "Yes" or "No". Response time was 4 s, participants responded by pressing the respective button on an fMRI-ready keyboard (Lumitouch, Photon Control Inc. Canada). After the response, or in case of a missing response after expiration of the response time, a feedback with the correct answer was displayed for 2 s, i.e., "The patient has a stomach Frontiers in Behavioral Neuroscience www.frontiersin.org February 2015 | Volume 9 | Article 34 | 3 ache" or "The patient does not have a stomach ache". The actual response of the participant was not commented upon. The food stimuli were presented in randomized order, each stimulus was presented ten times. Four stimuli were presented per context. Stimuli were counterbalanced with regard to their causing the aversive consequence of a stomach ache, with two stimuli per context causing stomach ache during acquisition, while the other two did not. During the extinction phase (80 trials), half of the stimuli were presented in the same context as during acquisition (condition AAA-no context change-40 trials) and the other half in the other context (condition ABA-context change-40 trials) in randomized order. In addition, stimuli were subdivided into two types: for actual "extinction stimuli", the consequence changed and the new consequence had to be learned, for "distractor stimuli", which were introduced in order to make overall learning more difficult, the consequence remained unchanged. In each context we used two extinction stimuli and two distractor stimuli. In all other respects, trials were identical to those during acquisition.
During the recall phase (40 trials), all stimuli were presented once again in the context of acquisition (five presentations per stimulus). Trials were identical to those during acquisition with the exception that, during the recall phase, no feedback with the correct response was given.

PROCEDURE
In a first fMRI session, participants performed the acquisition phase of the predictive learning task. After this session, the selective NA reuptake inhibitor atomoxetine was administered orally in a single dose of 60 mg. Control participants received an identical-looking placebo.
After drug administration, participants rested for 90 min. The second fMRI session was performed in a time window of about 90-120 min after administration of the drug. The task timing was based on the phase of peak plasma levels for atomoxetine (60-120 min after oral ingestion in adults) (Sauer et al., 2005;Chamberlain and Robbins, 2013).

IMAGING DATA ACQUISITION
Functional and structural brain scans were acquired using a whole-body 3T scanner (Philips Achieva 3.0 T X-Series, Philips, The Netherlands) with a 32-channel SENSE head coil. Bloodoxygen level dependent (BOLD) contrast images were obtained with a dynamic T2 * weighted gradient echo EPI sequence using SENSE (TR 3200 ms, TE 35 ms, flip angle 90 • , field of view 224 mm, slice thickness 3.0 mm, voxel size 2.0 × 2.0 × 3.0 mm). We acquired 45 transaxial slices parallel to the anterior commissure-posterior commissure (AC-PC) line which covered the whole brain. High resolution structural brain scans of each participant were acquired using an isotropic T1 TFE sequence (field of view 240 mm, slice thickness 1.0 mm, voxel size 1 × 1 × 1 mm) with 220 transversally oriented slices covering the whole brain.
The task was presented to the participants via fMRI-ready LCD-goggles (Visuastim Digital, Resonance Technology Inc., Northridge, CA, USA) connected to a laptop which ran specific software programmed in Matlab. Responses were given by means of an fMRI-ready keyboard (Lumitouch response pad, Photon Control Inc., Canada).

IMAGING DATA ANALYSIS
For preprocessing and statistical analysis of fMRI data, we used the software Statistical Parametric Mapping (SPM), Version 8 (Welcome Department of Cognitive Neurology, London, UK), implemented in Matlab 7.6.0 (Mathworks, Natick, MA, USA). Three dummy scans, during which BOLD signal reached steady state, preceded the actual data acquisition of each session, thus preprocessing started with the first acquired volume. Preprocessing at the single subject level consisted of the following steps: slice timing correction to account for time differences due to multislice image acquisition; realignment of all volumes to the first volume for motion correction, spatial normalization into standard stereotactic coordinates with 2 × 2 × 2 mm 3 using an EPI template of the Montreal Neurological Institute (MNI), smoothing with a 6 mm full-width half-maximum (FWHM) kernel, in accordance with the standard SPM procedure. The acceptable limit for head motion was 2 mm for translational movements and 0.5 • for rotational movements.
In a first level single subject analysis we calculated activation during extinction learning and recall phases in the conditions ABA and AAA, respectively. The contrasts were calculated within a combined anatomically defined mask which was constructed using the software MARINA (BION Bender Institute of Neuroimaging, University of Giessen, Germany) (Walter et al., 2003). The mask contained, as a priori regions of interest, PFC, hippocampus, amygdala, and insula. All data contained in this combined mask were analyzed together in a single analysis. We used an event-related design, modeling the events of each trial (stimulus and questions presentation, feedback presentation) using distinct stick functions convolved with the default HRF in SPM, with our analysis based on the stimulus presentation phase of each trial. The contrast images from these analyses were entered into second-level random-effects analyses to calculate in one-sample tests the activation patterns of the experimental and control groups for the different contrasts, using a threshold of p < 0.05 FWE-corrected (Family-Wise Error) for multiple comparisons with a minimal cluster size (k) of 10 voxels. Moreover, we calculated two-sample tests to directly investigate in which regions the experimental group showed enhanced activation compared to controls. To identify subtle group differences in these extinction-relevant regions in a hypothesis-led anatomically constrained manner, we used a more liberal threshold of p < 0.01 uncorrected.
In additional analyses, we evaluated the relation between BOLD signal changes in extinction-relevant brain regions and behavioral data. By means of one-sample tests we identified extinction-relevant ROIs with common activation across groups for contrasts of ABA and AAA trials during extinction learning and recall phase (vmPFC, hippocampus, amygdala, anterior cingulate) and extracted their mean signal intensities (in arbitrary units) using the MarsBar tool (Brett et al., 2002)  order to perform correlational analyses on potential relations between learning performance and brain activation.

BEHAVIORAL DATA ANALYSIS
For all three learning phases, log files were written that contained information on response latency, response type, and correctness of response for all learning phases. For calculation of the renewal effect, only responses to stimuli with consequence change (extinction stimuli) during the recall phase were analyzed. Based on the literature, the behavioral renewal effect in the predictive learning task should occur only in the condition ABA, in which extinction is performed in a context different from the context present during acquisition and recall phase. During the recall phase, a renewal effect occurs if a response is given that was correct during acquisition, but wrong during extinction (e.g., if in acquisition in context A cherries cause stomach ache, and in extinction learning in context B they no longer cause a stomach ache, then a renewal effect response during recall in context A would be consistent with cherries causing stomach ache). Statistical analyses were performed using the IBM SPSS Statistics for Windows software package, version 20.0 (Armonk, NY:IBM Corp). To test our directional hypotheses regarding performance improvements following the experimental treatment we used one-tailed t-tests. In order to evaluate the learning progress during extinction learning, we divided the extinction session into 8 blocks with 10 trials and performed an ANOVA with repeated measures for the factor block. Furthermore, we used post hoc tests to separately analyze group differences in performance during: (a) the first block of learning, during which participants experience the surprise event of changed contingencies between stimulus and outcome; (b) early extinction learning, consisting of blocks 2-5; and (c) late extinction learning, consisting of blocks 6-8. In our previous study using the predictive learning task we found that a considerable portion (about 40%) of the participants did not exhibit the renewal effect. This is a typical finding that also appears in this type of task outside an fMRI setting (Lissek et al., 2013). For further evaluation of their behavioral data, participants were therefore assigned to either the REN (showing renewal) or the NOREN (no renewal) group, respectively. Group assignment was based on their results in the ABA trials with consequence change during the recall phase, i.e., those trials designed to evoke the renewal effect. Participants who never showed a renewal effect (i.e., on any of these trials or 0% renewal responses) were assigned to the NOREN group, whereas participants who showed a renewal effect were assigned to the REN group.
The higher variability in PLAC during later extinction learning was due to three participants out of 20 showing an error level of 20-26% in these blocks, while the error level of the majority was 0-6.6%. The ATO group's lower variability during this phase was due to only two participants out of 20 showing 10-13% errors and the remaining 18 participants 0-3% errors.

Renewal effect
Stimulation of the noradrenergic system did not significantly enhance the renewal effect: we observed no significant differences between the atomoxetine and the placebo group with regard to the strength of the renewal effect, i.e., the preferred recall of associations correct for the acquisition phase in recall trials where extinction occurred in a different context. The PLAC group showed 34.16% (+/− s.e.m 9.246) renewal responses in the ABA condition, compared to the ATO group with 39.63% (+/− s.e.m. 10.629; t (38) = 0.390; p = 0.345). In recall trials for which previous extinction occurred in an identical context (AAA condition), ATO participants were significantly better in avoiding errors, instead responding with associations that were correct during extinction (ATO mean 0.00% errors +/− s.e.m. 0.000, PLAC Frontiers in Behavioral Neuroscience www.frontiersin.org February 2015 | Volume 9 | Article 34 | 5

FIGURE 2 | Behavioral results. (A)
Percentage of errors in extinction trials in the ATO (black) and PLAC (gray) groups. The ATO group made significantly less errors than the PLAC group over all extinction trials (overall), in extinction trials with consequence change only (overall cc), in ABA extinction trials with consequence change (ABA cc) and in AAA extinction trials with consequence change (AAA cc). (B) Extinction learning curves for ATO (black) and PLAC (gray) participants, with the extinction session divided into eight blocks of 10 trials each. During initial extinction in block 1, error rates did not differ between groups. However, in subsequent learning, the PLAC group was significantly slower in reducing errors during blocks 2-5. (C) Percent renewal effect responses in all participants (ABA all) and only in those participants who actually showed a renewal effect (ABA REN) in the ATO (black) and PLAC (gray) groups. ATO and PLAC groups did not differ with regard to the strength of the renewal effect they exhibited. mean 4.16% errors +/− s.e.m. 2.049; t (38) = 1.925; p = 0.031; See Figure 2C).

Gender differences
We did not observe any significant gender differences within the ATO group, which comprised nine womens and nine mens, neither in the error rate during acquisition (t (16) = 0.677; p = 0.508) and extinction learning (t (16) = −0.918; p = 0.372) nor in the strength of the renewal effect (t (16) = 0.161; p = 0.874).

Performance of participants who showed/ did not show the renewal effect
In both groups, those participants who showed or did not show the renewal effect were equally distributed: in the ATO group 55.55% of participants showed renewal (REN), 44.45% did not (NOREN) (χ 2 = 0.200; p = 0.655), in the PLAC group 50% showed renewal and 50% did not (χ 2 = 0.000; p = 1.00). There was no significant difference with regard to the strength of the renewal effect between the REN subgroups, with the ATO REN group exhibiting a mean of 71.33% renewal responses (+/− s.e.m. 11.527) and the PLAC REN group a mean of 68.33% renewal responses (+/− s.e.m. 10.077), t (18) = 0.196; p = 0.847 two-tailed.

IMAGING RESULTS
In one-sample tests, we analyzed brain activation during extinction learning and recall phases in the familiar or a novel context for the ATO and PLAC groups separately. In two-sample tests, we directly compared activation patterns in the ATO and PLAC groups.

Brain activation during extinction learning and recall in ATO and PLAC groups separately
During extinction learning in a novel context (condition ABA), both groups activated extended regions in bilateral dlPFC (Brodmann Area (BA) 8, 9 and 46) and bilateral OFC (BA 10, 47). In posterior hippocampus and superior temporal regions (BA 38, 22), activation was restricted to the left hemisphere in ATO and bilateral in PLAC. In fusiform gyrus (BA 37), ATO showed small clusters of bilateral activation compared to PLAC with a larger cluster of righthemispheric activation. In addition, the PLAC group exhibited a more extensive activation of bilateral insula compared to ATO, as well as activations in bilateral precuneus, righthemispheric cingulate gyrus (BA 32) and bilateral lingual gyrus (BA 19) which were absent in ATO. In contrast to PLAC, the ATO group also showed activation in right-hemispheric amygdala.
During extinction learning in a familiar context (condition AAA), both groups showed prominent activation in bilateral superior temporal (BA 22, 38) and fusiform gyri (BA 37), in bilateral OFC (BA 10, 47) and bilateral hippocampus. In addition, only the ATO group showed participation of Broca's area (BA 45), while PLAC activated the corresponding righthemispheric region (BA 44). Also, lingual gyrus (BA 19) activation was observed only in ATO. In contrast, only the PLAC group showed participation of bilateral precuneus and a substantially larger activation in bilateral insula-similar to the pattern exhibited during extinction learning in the novel context.
During recall of associations extinguished in an identical context (condition AAA), both groups reveal participation of several clusters in bilateral dlPFC (BA 8,9,46), bilateral hippocampus, fusiform, and lingual gyri. Activation in cingulate regions was bilateral in ATO and exclusively left-hemispheric in PLAC, in OFC left-hemispheric in ATO and right-hemispheric in PLAC, in superior temporal regions bilateral in ATO and righthemispheric in PLAC. Only the ATO group showed activation in Broca's area (BA 44) during this phase (see Table 1).

Effects of NA on brain activation during extinction learning-compared to placebo
During extinction learning in the AAA condition, ATO participants compared to PLAC show significantly higher activation in bilateral anterior and posterior hippocampus, bilateral insula (BA 13), orbitofrontal cortex (BA 47 and 10), left vmPFC (BA 10) and ACC/cingulate gyrus (BA 24,32), in left superior temporal gyrus (BA 22,38,41) as well as in bilateral amygdala. During extinction learning in the ABA condition, ATO exhibits increased activation compared to PLAC in right dlPFC (BA 46), right vmPFC (BA 10), right hippocampus, bilateral ACC (BA 32) as well as right insula.
Areas where ATO showed small clusters of lower activation than PLAC were bilateral dlPFC (BA 8) and vmPFC (BA 10/11) as well as left insula (BA 13) in ABA extinction. During AAA extinction, there was no region in which ATO exhibited lower activation than PLAC.
In ABA recall, ATO showed lower activation than PLAC in Broca's Area (BA 45) and a small cluster in left anterior hippocampus. In AAA recall, there was lower activation in ATO in regions in left anterior and right posterior hippocampus as well as in bilateral dlPFC (BA 9) (See Table 2; Figure 3).

CORRELATIONS BETWEEN BRAIN ACTIVATION AND PERFORMANCE
To determine whether activation in particular brain regions was related to performance in extinction and recall, we performed across groups analyses correlating performance data  with activation to the stimuli in several task-relevant ROIs derived from onesample tests comprising all participants. Activation during extinction learning of stimuli in a novel context (ABA) correlated significantly with later recall performance in the following brain regions: In left vmPFC, BA 10 (peak MNI coordinate −12 62 20) there was a positive correlation with the number of renewal effect responses during recall (r = 0.326; p = 0.040 two-tailed). Thus, the more active the vmPFC was during encoding of the new association in a novel context, the better the assignment of the association to its context in recall. In left anterior hippocampus (peak MNI coordinate −20 −4 −24), we found a negative correlation with the number of renewal responses during recall (r = −0.414; p = 0.008 two-tailed), suggesting that processes in left anterior hippocampus in these extinction trials were associated to a lower tendency of showing a renewal effect across all participants. Moreover, activity in right anterior cingulate BA 32 (peak MNI coordinate 2 40 12) was negatively correlated with the number of errors in recall of AAA trials (r = −0.409; p = 0.009 twotailed), i.e., a low cingulate activation during extinction learning in ABA trials was linked to more incorrect recall of associations in AAA trials. Activation during extinction learning of stimuli in the identical context (AAA) did not correlate significantly with recall performance in the recall phase. During recall of stimuli previously extinguished in the identical context (AAA condition), activation in right amygdala showed a significant negative correlation with the number of recall errors in AAA trials (peak MNI coordinate 32 0 −22) (r = −0.405; p = 0.009 two-tailed).

ACTIVATION OF THE NORADRENERGIC SYSTEM ENHANCES EXTINCTION LEARNING WHILE INCREASING ACTIVITY IN SEVERAL TASK-RELEVANT REGIONS
As hypothesized, an NA reuptake inhibitor significantly accelerated extinction learning in the ATO group compared to the PLAC group. Faster extinction learning occurred in both conditions ABA and AAA, suggesting that reversing an established cue-outcome association was easier for ATO participants, regardless of whether stimuli were presented in an identical or a novel context. Overall, this result suggests a higher potential for behavioral flexibility in the ATO group. Our findings correspond to results from animal studies that found instrumental extinction and long-term extinction enhanced by systemic administration of atomoxetine in rats (Janak and Corbit, 2010;Janak et al., 2012). In humans, the effect may well be based on the ability of atomoxetine to increase inhibitory control (Chamberlain et al., 2009), thus enabling a more efficient inhibition of obsolete associations and the respective responses. Moreover, atomoxetine has been shown to heighten phasic alertness in humans (Graf et al., 2011)-a general effect of the agonist which may provide a more salient representation of the task as well as enhanced error sensitivity.
Faster extinction learning in the ATO group was associated with increased activation in distinct hippocampal regions during both the ABA and the AAA condition, i.e., during extinction in a novel as well as in a familiar context, ATO participants showed higher activity in hippocampus than PLAC participants. The present results are consistent with the conclusions from our  previous study with the same predictive learning task (Lissek et al., 2013), stating that hippocampus encodes the relation between context and cue-outcome. Here, the stronger activation is presumably related to the ATO group's more efficient encoding of this relation, which in turn supported their faster extinction learning.
Moreover, corresponding to our hypothesis, the ATO group showed higher activation than PLAC in vmPFC during ABA and AAA extinction. This result suggests that, complementing our findings in the previous study of vmPFC participation in recall of extinction memory, the region also has a prominent role in extinction learning, which is presumably related to quickly adapting response decisions to the changed circumstances, taking context into consideration. The correlation between left vmPFC activity during ABA extinction and the number of renewal effect responses in ABA recall that we observed across groups delivers further evidence for a crucial participation of vmPFC during updating of the relation between cue-outcome association and context which enables a better assignment of an association to its context during recall. A study of human fear extinction learning similarly reported contextdependent activation in hippocampus and vmPFC, suggesting that hippocampus confers context dependance upon vmPFC (Kalisch et al., 2006), which implies that both structures participate in encoding and decoding of the relation between a context, a cue and an outcome. In our study, both vmPFC and hippocampus therefore appear to participate in contextual extinction learning, with hippocampus encoding context and vmPFC decoding this information in order to update and adapt responses appropriately. As opposed to this pronounced increase we observed a complementary reduction of activation in small clusters of vmPFC during ABA extinction. The left-hemispheric region in which ATO shows lower activation than PLAC largely corresponds to the region in which we found the above-mentioned correlation with the number of renewal responses during ABA recall, which might reflect the slightly, though not significantly, lower percentage of renewal responses in the ATO group.
Other regions that showed higher activation in the ATO group during extinction learning presumably also support facilitated extinction learning, such as distinct regions in insula and anterior cingulate that were more active in both the ABA and AAA condition. Anterior cingulate and anterior insula together have been suggested to constitute a salience detecting network (Sridharan et al., 2008;Vincent et al., 2008;Menon and Uddin, 2010;Craigmyle, 2013). In the context of an overall role of insula in processing salience (Menon and Uddin, 2010), activation of the region in the processing of feedback (regardless of valence) has been found (Bischoff-Grethe et al., 2009). Moreover a function in support of coordination and evaluation of performance in tasks with varying demands has been proposed (Eckert et al., 2009). Moreover, processing of changing reinforcement contingencies (D'Cruz et al., 2011) and error awareness (Ullsperger et al., 2010) have been demonstrated as insula functions. In our previous study (Lissek et al., 2013), insula activation was stronger to the novel context-cue compound, consistent to the proposed function of detecting salience. Within the individual groups as analyzed in the onesample tests, this effect was observed here too. However, in the direct comparison between groups, ATO exhibits relatively higher insula and anterior cingulate activation for both the ABA and AAA conditions. Thus, an NA reuptake inhibitor enhanced the response of these regions to both novel and familiar context-cue compounds, conceivably in the context of registering errors, an activity which might be related to the ATO group's lower error rate in both conditions. Increased activation of insula with atomoxetine was previously found associated with successful response inhibition in a stop-signal task (Chamberlain et al., 2009). Higher activation of anterior cingulate regions with atomoxetine compared to placebo was observed also during response inhibition in a go/no-go task (Nandam et al., 2014), suggesting that higher activation in these regions in the ATO group during extinction learning may be related to their superior response inhibition.
In addition, right anterior cingulate activation in ABA extinction correlated with enhanced AAA recall across groups, with higher activation during ABA trials linked to fewer errors in recall of AAA trials; an effect that appears to be related to the significantly better performance of ATO in AAA recall. It can be speculated that during extinction learning in ABA trials, cingulate attentional processing highlighted the importance of context for the change in stimulus-outcome associations, information that could be utilized later for correct responding in trials where the context did not change. However, it remains unclear why such an information transfer did not occur for ABA trials, since there was no significant correlation with performance in ABA recall. A more direct contribution to superior performance in AAA recall was delivered by the right amygdala, which appears to be involved in processing recall of an altered association within an identical context, as higher amygdala activation was associated with fewer errors in AAA recall.

ACTIVATION OF THE NORADRENERGIC SYSTEM DOES NOT AFFECT THE STRENGTH OF THE RENEWAL EFFECT, BUT IS ASSOCIATED WITH INCREASED ACTIVITY IN REGIONS INVOLVED IN RENEWAL
Contrary to our hypothesis, the NA reuptake inhibitor atomoxetine did not increase the percentage of renewal effect responses in the ABA condition relative to placebo. Conceivably, also the slower extinction learning progress in PLAC participants yielded a stable level of extinction memory sufficient for producing a comparable renewal effect in both Frontiers in Behavioral Neuroscience www.frontiersin.org February 2015 | Volume 9 | Article 34 | 11 groups. Moreover, the NA agonism-induced superior extinction learning performance does not appear to influence the decision required during the recall phase in the ABA condition, i.e., deciding whether to take context into account in selecting a response. The behavioral similarities in renewal performance of ATO and PLAC participants partially reflect in brain activation patterns. For example, we observed no differential activation in ventromedial PFC between ATO and PLAC during ABA recall. This area was previously (Lissek et al., 2013) found more active in retrieval of context-cue associations of the ABA condition in those participants who actually displayed a renewal effect compared to those who did not. Therefore, high vmPFC activation in ABA recall is apparently related to the decision to consider context in responding. Here, the lack of differential vmPFC activation for ABA recall in the direct comparison of ATO and PLAC groups may thus correspond to the finding that the groups show a similar level of renewal, and that the NA reuptake inhibitor does not affect processing of renewal.
In contrast, activation during recall in another region that is relevant for processing contextual information in extinction (i.e., the hippocampus) differs between the groups. ATO compared to PLAC shows higher activation in a region in left mid hippocampus in response to ABA trials, and in right mid hippocampus in AAA trials. On the other hand, ATO exhibits reduced activation in left anterior hippocampus in ABA and AAA recall and additionally in right posterior hippocampus in AAA recall. The activation level in left anterior hippocampus appears relevant for renewal only during ABA extinction learning, where across groups it correlated negatively with the strength of the renewal effect exhibited in ABA recall, but not during recall proper, since the differential recall activation in this region did not affect the strength of renewal. Thus, higher respectively lower hippocampal activation appears unrelated to the level of the renewal effect displayed by the groups, which corresponds to the result of our previous study of no differential hippocampal activation in REN and NOREN groups during ABA recall. Overall, these findings support our assumption that prominent hippocampal activation is crucial during extinction learning by marking relevance of context, but that during extinction recall it acts in concert with vmPFC activation to produce a renewal effect.
Further regions in which ATO showed higher activation than PLAC during ABA recall were bilateral cingulate (BA 24) as well as bilateral dlPFC (BA 9). None of these regions alone or in concert had an impact on the strength of the renewal effect. This conclusion corresponds to the finding that this observed activation pattern actually constitutes a subset of the pattern exhibited during ABA extinction recall by participants who did not show a renewal effect (Lissek et al., 2013). There, among a number of other regions, we too found higher activation in cingulate gyrus (BA 24), right dlPFC (BA 9) and right lateral OFC (BA 10). Therefore, the activation of those regions does not appear to contribute to generating a renewal effect.

CONCLUSION
To the best of our knowledge, this study is the first to demonstrate the effects of NA stimulation upon brain activation patterns associated with extinction learning and the renewal effect of extinction in healthy human participants.
In summary, our results deliver evidence for the involvement of the human noradrenergic system in the modification of established associations during extinction learning, which is presumably related to its attention-enhancing functions on the one hand and its role in response inhibition, which is associated with insula and anterior cingulate activity, on the other. Moreover, NA-induced activity increases in hippocampus and vmPFC contributed to more efficient encoding of the relationship between context and cue-outcome. By means of this orchestration, the noradrenergic system appears to support behavioral flexibility in extinction learning which enables swift reversals of established cue-outcome associations. In contrast, assignment of an association to a context and subsequent decision on the adequate response in recall of extinction memories, as exemplified in the renewal effect, presumably operate largely independently of noradrenergic mechanisms.

FUNDING
This work was supported by a grant from the DFG Deutsche Forschungsgemeinschaft (FOR 1581 Extinction Learning).