# IMPULSIVE COMPULSIVE SPECTRUM DISORDERS

EDITED BY : David Belin, Trevor W. Robbins, Eric Hollander and Margarita Moreno PUBLISHED IN : Frontiers in Behavioral Neuroscience

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-991-5 DOI 10.3389/978-2-88963-991-5

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# IMPULSIVE COMPULSIVE SPECTRUM DISORDERS

Topic Editors:

David Belin, University of Cambridge, United Kingdom Trevor W. Robbins, University of Cambridge, United Kingdom Eric Hollander, Albert Einstein College of Medicine, United States Margarita Moreno, University of Almeria, Spain

Citation: Belin, D., Robbins, T. W., Hollander, E., Moreno, M., eds. (2020). Impulsive Compulsive Spectrum Disorders. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-991-5

# Table of Contents


Elisa Rodríguez-Ortega and Inmaculada Cubero

*20 Nigrostriatal Dopaminergic Denervation Does Not Promote Impulsive Choice in the Rat: Implication for Impulse Control Disorders in Parkinson's Disease*

Robin Magnard, Yvan Vachez, Carole Carcenac, Sabrina Boulet, Jean-Luc Houeto, Marc Savasta, David Belin and Sebastien Carnicella


George Savulich, Emily Thorp, Thomas Piercy, Katie A. Peterson, John D. Pickard and Barbara J. Sahakian


Javier Orihuel, Emilio Ambrosio and Alejandro Higuera-Matas *69 Compulsivity in Alcohol use Disorder and Obsessive Compulsive*


Melissa T. Lee, Davis N. Mpavaenda and Naomi A. Fineberg

*110 A Working Hypothesis for the Role of the Cerebellum in Impulsivity and Compulsivity*

Marta Miquel, Saleem M. Nicola, Isis Gil-Miravet, Julian Guarque-Chabrera and Aitor Sanchez-Hernandez


Eric P. Zorrilla and George F. Koob

*154 Corrigendum: Impulsivity Derived From the Dark Side: Neurocircuits That Contribute to Negative Urgency*

Eric P. Zorrilla and George F. Koob


Sergio Ramos, Gabriela E. López-Tolsa, Espen A. Sjoberg and Ricardo Pellón

*236 Impulsivity and Compulsivity After Subthalamic Deep Brain Stimulation for Parkinson's Disease*

Sara Scherrer†, Andrew H. Smith, Jaimie Gowatsky, Christina A. Palmese, Joohi Jimenez-Shahed, Brian H. Kopell, Helen S. Mayberg and Martijn Figee

# Altered Eye-Movement Patterns During Text Reading in Obsessive–Compulsive Disorder and Internet Gaming Disorder

Tak Hyung Lee1† , Minah Kim2† , Yoo Bin Kwak <sup>1</sup> , Wu Jeong Hwang<sup>1</sup> , Taekwan Kim<sup>1</sup> , Jung-Seok Choi 2,3\* and Jun Soo Kwon1,2,4

<sup>1</sup>Department of Brain and Cognitive Sciences, Seoul National University College of Natural Sciences, Seoul, South Korea, <sup>2</sup>Department of Psychiatry, Seoul National University College of Medicine, Seoul, South Korea, <sup>3</sup>Department of Psychiatry, SMG-SNU Boramae Medical Center, Seoul, South Korea, <sup>4</sup> Institute of Human Behavioral Medicine, SNU-MRC, Seoul, South Korea

Obsessive-compulsive disorder (OCD) and internet gaming disorder (IGD), which are similar in that both involve repetitive behaviors and related with cognitive dysfunctions, frequently begin in early adolescence, which is a critical period for learning. Although the deterioration in cognitive functioning caused by these conditions may have adverse effects on information processing, such as text reading, there has been no comprehensive research on the objective indicators of altered reading patterns in these patients. Therefore, we evaluated eye-movement patterns during text reading in patients with OCD or IGD. In total, 20 patients with OCD, 28 patients with IGD and 24 healthy controls (HCs) participated in the reading task using an eye tracker. We compared the fixation durations (FDs), saccade amplitudes and eye-movement regressions of the three groups during reading. We explored relationships between the parameters reflecting altered reading patterns and those reflecting the severity of clinical symptoms. The average FDs and forward saccade amplitudes did not differ significantly among the groups. There were more eye-movement regressions in patients with OCD than in patients with IGD and HCs. No correlation was found between altered eye-movement patterns during reading and the severity of clinical symptoms in any of the patient groups. The significantly increased number of regressions (NRs) in the OCD group during reading may reflect these patients' difficulties with inferential information processing, whereas the reading pattern in the IGD group is relatively intact. These findings suggest that patients with OCD and patients with IGD have different eye-movement patterns during reading reflecting distinct cognitive impairments in the two patient groups.

Keywords: information processing, eye-movement, reading, obsessive-compulsive disorder, internet gaming disorder

### INTRODUCTION

Obsessive-compulsive disorder (OCD) and internet gaming disorder (IGD), the most prevalent conditions among adolescents, share symptoms involving repetitive behavior (Walitza et al., 2011; Robbins and Clark, 2015; Feng et al., 2017). OCD is characterized by obsessions and compulsions, which are intrusive, recurring, unwanted, or aversive thoughts and repetitive,

#### Edited by:

David Belin, University of Cambridge, United Kingdom

#### Reviewed by:

Lingdan Wu, Universität Konstanz, Germany Pedro E. Maldonado, Universidad de Chile, Chile

> \*Correspondence: Jung-Seok Choi choijs73@gmail.com

†These authors have contributed equally to this work

Received: 16 May 2018 Accepted: 03 October 2018 Published: 18 October 2018

#### Citation:

Lee TH, Kim M, Kwak YB, Hwang WJ, Kim T, Choi J-S and Kwon JS (2018) Altered Eye-Movement Patterns During Text Reading in Obsessive–Compulsive Disorder and Internet Gaming Disorder. Front. Behav. Neurosci. 12:248. doi: 10.3389/fnbeh.2018.00248 ritualized behavior or mental activity that is aimed at reducing distress (American Psychiatric Association and American Psychiatric Association. DSM-5 Task Force, 2013). IGD, which is characterized by a lack of interest in all everyday activities except internet games, was categorized as a condition requiring further study in Section III of the Diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5) in 2013 (American Psychiatric Association and American Psychiatric Association. DSM-5 Task Force, 2013; King and Delfabbro, 2014). These two conditions are similar in that both involve repetitive thoughts and behaviors and cognitive dysfunction, including impaired inhibitory control (Walitza et al., 2011; de Wit et al., 2012; King and Delfabbro, 2014) and the failure to resist an impulse or temptation to perform an act that is harmful to the person (Grant et al., 2010). From this perspective, OCD and IGD can be categorized as behavioral addictions (Holden, 2001; Potenza, 2006; Robbins and Clark, 2015). Patients with OCD have impairments in various domains of cognitive functioning, such as inhibitory control, set shifting, planning and working memory (Dittrich and Johansen, 2013; Shin et al., 2014; Abramovitch and Cooperman, 2015). Excessive internet use and difficulties in suppressing the cravings for internet games are associated with executive dysfunction in patients with IGD (Dong and Potenza, 2014; Zhou et al., 2016). Impaired visuospatial memory (VSM) is the most consistently reported cognitive impairment in patients with OCD. However, VSM deficits are rarely reported in patients with IGD. Instead, VSM may be enhanced by the repetitive exposure to visual stimuli and cognitive training involved in internet gaming (Oei and Patterson, 2013; Blacker et al., 2014).

Both OCD and IGD are most prevalent during early adolescence, which is a critical period for learning and social development (Kessler et al., 2007). In addition, both OCD and IGD has repetitive thought and attentional bias to a specific object or target related with their symptom (Tata et al., 1996; Bradley et al., 2016; Kim et al., 2018). Because those symptoms affect attentional capacity which is a very basic unit of cognitive functioning, broad range of cognitive functioning can be affected by symptoms of OCD and IGD. This is particularly important because the cognitive impairments that affect information processing could have a negative impact on normal development (Millan et al., 2012). Indeed, impaired cognitive functions in early adolescence could affect individuals in achieving educational success; this may in turn lead to later social and occupational dysfunction, which may reduce the long-term quality of life (Sawyer et al., 2002). One of the most important learning-related abilities in school-age children involves reading, which is affected in both OCD and IGD by impairments in higher order cognitive functioning (Carretti et al., 2009; Borella et al., 2010). Reading skills are essential for learning and good academic performance, as a large proportion of new information is acquired through reading (Pretorius, 2002; García-Madruga et al., 2014). Reading requires the comprehensive use of various domains of cognitive functioning, such as working memory, inhibitory control, lexical processing and attentional control. Therefore, reading problems may develop in both patients with OCD and those with IGD because of the aforementioned impairments in information processing or cognitive functioning.

The eye-movement patterns involved in the information processing underpinning the process of reading were recently measured using an eye tracker (Raney et al., 2014; Cop et al., 2015). The basic assumption of such a measurement method is that longer (i.e., the amount of time that a gaze is fixed on a certain position) and more numerous fixations are associated with longer periods of information processing (Raney et al., 2014). Excellent readers have short fixation durations (FDs), long saccades (i.e., gaze movements between fixations) and few instances of repetitive reading (regressions; Rayner, 2009). A study by Deans et al. (2010) found that patients with reading disorders (RDs) spent a longer total time reading and had longer FDs than healthy controls (HCs); additionally, patients with attention-deficit/hyperactivity disorder (ADHD) had a higher proportion of regressive and vertical saccades compared to HCs (Deans et al., 2010). Another study on RDs identified regressive saccades as a factor that could differentiate individuals with RDs from HCs (Nilsson Benfatto et al., 2016). Finally, in one study there were more regressions and more and longer fixations among those in the early stages of Alzheimer's disease compared to HCs (Fernández et al., 2013). Thus, the eye movements performed during reading may reflect changes in the cognitive processes involved in text comprehension associated with various diseases characterized by cognitive impairments.

OCD and IGD are both marked by dysfunctional information processing because of impaired cognitive functions. However, the eye-movement patterns during reading may differ in individuals with these conditions because of the unique characteristics of each. VSM is known to be impaired in OCD (Shin et al., 2004; Abramovitch et al., 2013) but not in IGD, as a previous study reported that VSM may be enhanced by the repetitive exposure to visual stimuli and cognitive training involved in internet gaming (Blacker et al., 2014; Steenbergen et al., 2015). Indeed, patients with OCD may be more likely to repeatedly read the same text because of comprehension difficulties caused by either obsessions or compulsions that render set shifting difficult or create a slow processing speed (Abramovitch et al., 2013). In contrast, patients with IGD may have faster reading speeds despite comprehension difficulties due to their repeated exposure to various images and game scenes during game play.

Therefore, we used an eye-tracking method to investigate whether the eye-movement patterns during text reading differed between groups with OCD and IGD. We hypothesized that, during reading, patients with OCD would have longer FDs and more numerous regressions than patients with IGD, whereas the reading patterns of patients with IGD would be relatively preserved despite their dysfunctional information processing.

#### MATERIALS AND METHODS

#### Participants and Clinical Assessments

Twenty patients with OCD, 28 patients with IGD, and 24 HCs participated in this study. Patients with OCD were recruited from the outpatient clinic at Seoul National University Hospital (SNUH). Patients with IGD were recruited from the addiction outpatient clinic at SMG-SNU Boramae Medical Center. OCD and IGD were diagnosed by experienced psychiatrists based on DSM-5 criteria. Eight patients with OCD were taking selective serotonin reuptake inhibitors (SSRIs) at the time of the study. We evaluated the severity of obsessive-compulsive (OC) symptoms using the Yale-Brown Obsessive Compulsive Scale (Y-BOCS; Goodman et al., 1989). All patients with IGD played internet games for more than 4 h per day for 1 year and were medication free. HCs played internet games for no more than 2 h per day and had no history of psychiatric illness. The severity of all participants' internet gaming addiction was assessed using Young's Internet Addiction Test (IAT; Young, 1996). The Barratt Impulsivity Scale-11 (BIS-11; Fossati et al., 2001) was used to measure impulsivity. We assessed the severity of depressive symptoms using the Beck Depression Inventory (BDI; Steer et al., 1999), and the severity of anxiety symptoms was assessed using the Beck Anxiety Inventory (BAI; Steer et al., 1993). We evaluated the intelligence quotient (IQ) using the abbreviated form of the Korean-Wechsler Adult Intelligence Scale-III (K-WAIS-III; Kim et al., 1994). Exclusion criteria included lifetime diagnosis of substance abuse or dependence, neurological disease, significant head injury accompanied by loss of consciousness, medical illness with documented cognitive sequelae, sensory impairment, or intellectual disability (IQ <70).

This study was carried out in accordance with the recommendations of GCP, the institutional review boards of SMG-SNU Boramae Medical Center and SNUH. The protocol was approved by the the institutional review boards of SMG-SNU Boramae Medical Center and SNUH . All subjects gave written informed consent in accordance with the Declaration of Helsinki.

### Reading Task and Eye-Movement Recording

The content of the reading task was selected from among the reading assignments provided by the Educational Broadcasting System of Korea (EBS). To ensure that the assignment would be easily understood by both adolescent and adult participants, we chose a natural science text that had elicited a rate of response to comprehension questions of more than 80% correct. This text addressed the mechanism of heavy rain formation, and the reading task was developed using Experimental Builder 2.1.45 (SR Research, Canada). Reading assignments were displayed on a 17 inches monitor with dimensions of 1024 × 768 pixels. The reading task consisted of five pages with 10 lines per page, yielding 44 lines in total. After finishing reading one page, subjects proceeded to the next page by clicking the mouse button. Eye movements were measured using EyeLink 1,000 (SR Research), with a 1,000 Hz sampling rate, while participants performed the reading task. The font size and line spacing were about 1◦ , the horizontal viewing angle was 27◦ , and the vertical viewing angle was 20◦ . Participants placed their jaws and foreheads on a chin rest at a distance of 70 cm from the monitor. To ensure that participants performed the task to the best of their ability, before starting the experiment we told them that they would be asked four questions related to the reading after finishing the task.

### Eye-Movement Data Analyses

Data analyses were performed using EyeLink Data Viewer 2.6.1 (SR Research) and customized MATLAB scripts. The variables of interest were FD, amplitude of forward saccades (AFS) and number of regressions (NRs). Fixations and saccades were identified by EyeLink 1,000 tracker parser processes. The parser processes differed from the manufacturer's settings, which were calibrated to the threshold values for displacement, velocity, and acceleration, which were set at 0.15◦ , 30◦ /s and 8,000◦ /s2 , respectively. FD is the value of the duration of fixed gazes. FSA is the size of the angle formed by the movement of the eye toward the text. NR reflects the number of times that the saccade traveled in the direction opposite the direction of text progression. **Figure 1** presents an example of the fixations, saccades, and NR superimposed over a screen displaying the reading task.

#### Statistical Analyses

Data were analyzed using the lmerTest and multcomp packages of R. We used one-way analysis of variance (ANOVA) to compare demographic and clinical characteristics and some of the eye-movement parameters (i.e., total duration and NR) across the three groups. Because FD and AFS had a skewed long-tail distribution in individuals, we used the median value for each subject for the Kruskal-Wallis test. We analyzed categorical data using chi-square tests. Post hoc Bonferroni tests were performed if indicated. Pearson's correlation analyses were performed to explore associations between altered eye-movement parameters during reading and scores on the Y-BOCS or IAT of each patient group. Also, to rule out the effect of depressive symptoms, we performed an exploratory correlation analysis between BDI

FIGURE 1 | Image of the sample scene of the reading task with the eye-movement variables superimposed. The text, written in Korean in the box in bold, was presented in the task. Gray circles mean that the fixation and size of the circle varied according to the duration of each fixation. Arrows with dotted lines indicate forward saccades. Arrows with bold lines indicate regressions.

scores and NR, both of which were significantly larger in OCD patients than the IGD group and HCs. The level of statistical significance was set at P < 0.05.

### RESULTS

### Demographic and Clinical Characteristics

There were no significant differences among the groups in terms of sex, age, years of education, IQ, or BIS-11 score. However, there were significant intergroup differences in IAT (F = 16.481, P < 0.001), BDI (F = 13.373, P < 0.001), and BAI (F = 10.093, P < 0.001) scores. Post hoc Bonferroni testing revealed that patients with IGD (t = 5.637, P < 0.001) and those with OCD (t = 3.045, P = 0.010) had higher IAT scores than HCs. The BDI score was highest in the OCD group (OCD vs. IGD, t = 2.638, P = 0.031; OCD vs. HC, t = 4.904, P < 0.001), intermediate in the IGD group (IGD vs. HCs, t = 2.632, P = 0.032), and lowest in HCs. The BAI score was higher in the IGD (t = 3.430, P = 0.003) and OCD (t = 4.057, P < 0.001) groups compared to HCs (**Table 1**).

### Reading Task and Eye Movements

The eye-tracking data of two patients with OCD who correctly answered only one of the four questions related to the reading were excluded from further analyses because the seriousness of their participation seemed questionable. The means (standard deviation) of skewness of all subjects were 1.34 (0.99) for FD and 2.66 (1.14) for the AFS. There were no significant intergroup differences in the total duration of reading (F = 2.479, P = 0.091), FD (χ <sup>2</sup> = 5.748, P = 0.056), or AFS (χ <sup>2</sup> = 2.591, P = 0.274). However, we found significant group differences in NR (F = 4.553, P = 0.014; **Table 2**). The distribution of data by group for the four variables is shown in **Figure 2**. Post hoc Bonferroni testing revealed that patients with OCD had higher NR than patients with IGD (t = 2.702, P = 0.026) and HCs (t = 2.678, P = 0.028) as shown in **Figure 3**. There were no significant correlations between NR and Y-BOCS total score (r = 0.203, P = 0.362) or between scores on the BDI and NR (r = −0.016, P = 0.948) in patients with OCD. Additionally, we performed analysis of covariance (ANCOVA) to the total duration and the NR, because the IQ showed the trend level group difference. In the result of ANCOVA, group differences in NR remained significant after IQ controlled as a covariate (F = 4.504, p = 0.015).

### DISCUSSION

We aimed to identify the distinctive patterns of eye movements of patients with OCD and patients with IGD during reading. We found that the time spent on lexical processing (i.e., FD) and the distance traveled when the gaze moved according to the direction of the text (i.e., AFS) did not differ significantly among the groups. However, the number of returns from the direction of reading (i.e., NR) was significantly higher in patients with OCD than in patients with IGD and HCs. There was no significant correlation between NR and the severity of OC symptoms in patients with OCD.

A more significant increase in NR was found in the OCD group compared to the IGD and HC groups. Regressions occur when readers miss, forget, or are unsure about what they have already read, and NR is influenced by the difficulty of the text (Rayner, 1998; Booth and Weger, 2013). Mathematical eye-movement models, such as saccade-generation with inhibition by foveal targets (SWIFT) and E–Z reader, also indicate that regression occurs when it is difficult to understand a specific word in a sentence (Engbert et al., 2002; Reichle et al., 2003). The increased NR during reading shown by patients with OCD suggests that their cognitive dysfunction may have impaired their ability to understand the content of text that they had read before moving their gaze (Cop et al., 2015). During text reading, most information processing occurs during fixations (Rayner, 2009). Only the OCD group showed increased NR in the current study, and the trend level of the FD of the three


Data are given as mean (standard deviation). ∗∗∗P < 0.001. Abbreviations: OCD, obsessive-compulsive disorder; IGD, internet gaming disorder; HC, healthy control; IQ, intelligent quotient; IAT, Korean version of the Young internet addiction test; BDI, Beck depression inventory; BAI, Beck anxiety inventory; BIS-11, Barratt impulsiveness scale version 11; Y-BOCS, Yale-Brown obsessive-compulsive scale; NA, Not Applicable. <sup>a</sup>Analysis of variance or χ <sup>2</sup> analysis. <sup>b</sup>With one missing values in OCD group.


TABLE 2 | Reading task performing duration and eye-movement analysis results during reading task across three groups.

Abbreviations: HC, healthy control; IGD, internet gaming disorder; OCD, obssesive-compulsive disorder. <sup>a</sup>Analysis of variance. Data are given as mean (standard deviation). <sup>∗</sup>P < 0.05. <sup>b</sup>Kruskal-Wallis rank sum test. Data are given as median (1 quantile; 3 quantile).

groups differed; thus, the ability of the OCD group to process information differed with that of other groups. Additionally, the reduced inhibitory control and set shifting ability related to the repetitive behavioral characteristics of OCD may have contributed to the increased NR in the OCD group; that is, the habitual forward movement of the gaze of patients with OCD, even when they had gained sufficient understanding of the sentence they had just read, may have affected these results. Moreover, patients with OCD may have reconfirmed their understanding of text they had already read because of a deterioration in their confidence in their cognitive abilities (Hermans et al., 2008). However, the lack of increased NR in patients with IGD may reflect the stability of the confidence of these patients and the fact that the only repetitive activities performed by this group involve internet games.

In contrast to our initial hypothesis, we did not find a significant correlation between increased NR and the severity of OC symptoms in patients with OCD. This may have been because of the trait status of abnormal reading patterns, which would mean that such patterns would not change as a function of a state change, such as symptomatic improvement. Because dysfunctions in several cognitive domains are traits of OCD, reading ability, which is closely related to cognitive functioning, is also likely to be a trait marker rather than a state marker (Abramovitch and Cooperman, 2015). Another possible explanation involves the cognitive complexity of eye-movement patterns during reading. Reading requires integrating and regulating many domains of cognitive functioning, including working memory, attention, word identification and language comprehension (Fernández et al., 2014). Therefore, it may be difficult to identify correlations between simple eye-tracking parameters and symptom severity without considering the complex interactions of various domains of cognitive functioning that may occur during reading. The heterogeneity of the

FIGURE 2 | The distribution of data by group for total duration (A), number of regression (NR) (B), fixation duration (FD) (C) and saccade amplitude (D). In each plot, solid line indicates the OCD group and dotted line indicates the IGD group and the dashed line indicates the HC group. The vertical line indicates the mean value of the group in (A) and (B). In (C) and (D), the vertical line indicates the median value of each groups Abbreviations: OCD, Obsessive-compulsive disorder; IGD, internet gaming disorder; HC, healthy control.

severity and characteristics of OC symptoms may also have contributed to the inconsistency between the results and the initial hypothesis. Although it is likely that NR is associated with repetitive behavior, some patients with OCD may suffer from obsessions in the absence of repetitive compulsive behaviors. In addition, many of the patients with OCD were taking medication, which may have reduced the severity of their symptoms to below the threshold of detection of our eye-tracking method. Such heterogeneity could have obscured the possible correlation between increased NR and the severity of OC symptoms.

In the current study, patients with IGD, unlike those with OCD, did not show altered eye-movement patterns related to impaired reading ability. IGD involves both compulsive characteristics (i.e., repetitive gaming behavior) and addictive characteristics (i.e., an increased desire to play a specific game; Dong and Potenza, 2014). The addictive characteristics may cause significant behavioral changes and cognitive abnormalities when a stimulus or desire related to the game of choice appears (Zhou et al., 2012); however, the domains of cognitive functioning that are impaired and/or the psychiatric symptoms that emerge may differ according to the characteristics of the game of choice (Na et al., 2017). For example, individuals addicted to first-person shooter (FPS) games have higher levels of impulsivity than nonaddicted persons (Metcalf and Pammer, 2014), and players of massive multiplayer online role-playing games (MMORPGs) have shown increased social anxiety (Park et al., 2016). However, playing video games improves visuospatial functions through repetitive training using visual cues (Oei and Patterson, 2013; Blacker et al., 2014), and regular playing of FPS games may improve decision-making ability or action cascading (Metcalf and Pammer, 2014; Steenbergen et al., 2015). MMORPGs may have positive effects on language learning through real-time online interactions with other players and the narrative or instructions embedded in the games (Zhang et al., 2017). Because the patients with IGD who participated in this study were addicted to games with different characteristics, the effects of addiction on eye-movement patterns during reading may have canceled themselves out, leading to an ostensible absence of abnormality. Another possible explanation is that because the patients with IGD in this study were recruited via internet advertisements, they may have had less severe addictions than participants in other studies, who were chosen from among those visiting a hospital for help and treatment (Kim et al., 2012; Zhou et al., 2016). Thus, the symptoms of the patients with IGD may not have been sufficiently severe to lead to measurable changes in reading patterns in the current study.

Because eye-movement patterns during reading reflect both the linguistic features of the text (i.e., word frequency, word length and sentence complexity) and the characteristics of the reader (i.e., reading ability and topic knowledge), they can be used to measure the text comprehension of a reader (Palmer et al., 1985; Rayner, 1998; Clifton et al., 2016). The altered eye-movement pattern of the OCD group implies an impaired reading ability that has also been observed in other psychiatric disorders, such as ADHD, RD and Alzheimer's disease (Deans et al., 2010; Fernández et al., 2013). Moreover, these same abnormalities are reflected in the increased NR observed during reading. However, other features, such as the increased average fixation time in RDs and the total fixation and duration times, vary according to conditions (Fernández et al., 2013, 2014). Condition-related differences in eye-movement patterns during reading may be due to the different types and degrees of cognitive dysfunction associated with each disorder. Therefore, eye-movement tests, including those that measure the cognitive factors that may be affected by the pathological features of the condition under examination, are likely to be useful behavioral biomarkers in a variety of psychiatric patients (Itti, 2015).

This study has several limitations. First, we did not directly assess the reading comprehension of the participants, as the number of correct answers to the four questions was used to judge only the seriousness with which the task was performed. Therefore, this study did not address important aspects of information processing reflected in text comprehension. Second, the patient groups in this study were heterogeneous in terms of symptom severity, medication status and symptom domain (i.e., main OC symptom domain or game of choice). Specifically, the high BDI scores of the OCD group may have contributed to the increased NR even though there was no significant correlation between BDI scores and NR in OCD subjects. Eight patients with OCD were taking SSRIs, but their main symptom domains were heterogeneous. Patients with IGD played various types of games and had relatively low IAT scores. Such heterogeneity (including the use of SSRIs by patients with OCD (Sayyah et al., 2016)) and the relatively low level of symptom severity in patients with IGD may have affected participants' information processing. Finally, the predominance of males in the three groups, especially the IGD group, and the relatively small sample size were other limitations of the current study.

To the best of our knowledge, this is the first study to differentiate altered eye-movement patterns of patients with OCD and patients with IGD according to characteristics of these diseases. We found that NR was increased in the OCD group, although there was no obvious change in the eye-movement pattern of the IGD group during text reading. The increased NR in the OCD group during reading may reflect difficulties with inference or set shifting due to characteristics of OCD. The relatively preserved eye-movement pattern of patients with IGD during text reading, despite their difficulties with information processing, may reflect the effects of cognitive—visual training during repeated game play. The findings of the current study suggest that patients with IGD and patients with OCD have different eye-movement patterns during reading that may reflect the distinct domains of cognitive dysfunction associated with each of these disorders. Additional studies that combine measurements of eye-movement patterns during reading with explorations of various domains of cognitive functioning would be of great interest.

### REFERENCES


### AUTHOR CONTRIBUTIONS

MK and J-SC was responsible for recruitment of patients and HC participants, the collection of demographic and clinical data. TL, MK, J-SC and JK contributed for study design and procedure. TL, YK, WH and TK collected eye-movement data. TL and MK performed the data analysis and wrote the manuscript draft. J-SC and JK supported interpretation of the study results. J-SC and JSK managed and supervised the whole procedure of this study. All authors have critically reviewed the content and approved the final version of the manuscript.

### FUNDING

This work was supported by a grant from the National Research Foundation of Korea (Grant No. 2014M3C7A1062894).


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Lee, Kim, Kwak, Hwang, Kim, Choi and Kwon. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Environmental Enrichment Modulates Drug Addiction and Binge-Like Consumption of Highly Rewarding Substances: A Role for Anxiety and Compulsivity Brain Systems?

Elisa Rodríguez-Ortega<sup>1</sup> and Inmaculada Cubero1,2 \*

<sup>1</sup>Departamento de Psicología, Universidad de Almería, Almería, Spain, <sup>2</sup>Centro de Evaluación y Rehabilitación Neuropsicológica (CERNEP), Universidad de Almería, Almería, Spain

Drug addiction is a chronic disorder comprising components of both impulsivity and compulsivity in the so called "addiction cycle" which develops over time from early non-dependent, repetitive, binge-consumption to later post-dependent compulsive consumption. Thus, frequent binge-like intake is a typical pattern of excessive drug intake characteristic of the pre-dependent phase of the addiction cycle, which represent an important risk factor to develop addiction in vulnerable individuals. In this framework, it is of paramount interest to further understand the earliest stage of the addiction cycle so novel approaches would emerge aimed to control repetitive episodes of bingeconsumption in non-dependent subjects, protecting vulnerable individuals from transition to dependence. Environmental enrichment (EE) is a preclinical animal model in which animals are housed under novel, social enriched conditions, which allows exercising and provides sensory and cognitive stimulation. EE promotes important improvements for a variety of cognitive processes and clear therapeutic and protective effects preventing ethanol (EtOH) and drug addiction as well. Interestingly, recent observations suggest that EE might additionally modulate binge-like intake of highly palatable caloric substances, including EtOH, which suggests the ability of EE to regulate consumption during the initial stage of the addiction cycle. We have proposed that EE protective and therapeutic effects on binge-consumption of palatable substances might primarily be mediated by the modulatory control that EE exerts on anxiety and impulsivity/compulsivity traits, which are all risk factors favoring transition to drug addiction.

Keywords: environmental enrichment, the addiction cycle, anxiety, impulsivity/compulsivity, binge-intake

## INTRODUCTION

Drug addiction is a chronic psychiatric disorder which exhibits components of both impulsivity and compulsivity and develops over time on three progressive stages in the so called ''addiction cycle'' (Koob and Volkow, 2009): (1) a binge/intoxication phase guided by the rewarding properties of drugs and impulsivity; (2) a withdrawal phase where excessive consumption escalates; and (3) a preoccupation/anticipation phase mainly guided by negative reinforcement, increased stress and anxiety and compulsivity (Koob and Volkow, 2009). Similarly to drugs, recent growing scientific evidence points to the existence of the new disorder called ''food addiction'' which exhibits several

#### Edited by:

David Belin, University of Cambridge, United Kingdom

#### Reviewed by:

Marco Venniro, National Institute on Drug Abuse (NIDA), United States Christian P. Müller, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany

> \*Correspondence: Inmaculada Cubero icubero@ual.es

Received: 24 September 2018 Accepted: 15 November 2018 Published: 29 November 2018

#### Citation:

Rodríguez-Ortega E and Cubero I (2018) Environmental Enrichment Modulates Drug Addiction and Binge-Like Consumption of Highly Rewarding Substances: A Role for Anxiety and Compulsivity Brain Systems? Front. Behav. Neurosci. 12:295. doi: 10.3389/fnbeh.2018.00295 neurobehavioral components matching drug addiction disorders (Avena et al., 2008; Gearhardt et al., 2011; Novelle and Diéguez, 2018), and where binge eating of highly palatable/caloric substances dominates the early stages of that process (Schulte et al., 2016).

Traditionally, drug and ethanol (EtOH) addiction research is dominated by studies addressing post-dependent stages of the addiction cycle, modeled by drug/EtOH dependence (Yardley and Ray, 2016; Spanagel, 2017) and drug/EtOH relapse preclinical procedures (Marchant et al., 2013; Vengeliene et al., 2014). Because continued binge-like consumption represents a risk behavior that favors transition to addiction (Koob and Le Moal, 2006; Courtney and Polich, 2009; Crabbe et al., 2011; Thiele and Navarro, 2014), we have alternatively focused our interest during the last years in the early pre-dependent stage of the addiction cycle dominated by repetitive binge-like intake (Alcaraz-Iborra et al., 2014, 2017; Alcaraz-Iborra and Cubero, 2015; Carvajal et al., 2015; Rodríguez-Ortega et al., 2018). It is our, and others authors (Thiele and Navarro, 2014) believe, that understanding those neurobehavioral processes involved in repetitive binge consumption in non-dependent animals would help us to develop new approaches preventing transition to dependence. In this regard, we have recently addressed in our laboratory the beneficial impact of providing environmental enriched housing conditions to adolescent and adult animals showing spontaneous binge-like consumption.

It is well known that housing conditions modulate diverse psychological processes and drug addiction (Nithianantharajah and Hannan, 2006). Preclinical research have demonstrated that environmental enrichment (EE), where animals are socially grouped and exposed to sensorial and motor enriched housing conditions (Crofton et al., 2015), prevents and also modulates ongoing EtOH intake and drug addiction as well (Nithianantharajah and Hannan, 2006). Consistent with this work, we have recently extended this knowledge by showing the therapeutic and protective role of EE on excessive binge-like consumption of EtOH (Rodríguez-Ortega et al., 2018). Interestingly, there is experimental evidence indicating that EE exposure also reduces anxiety (Peña et al., 2006, 2009; Sztainberg et al., 2010; Rodríguez-Ortega et al., 2018), compulsivity (Muehlmann et al., 2012; Bechard et al., 2016; Rodríguez-Ortega et al., 2018) and modulates EtOH (Deehan et al., 2007, 2011; de Carvalho et al., 2010; Rodríguez-Ortega et al., 2018) and sucrose intake (Brenes and Fornaguera, 2008; Grimm et al., 2010, 2013, 2016). Elevated anxiety (Wand, 2005), high sensitivity to stress (Goeders, 2003) and/or high compulsivity (Figee et al., 2016) might significantly increase vulnerability to develop addiction; on the other hand, we have reported that EE modulate spontaneous binge-intake. Taking together the aforementioned data, we have proposed the working hypothesis that EE housing conditions might primarily act on anxiety and impulsivity/compulsivity related brain systems during the intoxication early phase of the addiction cycle, protecting vulnerable non-dependent organisms from excessive binge-like intake and transition to dependence (Rodríguez-Ortega et al., 2018).

In the next paragraphs we first address the impact of EE exposure on the later dependent stage of the addiction cycle and then, we focus on more recent evidence indicating the additional benefits of EE exposure on the early, binge-dominated, pre-dependent stage.

## ENVIRONMENTAL ENRICHMENT EXPOSURE REGULATES DRUG AND SUGAR INTAKE DURING THE LATER STAGE OF THE ADDICTION CYCLE

### EE Protective Role in Drug Addiction

Recent studies employing animal models firmly submit that EE access during adolescence could act as a protective tool to prevent drug addiction development (Stairs and Bardo, 2009; Solinas et al., 2010). C57BL/6J mice raised under EE conditions during adolescence showed attenuated morphine-induced conditioned place preference (CPP), reduced hyperlocomotion and behavioral sensitization induced by acute morphine administration (Xu et al., 2007), and also showed blunted acute morphine-induced locomotor activity (Xu et al., 2014). Furthermore, rearing C57BL/6J mice in EE conditions reduced the reinforcing properties of heroin in a CPP test (El Rawas et al., 2009) and reduced Sprague-Dawley rats' amphetamine self-administration (SA) in a fixed ratio (FR) paradigm (Green et al., 2002). Moreover, early EE access reduced cocaine rewarding properties in a CPP test, reduced cocaine induced motor activation, blunted responses to a repetitive cocaine challenge (Solinas et al., 2009) and decreased intravenous cocaine SA in C57BL/6J mice and Sprague-Dawley rats (Green et al., 2010).

### EE Therapeutic Role in Drug Addiction

On the other hand, there is consistent scientific evidence supporting the therapeutic effect of EE access during adulthood to modulate drug consumption, drug reward and drug relapse. Thus, in Long Evans rats trained for continued heroin administration in a FR program, EE access blunted operant responses in cue-induced reinstatement of heroin seeking (Galaj et al., 2016). In Wistar rats, exposure to EE decreased behavioral deficits induced by methamphetamine and the risk of withdrawal unleashed by relapse (Hajheidari et al., 2015). EE also reduced heroin-, nicotin- and methamphetamine-seeking responses measured by a FR schedule in Sprague-Dawley rats (Sikora et al., 2018). Additionally, cocaine-induced behavioral sensitization and CPP to cocaine were blunted in adult C57BL/6 mice exposed to EE conditions and EE also protected cocaine-pre-exposed animals from cocaine-elicited CPP reinstatement (Solinas et al., 2008). In Sprague-Dawley rats, EE access blunted cocaine and stress induced cocaine reinstatement (Chauvet et al., 2009). EE eliminated cocaineseeking induced by context in C57BL/6 mice (Chauvet et al., 2011) and blunted the onset of cocaine craving incubation. In the same direction, EE eliminated an already stablished cocaine incubation (Chauvet et al., 2012), reduced reinstatement of cocaine operant responding elicited by a sensorial cue and also inhibited cocaine-seeking responses during extinction (Thiel et al., 2009).

### EE Therapeutic Role on Sucrose Intake and Sucrose Seeking

Interestingly, exposure to EE housing conditions do also modulate excessive consumption of highly palatable caloric substances. Isolated Sprague-Dawley rats consumed more sucrose than social or EE housed counterparts in a two bottlechoice (2BC) sucrose consumption test (Brenes and Fornaguera, 2008); isolated reared Lister hooded rats significantly drank more sucrose than socially reared rats when given sucrose in a 2BC paradigm in an ascending order of presentation (Hall et al., 1997). Additionally, single housed Wistar rats showed greater CPP for sucrose than their socially housed counterparts (Van den Berg et al., 1999). Moreover, EE access after sucrose SA training decreased sucrose seeking in Long-Evans rats, as measured by the response for a tone plus light cue previously conditioned to sucrose SA (Grimm et al., 2010). Also, acute and chronic access to EE was effective in reducing sucrose cue-reactivity and consumption in Long-Evans rats (Grimm et al., 2013, 2016). And isolated Sprague-Dawley rats were more sensitive to a cue-light compound paired with sucrose, when compared with enriched housed rats (Gill and Cain, 2011). Finally, Sprague-Dawley rats under EE housing conditions showed greater extinction of sucrose-maintained operant responding on a continuous reinforcement schedule than isolated rats (Stairs et al., 2006). Taking together, available data indicate that EE access clearly regulates sucrose consumption and sucrose seeking.

### ENVIRONMENTAL ENRICHMENT EXPOSURE REGULATES BINGE-LIKE CONSUMPTION IN NON-DEPENDENT ANIMALS DURING THE EARLY STAGE OF THE ADDICTION CYCLE

While there is consistent experimental evidence regarding the impact of EE on drug addiction, far less is known regarding the effect of EE exposure on excessive consumption of rewarding substances during early stages of the addiction cycle, dominated by binge-like intake. Preliminary data on EtOH research strongly suggest that EE exposure might provide therapeutic and protective effects to successfully modulate binge-like EtOH intake characteristic of the early stages of the addiction cycle. ''Drinking in the dark'' (DID; Rhodes et al., 2005, 2007) is a preclinical model of binge-like drinking which models human EtOH binge-intake as it triggers similar patterns of high voluntary EtOH consumption (BECs around 80 mg/dl) in short intervals (Cox et al., 2013; Alcaraz-Iborra et al., 2014; Thiele and Navarro, 2014; Carvajal et al., 2015). DID also models human binge consumption of caloric palatable substances. Thus, sucrose limited access on a DID procedure triggers more sucrose intake than 50% of the total amount consumed over 24 h under unlimited access (Sparta et al., 2008; Kaur et al., 2012). Given the ability of DID to model human binge-like consumption, it has been employed for studying the impact of EE on binge-like consumption of EtOH and other rewarding substances (sucrose) during the initial stages of the addiction cycle.

## EE Reduces EtOH Binge-Like Consumption

Recent evidence indicates that EE exposure might work to modulate excessive, binge-like EtOH consumption during the pre-dependent, early stage, of the addiction cycle. First, social and EE reduced EtOH preference and binge-like EtOH consumption in adult male C57BL/6 mice living in continuous (24 h) or restricted (3 h) EE conditions (Marianno et al., 2017). Also, C57BL/6 mice early socially housed showed reduced EtOH binge-like intake in a DID-2BC schedule when compared with their isolated housed counterparts (Lopez et al., 2011). EE access blunted high EtOH binge-like intake elicited by chronic social isolation in C57BL/6J mice (Lopez and Laber, 2015) and EE rearing during adolescence protected C57BL/6J mice from excessive EtOH binge-like drinking during adulthood (Rodríguez-Ortega et al., 2018). Furthermore, EE access significantly ameliorated steady EtOH binge-like consumption of adult mice housed in standard conditions (Rodríguez-Ortega et al., 2018) indicating a therapeutic role of EE on EtOH binge-like intake during that early, pre-dependent stage. Taking together, preliminary evidence points to the benefits of EE exposure, either during adolescence as a protective tool, or during adulthood as a therapeutic one, to modulate EtOH binge-like consumption in non-dependent organisms.

### ENVIRONMENTAL ENRICHMENT EXPOSURE MODULATES ANXIETY-LIKE RESPONSES AND COMPULSIVITY

High compulsivity (Figee et al., 2016), enhanced anxiety (Wand, 2005) and novelty-seeking responses (Iacono et al., 2008; Montagud-Romero et al., 2014; Arenas et al., 2016) are all premorbid neurobehavioral traits strongly linked to enhanced vulnerability to the onset of drug, food and EtOH addiction. Importantly, there is consistent scientific evidence pointing to the EE ability to regulate anxiety- and compulsivity-like behaviors as well as novelty-seeking traits. Thus, exposure to EE housing conditions reduced anxiety-like behaviors as measured by the Elevated Plus Maze (EPM) in both rats and mice (Peña et al., 2006, 2009; Sztainberg et al., 2010; Ragu Varman and Rajan, 2015; Bahi, 2017b), the Zero Maze (EZM) in rats (Nobre, 2016) and the light/dark box in mice (Sztainberg et al., 2010; Ragu Varman and Rajan, 2015). Moreover, EE housing conditions decreased motor exploration in SHR rats in an open field test (de Carvalho et al., 2010) and did blunt the onset of repetitive compulsive motor responses (Muehlmann et al., 2012; Bechard and Lewis, 2016; Bechard et al., 2016), which has been considered a manifestation of compulsivity in drug addiction (Figee et al., 2016). Finally, we have recently reported that animals under EE and long-term exposed to EtOH binge-consumption, show reduced anxiety- and compulsive-like responses, and reduced novelty-seeking behaviors, as assessed by EPM and the Hole Board (HB) test, respectively (Rodríguez-Ortega et al., 2018).

Given that, first, EE protects and ameliorates excessive EtOH binge-like drinking (Lopez et al., 2011; Lopez and Laber, 2015; Marianno et al., 2017; Rodríguez-Ortega et al., 2018), excessive sucrose consumption (Brenes and Fornaguera, 2008; Grimm et al., 2010, 2013, 2016); and second, because EE reduces anxiety (Peña et al., 2006, 2009; Sztainberg et al., 2010; Ragu Varman and Rajan, 2015; Nobre, 2016; Bahi, 2017a; Rodríguez-Ortega et al., 2018) and compulsivity-like responses (Bechard and Lewis, 2016; Bechard et al., 2016; Rodríguez-Ortega et al., 2018), we have proposed the hypothesis holding the existence of a causal nexus linking EE-mediated reduced anxiety/compulsivity and reduced EtOH binge-like consumption. Thus, we defend a primary impact of EE on anxiety/compulsivity/impulsivity brain systems during the pre-dependent early stages of the addiction cycle which, in turn might secondarily modulate binge-like intake of rewarding substances, finally preventing transition to addiction in vulnerable organisms.

#### ENVIRONMENTAL ENRICHMENT AND HUMAN DRUG INTAKE

Consistent with preclinical results, human research has suggested that EE exposure during the early and later, stages of the addiction cycle might be determinant to control excessive drug intake and prevent drug relapse. In this regard, community programs such as the Strategic Prevention Framework (SPF), has successfully controlled elevated EtOH binge drinking in adolescents and enforcement outcomes overtime (Anderson-Carpenter et al., 2016). Accordingly, the importance of controlling key environmental factors to prevent repetitive EtOH binge drinking episodes among students has been highlighted in national laws and drug policies, normative campaigns and targeted law enforcement (Clapp and Shillington, 2001; Clapp et al., 2003). Interestingly, programs such as Alcoholic Anonymous which encourage a controlled environment, free of drugs-cues and stress, together with enhanced social cognition and reinforcement (Schnabel, 2009), or the ''Drug courts'' program, which offers community-based treatment options (Turner et al., 2002), have both shown the utility of EE conditions to prevent EtOH relapse in addicted persons. Altogether, pre-clinical studies and human research, reinforce the idea that exposure to environmental enriched conditions during the different stages of the addiction cycle offer clear benefits to prevent and modulate binge-consumption, drug addiction and drug relapse.

Nevertheless, this is a young scientific field and we have a long way ahead. We need to further explore neurobehavioral and neurochemical mechanisms involved in EE positive effects, both protective and therapeutic, on the addiction cycle. We need to address whether EE exposure to non-addicted organisms impact

#### REFERENCES

Alcaraz-Iborra, M., and Cubero, I. (2015). Do Orexins contribute to impulsivitydriven binge consumption of rewarding stimulus and transition to drug/food on impulsive/compulsive consumption of also on other still non-explored drugs, during early stages of the addiction cycle, so new behavioral strategies can be developed to prevent transition to dependence. We need to further explore whether EE benefits on drug intake are primarily linked to EE-induced regulation of impulsivity/compulsivity and anxiety brain systems. We also need to understand how EE exposure during adolescence prevents during adulthood drug intake in different stages of the addiction cycle. Importantly, the role of EE on eating disorders and food addiction need to be deeply explored. In this regard, we are currently addressing in our lab whether EE exposure regulates binge-consumption of highly palatable caloric substances in non-dependent animals, so we can build testable hypothesis regarding the protector and therapeutic role of EE on the development of binge- and eating disorders and obesity. Finally, a new research avenue in this field, providing a better knowledge of EE neurochemical mechanisms, would help to develop new pharmacological modulators that mimic (environmimetics) or enhance EE positive effects (Nithianantharajah and Hannan, 2006).

### CONCLUSION

Preclinical studies and human research suggest that EE has a therapeutic and a protective role in drug addiction and exerts a modulatory control on excessive binge-like intake of palatable substances, including EtOH and sucrose. We know that EE also regulates excessive anxiety and compulsivity/impulsivity-like behaviors, known to be risk factors favoring the development of binge-consumption and transition to addiction. In this framework, we have proposed the working hypothesis that EE exposure might primarily modulate anxiety and impulsivity/compulsivity brain systems which, in turn, might secondarily moderate binge consumption of palatable substances preventing the progression toward addictive consumption during later stages of the addiction cycle. Future studies will further elucidate whether the observed EE therapeutic and protective effect on binge-consumption and anxiety/impulsivity/compulsivity, are causally linked.

#### AUTHOR CONTRIBUTIONS

IC and ER-O designed the conceptual framework in the manuscript and wrote the manuscript. All authors critically reviewed the content and approved the final version for publication.

### FUNDING

This work was supported by FEDER co-funding and Ministerio de Economia y Competitividad (Spain; grant PSI2015-64460-R to IC); Feder grant FEDERUALM05-23-006 (to IC).

dependence? Pharmacol. Biochem. Behav. 134, 31–34. doi: 10.1016/j.pbb.2015. 04.012

Alcaraz-Iborra, M., Carvajal, F., Lerma-Cabrera, J. M., Miguel, L., and Cubero, I. (2014). Binge-like consumption of caloric and non-caloric palatable substances

in ad libitum-fed C57BL/6J mice: pharmacological and molecular evidence of orexin involvement. Behav. Brain Res. 272, 93–99. doi: 10.1016/j.bbr.2014. 06.049


(EtOH)-drinking in male C57BL/6J mice augments subsequent voluntary EtOH intake but not other dependence-like phenotypes. Alcohol. Clin. Exp. Res. 37, 1688–1695. doi: 10.1111/acer.12145


withdrawn rats. Neurosci. Lett. 584, 151–155. doi: 10.1016/j.neulet.2014. 10.017


in C57BL/6J mice. Physiol. Behav. 84, 53–63. doi: 10.1016/j.physbeh.2004. 10.007


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Rodríguez-Ortega and Cubero. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Nigrostriatal Dopaminergic Denervation Does Not Promote Impulsive Choice in the Rat: Implication for Impulse Control Disorders in Parkinson's Disease

Robin Magnard<sup>1</sup> , Yvan Vachez <sup>1</sup> , Carole Carcenac<sup>1</sup> , Sabrina Boulet <sup>1</sup> , Jean-Luc Houeto<sup>2</sup> , Marc Savasta<sup>1</sup> , David Belin<sup>3</sup> and Sebastien Carnicella<sup>1</sup> \*

1 INSERM U1216 and Univ. Grenoble Alpes, Grenoble Institut des Neurosciences (GIN), Grenoble, France, <sup>2</sup>CIC-INSERM 1402, Service de Neurologie, CHU de Poitiers, Université de Poitiers, Poitiers, France, <sup>3</sup>Department of Psychology, Faculty of Biology, University of Cambridge, Cambridge, United Kingdom

Impulse control disorders (ICDs) are frequent behavioral complications of dopaminergic (DA) replacement therapies (DRTs) in Parkinson's disease (PD). Impulsive choice, which refers to an inability to tolerate delays to reinforcement, has been identified as a core pathophysiological process of ICDs. Although impulsive choices are exacerbated in PD patients with ICDs under DRTs, some clinical and preclinical studies suggest that the DA denervation of the dorsal striatum induced by the neurodegenerative process as well as a pre-existing high impulsivity trait, may both contribute to the emergence of ICDs in PD. We therefore investigated in a preclinical model in rats, specifically designed to study PD-related non-motor symptoms, the effect of nigrostriatal DA denervation on impulsive choice, in relation to pre-existing levels of impulsivity, measured in a Delay Discounting Task (DDT). In this procedure, rats had the choice between responding for a small sucrose reinforcer delivered immediately, or a larger sucrose reinforcer, delivered after a 0, 5, 10 or 15 s delay. In two different versions of the task, the preference for the large reinforcer decreased as the delay increased. However, and in contrast to our initial hypothesis, this discounting effect was neither exacerbated by, or related to, the extent of the substantia nigra pars compacta (SNc) DA lesion, nor it was influenced by pre-existing variability in impulsive choice. These results therefore question the potential implication of the nigrostriatal DA system in impulsive choice, as well as the DA neurodegenerative process as a factor contributing significantly to the development of ICDs in PD.

Keywords: Parkinson's disease, impulse control disorders, impulsive choice, delay discounting task, dopaminergic nigrostriatal system, 6-OHDA, rats

### INTRODUCTION

Parkinson's disease (PD) is a neurodegenerative disorder hitherto considered to stem from the loss of dopaminergic (DA) neurons in the substantia nigra pars compacta (SNc) and mainly characterized by cardinal motor symptoms (Samii et al., 2004). However, PD is also associated with a plethora of neuropsychiatric deficits, ranging from apathy and depression to impulse control disorders (ICDs; Voon et al., 2011; Sinha et al., 2013; Sierra et al., 2015; Houeto et al., 2016). ICDs are a complex group of impulsive/compulsive behaviors that includes gambling disorder,

#### Edited by:

Alicia Izquierdo, University of California, Los Angeles, United States

#### Reviewed by:

Suzanne Mitchell, Oregon Health & Science University, United States Tony Ye, University of Arizona, United States

> \*Correspondence: Sebastien Carnicella sebastien.carnicella@inserm.fr

Received: 01 October 2018 Accepted: 29 November 2018 Published: 13 December 2018

#### Citation:

Magnard R, Vachez Y, Carcenac C, Boulet S, Houeto J-L, Savasta M, Belin D and Carnicella S (2018) Nigrostriatal Dopaminergic Denervation Does Not Promote Impulsive Choice in the Rat: Implication for Impulse Control Disorders in Parkinson's Disease. Front. Behav. Neurosci. 12:312. doi: 10.3389/fnbeh.2018.00312 hypersexuality or compulsive shopping. ICDs are displayed by up to 14%–15% of PD patients under DA replacement therapies (DRTs) of whom quality of life they dramatically affect (Voon et al., 2009; Weintraub et al., 2010; Houeto et al., 2016). At a neurobiological level, despite the suggestion that an overstimulation of the mesocorticolimbic DA system by DRTs promotes ICDs in PD (Dagher and Robbins, 2009; Tang and Strafella, 2012), the underlying psychobiological and etiopathogenic factors that contribute to the development of ICDs only in vulnerable individuals remain unclear.

However, impulsive choice, a form of cognitive impulsivity which reflects an inability to tolerate delays to reinforcement, has been identified as a core pathophysiological process of ICDs (Voon et al., 2011; Houeto et al., 2016). Indeed, impulsive choice as measured in Delay Discounting Tasks (DDTs), which is characterized by the preference for small, immediate, rewards, over larger, delayed, rewards, is exacerbated in PD patients with ICDs (reviewed in Voon et al., 2011). Moreover, higher levels of impulsive choice have also been observed in de novo unmedicated or ''off'' medication PD patients without ICDs compared to healthy controls (Milenkova et al., 2011; Al-Khaled et al., 2015), suggesting that nigral DA cell loss itself may contributes to alter impulse control in PD (Voon and Dalley, 2011).

Recent studies in rats also support this hypothesis. Indeed, bilateral 6-hydroxydopamine (6-OHDA) lesions of the dorsal striatum reduced their tolerance for delayed reinforcers in a DDT (Tedford et al., 2015). In addition, α-synuclein-induced nigrostriatal neurodegeneration has been shown to increase other forms of impulsive behaviors (Engeln et al., 2016). However, the lesional approaches used in these studies provoked substantial motor deficits that may have bias measures of impulsivity. In addition, since only a subset of PD patients is affected by ICDs, the degeneration of the nigrostriatal DA system does not appear to be sufficient to promote ICDs, indicating a potential interaction with an endophenotype of vulnerability, as suggested in a previous study (Engeln et al., 2016).

Because impulsivity is tightly associated with ICDs and represents an endophenotype of vulnerability to develop compulsive behaviors and is a critical factor for the development of compulsive behaviors (Belin et al., 2008; Ansquer et al., 2014), it has been hypothesized that a high impulsivity trait may be associated with the disease progression and the emergence of ICDs in those vulnerable PD patients (Dagher and Robbins, 2009; Voon and Dalley, 2011; Houeto et al., 2016). Yet, the potential relation between the DA denervation and impulsivity remains to be established.

We therefore investigated in a longitudinal study (**Figure 1A**) the effect of nigrostriatal DA denervation on impulsive choice and its relation with endogenous level of impulsivity in DDTs. For this, we used a preclinical model in rats specifically designed to study PD-related non-motor symptoms (Magnard et al., 2016). Based on 6-OHDA-induced bilateral but partial lesions of the nigrostriatal DA system, this model has been demonstrated to reveal denervation-induced behavioral impairments, such as motivational- and affective-related deficits, without displaying significant impairments of motor functions (Carnicella et al., 2014; Drui et al., 2014; Favier et al., 2014).

### MATERIALS AND METHODS

#### Animals

Experiments were performed on male Sprague-Dawley rats (Janvier, France) 8 weeks old (weighting 300 g) at the beginning of the experiment. Twenty-one and 31 rats were used in DDT-within and -between experiments, respectively. They were individually housed under standard laboratory condition (12 h/light/dark cycle, with lights ON at 7 a.m.). They were food restricted at 90% of their free feeding weight during the DDT procedure, but had ad libitum access to water. Protocols complied with the European Union 2010 Animal Welfare Act and the new French directive 2010/63, and were approved by the French national ethics committee n◦ 004.

### Bilateral 6-OHDA Lesion

This procedure has been described extensively elsewhere (Carnicella et al., 2014; Drui et al., 2014; Favier et al., 2017). Briefly, food restriction was suspended 2 days before the beginning of the surgery. Rats were administered with desipramine hydrochloride (15 mg.kg−<sup>1</sup> subcutaneously; Sigma-Aldrich, St. Quentin-Fallavier, France) 30 min before they received an intracerebral infusion of 6-OHDA or vehicle (NaCl) in order to protect noradrenergic neurons. Rats were then anesthetized with a mixture of xylazine (15 mg.kg−<sup>1</sup> ) and ketamine (100 mg.kg−<sup>1</sup> ) both administered intraperitoneally. Rats were secured on a Kopf stereotaxic apparatus (Phymep, Paris, France) and 6 µg 6-OHDA (Sigma-Aldrich, St. Quentin-Fallavier, France) dissolved in 2.3 µl sterile 0.9% NaCl, or 2.3 µl sterile 0.9% NaCl (sham conditions), were injected bilaterally, through a 26 gauge cannula (Plastic One, Roanoke, USA) in the SNc, at a flow rate of 0.5 µl.min−<sup>1</sup> , at the following coordinates relative to bregma: AP, −5.4 mm; ML, ±1.8 mm; DV, −8.1 mm (Paxinos and Watson, 1998). After recovery from anesthesia, animals were returned to the facilities with food and water available ad libitum during 4 weeks in order to allow recovery and the 6-OHDA lesion to develop and stabilize prior to being re-subjected to food restriction and behavioral training.

#### Tyrosine Hydroxylase Immunohistochemistry and Quantification of Striatal DA Denervation

Rats were euthanized under chloral hydrate anesthesia at the end of the behavioral experiments. They were intracardially perfused with paraformaldehyde (PFA; 4%) as in Drui et al. (2014; DDT-within) or brains were post-fixed with PFA as in Favier et al. (2017, DDT-between), then frozen in cooled isopentane (−40◦C) and stored at −30◦C. Fourteen micrometer thick serial frontal sections of the striatum were processed with a cryostat (Microm HM 500, Microm, Francheville, France), collected on microscopic slides and stored at −30◦C.

Immunostaining was carried out as previously described (Favier et al., 2017). Sections collected on microscope slides were

delivered. After 35 s, the houselight is turned off and levers are retracted, for a 20 s intertrial interval (ITI), until the beginning of a new trial. During the task, each lever press leads also to levers retraction. (C) Schematic representation of the two DDT paradigms. The within-session delay discounting paradigm (within-DDT), is composed of five blocks, each composed of four forced choices (2 per lever) and 10 free choices. The delay increased progressively from one block to another. The task ends after 70 trials. In the between-session delay discounting paradigm (between-DDT), rats first performed 10 forced choices and then 30 free choices, experiencing only one delay per session. The task ends after 40 trials. These two experiments were independent, and two groups of rats were used, one per paradigm.

first air-dried and post-fixed with 4% PFA for 10 min, and then washed in PBS. Brain sections were subsequently incubated with an anti-tyrosine hydroxylase (anti-TH) antibody (mouse monoclonal MAB5280, Millipore, France, 1:2,500) and then with a biotinylated goat anti-mouse IgG antibody (BA-9200, Vector Laboratories, Burlingame, CA, USA; 1:500). Immunoreactivity was visualized with avidin-peroxidase conjugate (Vectastain ABC Elite, Vector Laboratories Burlingame, CA, USA).

TH immunoreactivity in the dorsal striatum and the nucleus accumbens (NAcc) was analyzed across the sections ranging from +2.2 to +0.7 mm from bregma with the ICS FrameWork computerized image analysis system (Calopix, 2.9.2 version, TRIBVN, Châtillon, France) coupled to a light microscope (Nikon, Eclipse 80i) and a Pike F-421C camera (ALLIED Vision Technologies, Stadtroda, Germany) for digitalization. Masks from the different striatal subregions were drawn with the computer analysis system to ensure that appropriate comparisons were made between homologous anatomical regions. Optical densities (ODs) were measured for each striatal region, and the mean OD was calculated with ICS FrameWork software (TRIBVN, 2.9.2 version, Châtillon, France). ODs were expressed as percentages relative to the mean optical density values obtained from the homologous regions of sham-operated animals. Only individuals displaying mean bilateral TH immunoreactivity (TH-IR) loss in the range of 50% to 85% in the dorsal striatum and less than 30% in the NAcc, were included in the analysis, as previously described (Drui et al., 2014; Favier et al., 2014). Based on these restrictive criteria, six and 13 6-OHDA lesioned rats were excluded for the within-DDT and the between-DDT experiments, respectively.

#### Delay Discounting Tasks

Sixteen operant chambers for rats (30 × 24 × 27 cm) from Med-Associates (St. Albans, VT, USA) were used. They were equipped with two retractable levers, a cue light located above each lever, a central house light, as well as a dual cup liquid receptacle (ENV-200R3AM) located between the two levers and connected through tubing (PHM-122-18) to syringe pumps

(PHM-100) for the delivery of the sucrose solution (5% or 10% w/v in tap water, Sigma-Aldrich, St. Louis, MO, USA).

Dorsal striatum; NAcc, Nucleus accumbens. Scale bar: 1 mm.

Two independent delay discounting procedures were carried out on two different batches of rats: one with the delay increasing within session (within-DDT) and the other with the delay increasing between sessions (one delay at a time, between-DDT; **Figure 1B**). The two tasks are described below, and were adapted from Evenden and Ryan (1996) and Mar and Robbins (2007) and based on pilot parametric experiments. The procedures started after 1 week of food restriction. Rats were randomly assigned to one operant chamber and the lever side (left or right) assigned to the large or small reinforcer was counterbalanced across operant chambers to prevent any bias of preference. Rats were exposed to one training session each day.

#### Phase 1: Operant and Forced Choice Training

Each session of this phase was divided in 70 and 40 trials for the within-DDT and between-DDT experiment, respectively. When a trial started, only one lever was extended and the above cue-light as well as the house light were illuminated to signal the opportunity to press. If the rat pressed the extended lever within 35 s, the lever was retracted, the cue-light turned off and the small reinforcer (120 µL of a 5% sucrose solution) delivered. After 35 s, the house light was turned off for a 20 s intertrial interval (ITI). If the rat failed to press the extended lever during the allocated 35 s period, the lever was retracted, the cue-light turned off at the beginning of the ITI, and an omission was counted.

As soon as the contingency between the instrumental response and the delivery of the small reinforcer (>85% of reinforced trials over three consecutive days) was acquired, rats were trained to acquire the other contingency whereby under similar forced choice sessions, they were required to respond on the other lever to obtain the larger reinforcer (120 µL of a 10% sucrose solution), until the same criteria was reached.

#### Phase 2: Discrimination and Free Choice Training

During this phase, rats were trained to choose between the large and the small reinforcer-associated lever, without any delay. Discrimination phase was considered as acquired when preference for the large reinforcer was above 85% during three consecutive days. As for phase 1, the trial started with the illumination of the house light (35 s long) and it is followed by a 20 s-off period signaled by the absence of light. The within- and between-session procedures are detailed below.

#### Within-Session

The task was divided into five blocks of 14 trials (total: 70 trials/session). Each block began with four forced choices (2 per lever) assigned in a random order, in which only one lever was extended at a time. The remaining 10 trials offered free choices during which both levers and cue-lights were extended and illuminated, respectively. Upon pressing one lever, both levers were retracted immediately and the associated reinforcer (120 µL of a 5% or a 10% sucrose solution) delivered. Each block had the same structure.

#### Between-Sessions

The task started with 10 forced choices (five per lever) assigned in a random order, in which only one lever was extended at the time. Then, rats had access to 30 free choices trials (total: 40 trials/session) with both levers extended and cue-lights illuminated as for the within-session protocol. As in this procedure, only one delay per session was experienced (see ''Phase 3: Delay Discounting'' section), the last 3 days of the discrimination phase were used to determine the preference for the large reinforcer at the 0 s delay.

#### Phase 3: Delay Discounting

The same blocks and structures as in the discrimination phase were used, except that delays between lever press and the delivery of the large reinforcer varied (**Figure 1C**).

#### Within-Session (Figure 1B Top)

Rats underwent 10 consecutive delay discounting test sessions, in which delay between lever press and the delivery of the large reinforcer increased between each block in an ascending order (see Tedford et al., 2015 for rationale) within the session. The delays per block were set at 0, 5, 10 and 15 s respectively. The last five sessions were averaged to determine the preference for the large reinforcer in function of delay, as performance was stable across the population (<20% variability between the mean of each group across these five sessions).

#### Between-Sessions (Figure 1B Bottom)

The delay preceding the delivery of the large reinforcer increased every five sessions, with the delays being the same as those used in

expressed as percentage of sham mean value. (D) Positive correlation between pre and post-surgery of individual AUC values for sham operated rats (empty circle) and 6-OHDA lesioned rats (full circle). Correlation is only significant for sham rats (p < 0.05; for 6-OHDA rats: p = 0.15). (E) 6-OHDA lesioned rats exhibited a higher percentage of omissions than sham rats during within-DDT. Data are represented as mean of omissions ±SEM in function of delay (averaged for the last five sessions). NS, non-significant, ∗∗p < 0.01; ∗∗∗p < 0.001; Sham (n = 9) vs. 6-OHDA (n = 6), reinf, reinforcer; AUC, area under the discounting curve; a.u, arbitrary units.

for the within-session procedure. The last three sessions for each delay were averaged to determine the preference for the large reinforcer, as performances were stable (<20% variability of the mean of each group across these three sessions).

The two DDTs were first performed before the lesion procedure in order to determine individual impulsivity traits and equilibrate the different experimental groups (ANOVAs conducted on this pre-surgery period revealed a main effect of delay: Fs > 10.56, ps < 0.001, but no group effect: Fs < 0.29, ps > 0.59 and no group × delay interaction: Fs < 0.14, ps > 0.93). Post-surgery resumption started on phase 2 (discrimination) in order to evaluate the effect of the lesion on impulsive choice and impulsivity trait.

as percentage of sham mean value. (E) Positive and significant correlation between pre and post-surgery of individual AUC values for sham and 6-OHDA lesioned rats (ps < 0.05). NS, non-significant, ∗∗∗p < 0.001; Sham (n = 12) vs. 6-OHDA (n = 6), reinf, reinforcer; AUC, area under the discounting curve; a.u, arbitrary units.

#### Data and Statistical Analyses

Data are presented as mean ±SEM or individual datapoints. Calculation of the area under the curves (AUC) in order to measure delay discounting in each individual, were adapted from Myerson et al., 2001. Briefly, delays, and large reinforcer preference were first expressed as the proportion of their maximum value in order to be comprised between 0 and 1. Then, the resulting discounting curve was subdivided discounting graph into a series of half trapezoids, from 0 to 5 s, 5 to 10 s and 10 to 15 s. The area of each trapezoid is thus equal to the following equation: x<sup>2</sup> − x1(y<sup>1</sup> + y2)/2, where x<sup>1</sup> and x<sup>2</sup> are successive delays associated with the large reinforcer preference y<sup>1</sup> and y2, respectively. Thus, the area under the discounting curve is the sum of the area of the three trapezoids. In contrast to Myerson et al. (2001), the value of the delay 0 was not normalized to 100% (i.e., 1), because the current strategy is considered better to capture the reaction and adaptation of each individual to the delay with regards to their initial preference for the large reinforcer. Although we found a strong correlation between the two measures of AUC (R <sup>2</sup> = 0.88 and 0.82 for the within-session and between-sessions DDTs experiments respectively), these two calculation methods may lead to slight differences. Hence, measures of DD-AUC using the normalization described by Myerson et al. (2001), are reported in **Supplementary Figure S2**.

Data were analyzed by t-test or two-way repeated measure ANOVAs, using SigmaStat software (SigmaStat 4.0 2016, Systat software Inc., San Jose, USA). When indicated, post hoc analyses were carried out using the Student-Newman-Keuls test. Correlations were performed and analyzed with parametric linear regression approaches (Pearson product moment correlation and comparison of linear regression coefficients). Assumptions for the normality of the distributions and the homogeneity of variance were verified using the Shapiro-Wilk and Levene test, respectively. Significance for p values was set at α = 0.05. Effect sizes for the ANOVAs are also reported using partial η 2 values (Levine and Hullett, 2002; Murray et al., 2015).

### RESULTS

Rats were first trained in a DDT under which the delay increased progressively within each test session (within-DDT), in order to measure individuals' basal level of impulsive choice and distribute them in the various experimental groups. Rats were subsequently exposed to either sham or bilateral infusions of 6-OHDA into the SNc, which resulted in partial selective DA denervation in the dorsal striatum that relatively spared the NAcc (68.8 ± 2.4% and 4.6 ± 1.8% loss of TH-immunolabeling, respectively, (t(10) = 21.42, p < 0.001; **Figures 2A,B**). This pattern of denervation is in line with previous studies (Drui et al., 2014; Favier et al., 2014) which have demonstrated it circumvents the motor deficits frequently associated with greater or other patterns of nigrostriatal DA lesions.

At resumption of within-DDT training, all rats maintained a preference for the large reinforcer over the smaller one in absence of delay, and irrespective of the lesion (**Figure 3A**; delay 0 s), indicating that, in agreement with previous results (Drui et al., 2014), the relative reinforcing value of natural rewards was not influenced by the nigrostriatal DA denervation. This preference for the large reinforcer decreased as the delay increased (main effect of delay: F(3,39) = 9.94, p < 0.001, partial η <sup>2</sup> = 0.43; **Figure 3A**). However, this discounting effect was not exacerbated in lesioned animals (no effect of lesion: F(1,39) = 0.05, p = 0.83, partial η <sup>2</sup> = 0.006, and no delay × lesion interaction: F(3,39) = 0.49, p = 0.69, partial η <sup>2</sup> = 0.04; **Figure 3A**), which led eventually to a similar AUC in both groups (no effect of lesion: F(1,26) = 0.03, p = 0.86, partial η <sup>2</sup> = 0.001, and no period × lesion interaction: F(1,26) = 0.01, p = 0.92, partial η <sup>2</sup> < 0.001; **Figure 3B**; see also **Supplementary Figure S2A**). Level of impulsivity seemed not to be related to the extent of the nigrostriatal DA denervation, as no correlation was evidenced between the percentage of TH-immunolabeling loss in the dorsal striatum and the level of impulsive choice (indexed by the AUC; **Figure 3C** and **Supplementary Figure S2B**). A positive correlation was nevertheless found between the AUC before and after surgery in sham rats (**Figure 3D**). Intriguingly, this relation significantly decreased in the 6-OHDA condition (significant difference in regression slope: t(11) = 3.35, p < 0.05; **Figure 3D**), which may indicate a narrowing of the variance of this trait or a change in the strategy used to compute the delay-preference function. However, this relation as well as the differences between sham vs. lesion groups, were not found with the conventional normalization method of Myerson et al. (2001; **Supplementary Figure S2C**).

In addition, although the number of omissions displayed by the sham group remained very low in both forcedand free-choice trials and across delays (**Supplementary Figures S1A,B**, **Figure 3E**, respectively), it was considerably increased in lesioned rats, especially as the session progressed and the delay increased (for free-choice trials, main effect of the lesion: F(1,39) = 31.83, p < 0.001, partial η <sup>2</sup> = 0.91 and delay × lesion interaction: F(3,39) = 15.75, p < 0.001, partial η <sup>2</sup> = 0.55). This increase in omissions may reflect an increased aversion to the cognitive/motivational demand when delays are introduced, which results in a delay-dependent disengagement from the task or a more general impairment in maintaining a motivated behavior over prolonged periods of time. Interestingly, a systematicity in the omissions profile was observed, as even within a block, omissions mostly occur at the end. Indeed, such SNc DA lesions have been shown to impaired the maintenance of preparatory and seeking behaviors (Magnard et al., 2016; Favier et al., 2017) and nigrostriatal DA denervation can induce profound attentional and/or cognitive deficits (Nieoullon and Coquerel, 2003; Aarts et al., 2011).

Because such a high level of omissions may have interfered with the discounting data by biasing the sampling of the choice responses by the rats, we modified the task to limit this effect. We hypothesized that testing only one delay at a time by increasing delays across sessions (between-DDT), would make each session shorter and less taxing. This new procedure was tested with another batch of rats with a similar pattern of DA denervation (**Figure 2C**). Even if lesioned rats displayed significantly higher levels of omissions than those of the sham group during forced- (**Supplementary Figures S1C,D**) and free-choice trials (main effect of lesion: F(1,48) = 5.62, p < 0.05, partial η <sup>2</sup> = 0.18, of the delay: F(3,48) = 126.49, p < 0.001, partial η <sup>2</sup> = 0.89, and delay × lesion interaction: F(3,48) = 5.64, p < 0.05, partial η <sup>2</sup> = 0.26; **Figure 4A**), this effect was markedly reduced, as the difference between the groups was attributable only to performance on the 10 s delay. Even under these conditions, which controlled for the potential confounding influence of high omissions in lesioned rats, these data confirmed that the partial bilateral SNc DA lesion did not exacerbate impulsive choice (main effect of delay: F(3,48) = 54.12, p < 0.001, partial η <sup>2</sup> = 0.77, but no effect of lesion: F(1,48) = 0.20, p = 0.66, partial η <sup>2</sup> = 0.01, or delay × lesion interaction: F(3,48) = 0.09, p = 0.96, partial η <sup>2</sup> = 0.006; **Figure 4B**). This was further supported by an absence of difference between sham and lesioned rats in the AUC (no effect of lesion: F(1,32) = 0.35, p = 0.55, partial η <sup>2</sup> = 0.01, and no period × lesion interaction: F(1,32) = 0.19 p = 0.66, partial η <sup>2</sup> = 0.001; **Figure 4C**; see also **Supplementary Figure S2D**) and, at the population level, by the absence of relationship between DA denervation and AUC (**Figure 4D** and **Supplementary Figure S2E**).

In addition, and in contrast with the previous experiment, the lesion did not change the correlation between pre- and post-surgery AUC, indicating no influence on impulsivity trait in individuals (no difference in regression slope: t(14) = 0.78, p > 0.40; **Figure 4E**; see also **Supplementary Figure S2F**).

#### DISCUSSION

Using a validated 6-OHDA lesion-based rodent model that was specifically designed for the investigation of non-motor, neuropsychiatric impairments related to PD (reviewed in Cenci et al., 2015; Magnard et al., 2016), we showed that bilateral and partial denervation of the nigrostriatal DA pathway neither induced nor exacerbate impulsive choice in two different DDTs, taking into account inter-individual variability at baseline.

The AUC, was used here as an empirical objective measure of discounting behavior and impulsivity trait in rats (Myerson et al., 2001; Odum, 2011). The discounting rate k factor has hitherto been a preferred index in clinical studies to assess intertemporal choice (e.g., Milenkova et al., 2011; Al-Khaled et al., 2015). Here, AUC was preferred to the k factor as an index of impulsivity, because it better accommodates the properties of the current dataset. Indeed, calculation of the k factor derives from the slope of the discounting function and relies on the indifference point (Broos et al., 2012), which is not necessarily crossed by all rats (especially low impulsive animals) under our experimental conditions. Together with the marked inter-individual differences observed in the present study in the time course of the discounting curve a fit-for-all model could not be implemented here. Nevertheless, the strong correlations observed between AUC obtained at different time points and test phases in sham conditions, especially in the between-sessions DDT, offer further evidence that it is a relevant and reliable measure of impulsive choice for longitudinal studies, as well as a useful and alternative tool for identifying endophenotypes of impulsive choice, in rats.

Although the discounting of the larger reinforcer over increasing delays was not influenced by nigrostriatal DA denervation, its inter-individual variability was markedly decreased by the lesion in the within- but not between-DDT. Interestingly, bilateral excitotoxic lesions of the dorsal striatum have also been shown to discretely flatten delay-preference function without affecting delay discounting in a within-session design version of the task (Dunnett et al., 2012). Because this effect progressively disappeared with extensive training, it had been attributed to a decrease ability to adjust behavior to the rapid modifications of task parameters across test sessions rather than to an alteration of impulsivity per se (Dunnett et al., 2012). Using similar experimental conditions (e.g., food restriction, similar pattern of DA denervation), the number of omissions drastically decreased in a between-DDT, in which the behavioral/motivational (less trials and shorter sessions) and cognitive (only one delay tested at any given time) demand, was reduced compared to a within-DDT. Consistently, Tedford et al. (2015) did not report any significant increase in omissions in another between-sessions DDT following dorsostriatal DA lesions. Together, these data suggest that nigrostriatal DA denervation may decrease the ability of the animal to properly engage in the task, and that between-session designs may be more appropriate than within-session designs to investigate the contribution of the nigrostriatal DA system to impulsive choice.

Although some clinical studies suggest a possible role of the neurodegenerative process in the development of impulsive behaviors in PD (Milenkova et al., 2011; Al-Khaled et al., 2015), conflicting results have been reported about the potential implication of a nigrostriatal DA deficit in different forms of impulsivity (Rokosik and Napier, 2012; Tedford et al., 2015; Engeln et al., 2016; Carvalho et al., 2017). This likely stems from the difficulty to disentangle an effect of nigrostriatal DA denervation on impulse control from its often dramatic effect on motor performance, alongside with the cognitive or motivational alterations potentially induced by DA lesions (Cenci et al., 2015).

Notably, it has been observed that a bilateral DA lesion of the dorsal striatum increased delay discounting in a betweensession, wherein a similar range of ascending delays as the one used in the present study was applied (Tedford et al., 2015). However, substantial methodological differences between the two studies may account for these seemingly contradictory results. First, the retrograde lesion approach used in Tedford's study led to motor dysfunctions, which could have impacted the coordinated sequences of actions necessary to perform the associated chained scheduled task. In addition, the use of a dorsostriatal retrograde lesion as opposed to the SNc anterograde lesion performed in the present study may lead to a different DA denervation pattern (e.g., Cenci et al., 2015) or different underlying DA deficits and compensatory mechanisms. Tedford et al. (2015) also used intracranial self-stimulation (ICSS) as a reinforcer in order to avoid satiety and other potential issues associated with food. However, chronic ICSS itself can induce profound neuroadaptations, such as overexpression of DA D<sup>1</sup> receptors in the NAcc (Simon et al., 2016), a structure in which DA receptors modulate impulsive choice (Basar et al., 2010). Therefore, the increased delay discounting reported in this study may result from a direct neurobiological interaction between ICSS and the lesion.

Although the present study focuses exclusively on nigrostriatal cell loss and the emergence of impulsive choice, it should be kept in mind that other neurochemical systems, such as the serotoninergic and noradrenergic systems, have been shown to be affected by the neurodegenerative processes of PD (Delaville et al., 2011; Maillet et al., 2016). Due to their implication in the control of impulsive behaviors, alteration of these two monoaminergic systems, which, in the case of noradrenaline, may even precede the degeneration of DA neurons (Delaville et al., 2011), are prone to contribute, independently, or in conjunction with DA denervation, to the development of impulsivity in PD (Dalley and Roiser, 2012; Kehagia et al., 2014; Ye et al., 2014; Dalley and Robbins, 2017).

Nevertheless, our study provides novel insights into the contribution of the nigrostriatal DA system to impulsive choice and useful methodological considerations for future studies. It also highlights the fact that further investigations are necessary to better apprehend the potential contribution of the DA neurodegenerative process in conjunction with impulsivity trait and DRTs to the development of ICDs in PD.

### AUTHOR CONTRIBUTIONS

RM, J-LH, MS, DB and SC designed research. RM, YV, CC, SB and SC performed research. RM, CC and SC analyzed data. RM, DB and SC wrote the manuscript with the help of the other authors.

### REFERENCES


#### FUNDING

This work was supported by the Institut National de la Santé et de la Recherche Médicale (INSERM), Association France Parkinson, Agence Nationale de la Recherche (ANR13 SAMA001401) and Grenoble Alpes University. DB is supported by an MRC programme grant (MR/N02530X/1).

#### ACKNOWLEDGMENTS

The authors would like to thank Eva Martinez for technical assistance during pilot experiments for the development of the delay discounting procedure.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbeh.20 18.00312/full#supplementary-material


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Magnard, Vachez, Carcenac, Boulet, Houeto, Savasta, Belin and Carnicella. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Cognitive Drivers of Compulsive Eating Behavior

#### Naomi Kakoschke<sup>1</sup> , Esther Aarts <sup>2</sup> and Antonio Verdejo-García<sup>1</sup> \*

<sup>1</sup>Monash Institute for Cognitive and Clinical Neurosciences (MICCN), School of Psychological Sciences, Monash University, Melbourne, VIC, Australia, <sup>2</sup>Donders Centre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands

Compulsivity is a central feature of obsessive-compulsive and addictive disorders, which share considerable overlap with excessive eating in terms of repetitive behavior despite negative consequences. Excessive eating behavior is characteristic of several eating-related conditions, including eating disorders [bulimia nervosa (BN), binge eating disorder (BED)], obesity, and food addiction (FA). Compulsivity is proposed to be driven by four distinct cognitive components, namely, contingency-related cognitive flexibility, task/attentional set-shifting, attentional bias/disengagement and habit learning. However, it is unclear whether repetitive behavior in eating-related conditions is underpinned by deficits in these cognitive components. The current mini-review synthesizes the available evidence for performance on compulsivity-related cognitive tasks for each cognitive domain among populations with excessive eating behavior. In three of the four cognitive domains, i.e., set-shifting, attentional bias and habit learning, findings were mixed. Evidence more strongly pointed towards impaired contingencyrelated cognitive flexibility only in obesity and attentional bias/disengagement deficits only in obesity and BED. Overall, the findings of the reviewed studies support the idea that compulsivity-related cognitive deficits are common across a spectrum of eatingrelated conditions, although evidence was inconsistent or lacking for some disorders. We discuss the theoretical and practical importance of these results, and their implications for our understanding of compulsivity in eating-related conditions.

#### Edited by:

David Belin, University of Cambridge, United Kingdom

#### Reviewed by:

Tommy Pattij, VU University Medical Center, Netherlands Antonio Ferragud, University of Cambridge, United Kingdom

#### \*Correspondence:

Antonio Verdejo-García antonio.verdejo@monash.edu

Received: 27 September 2018 Accepted: 31 December 2018 Published: 17 January 2019

#### Citation:

Kakoschke N, Aarts E and Verdejo-García A (2019) The Cognitive Drivers of Compulsive Eating Behavior. Front. Behav. Neurosci. 12:338. doi: 10.3389/fnbeh.2018.00338 Keywords: compulsivity, cognitive functioning, eating behavior, obesity, bulimia nervosa, binge eating, food addiction

## INTRODUCTION

Compulsivity is defined as ''the performance of repetitive, unwanted and functionally impairing overt or covert behaviors without adaptive function, performed in a habitual or stereotyped fashion, either according to rigid rules or as a means of avoiding perceived negative consequences'' (Fineberg et al., 2014, p. 70). Behavioral patterns of compulsive eating, defined as repetitive bouts, without homeostatic function, with adverse consequences, and as ways to relieve stress, are common across several eating-related conditions (Moore et al., 2017). These include: (1) eating disorders such as bulimia nervosa (BN) and binge eating disorder (BED); (2) obesity; and (3) food addiction (FA), which have very different diagnostic considerations (**Table 1**). However, is it important to acknowledge that the validity of FA is a highly debated and controversial concept within the scientific community (Ziauddeen and Fletcher, 2013; Hebebrand et al., 2014; Cullen et al., 2017). In this review article, we examine the cognitive underpinnings of this transdiagnostic compulsive eating phenotype. To do so, we adopt the four cognitive components of compulsivity proposed in the framework by Fineberg et al. (2014; i.e., cognitive flexibility, set-shifting, attentional


exclusively during course or Nervosa. BNand BED symptoms defined according to DSM 5 diagnostic criteria (American Psychiatric Association, 2013). BMI categories defined according to the World Health Organisation (2017). FA symptoms defined according to the by Gearhardt et al. (2016). Bold font denotes characteristics that fit a compulsive eating phenotype (i.e., repeated bouts, without adaptive–homeostatic function and/or driven by stress relief).

bias/disengagement, and habit learning), and review studies that measured at least one component in adults with BN, BED, obesity or FA. To ensure timeliness, we only reviewed research published in the last 5 years (for reviews of earlier work in discrete domains see: Wu et al., 2014; Stojek et al., 2018).

## REVIEW OF FINDINGS

In this section, we define each of the cognitive components of compulsivity and the tasks that measure them, and then review evidence of task performance in: (1) BN and BED; (2) obesity; (3) FA; and (4) overlapping conditions (e.g., obesity and BED; obesity and FA). **Figure 1** displays a summary of the findings.

## Contingency-Related Cognitive Flexibility

This component refers to ''impaired adaptation of behavior after negative feedback'' (Fineberg et al., 2014). It has been posited that compulsivity arises from perseverating on a behavior that was once rewarded, but then becomes associated with negative consequences, indicating less cognitive flexibility. Contingencyrelated cognitive flexibility has frequently been measured using the probabilistic reversal learning task (PRLT; Cools et al., 2002; Clarke et al., 2005), which involves choosing between two stimuli and learning that one is usually rewarded (positive outcome), while the other is usually punished (negative outcome). The rule then changes and participants need to adapt their behavior in response to the outcome change.

Although no studies have examined this component in BN, BED alone or FA, cognitive flexibility deficits have been observed in obesity. Specifically, individuals with obesity showed more difficulty inhibiting a previously learnt behavioral rule indicated by increased perseverative errors on the Rule Shift Cards task (Spitoni et al., 2017). Women with obesity also showed reversal

Note: proposal

learning deficits specific to food, but not monetary cues (Zhang et al., 2014). Contradictory findings have also been reported, whereby participants with obesity showed impaired punishment, but not reward learning relative to healthy controls (Coppin et al., 2014; Banca et al., 2016), while obese participants with BED showed impaired reward, but not punishment learning relative to those without BED (Banca et al., 2016).

#### Task/Attentional Set-Shifting

This component is defined as ''impaired switching of attention between stimuli'' (Fineberg et al., 2014). It involves frequent switching between sets of tasks or response types, which requires paying attention to multiple dimensions of stimuli. Of note, set-shifting is also contingency related, but it relies on stimulus-response sets rather than reward and punishment outcomes. The most common set-shifting measures were the Wisconsin Card Sorting Test (WCST) and the Trail Making Task Part-B (TMT-B), while the Intra-Dimensional/Extra-Dimensional set-shift task (Robbins et al., 1998) and the Task-Switching Paradigm (Steenbergen et al., 2015) were used less frequently. The WCST involves matching a card with specific features (e.g., color, shape) to one of four other cards using a ''matching rule,'' which changes over the course of the task. In the TMT-B, participants are asked to draw a line linking alternating numbers and letters (i.e., 1-A-2-B-3-C).

Most research on set-shifting has focused on eating disorders. Some studies found that set-shifting was not impaired in BN (Pignatti and Bernasconi, 2013), BED (Manasse et al., 2015), or sub-threshold BE symptoms (Kelly et al., 2013). However, Kelly et al. (2013) found that total number of binge episodes were positively correlated with perseverative errors on the WCST (i.e., poorer set-shifting). Furthermore, other studies found impaired set-shifting in patients diagnosed with BED or BN relative to healthy controls (Goddard et al., 2014; Aloi et al., 2015).

In obesity, studies examining set-shifting have produced inconsistent results. Specifically, some studies found no evidence of impaired performance (Chamberlain et al., 2015; Fagundo et al., 2016; Manasse et al., 2014; Schiff et al., 2016; Wu et al., 2016), while other studies found impaired set-shifting in participants with overweight or obesity relative to healthy controls (Gameiro et al., 2017; Steenbergen and Colzato, 2017) and eating disorder patients (Perpiñá et al., 2017). Studies have also shown impaired set-shifting in obese participants with BED, but not in those without (Banca et al., 2016), and obese participants with high, but not low FA symptoms (Rodrigue et al., 2018).

### Attentional Bias/Disengagement

This component entails ''impaired shifting of mental sets away from stimuli'' (Fineberg et al., 2014). Attentional bias involves the automatic orienting of attention towards certain stimuli; an aspect of selective attention (Cisler and Koster, 2010), while disengagement refers to an inability to direct/shift attention away from such stimuli, which may contribute to compulsive behavior via rigidity induced by disorder-relevant stimuli (Fineberg et al., 2014). Attentional bias is commonly measured with the Visual Probe Task (VPT), in which participants are instructed to respond to a dot that appears on the left or right side of a computer screen immediately following the presentation of a pair of stimuli, or the Emotional Stroop, in which participants are asked to name the ink color of a written word while ignoring its content.

Several studies have provided evidence of an attentional bias for unhealthy food cues in BN (Albery et al., 2016), BED (Sperling et al., 2017), or subthreshold BE symptoms (Popien et al., 2015), although one recent study found no evidence of attentional bias for unhealthy food in BED or BN relative to healthy weight controls (Lee et al., 2017). Some studies have also shown an attentional bias for unhealthy food in obese compared to healthy weight participants (Kemps et al., 2014; Bongers et al., 2015), while another study found no relationship between attentional bias toward food words and obesity-related indices (body mass index, BMI and abdominal fat; Janssen et al., 2017). Nevertheless, obese individuals with BED show a stronger attentional bias to unhealthy food cues than those without BED or normal-weight controls (Schag et al., 2013; Schmitz et al., 2014, 2015), and individuals with obesity and subthreshold BE symptoms showed more difficulty disengaging from such cues than those without BE (Deluchi et al., 2017). Participants with obesity and FA also had a larger attentional bias and more difficulty disengaging from unhealthy food cues relative to healthy weight controls without FA (Frayn et al., 2016).

### Habit Learning

This component involves ''lack of sensitivity to goals or outcomes of actions'' (Fineberg et al., 2014). Associative learning theories of instrumental behavior posit that actions are supported by two systems: a goal-directed and a habitual system (Balleine and Dickinson, 1998; de Wit and Dickinson, 2009). Compulsivity is hypothesized to arise from a shift away from goal-directed action toward habit due to an imbalance in these two underlying systems, i.e., an impaired goal-directed or overactive habit system. Evidence for an imbalance between these two systems can be tested with instrumental decision-making paradigms. In outcome devaluation tasks, participants have to refrain from responding to cues when the rewards associated with them have been devalued by selectively changing outcome contingencies as in the Slips-of-Action task (de Wit et al., 2012) or sensory-specific satiety (Balleine and Dickinson, 1998). The Two-Stage task uses a model-free/model-based reinforcement learning paradigm in which participants are instructed to make choices based on previously reinforced choices (model-free, ''habit''-like) or future goal states (model-based, ''goal-directed;'' Daw et al., 2011).

Results from studies on habit learning in obesity are inconsistent. Specifically, two studies have shown that individuals with obesity were less sensitive to action outcomes, i.e., action control was shifted towards habitual control and away from goal-directed control, which suggests that these two systems are unbalanced (Horstmann et al., 2011; Janssen et al., 2017). In contrast, two other studies using the Slipsof-Action task found that participants with obesity did not make more slips-of-action than healthy weight participants (Dietrich et al., 2016; Watson et al., 2017). However, another study demonstrated that obese individuals with BED showed greater impairments in goal-directed (model-based) than habitual (model-free) responses than obese participants without BED or healthy-weight participants (Voon et al., 2015a).

### DISCUSSION

Our review indicates some evidence of deficits across the four compulsivity-related cognitive processes among individuals with excessive eating-related problems. However, for most eating-related conditions (except for the overlapping condition, namely, obesity with BED) the data are inconclusive regarding impairments in the cognitive domains. These conflicting findings make it difficult to draw firm conclusions regarding the role of compulsivity-related cognitive deficits underlying problematic eating behavior across conditions. Nevertheless, the findings are first discussed for each compulsivity-related cognitive domain across the spectrum of eating-related problems. We then provide a conceptual discussion regarding the extent to which cognitive components related to compulsivity should be applied in the context of eating behavior, which is followed by an operational discussion of how we can move forward experimentally to advance our understanding of compulsivity-related cognitive functions.

The available research on contingency-related cognitive flexibility (i.e., reversal learning) shows a consistent pattern of results, namely, impaired reversal learning in obesity and BED. However, there were differences in terms of valence of impaired reversal learning (i.e., reward vs. punishment), which differed across conditions (i.e., obesity alone or obesity with BED). A potential explanation for the discrepant findings is that obese individuals with BED may be more likely to respond based on previously rewarded behaviors, while obese individuals without BED may be more likely to avoid responding based on previously punished behaviors (Banca et al., 2016). This idea is further supported by the finding of increased sensitivity to reward and enhanced risk taking in relation to reward expectation in obese individuals with BED, but not those without (Voon et al., 2015b). However, these findings do not align with the general view that BED is underpinned by negative reinforcement mechanisms (Vannucci et al., 2015). Nevertheless, it has been proposed that BED is characterized by generalized impairments in cognitive flexibility (Voon et al., 2015a). Thus, further studies are needed to unravel the role of reversal learning in obesity and BED. Finally, there was a lack of evidence for reversal learning in populations with BN or FA, and hence, the findings are limited to obese individuals with or without BED.

Within the domain of task/attentional set-shifting, studies also revealed mixed findings, which might be attributable to differences in sample composition (e.g., age and BMI) and methodology (i.e., self-reported vs. diagnosed BE; different cognitive tasks used to measure set-shifting ability). For example, the ID/ED task is proposed to measure multiple components of compulsivity, namely, reversal learning and set-shifting (Wildes et al., 2014), while the TMT-B measures only set-shifting ability. One possible explanation for the discrepant findings in the literature is that individuals with eating disorders or obesity might show deficits in some sub-components of set-shifting (e.g., engaging in vs. disengaging from a task-set), but not others (e.g., keeping the relevant task dimension online in working memory). Thus, the different facets involved in the various tasks used across studies may contribute to the contradictory results in this domain. In line with this idea, a recent meta-analysis demonstrated a small-to-medium effect size for inefficient set-shifting in BN, BED and obesity (Wu et al., 2014), which suggests that other factors may interact with set-shifting to predict compulsive eating behavior. Taken together, our review and the meta-analysis by Wu et al. (2014) suggest that set-shifting inefficiency is one compulsivity-related cognitive domain that may contribute to compulsive eating behavior.

The findings of this review also provide evidence for attentional bias/disengagement for disorder-specific cues, i.e., unhealthy food, in BED, obesity, and BED with obesity, although not all studies showed this effect, which is consistent with a recent review on attentional bias in BE-related disorders (Stojek et al., 2018). However, there was considerable variability in the tasks used to assess attentional bias, i.e., the Emotional Stroop or the VPT, the latter of which can distinguish between attentional bias and inability to disengage. Furthermore, the Stroop task requires executive functions other than attention, including inhibitory control (Balleine and Dickinson, 1998; de Wit and Dickinson, 2009), and thus, attentional bias may be linked to compulsive behavior more indirectly than the other cognitive components. Few studies assessed attentional bias/disengagement in BN or FA, which was also observed in the review by Stojek et al. (2018). Thus, future research should employ tasks that examine both attentional bias and disengagement from disorder specific stimuli across the spectrum of eating-related issues.

The tasks used to assess habit learning also demonstrated impairments in obesity and BED, although the studies in this domain were limited to these two eating-related populations. The finding that a propensity toward habit learning was shown with model-free vs. model-based and outcome devaluation tasks, but not the slips-of-action task indicates that these tasks may measure different aspects of habit learning. For example, behavior may be a consequence of an impaired goal-directed system or an overactive habit system, which can be distinguished using the Two-Stage task (Voon et al., 2015a). Moreover, the type of outcome devaluation in devaluation tasks matters. Due to possible obesity-related decreases in interoceptive sensitivity (Herbert and Pollatos, 2014), outcome devaluation via satiation (Horstmann et al., 2011; Janssen et al., 2017) might be less effective than outcome devaluation via instruction for overweight/obese individuals (Dietrich et al., 2016; Watson et al., 2017). While evidence for a propensity toward habit learning was more consistent in BED than obesity, more studies are needed before conclusions are drawn.

### Limitations and Future Research Directions

Our review highlights the emerging body of work on cognitive underpinnings, but well-established aspects of the compulsive eating phenotype, that still need to be incorporated in a cognitive model of compulsivity. Specifically, it is not clear how negative reinforcement mechanisms (i.e., emotional eating) or dietary restraint and related anxiety/stress, which are key drivers of compulsive eating in BN, BED and obesity, might interrelate with the cognitive components proposed by Fineberg et al. (2014). Research on habitual learning suggests that the balance between habit and goal-directed action control systems might depend upon factors such as stress (Schwabe and Wolf, 2011), while set-shifting deficits are modulated by anxiety (Billingsley-Marshall et al., 2013), and attentional bias toward unhealthy food cues is moderated by emotional eating (Hepworth et al., 2010). Future studies should test whether emotional eating and stress/anxiety interact with compulsivity-related cognitive deficits to predict the emergence of pathological compulsive eating.

Theoretically, the findings of the current review also have implications for our current understanding of eating problems. Specifically, eating disorders, namely, BN and BED, are considered psychiatric disorders, whereas obesity is typically considered a physiological condition. Our finding that eating disorders and obesity share common cognitive alterations related to compulsivity is consistent with the idea that obesity can be better conceptualized as a biobehavioral disorder characterized by physiological as well as neural, cognitive and behavioral problems that are present across the spectrum of eating disorders (Volkow and Wise, 2005; Wilson, 2010). However, it should be noted that obesity is a highly heterogeneous disorder, and that the ''compulsive eating'' phenotype, characterized by repetitive bouts, without homeostatic function, with adverse consequences, and as ways to relieve stress, fits some, but not all people with excess weight. Furthermore, we did not include studies on the complete spectrum of eating disorders that may include features of compulsive eating (e.g., BE/purging type Anorexia Nervosa (AN) or Other Specified Feeding or Eating Disorders, Purging Disorder, or Night Eating Syndrome). Nevertheless, our inclusion of disorders is in line with recent reviews on compulsive behavior as a central feature of certain eating disorders (i.e., BED), obesity, and the emerging concept of FA (Moore et al., 2017). In addition, this review focused only on the potential shared cognitive processes, and hence, whether there are overlapping neural and behavioral processes related to compulsivity across the spectrum of eatingrelated issues is yet to be determined. Importantly, the four cognitive domains of compulsivity are proposed to have distinct neural correlates. Although it was beyond the scope of the current review, future studies should aim to examine the neural underpinnings of the cognitive domains in an eating context.

Finally, we consider the practical relevance of these findings, including consideration of how compulsivity has typically been examined in the eating domain and the limitations of such methodological approaches. First, the cognitive tasks used in the reviewed studies have been borrowed from other fields, and thus, some tasks were used to measure multiple constructs (i.e., inhibition and setshifiting) or were not clearly operationalized in the context of compulsivity. Thus, future studies should use cognitive tasks specificially developed to measure the different components of compulsivity. Second, most of the reviewed studies examined group differences (i.e., clinical vs. healthy controls) in compulsivity-related cognitive performance. However, few studies investigated the relationship between performance on cognitive tasks and compulsive behavioral tendencies. Thus, future studies should include self-report questionnaires measuring phenotypic descriptions of compulsive behavior, including the Obsessive Compulsive Eating Scale (Niemiec et al., 2016) or the Creature of Habit Scale (Ersche et al., 2017).

In addition, there was a lack of experimental studies on compulsivity-related cognitive drivers of FA, despite its emerging conceptualization as a disorder characterized by compulsive eating behavior (Davis, 2017). Therefore, it is unclear whether so-called FA shares overlapping impairments in compulsivityrelated cognitive functioning with BN, BED and obesity. Indeed, most of the research on FA has focused on the clinical symptoms as measured with the YFAS; however, some recent studies have recently reported impaired impulsive action (i.e., go/no-go responses; Meule et al., 2012) and choice (i.e., delay discounting; VanderBroek-Stice et al., 2017) in FA. Future studies should examine compulsivity-related cognitive processing in FA to determine whether it is similarly characterized by such deficits.

A further limitation of the reviewed literature is that the studies relied heavily on cross-sectional rather than longitudinal designs. Therefore, the chronology of the cognitive components driving compulsivity in eating-related populations remains unclear. Specifically, cognitive performance deficits may be linked to the development and maintenance of compulsive eating behavior, and in turn, eating-related conditions. For example, it may be that an inefficient ability to adapt behavior after negative feedback or greater attentional engagement toward food cues confers increased risk of developing compulsive eating. Alternatively, these deficits may be a consequence of compulsive eating and as such, linked to the prognosis of the eating-related conditions and treatment outcomes. We hypothesize that this is likely a dynamic process in which there are trait vulnerabilities to develop compulsive eating behavior that are then exacerbated through reinforcement and maladaptive learning mechanisms. Future prospective and longitudinal studies should examine whether compulsivity is a vulnerability factor, which predates the development of obesity or eating disorders, or whether it overlaps with the onset of clinical symptoms, or both. It is also important to determine whether problematic eating behavior reflects a transition from impulsivity to compulsivity, as has been proposed in addiction models (Everitt and Robbins, 2016). Further to this point, the current review focused on studies that examined compulsivityrelated cognitive processes, so we did not review evidence for impulsivity-related cognitive processes. Thus, it is not clear how cognitive processes underlying impulsivity and compulsivity are related in the context of eating-related behaviors, or how they might interact with other processes such as decisionmaking.

Based on the aforementioned limitations, we make several recommendations for future research. First, future studies should examine all four compulsivity-related cognitive components within the same study in a particular population (e.g., patients with BED), rather than examining only discrete components. In parallel, research should examine these four components trans-diagnostically in the context of eatingrelated issues, which would allow us to determine whether there are shared underlying mechanisms driving compulsive eating behavior across disorders. Furthermore, some of the cognitive processes reviewed (i.e., set-shifting and reversal learning) are sub-components of the higher-order construct, cognitive flexibility (Wildes et al., 2014). Therefore, it would be useful to measure both of these sub-components in a single study to determine whether they interact in predicting compulsive behavior based on the proposed separate neural circuitry (Fineberg et al., 2014). Importantly, examining compulsivity-related cognitive processes at different stages of eating-related issues using prospective or longitudinal designs would enable the prediction of vulnerability to compulsive eating behavior. In addition, longitudinal research would have implications for informing the development of transdiagnostic prevention and treatment strategies designed to improve cognitive functioning, which may be a promising avenue for reducing compulsive behavioral tendencies across a range of disorders.

#### CONCLUSION

The findings of some of the included studies support the notion that impairments in compulsivity-related cognitive components

#### REFERENCES


may characterize a range of eating-related conditions, although the evidence was inconsistent or lacking for some disorders. The mixed findings in most domains likely resulted from divergent cognitive assessment tasks and possible interactions with dietary restraint, anxiety/stress, and emotional eating. Future research should comprehensively examine the cognitive components of compulsivity, include measures of compulsive eating, and use longitudinal designs to inform the clinical prediction of compulsivity-related symptoms and the development of interventions for compulsive eating.

#### AUTHOR CONTRIBUTIONS

NK and AV-G contributed to the conceptualization of the review. NK wrote the first draft of the manuscript. NK, EA and AV-G wrote sections of the manuscript. All authors contributed to manuscript revision, read and approved the submitted version.

#### FUNDING

NK was supported by a Faculty of Medicine, Nursing and Health Sciences Bridging Postdoctoral Fellowship from Monash University, Melbourne, VIC, Australia. EA was supported by a Food, Cognition and Behavior grant from the Netherlands Organization for Scientific Research (Nederlandse Organisatie voor Wetenschappelijk Onderzoek, NWO, grant 057-14-001). AV-G was supported by a Next Generation Clinical Researchers Career Development Fellowship Level II from the Australian Medical Research Future Fund (MRF1141214) and received a project grant (GNT1140197) from the National Health & Medical Research Council.


control following dopamine precursor depletion. Psychopharmacology 219, 621–631. doi: 10.1007/s00213-011-2563-2


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Kakoschke, Aarts and Verdejo-García. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Erratum: The Cognitive Drivers of Compulsive Eating Behavior

#### Frontiers Production Office\*

Frontiers Media SA, Lausanne, Switzerland

Keywords: compulsivity, cognitive functioning, eating behavior, obesity, bulimia nervosa, binge eating, food addiction

#### **An Erratum on**

#### **The Cognitive Drivers of Compulsive Eating Behavior**

by Kakoschke, N., Aarts, E., and Verdejo-García, A. (2019). Front. Behav. Neurosci. 12:338. doi: 10.3389/fnbeh.2018.00338

#### Approved by:

Frontiers Editorial Office, Frontiers Media SA, Switzerland

#### \*Correspondence:

Frontiers Production Office production.office@frontiersin.org

#### Specialty section:

This article was submitted to Pathological Conditions, a section of the journal Frontiers in Behavioral Neuroscience

> Received: 12 September 2019 Accepted: 12 September 2019 Published: 01 October 2019

#### Citation:

Frontiers Production Office (2019) Erratum: The Cognitive Drivers of Compulsive Eating Behavior. Front. Behav. Neurosci. 13:230. doi: 10.3389/fnbeh.2019.00230 Due to a typesetting error, the citations forJanssen et al. (2017) were removed from the final version. The citations should have appeared in the **Attentional Bias/Disengagement** section, paragraph two, ". . . (body Mass index, BMI and abdominal fat; Janssen et al., 2017)", in the **Habit Learning** section, paragraph two, ". . .which suggests that these two systems are unbalanced (Horstmann et al., 2011; Janssen et al., 2017).", and in the **Discussion** section, paragraph five, ". . . outcome devaluation via satiation (Horstmann et al., 2011; Janssen et al., 2017)."

The publisher apologizes for this error. The original article has been updated.

#### REFERENCES


Copyright © 2019 Frontiers Production Office. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Improvements in Attention Following Cognitive Training With the Novel "Decoder" Game on an iPad

George Savulich<sup>1</sup> , Emily Thorp<sup>1</sup> , Thomas Piercy <sup>1</sup> , Katie A. Peterson<sup>2</sup> , John D. Pickard<sup>2</sup> and Barbara J. Sahakian<sup>1</sup> \*

<sup>1</sup>Department of Psychiatry and Behavioural and Clinical Neuroscience Institute, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom, <sup>2</sup>Department of Clinical Neurosciences, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom

Work and study increasingly rely on the use of technologies requiring individuals to switch attention rapidly between emails, texts and tasks. This has led to healthy people having problems of attention and concentration and difficulties getting into the "flow," which impedes goal attainment and task completion. Possibly related to this, there is an increasing diagnosis of attention deficit hyperactivity disorder (ADHD) and prescriptions of drugs such as methylphenidate. In addition to ADHD, attention is impaired in other neuropsychiatric disorders, such as schizophrenia and in traumatic brain injury (TBI). Based on neuropsychological and neuroimaging evidence, we developed "Decoder," a novel game for targeted cognitive training of visual sustained attention on an iPad. We aimed to investigate the effects of cognitive training in 75 healthy young adults randomly assigned to a Cognitive Training (8 h of playing Decoder over 4 weeks; n = 25), Active Control (8 h of playing Bingo over 4 weeks; n = 25) or Passive Control (continuation of activities of daily living; n = 25) group. Results indicated that cognitive training with Decoder was superior to both control groups in terms of increased target sensitivity (A') on the Cambridge Neuropsychological Test Automated Battery Rapid Visual Information processing (CANTAB RVP) test, indicating significantly improved sustained visual attention. Individuals playing Decoder also showed significantly better performance on the Trail Making Test (TMT) compared with those playing Bingo. Significant differences in visual analogue scales were also found between the two gaming groups, such that Decoder received higher ratings of enjoyment, task-related motivation and alertness across all hours of game play. These data suggest that cognitive training with Decoder is an effective non-pharmacological method for enhancing attention in healthy young adults, which could be extended to clinical populations in which attentional problems persist.

Keywords: enhancement of attention, problems of attention/concentration, maintaining flow, cognitive training, Decoder game

#### Edited by:

Margarita Moreno, University of Almería, Spain

#### Reviewed by: Unai Diaz-Orueta, Maynooth University, Ireland Olga M. Bazanova, State Scientific-Research Institute of

Physiology & Basic Medicine, Russia \*Correspondence:

> Barbara J. Sahakian bjs1001@medschl.cam.ac.uk

Received: 28 September 2018 Accepted: 07 January 2019 Published: 21 January 2019

#### Citation:

Savulich G, Thorp E, Piercy T, Peterson KA, Pickard JD and Sahakian BJ (2019) Improvements in Attention Following Cognitive Training With the Novel "Decoder" Game on an iPad. Front. Behav. Neurosci. 13:2. doi: 10.3389/fnbeh.2019.00002

### INTRODUCTION

Neuropsychiatric disorders are disorders of cognition, motivation and their interaction (Sahakian, 2014). The most common disorder of neurodevelopmental origin is attention deficit hyperactivity disorder (ADHD), which affects up to 1 in 20 children in the United States (Faraone et al., 2003). In 2011, 11% of children aged 4–17 had received a diagnosis of ADHD (i.e., 6.4 million children; Visser et al., 2014). In approximately 80% of children with ADHD, symptoms persist into adolescence, with 60% still having ADHD as an adult (Kessler et al., 2005b). The most prominent cognitive deficit in ADHD is in attention or concentration, which was recognized as a ''subtype'' of the disorder in DSM-IV (American Psychiatric Association, 1994; Epstein and Loren, 2013), but has changed to a ''presentation'' in DSM-5 (American Psychiatric Association, 2013). Other problems in ADHD include deficits in working memory and increased impulsivity (Chamberlain et al., 2011, 2017). Although the degree to which core attentional impairments contribute to deficits in other cognitive domains is not well understood, all involve right frontal cortical dysfunction (Clark et al., 2007).

Methylphenidate, a common treatment for ADHD, and other ''cognitive-enhancing'' drugs such as modafinil are now frequently used at University and in the workplace to stay alert, maintain concentration and increase task-related motivation (Maher, 2008; Sahakian et al., 2015; Brühl and Sahakian, 2016; d'Angelo et al., 2017). A recent publication by Maier et al. (2018) reported the increasing ''lifestyle'' use of these drugs by healthy people for improving cognitive performance when studying or at work. Sahakian et al. (2015) and Brühl and Sahakian (2016) further reviewed the reasons for this increasing lifestyle use, including competition in a global environment; stress and frequent travel leading to poor quality sleep and jet lag; work where even small mistakes can have major consequences; and getting ''into the flow'' of work. It has been emphasized that the emergence of new technologies requiring rapid responses to emails and texts and the working on multiple projects simultaneously, that young people including students are having more problems with sustaining attention and frequently become distracted (Gazzaley and Rosen, 2016). This may be why the Care Quality Commission (2016) reported an over 50% increase in prescriptions for methylphenidate over a 5-year period. They attributed this rise to two main factors: increasing prescriptions for diagnosed childhood and adult ADHD and to its potential for diversion and misuse. As attention/concentration is an important cognitive domain for healthy people to function at work, home or University successfully, and because it is a core deficit in ADHD and in other neuropsychiatric disorders such as schizophrenia and in people with traumatic brain injury (TBI), the current study focused on its enhancement through cognitive training using a game. Whereas pharmacological cognitive enhancement raises a number of safety and ethical concerns in healthy people, non-pharmacological strategies offers low-risk, non-invasive alternatives (Savulich et al., 2017a).

Systematic reviews and meta-analyses have shown that cognitive training is an effective behavioral intervention in healthy people and in those with psychiatric disorders, although the degree of generalization to domains other than the one being trained is not always clear (e.g., Wykes et al., 2011; Kelly et al., 2014). Cognitive training is designed to stimulate cognitive function over time, leading to neuroplastic changes and improved functioning of the underlying neural network (Keshavan et al., 2014). In healthy people, cognitive training of working memory has been shown to increase dopamine receptor density and D<sup>1</sup> binding potential in prefrontal and parietal regions of the brain (McNab et al., 2009; Klingberg, 2010). In healthy older adults, training-related cognitive outcomes have shown associations with increased hippocampal activation (Kirchoff et al., 2012) and white matter integrity in the ventral attention network (Strenziok et al., 2014). Studies from our own laboratory have shown simultaneous effects on cognition and motivation following targeted cognitive training with evidencebased memory games (e.g., ''Wizard<sup>1</sup> ''), including patient groups with schizophrenia (Sahakian et al., 2015) and amnestic mild cognitive impairment (aMCI; Savulich et al., 2017b). In these studies, gaming technology was used to maximize cognitive training by maintaining high levels of enjoyment and task-related motivation, with no participant dropping out. Higher duration of cognitive training with a game was also associated with better memory outcomes in professional rugby players at high-risk of concussion (Sahakian et al., 2018).

In this study, we present data from healthy people who played a novel game called ''Decoder'' on an iPad in order to cognitively train sustained attention. Original features are the extensive neuropsychological and neuroimaging evidence-base for this game (Coull et al., 1996; Aron et al., 2003; Hampshire et al., 2010; del Campo et al., 2013; Pironti et al., 2014). In addition, Decoder was developed as a game for cognitive training in collaboration with a games developer to ensure that it maintained motivation with continued play and that it was individually titrated for difficulty. Throughout gameplay, gangs and missions are procedurally generated: each player gets their own unique game world. Each player can unlock regions and progress through the game in their own way. Decoder was designed for play on an iPad with a view to transfer to mobile phones for greater accessibility and utilization in any environment (e.g., at work, home, bus or train). As control groups for the Decoder training group, we included a group that played ''Bingo'' on an iPad for the same length of time (active control group) and a group that was assessed at baseline and retested at the conclusion of the study (passive control group). We hypothesized that only the group who played the Decoder game would improve their performance following cognitive training as assessed by target sensitivity (A') on the Cambridge Neuropsychological Test Automated Battery Rapid Visual Information processing (CANTAB RVP) test, a reliable and objective measure of sustained attention. To determine whether cognitive training of concentration had any deleterious effects on rapid shifts of attention, a different form of attention, we also examined performance on the Trail Making Test (TMT; Reitan and Wolfson, 1985). In neuroimaging studies,

<sup>1</sup>www.peak.net/advanced-training

performance on this task has been shown to activate the left dorsolateral prefrontal cortex and areas involved in motor control (Zakzanis et al., 2005). We predicted that playing Decoder would have no effect on TMT performance.

### MATERIALS AND METHODS

#### Participants

Seventy-five participants were recruited from the local Cambridgeshire area. Inclusion criteria were fluency in English; age 18–30; not currently taking any psychiatric medication or receiving a psychological treatment; and not having a current or past psychiatric diagnosis. All participants were screened on these criteria using the Mini-International Neuropsychiatric Inventory (Lecrubier et al., 1997) and the Adult ADHD Self-Report Scale (Kessler et al., 2005a).

#### Neuropsychological Assessment

#### The National Adult Reading Test (Nelson, 1982; NART)

The National Adult Reading test (NART) is a 50-item estimate of premorbid intelligence. Participants are instructed to read aloud 50 words of atypical pronunciation. Higher scores (0–50) indicate more correct responses (i.e., a higher estimate of intelligence).

#### Cognitive Measures of Attention

#### CANTAB Rapid Visual Information Processing Test<sup>2</sup> ; (RVP)

The RVP test is a measure of visual sustained attention. Participants are asked to detect sequences of digits (e.g., 2-4-6, 3- 5-7, 4-6-8). A white box appears in the middle of screen, of which digits from two to nine appear in a pseudo-random order, at a rate of 100 digits per minute. Participants are instructed to press a button every time they detect a sequence. The main outcome measure is A', a signal detection measure of sensitivity to the target, regardless of response tendency. The duration of the test is approximately 5 min.

#### Trail Making Test (Reitan and Wolfson, 1985; TMT)

The TMT is a measure of attention, visual search, scanning, processing speed and task switching. Participants are asked to connect a set of 25 targets as quickly but as accurately as possible in a sequential order. In Part A, the targets are numbers (e.g., 1, 2, 3, etc.) and in Part B, the targets alternate between numbers and letters (e.g., 1, A, 2, B, 3, C, etc.). Errors are corrected by the experimenter during the task. The main outcome measure is the time to completion (seconds).

#### Personality Trait Measures

All participants completed a battery of baseline personality trait measures including the Barratt Impulsiveness Scale (BIS; Barratt, 1994), the Impulsive Behavior Scale (UPPS-P; Whiteside and Lynam, 2001), the Sensation Seeking Scale (SSS; Zuckerman et al., 1964), the Apathy Evaluation Scale (AES; Glenn, 2005), the Beck Depression Inventory (BDI; Beck et al., 1996) and the Behavioral Inhibition/Avoidance Scale (BIS/BAS; Carver and White, 1994).

### Decoder

Decoder was developed in collaboration between the research team including a professional games developer and healthy young adults, with later engagement of patients with acquired brain injuries to ensure that it was fun, motivational, easy to understand and in line with a concept suitable for both healthy samples and clinical samples in line with public and patient involvement in order to enhance uptake. Our target cognitive process for enhancement was sustained attention (i.e., concentration). The object of the game is to assume the role of a Signal Intelligence officer tasked with breaking up global criminal gangs. To do this, the player must decode their communication by engaging in a visual sustained attention task using different combinations of number strings among distractors, in which the success of each mission is rewarded by exposing letters of the next criminal location (with higher scores revealing more letters). Each location generates a maximum of three missions, which are titrated for individual performance in real-time. High levels of enjoyment and motivation were prioritized during development of the game. Immediate engagement is achieved through the use of visual feedback and music to give the game a sense of flow, responsiveness and excitement. To keep the game motivating across several plays, a large number of variants, including short- (e.g., unlocking new regions) and long-term (e.g., earning all ranks) goals, are used. Personalization is achieved through selection of a character and backstory.

#### Procedure

This study received full ethical approval from the University of Cambridge Psychology Research ethics committee (reference Pre.2015.092). All participants provided written informed consent. This study comprised a three-group, randomized controlled design, including: the Cognitive Training Group (''Decoder''), the Active Control Group (''Bingo'') and the Passive Control Group (No Game). Potential participants were first screened for inclusion/exclusion criteria by telephone. Invited participants wishing to participant were then asked to attend a baseline testing session, in which they provided basic demographic information and completed the cognitive test battery, followed by all personality trait questionnaires. Participants were also asked about their technology use including the frequency of time spent using the Internet each week, frequency of playing computer games for enjoyment and prior regular use of cognitive training games or applications with the aim of improving cognition. Any participants reporting current use of any cognitive training games or applications were asked to stop throughout the duration of the study. Participants were then randomized to one of three groups.

Participants in the Cognitive Training Group were invited to attend eight, 1 h sessions of supervised cognitive training (i.e., play Decoder on an iPad, as described above). Similarly, participants in the Active Training Group were invited to attend

<sup>2</sup>www.cambridgecognition.com

eight, 1 h sessions of supervised Bingo on an iPad. Bingo was selected as the ''active control'' game because it uses the same type of stimuli as Decoder (i.e., number strings), but does not contain any element of cognitive training of sustained attention. In this version, participants were required to use a very low level of visual search, with reward only given when obtaining five numbers in a row. After each hour of gameplay, participants in the Cognitive Training and Active Control Groups were asked to rate their experience in terms of enjoyment, desire to continue, alertness and positive mood using 10-cm visual analogue scales. Participants in the Passive Control Group were not given access to either game and continued their daily lives as usual.

A maximum of 4 weeks after the baseline testing session, all participants then completed an outcome testing session, which was identical to the baseline testing session in terms of outcome measures.

#### Statistical Analyses

Demographic and baseline questionnaire measures were analyzed using one-way analysis of variance (ANOVA) for continuous variables and chi-square for categorical variables. Baseline and outcome variables were analyzed with an analysis of covariance (ANCOVA). As predictors, we used the participant's baseline scores on the same test (a continuous covariate) and Group (a fixed factor with three levels: Decoder, Bingo and No Game). For the CANTAB RVP, we also included prior regular cognitive training games/applications as a predictor (a fixed factor with two levels: Yes/No). This approach allowed us to assess the effects of gaming (Group) whilst controlling for within-group variance in pre-existing cognitive performance as well as the potential effects of other games/applications with purported cognitive benefits delivered using an iPad or mobile phone. Pairwise comparisons were made between groups following significant effects of gaming. Sidak correction was applied for adjustment of multiple comparisons.

#### RESULTS

#### Demographic Variables

The Decoder (n = 25), Bingo (n = 25) and No Game (n = 25) groups did not differ in basic demographic variables including age, gender, years in education and premorbid intelligence (**Table 1**). The groups also did not differ in personality traits including impulsiveness, impulsive behavior, sensation seeking, apathy, depression or behavioral avoidance/inhibition (**Table 1**). By design, frequencies of Internet use (X <sup>2</sup> = 0.25) and computer game play (X <sup>2</sup> = 0.33) were not significantly different between groups.

#### Attention

#### CANTAB RVP

Target sensitivity (A') significantly differed between groups at outcome, F(2,68) = 5.60, p = 0.006, η <sup>2</sup> = 0.14 (group means adjusted for baseline performance: Decoder = 0.96, SE = 0.01; Bingo = 0.93, SE = 0.01; No Game = 0.93, SE = 0.01; **Figure 1**). Pairwise comparisons showed that target sensitivity significantly differed between Decoder and Bingo, p = 0.04 and Decoder and No Game, p = 0.009, but not between Bingo and No Game, p = 0.95. There was an effect of regular cognitive training game/application experience, such that target sensitivity was significantly better in those without prior experience, p < 0.001 (group means adjusted for baseline performance:

FIGURE 1 | Target sensitivity [Cambridge Neuropsychological Test Automated Battery Rapid Visual Information processing (CANTAB RVP) A'] significantly differed between groups at outcome when controlling for baseline performance, F(2,68) = 5.60, p = 0.006. The Decoder group showed better detection of target sequences, with adjusted means differing significantly between Decoder and Bingo, p = 0.04, and Decoder and Control, p = 0.009. ∗ Indicates means significantly differed between groups.


Notes: NART, National Adult Reading Test; BIS, Barratt Impulsiveness Scale; UPPS, Impulsive Behavior Scale; SSS, Sensation Seeking Scale; AES, Apathy Evaluation Scale; BDI-II, Beck Depression Inventory-II; BIS/BAS, Behavioral Inhibition/Avoidance Scale.

with experience = 0.92, SE = 0.01; without experience = 0.96, SE = 0.96).

#### TMT

Time (seconds) taken to complete TMT numbers and letters was significantly different between groups at outcome, F(2,71) = 3.72, p = 0.03, η <sup>2</sup> = 0.06 (group means adjusted for baseline performance: Decoder = 32.28, SE = 2.02; Bingo = 39.73, SE = 2.02; No Game = 37.92, SD = 2.01; **Figure 2**). Pairwise comparisons showed that time to completion was significantly different between Decoder and Bingo, p = 0.03. Time to completion was not significantly different between Decoder and No Game, p = 0.15 or Bingo and No Game, p = 0.90.

The same pattern of result emerged when including gender as a covariate in the above models; there was no effect of gender for either task (p = 0.64 and p = 0.65).

#### Visual Analogue Scales

Across all hours of gameplay, the two game groups significantly differed in their average levels of enjoyment (Decoder = 6.57, SD = 2.08; Bingo = 4.70, SD = 2.18; t(48) = 3.11, p = 0.003), task-related motivation (Decoder = 7.07, SD = 2.12; Bingo = 4.64, SD = 2.77; t(48) = 3.59, p = 0.001) and alertness (Decoder = 6.35, SD = 2.12; Bingo = 4.59, SD = 2.18; t(48) = 2.90, p = 0.006). Positive mood did not significantly differ between the two groups after the 1st hour of game play (Decoder = 6.08, SD = 2.31; Bingo = 6.20, SD = 2.18; t(48) = 0.19, p = 0.85), but gradually increased for Decoder, and decreased for Bingo, and significantly differed after the 8th h of game play (Decoder = 6.92,

SD = 5.12; Bingo = 5.12, SD = 2.20; t(48) = 3.05, p = 0.004; **Figure 3**).

#### DISCUSSION

The main finding of this study was that attention/concentration was significantly improved by playing the Decoder game on an iPad, which was developed for cognitive training. This enhancement in attentional performance by the Cognitive Training Group was seen both in comparison to the passive control group and also the active control group who played Bingo for the same time period. In addition, subjective measures of enjoyment, motivation, alertness and positive mood remained at high levels in those who played Decoder after every hour of gameplay.

Attention prioritizes sensory processing according to task relevance (Sarter et al., 2001). Focused attention involves the ability to respond to specific stimuli (Commodari, 2017). It plays a key role in many cognitive functions such as problem solving and reasoning (Sohlberg and Mather, 1989). Divided attention refers to the optimal allocation of resources between different sets of input by rapid shifting of attentional focus, in an attempt to process information in parallel from multiple sources (Parasuraman, 1998). Different attentional processes have separate, but also overlapping neural networks (Nebel et al., 2005; Petersen and Posner, 2012; Esterman et al., 2014). It is possible that by improving one form of attention that it is at the expense of impairing another form of attention. Gazzaley and Rosen (2016) have argued that current styles of working and the multifaceted way that we engage with the environment is affecting our ability to sustain attention under conditions of distraction and interruptions which are often technology-related. Importantly, training of sustained attention/concentration did not impair performance on the TMT, which favors shifts in attention. Indeed, if anything, gameplay had a beneficial effect on this form of attention also. Decoder in the present study was aimed at healthy young adults wanting to enhance their attention or in those who easily get distracted. While we used iPads, the Decoder game will be available for all iPhone and iPad users of Peak Brain Training<sup>3</sup> in January 2019. In addition, it has been noted that healthy women have attentional problems specifically associated with the menstrual phase (Merritt et al., 2007; Pletzer et al., 2014, 2017), which might be improved by attention/concentration training.

While healthy young adults were used as participants in the current study, the aim of cognitive training with Decoder for future studies is to assess whether it can improve attentional performance, a key cognitive domain that is impaired in ADHD, schizophrenia and brain injury (Pironti et al., 2014; Lustig and Sarter, 2016; Savulich et al., 2018). The CANTAB RVP test, the main outcome of the present study, has been shown to utilize a fronto-parietal neural network (Coull et al., 1996). Using this test, attention has been shown to be a cognitive endophenotype in ADHD, as performance was impaired in

<sup>3</sup>www.peak.net

both adult patients with ADHD and also their unaffected first degree relatives, compared with healthy volunteers (Pironti et al., 2014). Furthermore, neuroanatomical abnormalities in gray matter volume in the right inferior frontal gyrus and white matter volume in the caudal portion of the right inferior frontal gyrus and fronto-occipital fasciculus were shared between patients with ADHD and their unaffected relatives (Pironti et al., 2014). Performance on the CANTAB RVP test has also shown to be a useful cognitive endophenotype for ADHD genetic studies over and above other tasks of motor speed and visual search (Gau and Huang, 2014). In neuroimaging and lesion studies in healthy people and in patients with ADHD, the right inferior frontal gyrus has been identified as a key area in the neural circuitry underlying response inhibition (Aron et al., 2003; Hampshire et al., 2010; Fan et al., 2018). Future research could determine the underlying neural network changes following attention/concentration and flow training with Decoder. For children with ADHD, there are a few games aimed at self-regulation and social cognition (Prins et al., 2013; Bul et al., 2015).

In addition to ADHD, attentional function is commonly impaired following concussion and TBI (Stierwalt and Murray, 2002). Patients with TBI have been shown to have impairments in sustained attention as measured by the CANTAB RVP (Salmond et al., 2005; Manktelow et al., 2017). The gamification of cognitive training would also have advantages for patients with problems of apathy or negative symptoms, as it can maintain motivation during the training period (see e.g., Sahakian et al., 2015; Savulich et al., 2017b).

In conclusion, we have demonstrated significant improvements in healthy young adults in attention following cognitive training with a game on an iPad developed specifically for this purpose, using evidence-based neuropsychological and neuroimaging studies. It may be that this will be a suitable game for those healthy individuals who find sustaining attention difficult in the workplace and will assist them in maintaining the ''flow'' despite distraction and interference in the environment.

#### AUTHOR CONTRIBUTIONS

ET, GS and KP recruited and tested the participants. TP and BS designed the Decoder game and TP programmed it. GS and BS analyzed and interpreted the data. BS and JP supervised the study. All authors wrote the manuscript.

#### FUNDING

This research was supported by the NIHR Brain Injury Health Care Technology Co-operative (HTC) and the NIHR MedTech and in vitro diagnostic Co-operative (MIC). GS is funded by the Wallitt Foundation and Eton College, with support from the NIHR Cambridge Biomedical Research Centre (BRC) Mental Health theme. TP is funded by the NIHR Cambridge Biomedical Research Centre (BRC) Mental Health theme. BS received funding from the NIHR Cambridge Biomedical Research Centre (BRC) Mental Health theme.

#### REFERENCES


Faraone, S. V., Sergeant, J., Gillberg, C., and Biederman, J. (2003). The worldwide prevalence of ADHD: is it an american condition? World Psychiatry 2, 104–113.


**Conflict of Interest Statement**: BS consults for Cambridge Cognition and Peak. We have technology-transferred the App to Cambridge Enterprise who intends to technology-transfer the App to the games company Peak so that it can become widely available for use on mobile devices. This has not occurred yet.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Savulich, Thorp, Piercy, Peterson, Pickard and Sahakian. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Sub-dimensions of Alcohol Use Disorder in Alcohol Preferring and Non-preferring Rats, a Comparative Study

Ana Domi, Serena Stopponi, Esi Domi † , Roberto Ciccocioppo\* and Nazzareno Cannella

Pharmacology Unit, School of Pharmacy, University of Camerino, Camerino, Italy

#### Edited by:

David Belin, University of Cambridge, United Kingdom

#### Reviewed by:

Marco Venniro, National Institute on Drug Abuse (NIDA), United States Nathan James Marchant, University Medical Center Amsterdam, Netherlands

#### \*Correspondence:

Roberto Ciccocioppo roberto.ciccocioppo@unicam.it

#### †Present address: Esi Domi,

Center for Social and Affective Neuroscience, Institute for Clinical and Experimental Medicine, Linkoping University, Linkoping, Sweden

> Received: 16 October 2018 Accepted: 07 January 2019 Published: 30 January 2019

#### Citation:

Domi A, Stopponi S, Domi E, Ciccocioppo R and Cannella N (2019) Sub-dimensions of Alcohol Use Disorder in Alcohol Preferring and Non-preferring Rats, a Comparative Study. Front. Behav. Neurosci. 13:3. doi: 10.3389/fnbeh.2019.00003 Recent animal models of alcohol use disorder (AUD) are centered in capturing individual vulnerability differences in disease progression. Here, we used genetically selected Marchigian Sardinian alcohol-preferring (msP) and Wistars rats to apply a multidimensional model of AUD adapted from a previously described DSM-IV/DSM-5 multisymptomatic cocaine addiction model. As proof of concept, we hypothesized that msP rats, genetically selected for excessive drinking, would be more prone to develop dependence-like behavior compared to Wistars. Before exposure of animals to alcohol, we monitored basal anxiety in the elevated plus maze (EPM). Animals were then trained in prolonged operant alcohol self-administration, consisting of 30-min daily sessions for 60 days in total. Each session consisted of two 10-min periods of alcohol reinforcement separated by 10-min interval of non-reinforcement. Following training, we applied three criteria of individual vulnerability for AUD: (1) persistence of lever pressing for alcohol when it was not available; (2) motivation for alcohol in a progressive ratio (PR) schedule of reinforcement; and (3) resistance to punishment when alcohol delivery was anticipated by a foot-shock (0.3 mA). We obtained four groups corresponding to the number of criteria met (0–3 crit). Rats in the 0crit and 1crit groups were characterized as resilient, whereas rats in the 2crit and 3crit groups were characterized as prone to develop a dependent-like phenotype. As predicted, the 2–3crit groups were enriched with msP rats while the 0–1crit groups were enriched in Wistar rats. In further analysis, we calculated the global addiction score (GAS) per subject by the sum of the normalized score (z-score) of each criterion. Results showed GAS was highly correlated with animal distribution within the 3 criteria. Specifically, GAS was negative in the 0–1crit groups, and positive in the 2–3crit groups. A positive correlation between basal anxiety and quantity of alcohol intake was detected in msP rats but not Wistars. In conclusion, we demonstrated that the 0/3criteria model is a suitable approach to study individual differences in AUD and that msP rats, selected for excessive-alcohol drinking, show a higher propensity to develop AUD compared to non-preferring Wistars.

Keywords: alcoholism, ethanol, self-administration, alcohol-seeking, punished responding, DSM-5

## INTRODUCTION

Alcohol Use Disorder (AUD) is associated with increased health risks and social harm with dramatic impact to the global disease burden (Rehm, 2011). In 2014, the World Health Organization reported that alcohol contributes to more than 200 diseases, such as alcohol dependence, liver cirrhosis and cancers, as well as alcohol related injuries (WHO, 2014). In attempting to capture the clinical condition of AUDs, a range of procedures have been developed to model alcohol dependence-related traits in rodents (Spanagel, 2000; Tabakoff and Hoffman, 2000; Hopf and Lesscher, 2014). In humans, addictive behavior is characterized by a shift from recreational to compulsive drug seeking as described in the DSM-IV (American Psychiatric Association, 2000). Long-term alcohol consumption induces neuroadaptations that are associated with loss of control, compulsive drug taking and negative emotional states (i.e., anxiety, depression; Wolffgramm and Heyne, 1995; Koob and Le Moal, 1997; Koob, 2013). Particularly, compulsivity, defined by DSM-IV/5 as use of alcohol despite harmful social, health and economic consequences, is a major component in the transition to alcoholism (Spanagel, 2009; Koob and Volkow, 2010; Hasin et al., 2013; McKim et al., 2016). Over the years, the characterization of different lines of rats and mice genetically predisposed to alcohol drinking have helped to elucidate several aspects of AUD neurobiology (McBride and Li, 1998; Bell et al., 2006; McBride et al., 2014). However, a clear understanding of the factors leading the development of dependence itself is still lacking, including compulsivity associated with disease progression (Crabbe, 2010). Such gaps between animal models and the human condition in AUDs are reflected in the limited efficacy of available pharmacological treatments to attenuate compulsive drinking (Volpicelli et al., 1995; Kranzler, 2000; Franck and Jayaram-Lindström, 2013).

Recent research has been oriented towards the development of preclinical models that more closely mimic the complexity of human alcohol addictive behaviors by going beyond the simple alcohol drinking procedures useful to study alcohol reward. By capturing multiple aspects that define AUD such as compulsive alcohol seeking and the inability to abstain from its use despite negative consequences, these models provide new insights into individual vulnerability to develop addictive-like behaviors (Hopf et al., 2011; Radwanska and Kaczmarek, 2012; Seif et al., 2013; Radke et al., 2017; Augier et al., 2018; Giuliano et al., 2018).

One of the features of drug addiction is the inter-individual vulnerability to lose control of drug consumption. This loss of control depends upon genetics, environment, personality traits, psychiatric comorbidities and the interplay of all these factors (Enoch, 2013; Morrow and Flagel, 2016; Egervari et al., 2018). In both humans and laboratory animals the predisposition to develop addiction-like behavior is present in only a small subpopulation of subjects (Anthony et al., 1994; Piazza and Deroche-Gamonet, 2013). In order to identify the interindividual differences in vulnerability to shift from controlled to compulsive drug intake that define this subpopulation, Deroche-Gamonet et al. (2004) developed a multidimensional animal model of drug addiction (Belin-Rauscent et al., 2016; Deroche-Gamonet et al., 2004; Belin et al., 2008, 2009, 2011). This model characterized a cocaine addiction-prone phenotype in rats, based on the DSM-IV diagnostic criteria of addiction (American Psychiatric Association, 2000), by measuring three traits: (1) inability to refrain from drug seeking; (2) high motivation for the drug; and (3) maintenance of drug use despite negative consequences (Deroche-Gamonet et al., 2004; Kasanetz et al., 2010; Belin et al., 2011). Here, we adapted the DSM-IV/5 based three-criteria model of cocaine addiction to characterize an alcohol-addiction prone phenotype in the rat. We used Marchigian Sardinian alcohol preferring (msP) rats and non-preferring Wistar rats to assess whether a genetic predisposition to ethanol preference contributes to the development of dependence-like behavior. msP rats represent an animal model of genetic predisposition to high ethanol drinking and relapse associated with anxious and depressive-like traits (Ciccocioppo et al., 1999, 2006; Hansson et al., 2006; Cippitelli et al., 2015; Stopponi et al., 2018). Based on these conceptualizations we predicted that msP rats, genetically selected for excessive drinking, would be more prone to develop dependence-like behavior compared to Wistars.

### MATERIALS AND METHODS

### Animals

Experiments were performed using male Wistar (n = 31; Charles River, Calco, Italy) and msP (n = 32; bred at the School of Pharmacy, University of Camerino) rats. Rats weighed 200–250 g at the beginning of the study. Rats were housed in pairs under a reversed 12:12-h light/dark cycle (lights off at 9:00 AM) with constant temperature (20–22◦C) and humidity (45–55%). Food and water were provided ad libitum. Ethanol (95%, Carsetti, Camerino, Italy) was diluted to 10% (v/v) in tap water for chronic, intermittent EtOH exposure and for self-administration behavioral testing. Animals were treated in accordance with the guidelines of the European Community Council Directive for Care and Use of Laboratory Animals. The experimental procedures were approved from the Italian Ministry of Health (authorization n◦ 414/2016-PR).

#### Elevated Plus-Maze

Before being exposed to alcohol, rats were tested in the elevated plus-maze (EPM) to measure anxiety-like traits. The apparatus was constructed of wood and painted black. It consisted of two open arms and two enclosed arms (40 cm high walls) arranged so that the similar arms were opposite each other. The maze, elevated 50 cm above the floor, was located in a sound attenuated room illuminated by a red dim light (∼30 lux). The 5 min test began placing the animal in the center of the maze, facing a closed arm. The number of open and closed-arm entries and the time spent in each arm was recorded. Data were expressed in percentage (open or closed time/total time × 100; open or closed entries/total entries × 100). The percentage of time spent in open arms and the number of open arm entries (with entries defined as placement of all four paws into the respective area) were used as measures of anxiety-like behavior, while the number of total arm entries was used as an indicator of general motor activity (Pellow et al., 1985; Cippitelli et al., 2011; Cannella et al., 2016; Domi et al., 2016; Stopponi et al., 2018).

## Alcohol Training Procedure

Prior to operant responding training, rats were exposed to an intermittent two-bottle choice alcohol drinking procedure (choice between 10% alcohol and water) for 3 weeks. This training protocol was adopted to avoid sucrose fading or water deprivation procedures and facilitate the acquisition of operant responding. Alcohol self-administration was performed in rat operant conditioning chambers (Med Associate St Albans, VT, USA) enclosed in sound-attenuating, ventilated, environmental cubicles. Each chamber was equipped with two retractable levers located in the front panel of the chamber with two stimulus lights placed above each lever in addition to a house light and a tone generator. The operant chambers were controlled, and data collected with MED-PC<sup>r</sup> IV windows-compatible software.

Rats were trained to press the active lever for EtOH 10% on a fixed-ratio 1 (FR1) schedule of reinforcement until a stable baseline was reached (one daily sessions for 7 days). Animals were then moved to a FR3 schedule of reinforcement until addiction criteria were tested. Training sessions were 30 min in duration, during which a 10-min reward-available period (drug-period) was followed by a 10-min reward-unavailable period (no-drug-period) which was followed by a second 10-min drug-period. Pressing the right (active) lever during the drug period resulted in the delivery of 0.1 ml of 10% ethanol in a receptacle connected to a syringe pump, which was followed by the activation of a cue light above the lever for 5 s and a 10 s time-out period. During the no-drug period, signaled by activation of the house light, pressing the active lever had no consequences. Responses on the left or ''inactive'' lever were recorded during the entire session but did not result in any programmed consequences.

### Evaluation of the Three Criteria for AUD Like-Behavior

At completion of the training we applied three main criteria to monitor individual vulnerability for AUD:


(i.e., the number of lever responses or the ratio required to receive one dose of 10% ethanol) was increased as follows: for each of the first four ethanol deliveries the ratio was increased by 1; for the next four deliveries the ratio was increased by 2 and for all of the following deliveries the ratio was increased by 4 (1, 1, 1, 1, 2, 2, 2, 2, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 60, 64, 68, 72 etc.; Economidou et al., 2006). Each alcohol delivery was paired with a 5 s illumination of the cue light. Sessions were terminated when 30 min had elapsed since the last reinforced response. The maximal number of responses that a rat produced to obtain one infusion was referred to as the break point.

3. To measure resistance to punishment, rats were placed for 10 min (corresponding to the first drug-period of a standard training session) in the SA chamber were the punishment was a foot-shock (0.3 mA, 0.5 s). The intensity of the shock was chosen based on a pilot study in which different cohorts of msP and Wistar rats were exposed to shock intensities of 0.1 mA, 0.3 mA and 0.6 mA. Results showed that rats were not sensitive to 0.1 mA intensity while at 0.6 mA foot shock completely abolished alcohol self-administration in all the animals. Here, in a FR3 schedule, the first active lever press led to the illumination of a new, different stimulus light (green light), signaling the presence of a shock session. The second active lever press produced a foot-shock of 0.3 mA via a metal grid connected to a shock generator. The third active lever press produced the delivery of 0.1 ml of 10% ethanol associated with the cue light. If within a minute, animals did not complete an FR3 the green light turned off and the sequence was reinitiated.

PR and punishment sessions were performed on days 45 and 55 respectively.

A rat was considered positive for a particular addiction-like criterion when the score for this behavior was in the top 34% percent of the distribution. This criterion was arbitrarily chosen based on seminal work from Deroche-Gamonet et al. (2004) and considering that a change of the selection threshold from 25 to 40% has minimal effect on individual rat-group allocation (Deroche-Gamonet and Piazza, 2014). We obtained four groups of rats (0crit, 1crit, 2crit and 3crit) defined by the number of positive criteria met.

As a second level of analysis, we measured the global addiction score (GAS) by calculating the sum of the normalized score (z-score) of each criterion for each subject (Belin et al., 2009).

### Statistical Analysis

Data are expressed as mean ± standard error (SEM). All behavioral experiments were analyzed by mean of Student's t-test comparison, one-way, factorial or repeated-measures analysis of variance (ANOVAs) and covariance (ANCOVA) according to experimental design. We examined for significant violations for assumptions of homogeneity of variances by using Levene's and Bartlett's test. In case of deviation from homogeneity of the variance was significantly detected, the Mann-Whitney and Kruskal Wallis non-parametric analysis were used (footshock resistance vs. genotype) and (footshock resistance vs. 0, 1, 2 and 3 criteria), respectively. Significant difference was set at p < 0.05. Post hoc comparisons were carried out by Newman-Keuls test when appropriate. To asses the escalation of alcohol seeking during the no-drug period we used a k-means cluster analysis with 10 iterations and with maximization of distances between groups defined a priori as 3. This approach was taken to verify the existence of a subgroup of animals that increased ''active'' lever presses over time (from day 1–44). Moreover, for each rat we calculated the slope of ''active'' responses during the no-drug period over the 44 days (divided in four intervals of 11 days each). Positive values of the slope represent an increase in lever presses over time while negative values reflect a decrease in lever presses over time (Dilleen et al., 2012; Ducret et al., 2016).

### RESULTS

### Anxiety-Like Behavior in msP and Wistar Rats

The msP rats exhibited significantly higher anxiety-like behavior spending less time in the open arms compared to Wistar rats (t(61) = 5.62, p < 0.001; **Figure 1A**). No effect was found in the total number of entries indicating no difference in locomotion (t (61) = 1.88, NS; **Figure 1B**).

#### Acquisition of Alcohol Self-Administration in msP and Wistar Rats

Rats were trained to self-administer alcohol for 60 days. Both msP and Wistar rats acquired and maintained stable alcohol self-administration levels. ANOVA of number of rewards earned revealed a significant effect of line (F(1,61) = 83.3; p < 0.001), significant effect of session (F(59,3599) = 100.31; p < 0.001) and a significant line × session interaction (F(53,3599) = 5.71; p < 0.001). msP rats self-administered significantly more ethanol compared to Wistars. Post hoc Neuman-Keuls test (**Figure 2A**) revealed a significand difference in the following days: day 2 (p < 0.05), 24 (p < 0.01), 3–23 and 25–60 (p < 0.001). Analysis of lever responses over training demonstrated that both msP and Wistar rats discriminated between active and inactive lever and the difference between rat lines was specific to the active lever [line (F(1,60) = 39.32; p < 0.001), lever (F(1,60) = 709.05; p < 0.001), session (F(59,3540) = 138.15; p < 0.001) rat line × lever × session interaction (F(59,3540) = 7, 65; p < 0.001); **Figure 2B**].

## Evaluation of the Three Criteria AUD Like-Behavior

#### Persistence in Alcohol Seeking

Persistence in alcohol seeking was measured as the number of active lever presses occurring during the 10 min of non-drug availability. We used a k-means cluster analysis with the number of clusters set a priori to three and the variables defined as the 44 days of self-administration (i.e., prior to the PR and foot-shock session tests) divided into 4 intervals of 11 days each (Int.1 = day 1–11, Int.2 = day 12–22, Int.3 = day 23–33, Int.4 = day 34–44; **Figure 3A**). In assessing whether during the 44 days of operant training animals had progressively increased their lever presses during the drug free period, ANOVA revealed a significant difference between intervals (F(3,180) = 22.02; p < 0.001) and a significant interval × cluster interaction (F(6,180) = 26.06; p < 0.001). Subjects in cluster 1 (14 cases: 8 msP and 6 Wistar rats) decreased their lever presses over time and were defined as low persistent (LP). Animals in cluster 2 (20 cases:12 msP and 8 Wistar rats) markedly increased persistence to response over time as revealed by the post hoc Neuman-Keuls analysis (Int.1 vs. Int. 4 p < 0.001) and were defined as high persistent (HP). Subjects in Cluster 3 (29 cases: 12 msP and 17 Wistar rats) defined as Intermediate (IM) maintained a consistent rate of lever presses during the 44 days of operant training. For the subsequent measure of this criterion, we considered for each rat the slope of lever presses during the no-drug period over the four intervals of time and compared it between msP and Wistars (**Figure 3B**). Despite the fact that msP rats had higher slope values in average (msP = 1.47 ± 0.39, Wistar = 1.11 ± 0.38), the Student ttest revealed no difference between msP and Wistar rats in persistence to response (t(61) = 0.67, p = ns).

#### Motivation

In PR contingency we compared the breakpoint reached by msP and Wistars by ANCOVA using the average intake during the last three self-administration sessions as covariate. ANCOVA found no significant effect of line (F(1,60) = 7.8; p = NS). However, the breakpoint was higher in msP rats than Wistar rats (msP = 39.75 ± 1.6, Wistar = 28.84 ± 2.06), indicating a higher motivation to self-administer alcohol in the alcohol preferring line. Indeed, since a higher motivation prompts the msP to self-administer higher amount of alcohol both during the acquisition and the test phase, it is not surprising that using the alcohol intake as covariate would cancel the difference in break point. Confirming this interpretation, when intake is not used as covariate a strong significant difference in break point between the two lines is detected (t(61) = 4.2, p < 0.001; **Figure 3C**).

#### Resistance to Punishment

In the punished reward test, animals were presented an aversive stimulus (foot-shock) associated with subsequent administration of alcohol. Observed at group level, punishing of operant responding decreased the motivation for alcohol in both genotypes, but at the individual level the number of rewards self-administered spanned from 0 to 100% of baseline (**Figure 3D**). The average rate of rewards earned in the punished reward tests were 42% ± 4.15 for msP and 39% ± 3.95 for Wistars calculated as the average of the first 10 min of the last four baseline sessions vs. the first 10 min of the punished schedule session. Mann-Whitney U-test revealed no difference between genotypes in alcohol seeking despite punishment expressed in percentage of their baseline (U = −0.85, two-sided exact p = 0.4).

#### Distribution of msP and Wistar Rats by Their Addiction-Like Behavior Score

msP and Wistar rats were scored for each addiction-like behavior and were assigned to a ''positive criterion'' subgroup if their individual score was in the top of 34% of the total distribution. Rats were then separated into four groups depending on the number of positive criteria met (from 0crit to 3crit). msP rats represented the majority of the 3crit [12.69% of the whole rats' cohort, 9.52% were msP (n = 6) and 3.17% were Wistars (n = 2)] and the 2crit groups [19.05% of the whole rats' cohort, 14.19% were msP (n = 9) and 4.76% where Wistars (n = 3)]. Conversely, Wistar rats represented the majority of the 1crit [34.92% of the whole rats' cohort; 12.70% were msP (n = 8) and 22.22% were Wistars (n = 14)] and the 0crit groups [33.33% of the whole rats cohort; 14.29% were msP

FIGURE 3 | (A) K-means clustering with three clusters of the total population (msP and Wistar rats) used to classify animals based on their performance in the drug free period and using as variables the 44 daily sessions divided in four intervals of 11 days each. Cluster 1 defined as low persistent (LP; 14 cases: 8 msP and 6 Wistar rats); cluster 2 defined as high persistent (HP; 20 cases: 12 msP and 8 Wistar rats); cluster 3 defined as intermediate (29 cases: 12 msP and 17 Wistar rats). Values are presented as mean (±SEM). ∗∗∗p < 0.001, significant between Int.1 and Int.4 in Cluster 2. (B) Slope values of the number of active lever presses during the drug free period in msP and Wistar rats over the 44 daily sessions divided in four intervals of 11 days each (Int.1, Int.2, Int.3 and Int.4 as points of the slope line). Values are presented as mean (±SEM). (C) Motivation for 10% ethanol measured by the break point during a progressive-ratio (PR) schedule of reinforcement in msP and Wistar rats. Values are presented as mean (±SEM). ∗∗∗p < 0.001, significant as compared to msP and Wistar rats in the break point achieved. (D) Number of reinforcements in a 10 min foot-shock session for msP and Wistar rats represented as percentage of the baseline (average of the first 10 min of days 42–44). Values are presented as mean (±SEM). The dotted line indicates the 66th percentile of distribution of (B–D).

(n = 9) and 19.05% were Wistars (n = 12); **Figure 4A**]. The criteria for which 1crit and 2crit rats were positive are shown in **Table 1**.

Based on the sum of normalized scores (z-scores) assigned to each criterion (**Figure 4B**), we obtained a GAS for individual rats. The average of this addiction score for each subpopulation was negative for the 0 and 1 crit groups (0crit = −2.07 and 1crit = −0.08) and positive for 2 and 3 criteria groups (2crit = 1.73 and 3crit = 3.1; **Figure 4C**). We found a main effect for addiction scores among the criteria subgroups as revealed by one-way ANOVA (F(3,59) = 37.81; p < 0.001). Neuman-Keuls post hoc test showed a significant difference between each addiction score (0crit vs. 1crit, 2crit and 3crit p < 0.001; 1crit vs. 2crit and 3crit P < 0.001; 2crit vs. 3crit p < 0.05).

### Differences in the Three Measures of AUD-Like Behavior in 0crit, 1crit, 2crit and 3crit Groups

One-way ANOVA applied to persistence in alcohol seeking revealed a significant difference between groups (F(3,59) = 14.83; p < 0.001; **Figure 4D**). The slope of active lever pressing during the no-drug-period was about nil in 0crit (−0.05 ± 0.33) and progressively increased as a function of the criteria met: 1crit (0.85 ± 0.38), 2crit (2.76 ± 0.46) and 3crit (3.87 ± 0.65). Neuman-Keuls post hoc test showed that 3crit rat exhibited higher active lever pressing slope compared to 0crit (p < 0.001) and 1crit (p < 0.001) groups but not compared to 2crit group (p = ns). The 1crit rats differed as well from 2crit rats (p < 0.001) but not from 0crit group (p = ns).

In the motivation for alcohol (**Figure 4E**), one-way ANOVA revealed a significant between groups difference in the breakpoint (F(3,59) = 11.71; p < 0.001). As shown by Neuman-Keuls post hoc test, 3crit rats exhibited higher breakpoint (43.5 ± 1.4) compared to the 0crit (26.48 ± 1.9; p < 0.001) and 1crit (33.36 ± 2.18; p < 0.01) groups but not compared to 2crit rats (44 ± 3.27; p = ns). In addition, the 2crit rats differed from 0crit (p < 0.01) and 1crit (p < 0.05) rats but not from 3crit rats (p = ns).

Kruskal Wallis H test of punished alcohol seeking revealed an overall effect of groups (H 3, N = 63 = 25,064 p < 0.001; **Figure 4F**). Multiple comparisons showed that 3crit rats presented a higher rate of punished reinforcers (71.32 ± 5.27) compared to 0crit (18.92 ± 3.4; p < 0.001), 1crit (47.01 ± 6.4; p < 0.01) and 2crit (46.86 ± 6.75; p < 0.05) rats. Moreover, 2crit group differed from 0crit group (p < 0.01) and 1crit group differed from 0crit group (p < 0.01).

TABLE 1 | Description of the positive criteria met by msP and Wistar rats within the 1crit and 2crit groups.


### Alcohol Consumption and AUD-Like Behavior

We evaluated the relationship between alcohol intake in msP and Wistar rats with the propensity to develop addiction-like behavior vs. resistance (2–3crit vs. 0–1crit) (**Figure 5**). Factorial ANOVA revealed a significant effect of genotype (F(1,59) = 20.77; p < 0.001) and criteria subgroup (F(1,59) = 11.00; p < 0.001) and a significant interaction between genotype and the criteria subgroup on alcohol intake (F(1,59) = 5.53; p < 0.05). Newman-Keuls post hoc test showed that 0–1crit Wistar rats had lower levels of alcohol intake compared to 2–3crit Wistar rats (p < 0.01) while the 0–1crit msP rats did not differ in alcohol intake from the 2–3crit msP rats (p = ns). Moreover, 0–1crit Wistar rats differ from 0 to 1crit msP (p < 0.001) while there is not a statistically significant difference between 2–3crit Wistar and 2–3crit msP rats on alcohol intake (p = ns).

### Anxiety and the Vulnerability to Develop Addiction-Like Behavior

In the literature, high anxiety behavior has been associated with the development of drug addiction (Stewart and Conrod, 2008; Ipser et al., 2015). As revealed by Pearson's analysis, a significant (r = −0.38, p < 0.05) negative correlation between alcohol intake during ethanol self-administration and the percentage of time spent in the open arms was found in msP rats in which high levels of alcohol drinking were associated with higher anxiety. No significant correlation was detected in Wistars (r = −0.04, p = ns). We also compared the three addiction-like criteria with the percentage of time spent in open arms of the EPM. Pearson's analysis indicated no correlation between anxiety levels and the three addiction-like criteria in msP (persistence in alcohol seeking: r = 0.084; p = ns, motivation for alcohol: r = −0.137; p = ns, resistance to

punishment: r = −0.052; p = ns) or Wistar rats (persistence in alcohol seeking: r = 0.072; p = ns, motivation for alcohol: r = 0.066; p = ns, resistance to punishment: r = −0.29; p < 0.001).

#### DISCUSSION

2–3crit msP rats.

Modeling human AUDs in rodents has been a challenge in preclinical studies since alcohol, unlike cocaine or opioids, is a weak reinforcer and requires protracted exposure for the development of dependence (Spanagel and Hölter, 1999). A valid animal model that attempts to reflect the human condition in addiction should capture multiple aspects that characterize substance use disorders (SUDs) such as compulsive drug seeking and taking, increased motivation for the drug, and continued intake despite negative consequences (Hopf and Lesscher, 2014).

In this study, based on the DSM-IV/5 diagnostic criteria, we used a multidimensional model including different behavioral features of addiction to characterize an alcohol addiction prone phenotype in rats. We also sought to evaluate the role of genetic predisposition to excessive drinking in shaping individual variability in developing alcohol addictive like-behavior by comparing alcohol preferring msP rats with non-preferring progenitor Wistar rats.

Our data showed an enhanced propensity of the msP rats to exhibit higher scores in the three defined criteria for individual vulnerability to AUD, as measured in: (1) persistence in alcohol seeking when alcohol is not available; (2) motivation for alcohol in a PR schedule of reinforcement; and (3) resistance to punishment when alcohol delivery is anticipated by a footshock. The percentage of rats positive in all three criteria was approximately 13% with msP rats representing the majority of the 3crit group compared to Wistar rats, 9.52% vs. 3.17%, respectively. The percentage of rats characterized as 3crit is similar to the small proportion of individuals that develop alcohol dependence after protracted exposure (Anthony et al., 1994; Wagner and Anthony, 2002). The 2crit group was also enriched in msP rats, while Wistars constituted the majority of the 0crit and 1crit groups. The inter-individual variability we observed was present not only between but also within genotypes demonstrating that this protocol is a suitable approach to study individual differences in alcohol dependence in largely homogeneous rat populations.

Persistence of alcohol seeking was evaluated daily throughout the training period. We ran a cluster analysis to identify a subgroup of rats that increased their response during the drug free period over time, as other laboratories have failed to capture this behavioral criterion (Waters et al., 2014). By clustering rats based on their persistence in lever pressing in the absence of alcohol availability, we were able to determine a subpopulation of rats (38% of msP and 26% of Wistars) that were highly persistent in lever pressing. Recent studies on rodent models of AUDs have interpreted the intrasession drug free period as a measure of alcohol seeking behavior (Jadhav et al., 2017; Radke et al., 2017). However, in those studies persistence to response when alcohol was not available was assessed only in three to five sessions making it difficult to evaluate how uncontrolled drug seeking develops over time (Belin et al., 2016). Here, in line with earlier studies using the 0/3crit model in cocaine use, we demonstrated that persistence of alcohol seeking is a trait that develops over time but only in a subset of animals.

In assessing motivation for alcohol by using the PR schedule of reinforcement, we found that msP rats, compared to Wistar rats, exhibited increased motivation to self-administer alcohol as shown by higher breakpoints during the PR session (Ciccocioppo et al., 2006). This schedule, more than the persistence of alcohol seeking or resistance to punishment, is linked to consummatory behavior, a trait used to select msP rats. It is not surprising, therefore, to observe such a remarkable difference from non-selected Wistars. Previous studies have also demonstrated that the breakpoint is sensitive to genetic selection procedures (Czachowski and Samson, 2002).

As described in the DSM-IV and DSM-5, compulsive drug seeking or drug-taking despite negative consequences is another hallmark of drug dependence (American Psychiatric Association, 2000, 2013).

A recent preclinical model developed in Cambridge laboratories, has been able to identify a subset of vulnerable individuals that display compulsive alcohol seeking in the face of punishment (Giuliano et al., 2018). By using probabilistic footshock punishment of the seeking response in a seeking-taking chained schedule of reinforcement they were able to distinguish the punisher from the reward. Indeed, it has been demonstrated that when the footshock co-occurs with the drug delivery its effectiveness as a punisher is reduced (Dickinson and Pearce, 1976; Pelloux et al., 2007). Several models use footshock punishment (various shock intensities have been used) paired with the delivery of a constant dose of alcohol (Seif et al., 2013; Jadhav et al., 2017; Radke et al., 2017; Augier et al., 2018). To distinguish the punishing from the reinforcing proprieties of alcohol here, rats were punished with 0.3 mA footshock that preceded alcohol taking response without pairing the shock with ethanol delivery. Results showed that punishing operant responding markedly decreased alcohol seeking in both msP and Wistar rats. This phenomenon was previously described when footshock punishment was used in rats trained in cocaine self-administration (Deroche-Gamonet et al., 2004; Kasanetz et al., 2010). However, individual animals showed different behavioral suppression levels that spanned from 0 to 100% of baseline. Most importantly, in both lines a subgroup of rats continued to self-administer alcohol despite the negative consequences of footshock.

Consistent, with previous works adopting the 0–3crit model of cocaine (Belin et al., 2008, 2011; Kasanetz et al., 2013; Cannella et al., 2017, 2018) and alcohol (Radke et al., 2017) addiction, here we used a single foot-shock session to assess resistance to punishment. As demonstrated by other studies, using multiple punishment sessions could have been an alternative to better separate the population in shock-resistant and shock-sensitive subgroups (Seif et al., 2013; Augier et al., 2018; Giuliano et al., 2018; Marchant et al., 2018). However, a single shock-test session is also informative of individual resistance to punishment. Moreover, a recent work demonstrated that in the context of the 0–3crit model applied to alcohol, multiple shock tests can compress rather than increase the range of distribution of resistance to punishment. This study also revealed that resistance to punishment of the 0crit to 3crit groups decreases over time, although the groups differences are maintained (Jadhav et al., 2017).

As a second level analysis, by summing the normalized score (z-score) applied to single criteria the GAS for individual subject was calculated. Results showed that each criteria group differed significantly in GAS. Specifically, GAS was negative in the 0crit/1crit groups identified as resistant whereas it was positive in the 2crit/3crit groups that were characterized as prone to develop an alcohol dependent-like phenotype. We found a high correlation between the GAS and the distribution of the animals within the three criteria, indicating the interdependence between these two measures.

Epidemiologic studies have suggested that genes play an important role in the vulnerability to alcohol abuse and the subsequent risk to develop alcohol dependence (Crabbe et al., 2006; Mayfield et al., 2008; Schuckit, 2009). In recent years there has been considerable debate on whether geneticallypredisposed alcohol drinking rodents may adequately model AUDs. To this end an ideal genetic animal model of alcoholism should carry the same genetic traits linked to alcoholism in humans, and ideally those traits should correlate with the expression of similar subphenotypic characteristics. One of the genetic traits of msP rats is the over-expression of the corticotropin-releasing factor system, linked to the presence of two single nucleotide polymorphisms (SNPs) of the CRF1 receptor (CRF1-R) gene leading to receptor overexpression (Hansson et al., 2006; Ayanwuyi et al., 2013; Cippitelli et al., 2015). Polymorphisms at level of the promoter region of the CRF1-R gene have been reported in humans with AUDs, suggesting a genetic trait in common with msP rats (Treutlein et al., 2006). Notably, resembling a large subset of alcoholic patients, msP rats also drink excessive amounts of alcohol (7–8 g/kg/day) and exhibit an anxious and depressive-like behavioral phenotype. In msP rats this phenotype is, at least in part, linked to a hyperactive CRF1- R function (Schuckit and Hesselbrock, 1994; Grant et al., 2004; Ayanwuyi et al., 2013; Cippitelli et al., 2015). Here, we confirmed that, compared to unselected Wistars, the msP rats have a higher basal level of anxiety. However, when we attempted to associate open arm time (a measure of anxiety) with animal distribution within the 3 criteria, no significant correlations were detected. This, together with earlier findings, suggest that high anxiety is linked to a genetic predisposition to excessive drinking but not with propensity to develop alcohol abuse traits (compulsive-like alcohol use, persistence in seeking, and motivation). Dilleen et al. (2012) demonstrated recently that a high anxiety trait predicts loss of control over cocaine, but not heroin self-administration, suggesting that outcome may depend on the psychoactive drug used.

msP rats represented the majority of subjects belonging to the 2crit and 3crit groups. However, Wistar rats satisfying the 2/3crit, self-administered as much alcohol as the msPs rats, and significantly more than 0/1crit Wistars. The msP rats in the 0/1crit group consumed the same high amount of alcohol of the 2/3crit group. These data indicate that a genetic predisposition to ethanol preference and excessive drinking may not be necessarily associated with a propensity to develop addictive-like traits modeled by the 0/3crit paradigm used here. Interestingly, in humans it has been shown that a large number (about 90%) of people with excessive drinking habits do not meet the criteria for alcohol dependence (Esser et al., 2014). Our results are therefore in line with human data and with results from another recent study which found that the propensity for P alcohol preferring rats to drink high levels of alcohol was dissociable from the development of compulsive alcohol seeking (Giuliano et al., 2018). Together these data suggest that the study of individual vulnerability is an important approach to investigate AUD, as it is able to dissociate excessive alcohol drinking from the propensity to develop the dependence, thus mimicking the human condition. Ultimately, the study of individual vulnerability and the employment of this model in pharmacological studies may help refine the investigation of novel chemical entities for AUD by exploring their efficacy on specific traits not limited to drinking, and that more closely mimic the human condition.

#### AUTHOR CONTRIBUTIONS

AD and RC were responsible for the study concept and design. AD performed the experiments. AD, NC, RC, ED and SS assisted with the data analysis, interpretation of findings and drafted the manuscript. All authors critically reviewed the content and approved the final version for publication.

### FUNDING

This study was supported by National Institute on Alcohol Abuse and Alcoholism (NIH) AA014351 (Friedbert Weiss and RC) and AA017447 (Marisa Roberto and RC).

## REFERENCES


### ACKNOWLEDGMENTS

We thank Prof. David Belin for his expert opinion on the evaluation of the ''persistence in alcohol seeking'' criterion and his comments on the statistical analysis. We thank Agostino Marchi and Rina Righi for the technical support and Marina Minucci for her contribution to the behavioral experiments.


and alcohol. Neuropsychopharmacology 26, 479–488. doi: 10.1016/s0893- 133x(01)00367-0


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Domi, Stopponi, Domi, Ciccocioppo and Cannella. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Impulsive Action and Impulsive Choice Are Differentially Associated With Gene Expression Variations of the GABA<sup>A</sup> Receptor Alfa 1 Subunit and the CB<sup>1</sup> Receptor in the Lateral and Medial Orbitofrontal Cortices

#### Edited by:

Nuno Sousa, University of Minho, Portugal

#### Reviewed by:

Trevor W. Robbins, University of Cambridge, United Kingdom Sebastien Carnicella, INSERM U1216 Grenoble Institut des Neurosciences (GIN), France

\*Correspondence:

Emilio Ambrosio eambrosio@psi.uned.es Alejandro Higuera-Matas ahiguera@psi.uned.es

#### †Present address:

Ana Contreras, Laboratory of Pharmacology, Department of Health and Pharmaceutical Sciences, School of Pharmacy, Universidad CEU-San Pablo, Madrid, Spain

> Received: 08 November 2018 Accepted: 28 January 2019 Published: 20 February 2019

#### Citation:

Ucha M, Roura-Martínez D, Contreras A, Pinto-Rivero S, Orihuel J, Ambrosio E and Higuera-Matas A (2019) Impulsive Action and Impulsive Choice Are Differentially Associated With Gene Expression Variations of the GABA<sup>A</sup> Receptor Alfa 1 Subunit and the CB<sup>1</sup> Receptor in the Lateral and Medial Orbitofrontal Cortices. Front. Behav. Neurosci. 13:22. doi: 10.3389/fnbeh.2019.00022 Marcos Ucha, David Roura-Martínez, Ana Contreras† , Sheyla Pinto-Rivero, Javier Orihuel, Emilio Ambrosio\* and Alejandro Higuera-Matas\*

Department of Psychobiology, School of Psychology, National University for Distance Education (UNED), Madrid, Spain

The orbitofrontal cortex (OFC) is a key brain region for decision-making, action control and impulsivity. Quite notably, previous research has identified a double dissociation regarding the role of this cortical territory in impulsive choice. While medial orbitofrontal lesions increase preference for a large but delayed reward, lateral orbitofrontal lesions have the opposite effect. However, there are no data regarding this anatomical dissociation in impulsive action. The neurochemical basis of impulsivity is still being elucidated, however, in recent years a role for the endocannabinoids and the related glutamatergic and GABAergic neurotransmitter systems has been suggested. Here, we submitted male Wistar rats to a delay-discounting task (DDT) or a two-choice serial reaction time task (2-CSRTT) and classified them as high impulsive or low impulsive in either task using cluster analysis. We then examined the gene expression of several elements of the endocannabinoid system or different subunits of certain glutamatergic or GABAergic ionotropic receptors (AMPA, NMDA, or GABAA) in the lateral or medial divisions of their orbitofrontal cortices. Our results confirm, at the gene expression level, the dissociation in the participation of the medial, and lateral divisions of the orbitofrontal cortex in impulsivity. While in the 2-CSRTT (inhibitory control) we found that high impulsive animals exhibited lower gene expression levels of the α1 GABA<sup>A</sup> receptor subunit in the lateral OFC, no such differences were evident in the medial OFC. When we analyzed DDT performance, we found that high impulsive animals displayed lower levels of CB<sup>1</sup> gene expression in the medial but not in the lateral OFC. We propose that GABAergic dynamics in the lateral OFC might contribute to the inhibitory control mechanisms that are altered in impulsive behavior while endocannabinoid receptor gene transcription in the medial OFC may subserve the delay-discounting processes that participate in certain types of impulsiveness.

Keywords: impulsivity, orbitofrontal cortex, delay-discounting, two-choice serial reaction time task, inhibitory control, ionotropic receptors, GABA, endocannabinoid system

## INTRODUCTION

fnbeh-13-00022 February 18, 2019 Time: 15:58 # 2

Understanding the mechanisms behind the control of behavior is one of the biggest challenges of modern Neuroscience. The natural tendency to make rapid decisions without foresight is a multifaceted trait commonly known as impulsivity. The capacity to make rapid decisions and act quickly without hesitation can be beneficial in many situations. However, when this tendency becomes extreme it can be detrimental and symptomatic of several psychopathological conditions such as attention deficit hyperactivity disorder or substance abuse (Dalley and Robbins, 2017).

During the last decades, researchers have explored different approaches to objectively measure impulsivity in humans and other mammals. There is an ample variety of tests based on decision making (such as delay and probability discounting tasks) and tests based on inhibiting motor actions [such as the fivechoice serial reaction time task (5-CSRTT) or go-no go tasks]. Considering the last decade of research and on the grounds of the neuroanatomical circuits essential to each test, impulsiveness is categorized into "waiting impulsivity" [measured with the delay-discounting task (DDT) and the 5-CSRTT], "stopping impulsivity" or the difficulty to stop an already initiated action (go/no-go tasks) and the preference for uncertain but bigger outcomes, known as "risky impulsivity" (probability discounting tasks). Although all these kinds of impulsivity share some common neural mechanisms they also rely on independent pathways (for an excellent review read Dalley and Robbins, 2017).

Waiting impulsivity is usually assessed using delaydiscounting or choice reaction time-based tasks. Nevertheless, these tasks could actually be assessing distinct subtypes of waiting impulsivity as they rely on subtly different neural mechanisms. For example, although both tasks are mediated by the nucleus accumbens (NAcc), the capacity of delaying gratification is more dependent on the core, while the inhibition of premature responses relies on the integrity of the shell (Basar et al., 2010). In addition, both subtypes of waiting impulsivity predict different aspects of drug addiction (Belin et al., 2008; Diergaarde et al., 2008).

The OFC has long been associated with several functions related to decision making (Wallis, 2007), including impulsivity (Chudasama et al., 2003; Berlin et al., 2004), but the concrete role of this area remains elusive. This elusiveness could be a consequence of the functional dissociation of the lateral and medial OFC shown both in humans (Elliott et al., 2000; Sescousse et al., 2010) and primates (Noonan et al., 2010). In rodents, the study of Mar et al. (2011) revealed a similar functional dissociation between the lOFC and the mOFC. The lesions in the lOFC elicited an increase in waiting impulsivity in a DDT whereas lesions of the mOFC caused the opposite effect. It may be tempting to speculate that this orbitofrontal dissociation could be related to the aforementioned segregation of functions between the core and shell of the NAcc, however, it seems that in the rat (contrary to the monkey) the NAcc is almost devoid of proper orbitofrontal connections [only the lateral portions of the shell receive some projections from the lOFC (Schilman et al., 2008)].

In this study, we set out to assess whether the expression of genes related to glutamatergic, GABAergic or cannabinoid neurotransmission in the lOFC or mOFC was related to the two varieties of waiting impulsivity that are captured by the DDT or the two choice serial reaction time task (2-CSRTT).

Most of the previous studies regarding impulsivity and the OFC have focused on other neurotransmitters such as dopamine and serotonin (Winstanley et al., 2006; Dalley et al., 2008). We chose to study the gene expression of several subunits of glutamatergic and GABAergic ionotropic receptors because of their direct relationship with the excitation or inhibition status of the region where they are being expressed and because little is still known about their roles in impulsivity. In addition, we have assessed endocannabinoid related gene expression because the endocannabinoid system plays a key role in the modulation GABA and glutamate release from the presynaptic terminals.

## MATERIALS AND METHODS

#### Animals

Adult male Wistar rats (n = 36, 18 per experiment) (Charles River Laboratories) were housed in groups of 3 in a controlled facility with a temperature of 22 ± 2 ◦C and relative humidity of 50% ± 10 on an inverted 12 h/12 h light/dark cycle (lights on at 8:00 pm). The rats weighted around 300 g at the beginning of the experiments and were kept at around 90–95% of their original weight by restricting their access to food (standard commercial rodent diet A04/A03: Panlab). They had ad libitum access to water through all the duration of the experiments. All the animals were maintained and handled according to European Union guidelines for the care of laboratory animals (EU Directive 2010/63/EU governing animal experimentation).

#### Apparatus

The behavioral tests were performed using six operant conditioning chambers (l = 300 mm; w = 245 mm; h = 328 mm) (Med Associates). The front part of each box was equipped with two levers 14 cm apart and a pellet dispenser with a nose poke detector between them. There were also light cues above each lever, a house light close to the top of the boxes and a white noise generator. The chambers were controlled using the software MedPC by a computer connected to a compatible interface (Med Associates).

### Behavioral Tasks

#### Acquisition of Lever Press Response

All the rats received instrumental training sessions with food pellets (grain-based rodent tablet, TestdietTM) and a light cue indicating the active lever on a fixed ratio 1 schedule. The sessions lasted 30 min and continued daily until the animals developed an acceptable lever press behavior (at least 30 lever presses), and then the same training was performed with the other lever (the order of the levers was counterbalanced across the conditioning chambers). Once the animals reached the criterion for both levers, they were trained with both active levers simultaneously (both cue lights on/both levers reward) until the Left/Right lever ratio was 1:1 ± 10%.

#### Behavioral Measurements of Impulsivity

#### **Delay-discounting task**

fnbeh-13-00022 February 18, 2019 Time: 15:58 # 3

For the study of "impulsive choice," we used an adaptation of the protocol of the DDT described by Mar and Robbins (2007). Each session lasted 100 min and consisted of five blocks of 12 trials each. Trials are presented every 100 s (i.e., 60 trials in 100 min). One of the levers (the "immediate lever") initiated the delivery of one food pellet when pressed while the other (the "delayed lever") delivered four of them. The immediate and delayed levers were in the same location (left or right) for each animal, but their position was counterbalanced between animals. The delay between lever press and the delivery of the reward was always 0 s for the immediate lever, whereas the delay associated to the delayed lever was increased across blocks in order to assess the tolerance to delay of the rats. The first two trials of each block were forced (i.e., only one lever was active and its corresponding cue light was illuminated). During the rest of the trials both levers were available, a fact that was signaled by the illuminated cue lights above each lever. Once a lever was pressed within the 10 s interval given, the cue lights were turned off and an inter-trial interval (ITI) commenced. If the rat failed to respond during the 10 s period, all lights were turned off, punishing the omitted response. During the first training sessions, both levers delivered a reward immediately, and these sessions continued until the rats showed a clear preference for the lever that delivered the large reward (> 90% choice). Once the criterion was met, the rats started the test sessions in which the delay of delivery for the delayed lever was increased with every block change (0, 5, 10, 20, and 40 s, respectively). At the end of each block, a tone cue was presented to mark the beginning of the next block. The choice ratio for each block was calculated by dividing the number of delayed responses in all the free-choice trials of the block (a maximum of 10 free choice trials per block) by the number of free choice trials completed. We used the average of the choice ratio during three consecutive blocks as a reliable estimate of choice behavior.

The sessions were repeated daily until the rats achieved a stable delay-discounting performance. Due to the variability of discounting curves between rats, the criterion for stability was defined by the average behavior of all the rats. We performed a two-way repeated measures ANOVA with the average choice ratios during two contiguous 3-sessions blocks as the BLOCK dependent variable and 3-SESSIONS and DELAY as withinsubject factors. Stability was met when no significant effect of the 3 SESSION BLOCK was found but a significant effect of DELAY was observed. This was achieved after twenty sessions of delay-discounting training.

Waiting impulsivity was operationalized here by the k parameter, calculated by fitting the choice ratio of the last three sessions block to a non-linear exponential function (CR = e−k(DELAY) ). The k parameter determines the rate of decay of the exponential function, i.e., the rate at which the lever choice changes from delayed to immediate across delays. Consequently, larger k-values indicate a faster rate of lever choice change and more impulsive behavior (Odum, 2011). There are other methods to compare the behavior of delay-discounting curves across groups or subjects, like the normalized area under the curve (AUC) (Myerson et al., 2001) or the AUC without normalization (Magnard et al., 2018). Similarly to the k parameter, these two metrics provide an index that is comparable between studies. In addition to computing the k parameter, we have also extracted both AUCs measures and tried to cluster the rats using the two indices. The main correlation of this study was preserved using the both AUCs. However, the groups resulting from the clustering process had very different sample sizes and were not considered in this study (see **Supplementary Information**).

#### **Two choice serial reaction time task**

The two choice serial reaction time task (2-CSRTT) used here is an adaptation of the popular five-choice serial reaction time task (Bari et al., 2008). The 2-CSRTT has been shown to be sensitive to an amphetamine challenge which increased premature responding in the task while leaving other parameters unaffected (Van Gaalen et al., 2009). This task was carried out in the same conditioning boxes described for the DDT.

The task started once the nose poke detector sensed an entry in the pellet dispenser. One of the stimulus lights was turned on for a variable period of time. If the lever under the light was pressed during the response interval time (CORRECT response), a pellet was delivered and, after an ITI, the next trial started. If the rat pressed the wrong lever (ERROR response), pressed a lever before any stimulus (PREMATURE response), or did not press any lever at all (OMISSION response), then the house light was turned off and rewards were not available during 5 s as a punishment. The sessions finished after 100 trials or 30 min, whichever came first. Once a rat completed one session with more than 75% of correct responses and less than 20% of omissions, the next phase of the experiment started. The experiment consisted of 12 training phases and a test phase. As the phases progressed, the stimulus duration and response interval time were shortened, while the ITI was extended [as detailed in the excellent description of the protocol by Bari et al. (2008)]. In the test phase, the ITI was drastically increased to 9 s to increase the number of premature responses and unmask the latent impulsivity trait. We used this variable (number of premature responses during the test phase) as a measure of the motor component of waiting impulsivity of each rat.

#### Sample Processing

After the behavioral assessments, the animals of both experiments were left ad libitum in their home cages for one week, in order to prevent any effect of the behavioral tests on gene expression. Then, they were mildly anesthetized with isoflurane and euthanized by decapitation. Using tools and surfaces previously treated with RNAseZap (Ambion) to prevent RNA degradation, the brain was extracted and the mOFC and lOFC were dissected out of 1 mm slices obtained by using a brain matrix and the adequate equipment. The dissected areas are depicted in **Figure 1**. The samples were then snap frozen in dry ice and stored at −70◦C for further processing. Five brains of the delay-discounting experiment were lost due to a faulty freezer.

#### RT-qPCR

RNA was isolated using the commercial kit RNeasy Lipid Tissue Mini Kit (Qiagen). Samples were retrotranscribed using a commercial kit (Biorad iScriptTM cDNA Synthesis Kit). PCR assays were performed on a real-time PCR detection system (CFX9600, Bio-Rad) with an SSO Advanced SYBR mix (Bio-Rad) using the primers indicated in **Supplementary Table S1**. We assessed the expression of subunits of the NMDA glutamatergic receptor (R1 and 2A), AMPA receptor (GluA1 and GluA2), GABA<sup>A</sup> receptor (alpha 1, alpha 2, delta, and gamma 2) an of elements of the endocannabinoid system (the CB<sup>1</sup> receptor, the anandamide synthesis enzyme NAPE-PLD, the anandamide-degrading enzyme FAAH, the 2-arachidonoyl glycerol (2-AG) synthesis enzyme diacylglycerol lipase and the 2-AG degrading enzyme monoacylglycerol lipase). The relative expression of the target genes was calculated according to Pfaffl (2001), using Gapdh as a reference gene and the reaction efficiencies were obtained using LinRegPCR software (Ruijter et al., 2009).

#### Statistical Analyses

The animals were classified according to their impulsivity using hierarchical cluster analysis with Ward's method. Although other approaches, like a quartile categorization, could be applied to isolate extreme sub-populations in our sample, we were interested in studying the whole population so that we could compare these results with those obtained in the correlational analysis (which must include the whole behavioral and neurochemical continuum of the entire population). We also refrained from using a quartile approach because doing so would incur in loss of power due to resulting smaller sample size.

We analyzed the differences in the behavior of the clustered groups with a two-way repeated measures linear mixed models approach with either lever preference (for DDT) or premature responses (for 2-CSRTT) as the dependent variable, CLUSTER as the between-subject factor and DELAY or SESSION as the within-subject factor. We also used Student t-tests to test if the averages for k or the premature responses during the day of the test were significantly different between the clustered groups. Subsequently, we checked for statistical differences in gene expression between both groups using either the Student's t-test for the homocedastic and normal data or Mann Whitney's U when the parametric assumptions were not met. We applied a false discovery rate (FDR) correction using the Benjamini-Hochberg procedure with an FDR level of 0.1. We report Cohen's d as the effect size estimator for parametric and r for nonparametric data. All the uncorrected p-values are available in the **Supplementary Materials**. Finally, we measured the relationship between the expression of the genes which were found to have differential expression between groups and either measure of impulsivity using Pearson's r when the populations of both variables were normally distributed and Kendall's τ for the nonparametric data.

All the statistical analyses were performed using SPSS 24 (IBM) or InVivoStat (Bate and Clark, 2011) and the level of significance was set to α = 0.05. All the graphs were designed using the PRISM 6 software (GraphPad Software, Inc.) or Photoshop (Adobe Systems Inc.).

#### RESULTS

#### Delay-Discounting

Regarding impulsivity measured with the DDT, we used the k-values of the animals to segregate them in two groups: 7 rats were assigned to the High Impulsive (HI-DD) group and 6 to the Low Impulsive (LI-DD) group (**Figures 2A,B**). As expected, HI-DD rats showed steeper discounting curves than LI-DD animals (significant CLUSTER∗DELAY interaction (F4,<sup>44</sup> = 7.48; p < 0.001), significant effect of the CLUSTER factor (F1,<sup>11</sup> = 12.57; p < 0.01) and significant DELAY factor

TABLE 1 | Results of the two-way repeated measures linear mixed model of the six last training sessions of the 2-CSRTT.


represented by the asterisks in the legend and in the horizontal axis. (C) k-value of high impulsive and low impulsive animals. ∗∗p < 0.01 as compared to the low impulsive group. Line and bar graphs represent the mean ± standard error of the mean. Symbols in bar graphs represent individual data points from each rat.

(F4,<sup>44</sup> = 51.56 p < 0.0001). We also compared the average k-value of both groups and verified that they differed significantly (t<sup>11</sup> = −5.77; p < 0.001; d = −3.16; **Figure 2C**).

After the FDR correction, we found that the rats of the HI-DD group expressed higher levels of Cnr1 in the mOFC than the rats of the LI-DD group (t<sup>8</sup> = −4.13; p < 0.01; d = −2.71; **Figure 4A**). We also found a significant positive correlation between k and the expression of Cnr1 in the mOFC (r = 0.77; p < 0.01 uncorrected). Accordingly, the animals that expressed higher levels of expression of these genes displayed higher impulsivity in this task (**Figure 4B**). There were no differences in Cnr1 gene expression between HI-DD and LI-DD in the lOFC (**Figure 4C**) or in the Gabra1 gene expression in either territory of the OFC (**Figures 5D,E**).

#### Two-Choice Serial Reaction Time

We also sorted another set of rats that performed the 2-CSRTT according to their premature responses in the long-ITI test day, they clustered in two groups: a high impulsive group of 11 rats (HI-2C) and a low impulsive group of 7 rats (LI-2C) (**Figure 3A**). The repeated measures linear mixed model analysis revealed no differences between both groups in either the premature, correct, incorrect, omitted or premature responses (**Table 1** and **Figure 3B**). During the test, no differences were found between both groups in the number of omissions, incorrect or perseverative responses but the number of premature responses during the test was significantly different between both groups (t<sup>16</sup> = −6.385; p < 0.001; d = −3.07; **Figure 3C**).

The analysis of the differences between the groups extracted by cluster analysis revealed that the expression of Gabra1 in the lOFC was lower in the HI-2C as compared to LI-2C rats (t<sup>15</sup> = 3.19; p < 0.01; d = 1.79; **Figure 5A**). We also found that the premature responses during the test were inversely related to the expression of Gabra1 in the lOFC (r = −0.48; p < 0.05 uncorrected). The animals that expressed lower levels of Gabra1 were less prone to make premature responses and

hence, less impulsive (**Figure 5B**). There were no Gabra1 gene expression differences between HI-2C and LI-2C in the mOFC (**Figure 5C**) or in Cnr1 expression in either territory of the OFC (**Figures 4D,E**).

### DISCUSSION

This study was aimed at determining if the expression of certain genes related to glutamatergic, GABAergic or endocannabinoid neurotransmission was associated to two different components of waiting impulsivity (delay-discounting and premature responding) and if there was neuroanatomical segregation between the medial and lateral divisions of the OFC in this relationship. For this purpose, we classified two separate groups of rats according to their performance in each task. A hierarchical clustering approach was chosen as the sorting strategy because, as observed from the figures, there was not a large variance between groups. We then compared the expression of selected genes related to neurotransmission in the medial and lateral orbitofrontal cortices between the resulting groups, searching for potential differences that could be specific to each variety of impulsivity.

Our results suggest that the gene expression signature of these two elements of waiting impulsivity is indeed different. We have found that, at the level of the genes studied here, the motor impulsivity component measured in the 2-CSRTT was mostly related to GABAergic gene expression in the lOFC, while the choice impulsivity assessed in the DDT was correlated with endocannabinoid gene expression in the mOFC.

The OFC has been strongly implicated in impulsiveness, goaldirected behavior and decision making-processes, although its key role in these psychological phenomena has been recently challenged (Stalnaker et al., 2015). With regard to impulsive behavior, the lesion studies that have been performed using DDT measurements of impulsive choice show conflicting results (Mobini et al., 2002; Chudasama et al., 2003; Winstanley et al., 2004; Rudebeck et al., 2006; Mar et al., 2011). The functional heterogeneity in the OFC has been suggested to be one of the reasons for such discrepancies (Mar et al., 2011; Stopper et al., 2014).

The mOFC has been proposed to be a hub where the different value signals of subjective goals are integrated (Kable and Glimcher, 2009). Indeed, mOFC-lesioned monkeys have difficulty making choices when the value of two options is close (Noonan et al., 2010) and studies with human patients have shown that mOFC lesions affect reward valuation and selfcontrol in intertemporal choice tasks (Peters and D'Esposito, 2016). Rat lesion studies also provide evidence for a role of the mOFC in impulsive choice whereby mOFC damage increases

the preference for a large but delayed reward (Mar et al., 2011). We have found that expression of Cnr1 in the mOFC was directly related to the waiting impulsivity that is captured by the DDT. The relationship between the endocannabinoid system and the different varieties of impulsivity is complex [see Moreira et al. (2015) for an excellent review]. Some previous reports suggested that the activation of CB<sup>1</sup> receptors in the OFC promote impulsive choice (Khani et al., 2015; Fatahi et al., 2018), however, these studies mainly targeted the lateral and ventral divisions of the OFC making any comparison to the present results problematic. There are also previous studies assessing the effects of systemic injections of CB<sup>1</sup> receptor agonists that suggest that THC administration reduced choice impulsivity measured with the DDT (Wiskerke et al., 2011). Interestingly, another study showed no effect after treatment with a cannabinoid agonist WIN 55,512-2 (Pattij et al., 2007). It is important to note that CB<sup>1</sup> receptors are mostly presynaptically localized in axon terminals, so the gene expression differences found here (arising from mRNAs in the cell bodies) could be modulating neurotransmission distally, in terminal areas such as the hippocampus, a structure that is strongly connected to the mOFC (Fettes et al., 2017). In any case, the higher levels of Cnr1 gene expression in high impulsive animals in the mOFC may suggest that this subpopulation could be especially vulnerable to the disrupting effects of cannabinoids on those cognitive processes that depend on the normal function of the mOFC, such as reward valuation or self-control. It could also mean that, based on their differential expression of cannabinoid receptors, high impulsive individuals might reduce their impulsivity (or at least the tolerance to delay component of impulsivity) to a higher degree than low impulsive individuals, after marihuana use. This hypothesis merits further testing.

Previous studies, both in humans (Elliott et al., 2000) and monkeys (Iversen and Mishkin, 1970; Noonan et al., 2010) have shown that the lOFC is specifically required when a response previously associated with reward has to be suppressed (but see Gourley et al., 2010) and, conversely, its inactivation leads to impaired adjustment of behavior after non-rewarded actions (Dalton et al., 2016). While lesions of the lOFC have been shown to increase impulsive choice (Mar et al., 2011), to the best of our knowledge, a clear (and specific) role for the lOF in premature responding has not yet been established.

Gabra1 expression was lower in the animals that made more premature responses in the 2-CSRTT. In forebrain pyramidal neurons, GABA<sup>A</sup> receptors containing the alpha 1 subunit are mainly expressed throughout the somatodendritic region while those containing the alpha 2 subunit are mostly localized to the axon initial segment (Nusser et al., 1996; Loup et al., 1998).

This differential expression of the subunit in high and low impulsive animals could translate into net differences in the cellular localization of the receptor in both populations and this might have implications for how inhibitory signals are integrated by the cortical pyramidal neurons where these receptors are expressed. There are other previous studies that have involved the GABAergic system in impulsive action. For example, Jupp et al. (2013) found that GABA<sup>A</sup> binding in the anterior cingulate cortex was negatively correlated with premature responding in the 5-CSRTT and Caprioli and co-workers established a role of the GABA synthesis enzyme GAD (glutamic acid decarboxylase) within the nucleus accumbens core in premature responding (Caprioli et al., 2014) In addition, GAD inhibition in the medial prefrontal cortex impaired impulse control measured in the 5- CSRTT (Paine et al., 2015).

### CONCLUSION

In conclusion, we here provide the first evidence for a dissociation between the medial and lateral division of the OFC in impulsive action and impulsive choice and suggest that CB<sup>1</sup> receptors in the mOFC are positively coupled to the expression of impulsive choice while GABA<sup>A</sup> receptors in the lOFC are markers of impulsive action. Functional studies interfering with or augmenting the expression of these genes must now be conducted in order to ascertain if there is a causal relationship between the gene transcription variations here reported and the different varieties of waiting impulsivity that we have studied in this work.

### AUTHOR CONTRIBUTIONS

AH-M conceived the research and carried out the initial experiments. MU performed the experiments and analyzed the data. AC performed the impulsive action behavioral experiments and qPCRs. DR-M performed qPCR experiments and helped

### REFERENCES


with data analysis. SP-R helped with the qPCR experiments. JO assisted with data analysis and interpretation. AH-M, EA, and MU wrote the manuscript with the feedback of the rest of the authors.

## FUNDING

This work has been funded by the Spanish Ministry of Economy and Competitiveness (PSI2016-80541-P to EA and AH-M); Spanish Ministry of Health, Social Services and Equality (Network of Addictive Disorders – Project n◦ : RTA-RD12/028/0020 of the Institute of Health Carlos III to EA and Plan Nacional Sobre Drogas, Project n◦ : 2012I057 to EA and 2017I042 to AH-M); The General Directorate of Research of the Community of Madrid (Project n◦ : S-2011/BMD-2308; Program of I + D + I Activities CANNAB-CM to EA); The UNED (Plan for the Promotion of Research to EA and DR-M); the European Union (Project n◦ : JUST/2013/DPIP/AG/4823-EU MADNESS) and the BBVA Foundation (2017 Leonardo Grant for Researchers and Cultural Creators to AH-M). MU received a predoctoral fellowship awarded by the Ministry of Science and Innovation and DR-M received a predoctoral fellowship awarded by the Research Office of the National University for Distance Education (UNED).

### ACKNOWLEDGMENTS

We thank Rosa Ferrado, Luis Carrillo, and Gonzalo Moreno for their excellent technical assistance.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbeh. 2019.00022/full#supplementary-material



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Ucha, Roura-Martínez, Contreras, Pinto-Rivero, Orihuel, Ambrosio and Higuera-Matas. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Compulsivity in Alcohol Use Disorder and Obsessive Compulsive Disorder: Implications for Neuromodulation

Elisabetta Burchi1,2 \*, Nikolaos Makris3,4, Mary R. Lee<sup>5</sup> , Stefano Pallanti<sup>6</sup> and Eric Hollander<sup>1</sup>

<sup>1</sup> Department of Psychiatry and Behavioral Sciences, Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, NY, United States, <sup>2</sup> Department of Health Sciences, University of Florence, Florence, Italy, <sup>3</sup> Center for Morphometric Analysis, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States, <sup>4</sup> Psychiatry Neuroimaging Laboratory, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States, <sup>5</sup> Section on Clinical Psychoneuroendocrinology and Neuropsychopharmacology, National Institute on Alcohol Abuse and Alcoholism and National Institute on Drug Abuse, Bethesda, MD, United States, <sup>6</sup> Department of Psychiatry and Behavioral Sciences, Stanford University Medical Center, Stanford, CA, United States

Alcohol use Disorder (AUD) is one of the leading causes of morbidity and mortality worldwide. The progression of the disorder is associated with the development of compulsive alcohol use, which in turn contributes to the high relapse rate and poor longer term functioning reported in most patients, even with treatment. While the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) defines AUD by a cluster of symptoms, parsing its heterogeneous phenotype by domains of behavior such as compulsivity may be a critical step to improve outcomes of this condition. Still, neurobiological underpinnings of compulsivity need to be fully elucidated in AUD in order to better design targeted treatment strategies. In this manuscript, we review and discuss findings supporting common mechanisms between AUD and OCD, dissecting the construct of compulsivity and focusing specifically on characteristic disruptions in habit learning and cognitive control in the two disorders. Finally, neuromodulatory interventions are proposed as a probe to test compulsivity as key pathophysiologic feature of AUD, and as a potential therapy for the subgroup of individuals with compulsive alcohol use, i.e., the more resistant stage of the disorder. This transdiagnostic approach may help to destigmatize the disorder, and suggest potential treatment targets across different conditions.

Keywords: alcohol use disorder, compulsivity, obsessive compulsive disorder, habit learning, cognitive control, neuromodulation, transcranial magnetic stimulation

#### INTRODUCTION

Alcohol use disorder (AUD), a problematic pattern of alcohol use accompanied by clinically significant impairment or distress (American Psychiatric Association, 2013), is one of the leading causes of morbidity and mortality worldwide (GBD 2016 Alcohol Collaborators, 2018). Globally, with 100.4 million estimated cases, AUD was the most prevalent substance use disorder (SUD) in 2016, with 99.2 million and three quarters of all substance use-attributable disability-adjusted

#### Edited by:

Walter Adriani, Istituto Superiore di Sanità (ISS), Italy

#### Reviewed by:

Erin Jane Campbell, The Florey Institute of Neuroscience and Mental Health, Australia Francesca Pacitti, University of L'Aquila, Italy

#### \*Correspondence:

Elisabetta Burchi bettina.burchi@gmail.com

Received: 30 January 2019 Accepted: 22 March 2019 Published: 11 April 2019

#### Citation:

Burchi E, Makris N, Lee MR, Pallanti S and Hollander E (2019) Compulsivity in Alcohol Use Disorder and Obsessive Compulsive Disorder: Implications for Neuromodulation. Front. Behav. Neurosci. 13:70. doi: 10.3389/fnbeh.2019.00070

life-years (DALYs). In the United States, 12-month and lifetime AUD prevalence is, respectively, estimated to be up to 14 and 30% of the adult population, with 9.8% of all deaths attributable to acute or chronic alcohol use (World Health Organization, 2018).

Despite there being effective treatments available, only an estimated 21.3 and 34.7% of patients with severe (≥6 DSM criteria) 12-month and lifetime AUD seek treatment in the United States (Grant et al., 2016). Further, high rates of relapse and poor longer term functioning are reported in the minority of patients that get some treatment (Maisto et al., 2018). This is particularly true for the subgroup of individuals with severe AUD (Tuithof et al., 2013), that is also characterized by a longer duration of untreated AUD, a lower level of spontaneous recovery as well as higher rates of psychiatric comorbidity (Grant et al., 2016; Saha et al., 2018). In addition to low rates of treatment utilization, AUD is indeed a heterogeneous disorder (Jellinek, 1960; Moss et al., 2007) and current therapeutic approaches are not developed to address this clinical variation. Hence, the magnitude of the therapeutic effect of the available AUD interventions is, overall, relatively modest (Kranzler and Soyka, 2018).

Next to the need to educate the public and policy makers in order to destigmatize the disorder and increase treatment rates (Keyes et al., 2010), the gaps in research and clinical care call for a shared framework to further characterize this heterogeneous disorder and lead to the development of new therapeutics, ultimately giving the clinician a biologically based roadmap to guide assessment and prioritize treatment (National Institute on Alcohol Abuse and Alcoholism, 2017). In this direction, the National Institute on Alcohol Abuse and Alcoholism (NIAAA) has implemented an Alcohol Addiction Research Domain Criteria (AARDoC) in order to identify specific domains relevant for alcohol addiction (Litten et al., 2015). Recently, incentive salience, negative emotionality and executive function have been proposed as key dimensional domains for the assessment of SUDs and AUD, which map onto the three stage cycle of the development of addiction over time (Koob and Le Moal, 1997; Koob and Volkow, 2016; Kwako et al., 2018). The three stage cycle (Koob and Le Moal, 1997) represents a model that provides an understanding of the development of AUD overtime, and may help organize research in AUD complementing the atemporal RDoC framework. The three stage cycle is consistent with the conceptualization of alcohol addiction as an aberrant form of learning, where alcohol exposure leads in time to alteration in the neurocircuitry underlying stress response, reward and cognitive functioning, all of which ultimately leads to compulsive substance use (Wise, 1987; Di Chiara, 1999; Berke and Hyman, 2000; Everitt et al., 2001; Lubman et al., 2004). Hence, compulsive drug seeking has been identified as the central, defining property of alcohol and substance use disorders (Wise and Koob, 2014; NIAAA; National Institute of Drug Abuse). The development of repetitive drug patterns of use, in turn, has been proposed as a main pathophysiologic factor contributing to the high relapse rates characteristic of addiction (Heyman, 2013).

While the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) defines AUD by a cluster of symptoms, parsing its phenotype by domains of behavior, such as


compulsivity, may be a critical step to improve outcomes of this condition. On the other hand, neurobiological underpinnings of compulsivity still need to be fully elucidated in addiction -and AUD in particular- in order to design specific targeted treatment strategies.

Compulsivity (**Box 1**) is a construct that encompasses motor, cognitive, affective and motivational processes. Multiple models have addressed the central question regarding the transition from casual to compulsive drug use over the development of addiction. The traditional hedonic/withdrawal hypothesis of addiction explained compulsive drug use as earlier pleasure seeking and later attempts to avoid unpleasant withdrawal symptoms (Koob and Le Moal, 1997, 2001), but this could not fully explain the high rates of relapse after long period of drug abstinence (Robinson and Berridge, 1993, 2000). More recently this theory has been complemented by models showing how other phenomena take place in the development of addiction, such as incentive sensitization (Robinson and Berridge, 2003), aberrant learning (Robbins and Everitt, 2002), and loss of cognitive control (Jentsch and Taylor, 1999). At the brain connectivity level, these phenomena have demonstrated to map onto limbic-cortico-striatal networks (Ma et al., 2010), with important overlap with neuroimaging findings in OCD (van den Heuvel et al., 2016).

Following the characterization of the neural basis of compulsivity developed in the field of Obsessive Compulsive Disorder (OCD) research, we discuss the findings supporting common mechanisms between AUD and OCD, focusing on specific aspects of the construct, namely habit learning and cognitive control. Finally, neuromodulatory interventions are

proposed as a probe of this hypothesis and as a potential therapy for the subgroup of individuals with AUD who demonstrate compulsive alcohol seeking and use, i.e., the later more severe stage of AUD. In particular, deep transcranial magnetic stimulation (dTMS), recently approved by the Food and Drug Administration to treat compulsivity in OCD, is specifically discussed as a promising non-invasive intervention to treat compulsivity in AUD.

Such a transdiagnostic approach aims to destigmatize AUD, and to help clarify common pathophysiological mechanisms underpinning compulsive behavior across different disorders (Fineberg et al., 2016; Gillan et al., 2016). Consistent with a need for new paths of research in the field of treatments for addictions, the ultimate aim of this manuscript is to highlight and discuss the scientific rationale supporting the development of evidence based treatments for compulsivity in AUD, eventually as a model for other disorders of addiction.

#### THE HABIT MODEL OF COMPULSIVITY

Compulsivity has been defined as a behavioral trait in which actions are persistent and repeated despite adverse consequences (Robbins et al., 2012) (**Box 1**). An initial hypothesis regarding compulsive behavior in OCD -where compulsivity has traditionally found its paradigmatic expression- purported that it represents a goal-directed process associated with a "cognitive bias" or disrupted assignment of value toward available alternative behaviors which are performed to reduce the likelihood that a feared consequence will take place (Salkovskis, 1985; Rachman, 1997). Later, OCD was conceptualized as a disorder of maladaptive habit learning (Graybiel and Rauch, 2000) on the basis of overlap between the frontostriatal circuits underlying repetitive behavioral habits and OCD (Alexander et al., 1986). Data from subsequent preclinical and clinical studies have actually elucidated the neural basis of habit formation in humans or the transition from goal-directed to habitual behavior as a shift away from signaling in ventral associative frontostriatal circuits comprising ventromedial prefrontal cortex (vmPFC), medial orbitofrontal cortex (mOFC) and caudate, toward that in dorsal sensorimotor frontostriatal circuits, including posterior putamen and premotor cortex (Balleine and O'Doherty, 2010; Gillan and Robbins, 2014). In particular, estimated white matter strength in the vmPFC seeded from the caudate have been found to predict flexible goal-directed action, while estimated white matter tract strength in the premotor cortex seeded from the posterior putamen predicts vulnerability to habitual behavioral control in healthy humans (de Wit S. et al., 2012). Patients with OCD consistently exhibit a cognitive bias toward habit learning over goal-directed behavior (Gillan et al., 2011, 2014; Vaghi et al., 2019) associated with relative higher task-related activity in dorsal vs. ventral striatum and weakened resting state caudate-ventrolateral prefrontal cortex (vlPFC) and putamendorsolateral prefrontal cortex (dlPFC) connectivity (Banca et al., 2015; Vaghi et al., 2017). In particular, decreased activity in caudate-prefrontal circuits accompanied by hyperactivation of subthalamic nucleus/putaminal regions have been associated to symptom generation in OCD (Banca et al., 2015). Hyperactivity of putamen preceded deactivation during avoidance/relief events indicating a pivotal role of the putamen in regulation of behavior and habit formation in OCD (Banca et al., 2015). Consistently, task dependent functional connectivity during goal-directed planning showed reduced hypoactivation of the right dlPFC coupled with reduced functional connectivity between this latter and the putamen (Vaghi et al., 2017). Other findings indicate a dissociation between actual behavior and subjectively reported action-outcome knowledge, reinforcing the notion that OCD may be driven by a disrupted goal-directed system over habitual response, rather than by dysfunctional beliefs (Vaghi et al., 2019). This diminished outcome sensitivity has also been associated with diminished caudate-parietal connectivity (Harrison et al., 2013).

Based on these studies, evidence favors the habit hypothesis for compulsivity in OCD: this model posits that rather than goal-directed avoidance behaviors, compulsions are a result of excessive habit formation (Gillan and Robbins, 2014; Burguière et al., 2015). Irrational threat beliefs (obsessions) characteristic of OCD may be a consequence, rather than a cause, of compulsive behavior. In other words, obsessions may be an attempt to resolve the discrepancy between patients' value attribution and their cognitively inexplicable urge to perform compulsive responses to determined stimuli.

Notably, deficits in goal-directed control and over–reliance on habits are a model of compulsivity that is applicable to other psychiatric disorders where compulsions feature prominently (Hollander, 1993; Hollander and Wong, 1995; Fontenelle et al., 2011; Voon et al., 2015) in particular to drug addiction (Robbins and Everitt, 1999; Gerdeman et al., 2003; Everitt and Robbins, 2005; Hogarth et al., 2013).

### HABIT HYPOTHESIS OF COMPULSIVITY IN AUD

#### Compulsivity and Habit Learning in Addiction

Drug addiction has been defined as representing a dysregulation of motivational circuits caused by development of aberrant incentive salience and habit formation, accompanied by compromised executive function over the use of the substance (Koob and Volkow, 2016). Such trajectory correlates with a progression from reward-driven to habit-driven drug-seeking behavior overtime (Koob and Le Moal, 2005; Everitt and Robbins, 2013).

Borrowing from the habit hypothesis of OCD, the experience of "wanting" the drug, progressively detached from the experience of "liking" it (Berridge and Robinson, 2016), may be understood as post hoc cognitive rationalizations of a goalinsensitive, stimulus-driven behavior (Gillan and Robbins, 2014). While the expectancy of reward may be crucial to initiate drug self-administration in the early stage of the disorder, the drive toward consumption of the drug triggered by the cue becomes quite indipendent from the expectancy of pleasure in the late stages of the disorder. Thanks to conditioned reinforcement,

over time the "wanting" for the drug becomes triggered by the incentive motivational properties of drug cues, independently of a cognitive desire for a declarative goal (Berridge and Robinson, 2016). This shift in phenomenology has been associated with a cascade of neuroadaptations that engage the dorsal striatal habit system over the ventral striatal loop of reward and motivation mediated by phasic dopamine release in the dorsolateral striatum (see for review of clinical studies Volkow and Fowler, 2000; Newlin and Strubler, 2007; Dolan and Dayan, 2013; Grant and Kim, 2014; Koob and Volkow, 2016). In particular, translational studies have shown how a gradual progression from hedonic to habitual and compulsive drug use over time (i.e., a gradual progression from a response that is first dependent and then independent from evaluation of the relationship between action and outcomes eventually becomes a response despite adverse consequences) is associated with (i) habit overlearning and (ii) parallel reduction in goal-directed behaviors and inhibitory control that correspond to a shift in recruitment of ventral to dorsal regions of the striatum (Yin et al., 2004; Belin and Everitt, 2008; Balleine et al., 2009; Lesscher and Vanderschuren, 2012; Willuhn et al., 2012). Intermittent drug–induced dopamine (DA) signaling promotes the ability of drug-paired cues to increase DA levels and recruit striatal-globus pallidal-thalamocortical loops that engage the dorsal striatum. This shift from a system dedicated to updating predictions of value (ventral) to a system dedicated to the optimization of reward-related responses (dorsal) augments progression through the addiction cycle and helps explain craving and compulsive drug use (Koob and Volkow, 2016).

#### Compulsivity and Habit Learning in AUD

A confluence of preclinical and clinical data strongly supports that the pathogenesis of AUD involves a shift from associative striatum (caudate) to sensorimotor striatum (posterior putamen) in response to alcohol reward responding, likely driven by simultaneous reductions in goal-directed control over actions and increase in habit associations. Learning about response-outcome (R-O) associations has been investigated using instrumental learning paradigms (Balleine and Dickinson, 1998). In instrumental discrimination tasks, which have been developed to distinguish between goal-directed and habit-based learning (de Wit et al., 2007), stimuli (or cues) are congruent, unrelated or incongruent with subsequent outcomes. Whereas performance (or learning) on congruent and control trials can be supported by both the goal-directed (S-R-O) and habitual system (S-O), performance on the incongruent discrimination (in which each pictures functions as a stimulus and an outcome for opposing responses) relies solely on the habit system. The dominance of habitual control over flexible goal-directed responding has traditionally been assessed using revaluation tests, by manipulating outcome value and observing consequent effects on response (Dickinson, 1985). In a typical outcome devaluation methodology (Adams, 1982), after a training phase, the value of the reinforcer of action (O) is typically reduced (extinction phase) affecting internal or external motivational states, and the experimenter assesses if the behavior (R) appropriately updates, in light of this change, measuring the strength of the R-O association. An alcohol addiction model study conducted in rats showed that instrumental alcohol seeking became insensitive to devaluation after 4 weeks of training, mirrored by a shift in control from the dorsomedial striatum to the dorsolateral striatum (Corbit et al., 2012). Another preclinical study using the instrumental learning paradigm showed how alcohol reinforced behavior was less sensitive to devaluation compared to when behavior was reinforced with food, suggesting that alcohol consumption may be particularly susceptible to habit formation (Dickinson et al., 2002). An fMRI clinical study using a cuereactivity paradigm showed significantly higher cue-induced activation of the dorsal striatum in heavy drinkers compared to social drinkers and higher cue-induced activation of ventral striatum and prefrontal areas in social drinkers compared to heavy drinkers (Vollstädt-Klein et al., 2010). These findings were interpreted as an indirect indication of a shift from ventral to dorsal striatal involvement in the development of AUD, associated with the increasing role of habit–like drug seeking behavior over the course of the disorder. Another clinical study using neuroimaging and learning tasks showed how early abstinent patients presented greater habit formation compared to controls and a progressive shift toward greater goal-directed behaviors with prolonged abstinence (Voon et al., 2015). Another study conducted in AUD patients was the first providing direct behavioral and neurophysiological evidence for an imbalance between goal-directed and habitual control in humans with a substance use disorder (Sjoerds et al., 2013). Subject underwent fMRI during completion of an instrumental learning task characterized by a discrimination learning phase and an outcome-devaluation test phase designed to study the balance between goal-directed and habit learning: patients with AUD compared to healthy controls showed a strong engagement of the neural habit pathway, comprising dorsolateral/posterior parts of the striatum (posterior putamen, caudate tail/body) and a relatively weak engagement of the goal-directed pathway in the vmPFC and dorsomedial/anterior parts of the striatum (caudate head, anterior putamen) during instrumental learning even in the context of AUD-irrelevant stimuli. Moreover, vmPFC activation was negatively associated with AUD duration. Another study investigating the effects of alcohol on devaluation sensitivity for food reward suggested a general effect of alcohol toward habit-formation (Hogarth, 2012).

All these studies give direct (Sjoerds et al., 2013) and indirect (Dickinson et al., 2002; Vollstädt-Klein et al., 2010; Corbit et al., 2012; Hogarth et al., 2012) demonstration of a shift from ventral to dorsal striatal involvement in the development of AUD, associated with an increasing role of habit–like drug seeking behavior over the course of the disorder. Moreover, alcohol consumption has demonstrated to be particularly susceptible to habit formation over other reinforcers (Dickinson et al., 2002) and it has also shown to exert a general direct effect toward habit-formation attenuating goal directed control over action selection (Hogarth, 2012).

Of note, habit circuitry, including the dorsolateral striatum, is also implicated in punishment resistance (Jonkman et al., 2012), characteristic feature of compulsive behaviors, namely behaviors performed in spite of adverse consequences. Indeed,

it is important to notice that the construct of "habit" doesn't completely overlap with the construct of compulsivity, where habitual alcohol use involves behavior that continues despite outcome devaluation, whereas compulsive alcohol use encompasses continued use despite adverse consequences (Hopf and Lesscher, 2014; Marchant et al., 2018). Studies investigating neural correlates of compulsive alcohol seeking using rodent model of aversion-resistant alcohol seeking showed involvement of mPFC, insula and nucleus accumbens (Seif et al., 2013) and hyperactivation of these areas have been found in heavy drinkers as opposed to light drinkers when viewing threat-paired alcohol cues (Grodin et al., 2018).

Next to extensive preclinical evidence suggesting overlap in the striatal regions involved in habitual and compulsive behavior (Everitt and Robbins, 2005; Belin et al., 2009; Pierce and Vanderschuren, 2010; Jonkman et al., 2012; Seif et al., 2013), a recent study conducted on rats for the first time demonstrated the causal importance of the functional recruitment of the anterior dorsolateral striatum (aDLS), a region strongly associated with the consolidation and performance of stimulus-response habits (Yin et al., 2004; Zapata et al., 2010; Corbit et al., 2012), in the switch from controlled to compulsive alcohol use (Giuliano et al., 2019). This study found that individual differences in the reliance of alcohol seeking habits on aDLS dopamine predict and underlie the emergence of compulsive alcohol seeking, providing first evidence that compulsive alcohol seeking stems from an inability to disengage aDLS control over seeking behavior when faced with negative outcomes. This result shows that the maladaptive nature of alcohol seeking in those individuals that become compulsive lays in the rigidity of those aDLS dopamine-dependent habits.

### Compulsivity and Cognitive Control in AUD and OCD

From a neurocognitive point of view, AUD is characterized by an imbalance between overwhelming drive toward alcohol consumption and inability to inhibit alcohol consumption, i.e., a disruption in cognitive control over alcohol use (Baler and Volkow, 2006; Jentsch and Pennington, 2014; Koob and Volkow, 2016). The RdoC domain of cognitive control is defined as a modulatory system that supervises all the activities in the service of goal-directed behavior, when prepotent modes of responding are not adequate to meet the demands of current context. Cognitive (or effortful) control encompasses the ability to select relevant stimuli, inhibit responses influenced by distracting elements, select appropriate responses, monitor the outcome of those responses, and adjust behavior as needed in the face of changing situations.

Despite well documented dysfunctions across multiple cognitive functions (Stavro et al., 2013), deficits in response inhibition have been proposed as the most promising marker of cognitive control impairments measured by behavioral tasks in AUD (Wilcox et al., 2014). Response inhibition, a subdomain of cognitive control and defined as the ability to suppress a pre-potent behavior that is inappropriate or no longer required, has been typically assessed using a range of neuropsychological paradigms including those measuring motor response inhibition (further differentiated into action restraint and action cancelation) and cognitive inhibition (interference control) (Bari and Robbins, 2013). It has been suggested that interference control, action restrain and action cancelation represent early, intermediate and late processes of response inhibition (Sebastian et al., 2013). It has also been proposed that inhibitory control is highest in the action cancelation, making of the Stop Signal Task (SST) the more sensitive task to measure inhibitory control, tapping both impulsivity and compulsivity traits (Smith et al., 2014). In a SST, a "stop" signal appears after the onset of a "go" signal on a subset of trials, requiring the participant to interrupt an ongoing motor response to the go signal that has already been triggered, and the primary dependent measure is the stop-signal reaction time (SSRT). Significant deficits in response inhibition have been observed for all tasks in AUD (Karch et al., 2008; Li et al., 2009) especially using the SST (Smith et al., 2014). Typically, task dependent functional connectivity studies have demonstrated that frontostriatal pathways are critical for response inhibition that is weakened over the progression of AUD: individuals with more severe AUD exhibit impaired connectivity between dorsal striatum and anterior cingulate cortex (ACC), mPFC and mOFC (Courtney et al., 2013; Lee et al., 2013). Studies investigating response inhibition have then further demonstrated the importance of dlPFC-dorsal striatum connectivity for behavioral regulation in AUD especially in the late stage of the disorder (Schulte et al., 2012; Courtney et al., 2013; Müller-Oehring et al., 2013). Moreover, impaired response inhibition in AUD has been related to more intense cue-induced alcohol craving (Papachristou et al., 2013). Importantly, impairments in response inhibition may help differentiate AUD subtypes and predict clinical outcomes (Nigg et al., 2006; Saunders et al., 2008; Schuckit et al., 2012).

In the past several years, deficits in response inhibition have also been described and associated to disrupted functional activations in frontostriatal circuits in OCD involving presupplementar motor area (pre-SMA), inferior frontal gyrus, ACC, striatum, thalamus (Maltby et al., 2005; Penadés et al., 2007; Roth et al., 2007; Woolley et al., 2008; Page et al., 2009; de Wit S.J. et al., 2012; van Velzen et al., 2014) and proposed as a neurocognitive endophenotype in OCD (Chamberlain et al., 2007; Menzies et al., 2008; de Wit S.J. et al., 2012). Evidence in both OCD and AUD suggests that disruptions of indirect cortico-striatal-thalamic-cortical (CSTC) pathways may mediate both compulsive behaviors and failure in response inhibition (**Table 1**) (Aron and Poldrack, 2006; Chambers et al., 2009; Aron, 2011). However, the use of behavioral tasks to dissect cognitive control has some limitations, hampering insights into integrated global brain functioning in non−task related circumstances, which contributes to behavioral variability. Techniques such as a priori defined seed−based connectivity help to elucidate how striatal brain areas are integrated into a broader functional network and how that is related to the duration and severity characteristics of alcohol addiction (Schmaal et al., 2013; Müller-Oehring et al., 2015; Galandra TABLE 1 | Fronto-striatal functional connectivity and cognitive control in AUD and OCD.

Resting state functional connectivity studies

fnbeh-13-00070 May 8, 2019 Time: 15:39 # 6


Task -dependent functional connectivity studies


#### AUD OCD

Resting state functional connectivity studies


Task dependent functional connectivity


DS, dorsal striatum; dACC, dorsal anterior cingulate cortex; PCC, posterior cingulate cortex; mPFC, medial prefrontal cortex; mOFC, medial orbito frontal cortex; dlPFC, dorso lateral prefrontal cortex; vlPFC, ventro lateral prefrontal cortex; SST, Stop Signal Task.

et al., 2019). A very recent study coupling resting-state fMRI with an in-depth neuropsychological assessment of the main cognitive domains (Galandra et al., 2019), gives support to the hypotheses that cognitive impairments in AUD (Stavro et al., 2013; Le Berre et al., 2017) are not explained by specific susceptibility of frontal regions to alcohol neurotoxic effects, but rather to dysfunctional connectivity between the cortical (dlPFC and dACC) and subcortical (basal ganglia) nodes of the networks underlying cognitive control on goal-directed behavior. Indeed, while functional connectivity between the basal ganglia and both the dlPFC and dorsal ACC (dACC) nodes was positively related to executive performance in the whole sample, the strength of the very same connections was significantly reduced in patients reflecting the amount of alcohol consumption. The significance of this decrease in connectivity has been investigated by using a graph−theoretical approach, showing that prolonged alcohol dependence is associated with decreased global brain network efficiency and less anterior striatal segregation (Sjoerds et al., 2017). These studies suggest that executive impairment in AUD patients may reflect altered frontostriatal connectivity which underpins top-down modulation of behavior by mediating the switch between automatic and controlled processing. While reward prediction error signal was found intact suggesting proper behavioral options value decoding in the striatum, impaired dlPFC-striatal connectivity has been associated with abnormal processing of financial gains and losses, suggesting that dysfunctional intrinsic connectivity might also underpin defective behavioral learning during task-performance in AUD (Park et al., 2010). Hence, alterations in frontostriatal connectivity underlining impaired learning and decision making may in turn contribute to characteristic poor treatment outcome in AUD. Relapse in AUD has been associated with pronounced atrophy in bilateral OFC, right mPFC and right ACC, impaired connectivity between dorsal striatum and mOFC, and reduced mPFC activation during goal-directed behavior (Grüsser et al., 2004; Beck et al., 2012; Lee et al., 2013; Durazzo and Meyerhoff, 2017; Sebold et al., 2017). On the other hand, other studies have underlined the association between abstinence and normalization in frontostriatal circuits: mOFC-dorsal striatum functional connectivity, impaired in alcohol-dependent patients, showed to increase and normalize in correlation with the duration of abstinence (Lee et al., 2013), and to be associated with a progressive shift toward goal-directed behavior over habitual control (Voon et al., 2015). With progression from short to long-term abstinence, the synchrony within the reward network (subgenual ACC, caudate, nucleus accumbens, and thalamus) and within the executive control network (dlPFC, ACC, and nucleus accumbens) were found to, respectively, decrease and increase progressively (Camchong et al., 2013).

Meta-analysis of seed-based resting-state fMRI studies in OCD found consistent hypoconnectivity within frontoparietal (peaking into dlPFC) and salience network (peaking supramarginal gyrus), and between salience, frontoparietal and default-mode network (Gürsel et al., 2018); consistent general dysconnectivity was found within default-mode (peaking in dmPFC and ACC) and frontoparietal network (peaking in the

inferior parietal lobule; LC, locus coeruleus; SMA, supplementary motor area; VTA, ventral tegmental area.

BOX 2 | Cognitive control and the potentiality of targeting the dACC. Within the RDOC construct of cognitive control and, more specifically the subdomain of response inhibition/response selection, the neural network of the anterior cingulo-insular or salience network (aCIN) seems to be critically relevant (Downar et al., 2016). The dACC stands midway along a hierarchy of medial prefrontal regions that guide goal selection, thought selection and action selection based on internal drives as opposed to external cues. The dACC is active when choosing freely between to different cognitive tasks and during voluntary internally cued modulation of emotional state (Downar et al., 2016). The anterior cingulate cortex (ACC) interfaces attention, executive function, drive, affect and motor control. Convergent data from neuroimaging, neuropsychological, genetics, and neurochemical studies have implicated, bilaterally, dysfunction of dorsolateral prefrontal and anterior cingulate cortical structures, which constitute the cortical arm of the frontostriatal network (caudate and putamen being its subcortical counterpart) driving executive function (Solanto, 1984; Castellanos, 1997; Zametkin and Liotta, 1998; Giedd et al., 2001; Seidman et al., 2006; Makris et al., 2007), in cognitive dyscontrol. The ACC (Brodmann's area 24 or BA 24) is an important regulator of other cortical and subcortical brain regions as well, and its disconnection appears to be consistent with an executive dysfunction. The ACC is connected with other cortical areas ipsilaterally and controlaterally as well as with subcortical structures. The topographic organization of these axonal connections is precise within the subcortical white matter and architectonically arranged within the overlaying cortical layers of the ACC (Figure 1). This mesh of connections converging to and diverging from the neuronal bodies of the ACC constitutes a central networking node within the overall connectional map of the brain crucial for the interface of drive, emotion, cognition and motricity (Paus, 2001) as well as for the modulation of cognitive control (Cohen et al., 2000). Short (U-fibers) and medium range association fibers connect BA 24 with adjacent cortical areas. The more complex long association fibers belong to three discrete fiber bundles, namely the cingulum bundle (CB) (Mufson and Pandya, 1984), the uncinate fascicle (UF) (Petrides and Pandya, 1988) and the extreme capsule (EmC) (Petrides and Pandya, 1988). Through these three fiber pathways the anterior cingulate cortex is connected in a bidirectional fashion as follows (Schmahmann and Pandya, 2007). The CB connects BA 24 with rostral posterior cingulate BA 23, paracingulate BA 32 and orbital frontal BA 14, 12, and 11 as well as prefrontal BA 8 and 6 (Baleydier and Mauguiere, 1980). Via the UF, BA 24 connects with subcallosal BA 25 and basal forebrain regions as well as the amygdala and the perirhinal cortex. The EmC provides connections with the dysgranular insula and the insular proisocortex. Striatal fibers by virtue of the subcallosal fascicle of Muratoff connect BA 24 to the caudate's head and, through the external capsule, with the putamen and claustrum. Thalamic connections are with anterior, the medial dorsal as well as the midline thalamic nuclei. Other connections are with the parahippocampus, the hippocampus, the zona incerta and the pons as well as with the locus coeruleus and the ventral tegmental area (Paus, 2001). Motor connections are with the supplementary motor area, the motor cortex and the spinal cord (He et al., 1995). Chronic failure of the ACC network may have profound implications contributing to deficit of cognitive control. Convergent evidence from fMRI (Carter et al., 1998, 1999; Botvinick et al., 1999; Cohen et al., 2000) and evoked potential (Gehring and Knight, 2000) experimentation in humans suggest that ACC is associated with monitoring of conflict and modulation of cognitive control as well as modulation of allocation of attention in real time. As illustrated in Figure 1, interactions between the dorsolateral prefrontal (DLPFC), inferior parietal lobule (IPL), orbital frontal cortex (FOC), amygdala and brainstem centers such as the locus coeruleus or the ventral tegmental area, enable the ACC to integrate sensitive information in real time to monitor conflict of competitive cognitive tasks, modulate cognitive control and produce balanced behavior (Yeterian and Pandya, 1985; Cohen et al., 2000). It appears that alterations in the ACC and its associated frontostriatal network (primarily the caudate nucleus and putamen being its subcortical counterpart), which are driving executive function, are critical in executive dysfunction. Therefore, neuromodulatory interventions seem to be an important therapeutic means to modulate ACC function and restore balance in behavior when cognitive control is lost in such conditions as AUD.

striatum), as well as between frontoparietal, default-mode, and salience networks (Gürsel et al., 2018).

These findings match perfectly with the structures comprising the CSTS circuits, namely OFC, ACC, vmPFC, striatum, and thalamus (Pauls et al., 2014).

While the CSTC loop hypothesis has dominated the OCD literature over the last decades (Alexander et al., 1986), as the nature of integration has become apparent for the processing of information in the striatum, such that information is carried in "spirals," rather than isolated "loops" (Milad and Rauch, 2012) more attention has been paid to potential "hubs" in the neurocircuitry thought to underlie OCD.

The dorsal ACC has been proposed as a connective hub of cognitive control (Shackman et al., 2011) in a ideal position to receive sensory input and act on that information via downstream motor regulation (**Figure 1** and **Box 2**). Proposing cognitive control as a main domain in OCD pathophysiology, dACC dysfunction and consequent aberrant cognitive control signal

specification has been proposed to drive the pursuit of tasks that do not accord to long-term goal, underlying the core pathophysiology of OCD (McGovern and Sheth, 2017).

Functional connectivity studies have suggested that cognitive and behavioral alterations observed in AUD might reflect functional imbalances within a CTST loop involving the key nodes of the reward (Park et al., 2010; Camchong et al., 2013; Müller-Oehring et al., 2013), salience (Sullivan et al., 2013; Zhu et al., 2017) and executive networks (Weiland et al., 2014; Zhu et al., 2017; Galandra et al., 2019), namely striatum, dlPFC, and dACC. Activity of the cingulate cortex is also emerging as a marker of subsequent alcohol relapse (De Ridder et al., 2011; Zakiniaeiz et al., 2016).

### NEUROMODULATORY STRATEGIES TO TARGET COMPULSIVITY IN AUD

#### State of Art in the Treatment of AUD

A variety of therapeutical approaches are currently available for AUD, such as behavioral treatments, medications and mutualsupport groups (NIAAA treatment Navigator page). While, due to the anonymous nature of mutual-support groups, it is difficult to determine their success rates compared with those led by health professionals, cognitive behavioral therapy (CBT), motivational enhancement therapy (MET), marital and family counseling and other brief interventions have demonstrated to be beneficial in AUD. In addition to behavioral treatments, there are currently three FDA approved medications for the post withdrawal maintenance of alcohol abstinence, namely disulfiram, naltrexone, and acamprosate (Kranzler and Soyka, 2018). Disulfiram works negatively reinforcing aversion toward alcohol by inhibiting the enzyme acetaldehyde dehydrogenase and resulting in unpleasant effects when combined with alcohol (Mutschler et al., 2016). Naltrexone is an opiate antagonist that is hypothesized to work buffering the endorphin-mediated rewarding effects of alcohol (Mason et al., 2002). Acamprosate was thought to reduce craving for alcohol by acting as GABA agonist and modulator of glutamate NMDA activity, both disrupted by chronic alcohol use (Mason et al., 2002) and recent evidence suggests that its anti-relapse effects may act via calcium (Spanagel et al., 2014). However, given the broad distribution of neurotransmitter systems in the brain, it is particularly difficult to have a targeted action on neural circuits using pharmacotherapeutics.

Although a review of pharmacotherapy is out of the scope of this paper and mechanisms of action have to be fully elucidated (Koob and Mason, 2016), the available medications for AUD mainly try to reduce the positive reinforcing properties of alcohol or the negative reinforcing aspects of chronic alcohol use by relieving craving (Heilig et al., 2010).

Research has provided evidence that the propensity to engage in drug/alcohol-seeking is determined by the expected value and probability of getting the drug (Hogarth, 2012). Whereas the overall propensity to engage in a goal-directed choice is determined by the expected outcome value of the outcome, in habit learning the capacity of cues to elicit this choice is determined by the expected probability of getting the outcome. Hence, in habit learning, devaluing the outcome does not affect the response. In other words, the actual treatments may have partial efficacy because they may work on tonic (expected alcohol value), but not on cue-elicited phasic DA signaling (Hogarth et al., 2014). In order to improve treatment outcome, new interventions should be developed to directly target inflexible, habitual cue-elicited drug-seeking behavior.

### Neuromodulation to Treat Compulsivity in AUD

Following the conceptual framework outlined above, neuromodulatory interventions able to selectively target frontostriatal circuitries may held therapeutic promise for the treatment of compulsive alcohol seeking in AUD. Promisingly, effective neuromodulatory interventions for compulsive symptoms have been associated with normalization in connectivity and restored behavioral control from the striatum to prefrontal cortical regions (Mian et al., 2010; Figee et al., 2013; Dunlop et al., 2016).

Whereas transcranial direct current stimulation (tDCS) needs more investigation for the treatment of compulsivity given the lack of sham controlled studies, repetitive Transcranial Magnetic Stimulation (rTMS) (**Box 2**) as non-invasive and relatively site specific, has been the most studied neuromodulation technique in the treatment of compulsive behavior in both AUD and OCD (Campbell et al., 2018; Shivakumar et al., 2019). As the time of writing, there have been 10 studies probing rTMS as a tool to change alcohol consumption and explore the associated neurocircuit changes in AUD (**Table 2**). Different outcome measures were used to assess the impact of different stimulation protocols on relapse, wanting of the drug ("craving"), cognitive control and associated funtional connectivity (**Table 2**). Traditional rTMS studies targeting the right dlPFC had some success at reducing alcohol craving (Mishra et al., 2010, 2015; Herremans et al., 2015) in accordance with the results obtained by a sham controlled tDCS study applied to the same area (Boggio et al., 2008). To date, the majority of interventional rTMS studies in OCD have been conducted stimulating the dlPFC as well, with mixed results. On the other hand, the most recent meta-analysis to assess whether the effectiveness of rTMS in improving OCD symptoms is moderated by its application over different cortical targets revealed that rTMS applied over the SMA yields greater improvements than rTMS applied over the dlPFC or OFC (Rehn et al., 2018). This therapeutic effect has been attributed to the normalization of hyperactive orbito-fronto-striatal circuits induced by low frequency-rTMS (LF-rTMS) (Mantovani et al., 2010). The SMA plays a central role in motor planning and response-inhibition (de Wit S.J. et al., 2012) and has extensive connections to regions involved in cognitive and emotional processes. Studies suggest that hyperactivity in this area may be associated with deficient inhibitory control over repetitive behaviors (Oliveri et al., 2003), thus making it an attractive target for the inhibitory effects of LF-rTMS. The efficacy of neurosurgical treatments for the treatment of resistant OCD patients suggests other


fnbeh-13-00070 May 8, 2019 Time: 15:39 # 9

promising deep targets for treating compulsivity. Stereotactic lesions in the dACC (cingulotomy) have shown long term efficay in the treatment of refractory OCD with average full response of 41% (Sheth et al., 2013; Brown et al., 2016). Based on good results with stereotactic ablation, Deep Brain Stimulation (DBS) has been explored and DBS of anterior and ventral capsule, ventral striatum or subthalamic nucleus (STN) has shown a global percentage of responders of 60.0% with long-term efficacy (Bais et al., 2014). Stimulation of STN has been reported to decrease OFC and mPFC metabolism, as well as ACC activity, while stimulation of striatal areas has been associated with decrease in OFC and subgenual ACC activity (Alonso et al., 2015).

Despite the limited number of cases examining DBS in patients with AUD, DBS of the nucleus accumbens has shown long term treatment benefit, reducing alcohol craving for up to 8 years associated with modulation of anterior mid−cingulate cortex functioning and cognitive control (Kuhn et al., 2011; Müller et al., 2016). In light of the striking results obtained with DBS of striatal areas and the evidence of aDLS dopamine dependent alcohol seeking (Giuliano et al., 2019), the recent development of new coils designed to target deep cerebral areas with rTMS, may be particularly interesting, avoiding the complications and adverse events related to invasive techniques and possibly gaining some of their therapeutic advantages. Indeed, there is already evidence that cortical stimulation with deep TMS (dTMS) may modulate sub-cortical striatal activity in AUD (Addolorato et al., 2017). While rTMS proved to indirectly modulate the insula and the ACC by stimulating the dlPFC (Nahas et al., 2001) or the frontal pole (Hanlon et al., 2017), H coil design series now promise to target these deeper structures directly, with greater effectiveness. A recent big randomized study investigated the efficacy of dTMS in OCD (Carmi et al., 2018): after provocation of symptoms, 99 treatment resistant OCD patients were treated with either high frequency (HF) or sham dTMS over the mPFC and ACC for 6 weeks and 29 sessions. Despite the lack of neuroimaging, this study conveys relevant results in terms of response rate (38,1%) and maintenance (4 weeks follow up) and to date it is the largest TMS controlled study ever conducted in OCD which led to FDA approval to market the dTMS system for treatment of OCD.

In addition, the mPFC and ACC have been associated with initiation of compulsive behavior in OCD (Rauch et al., 1994; Adler et al., 2000; Viol et al., 2019), they have also been individuated as core regions activating during alcohol cue processing (Heinz et al., 2009; Zakiniaeiz et al., 2016; Grodin et al., 2018; Hanlon et al., 2018) and may be considered as promising new targets for dTMS in AUD (Ceccanti et al., 2015; Herremans et al., 2016).

### SUMMARY AND PERSPECTIVE

Findings suggest that later stages of AUD may be better conceptualized as a disorder characterized by compulsive features, namely overreliance on stimulus-driven habits at the expense of flexible, goal-directed action, leading to frequent and persistent substance use despite serious negative consequences (Belin and Everitt, 2008; Fontenelle et al., 2011). In other words, although alcohol seeking is initially a goal-directed behavior consolidated by operant conditioning, in which alcohol is sought for its rewarding effect, it becomes ultimately a maladaptive optimized response elicited by alcohol-associated stimuli, characterized by over-active striatal habit forming circuitries coupled with lack of sufficient inhibitory control (Chamberlain et al., 2005; Vanderschuren and Everitt, 2005; Menzies et al., 2008; Smith et al., 2014). As seen in OCD, there is a lack of extinction of obsessions (Lovibond et al., 2009), in AUD there is disruption in extinction learning of ethanolseeking behavior with persistency of the behavior overtime despite adverse consequences (Gass et al., 2017; Grodin et al., 2018). Development of compulsivity may thus explain part of the treatment resistance and relapse in AUD (Sinha et al., 2011; Courtney et al., 2013; Lee et al., 2013). Despite the residual presence of a reward component in driving the behavior, craving for alcohol in the late stages of the disorder is comparable to a compulsion, in phenomenology – in the emergence of urges in response to alcohol-related cues and the inability to resist themand in neurocircuitry.

The field of cognitive neuroscience can provide measures that are a reflection of the underlying neurobiology, and may eventually inform treatment selection. Disruption in inhibitory control has been proposed as endophenotype and pathophysiologic factor in the development of OCD and AUD (Chamberlain et al., 2007; Yücel and Lubman, 2007; Menzies et al., 2008; de Wit S.J. et al., 2012; Jentsch and Pennington, 2014; Leeman et al., 2014; Wilcox et al., 2014; Gillan et al., 2016). In particular, impaired response inhibition is associated with

BOX 3 | Repetitive transcranial magnetic stimulation as a tool for personalized psychiatry. In rTMS, single TMS pulses are delivered at various frequency (typically 1–20 Hz) in either a fixed or bursting pattern from 600 to 4000 pulses per session. There is general agreement that low frequency (LF) stimulation (e.g., 1 Hz) causes long term depression of cortical excitability, whereas higher frequency (HF) stimulation (e.g., 10–20 HZ) induces long term potentiation of cortical excitability. These effects can be achieved through teta burst stimulation (TBS). With continuous TBS (cTBS), three pulse bursts at 50 Hz are applied at a frequency of 5 Hz. In most protocols, this cycle continues until 600 pulses have been delivered (20 s). For intermittent TBS (iTBS), bursts are applied at the same rate (five groups of three pulse bursts per second) for 2 s, followed by an 8-s pause. In most protocols this 10-s cycle occurs until 600 pulses have been delivered (190 s). When performed over the primary motor cortex, 600 pulses of cTBS inhibit cortical excitability, whereas 600 pulses of iTBS amplify cortical excitability (Huang et al., 2005). The advantage of TBS protocols is that effect sizes comparable to fixed frequency protocols can be achieved significantly faster (1–2 min versus 20–30 min). The spatial resolution and penetration depth of a TMS pulse depend on the coil. Typical figure-of-eight coil affects approximately 10 cm<sup>2</sup> of cortical surface, while H shape coil design approximately 100 cm<sup>2</sup> . Most flat coil designs have penetration depths from 1 to 2 cm, whereas the H-coil designs has higer depths of 2–3 cm (Deng et al., 2014). The introduction of H-coils has offered the opportunity of non-invasively modulate activity in brain targets that previously were accessible only by neurosurgial procedures. The combined use of neuronavigation and neuroimaging with rTMS, makes of this latter a feasible therapeutic tool for personalized psychiatry.

severity, duration of illness, impaired goal-directed planning and reduced frontostriatal connectivity in AUD. These findings suggest that in AUD response inhibition may be used as a useful marker of cognitive impairment, disruption in connectivity between frontal (dlPFC and dACC) and striatal nodes, as well as a potential treatment target. Recent studies have demonstrated how brain stimulation techniques may affect response inhibition, suggesting how non-invasive neuromulation may be particularly promising in order to develop treatments whose effect in drinking outcome are mediated by improvement in cognitive control (Nakamura-Palacios et al., 2012; Nardone et al., 2012). Future work is still needed to clearly identify the most reliable and valid markers for the deficits, and the degree to which deficits or changes in cognitive control moderate or mediate response to particular treatments in AUD. Recent findings point to the hypotheses that cognitive impairments in AUD are related to dysfunctional connectivity between cortical (dlPFC and dACC) and subcortical nodes (basal ganglia). Thus, studies aimed to characterize instrinsic resting state connectivity and hubs in the reward, salience and executive networks in AUD patients may be particularly useful for the development of new treatment strategies.

Treatment with rTMS is still in its infancy but is a promising tool for developing effective and viable circuit-specific treatment strategies in AUD. The current evidence suggests that targeting inflexible seeking responses may offer a therapeutic strategy to promote abstinence and prevent relapse in AUD. The results of the available studies, in accordance with recent network models of addiction (Dunlop et al., 2017) seems to point to the potential of mPFC and dACC (**Box 3** and **Figure 1**) as TMS targets for the treatment of compulsive alcohol seeking in AUD. The available findings suggest that those areas may be pivotal in order to enhance cortico-striatal-thalamic connectivity and capacity for response selection/inhibition, with the potential to act on the two core aspects of incentive salience and habit learning (**Box 3** and **Figure 1**). Future efforts to improve outcome for rTMS in AUD will likely benefit from rigorous manipulation of the cognitive state during neuromodulation.

### CONCLUSION

Despite effective treatments that are available in AUD, there remain high rates of relapse and poor long term functioning even in those patients who get therapy. We describe evidence supporting the role of compulsivity (persistent use despite adverse consequences), in the development of AUD resistant

#### REFERENCES


to approved interventions. Following a transdiagnostical approach and using OCD as comparison, we highlight features of compulsivity in AUD, showing how phenotypical similarities between the two disorders involve overlapping pathophysiological mechanisms. After identifying compulsivity as a promising and neglected target domain for new treatment approaches in AUD, we discuss neuromodulatory interventions in order to improve recovery in AUD.

For disorders, such as AUD, whose development relies on learning, rTMS gives the unique opportunity to non-invasively act on target neurocircuits with the best space-time resolution. The possibility to couple rTMS with specific tasks designed to activate specific associated circuits allows unique opportunity for personalized therapy. The current evidence seems to point to the potential of mPFC and dACC as TMS targets for the treatment of compulsive alcohol seeking in AUD. Further longitudinal studies combining neuromodulation with neuroimaging and neurocognitive measures are needed to shed light on additional mechanisms underlying rTMS effects in AUD patients, and to test the hypotheses that targeting compulsivity, the habit system and inhibitory control, normalizing fronto-striatal function may convey treatment benefit in AUD.

In conclusion, the conceptualization of AUD within a dimensional and learning framework opens new opportunities for research and clinical management:


### AUTHOR CONTRIBUTIONS

EB and EH contributed to the conceptualization of the manuscript. EB wrote the manuscript. NM wrote **Box 3**. All authors contributed to manuscript revisions, and read and approved the submitted version.


disorder: a meta-analysis of treatment outcome and predictors of response. PLoS One 10:e0133591. doi: 10.1371/journal.pone.0133591



behavior and habit learning in obsessive-compulsive disorder. Am. J. Psychiatry 168, 718–726. doi: 10.1176/appi.ajp.2011.10071062


prefrontal HF-rTMS session. Alcohol Alcohol. 48, 552–557. doi: 10.1093/alcalc/ agt054


neuropsychological studies of obsessive-compulsive disorder: the orbitofrontostriatal model revisited. Neurosci. Biobehav. Rev. 32, 525–549. doi: 10.1016/j. neubiorev.2007.09.005


traits in relation to a family history of alcoholism. Alcohol. Clin. Exp. Res. 32, 888–894. doi: 10.1111/j.1530-0277.2008.00648



Zhu, X., Cortes, C. R., Mathur, K., Tomasi, D., and Momenan, R. (2017). Modelfree functional connectivity and impulsivity correlates of alcohol dependence: a resting-state study. Addict. Biol. 22, 206–217. doi: 10.1111/adb.12272

**Conflict of Interest Statement:** EH received a research grant to study dTMS in OCD from BrainsWay.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Burchi, Makris, Lee, Pallanti and Hollander. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Erroneously Disgusted: fMRI Study Supports Disgust-Related Neural Reuse in Obsessive-Compulsive Disorder (OCD)

Kathrin Viol 1,2\*, Benjamin Aas 3,4 , Anna Kastinger 2,5 , Martin Kronbichler 6,7 , Helmut Johannes Schöller 1,2 , Eva-Maria Reiter 7,8 , Sarah Said-Yürekli 6,7,9 , Lisa Kronbichler 6,7 , Brigitte Kravanja-Spannberger <sup>2</sup> , Barbara Stöger-Schmidinger <sup>2</sup> , Wolfgang Aichhorn1,2 and Guenter Karl Schiepek 1,2,3

1 Institute for Synergetics and Psychotherapy Research, Paracelsus Medical University, Salzburg, Austria, <sup>2</sup>Department of Psychosomatics and Inpatient Psychotherapy, Paracelsus Medical University, Salzburg, Austria, <sup>3</sup>Department of Psychology, Ludwig Maximilian University of Munich, Munich, Germany, <sup>4</sup>Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital, Ludwig Maximilian University, Munich, Germany, <sup>5</sup>Department for Psychosomatics and Inpatient Psychotherapy, Paracelsus Medical University, Salzburg, Austria, <sup>6</sup>Centre for Cognitive Neuroscience, University of Salzburg, Salzburg, Austria, <sup>7</sup>Neuroscience Institute, Christian Doppler Clinic, University Hospital Salzburg, Salzburg, Austria, <sup>8</sup>Department for Radiotherapy and Radio-Oncology, Christian Doppler Clinic, University Hospital Salzburg, Salzburg, Austria, <sup>9</sup>Department of Neurology, Christian-Doppler Medical Center, Paracelsus Medical University, Salzburg, Austria

#### Edited by:

David Belin, University of Cambridge, United Kingdom

#### Reviewed by:

Alexander Nikolaevich Savostyanov, State Scientific-Research Institute of Physiology & Basic Medicine, Russia Stephan Bohlhalter, University of Bern, Switzerland

> \*Correspondence: Kathrin Viol k.viol@salk.at

Received: 23 November 2018 Accepted: 03 April 2019 Published: 24 April 2019

#### Citation:

Viol K, Aas B, Kastinger A, Kronbichler M, Schöller HJ, Reiter E-M, Said-Yürekli S, Kronbichler L, Kravanja-Spannberger B, Stöger-Schmidinger B, Aichhorn W and Schiepek GK (2019) Erroneously Disgusted: fMRI Study Supports Disgust-Related Neural Reuse in Obsessive-Compulsive Disorder (OCD). Front. Behav. Neurosci. 13:81. doi: 10.3389/fnbeh.2019.00081 Objective: fMRI scans of patients with obsessive-compulsive disorder (OCD) consistently show a hyperactivity of the insular cortex, a region responsible for disgust-processing, when confronted with symptom-triggering stimuli. This asks for an investigation of the role of disgust and the insula in OCD patients.

Methods: Seventeen inpatients with OCD and 17 healthy controls (HC) underwent fMRI scanning. Whole-brain contrasts were calculated for "Disgust vs. Neutral" for both groups, plus an analysis of variance (ANOVA) to assess the interaction between group and condition. Additionally, the emotional dimensions of valence and arousal, along with the ability to cope, were assessed by picture ratings.

Results: The picture ratings confirmed the patients' heightened sensitivity to disgust with higher values for arousal and inability to cope, but not for valence. fMRI scans revealed no hyperactivity of the insula in patients compared to controls for the condition "Disgust vs. Neutral," indicating no basic hypersensitivity to disgusting stimuli. Increased activity in the precuneus in controls for this condition might correspond to the downregulation of arousal.

Conclusions: The absent differences in neural activity of the insula in patients compared to controls for the disgust-condition, but heightened activity for symptom-provoking conditions, suggests that the illness is due to an erroneous recruitment of the insula cortex for OCD-stimuli. The finding is interpreted within the framework of the neural reuse hypothesis.

Keywords: OCD, fMRI, disgust, insular cortex, precuneus, neural reuse, picture rating

## INTRODUCTION

With a prevalence of 2%–3%, obsessive-compulsive disorder (OCD) is one of the most common psychiatric diseases and has a serious impact on the quality of life (Crino et al., 2005). Individuals with OCD experience a persistent intrusion of unwanted thoughts or images (obsessions) and/or the urge for repetitive, ritualistic behaviors or mental acts (compulsions) that need to be neutralized in order to reduce anxiety or distress (American Psychiatric Association, 2013). Although the etiology of OCD has not been fully understood yet, a line of research suggests that disgust proneness, i.e., a disposition or personality trait towards enhanced reactivity to disgust, may play an important role (Olatunji et al., 2011). The theory has been backed by empirical studies revealing elevated self-reported ratings for disgust in clinical and non-clinical samples (e.g., Olatunji et al., 2011; Berle and Phillips, 2006; Whitton et al., 2015). The aim of this study was to investigate if a heightened sensitivity to disgusting pictures can also be found in our sample and if this hypersensitivity corresponds to differences in disgust processing in the brain in OCD patients. We were especially interested in the role of the insular cortex since this region has been identified as relevant for disgust processing (next to a variety of other functions) in patients as well as in healthy participants. Group differences were found in both functional imaging and lesion studies (e.g., Wright et al., 2004; Knowles et al., 2018), especially when highly disgusting stimuli were used (Oaten et al., 2018).

Next to its response to disgust, the insular cortex is commonly activated in OCD patients when confronted with OCD-related stimuli. These OCD-related pictures are designed to provoke the symptoms of patients and show situations that OCD patients experience as triggering but are commonly perceived as neutral by healthy participants. For example, a clean toilet is often reported to provoke the patients' urge to continue cleaning (in the disgusting pictures condition, in contrast, a dirty toilet would be shown that provokes emotions of disgust also in healthy participants). Numerous studies have reported the insula's activation in such OCD-related conditions in patients but not in controls (e.g., Schienle et al., 2005; Schiepek et al., 2013; Thiel et al., 2014), especially also for the sample of this study (Viol et al., 2019). This suggests that the hypersensitivity to disgust in the pictures ratings of OCD patients corresponds to a hyperactivity of the insular cortex. However, the difference in the activity of the insula between patients and controls in a pure disgustbased paradigm revealed heterogeneous results. While Schienle et al. (2005) and Berlin et al. (2015) found greater activation of the insula in response to disgust-inducing images in OCD patients compared to controls, no such group difference was found in Thiel et al. (2014) and Weygandt et al. (2012). Moreover, no correlation could be found between OCD symptom severity and insular activity (e.g., Berlin et al., 2017). Overall, it is still unclear if OCD is related to abnormal disgust processing, and if a general hyperactivity of the insular cortex might cause the patients' hypersensitivity to disgust.

A note should be made on the heterogeneity of situations that provoke symptoms in OCD patients. Of course, a possible overlap between disgust-reaction and OCD-symptoms seems obvious for patients from the washing subtype, who fear contamination by disgust-related stimuli in the environment. However, it has been suggested that disgust may also play a fundamental role for the other OCD subtypes, i.e., symmetry/ordering, hoarding, and checking (e.g., Thorpe et al., 2003; Taylor and Liberzon, 2007). This observation makes sense in the context that feelings of disgust may not only arise from sensory experiences (e.g., taste, smell), but also from more abstract concerns, e.g., moral judgments (Rozin et al., 1999). Increased scores of disgust ratings also correlated with self-reported checking, ordering, neutralizing, and obsessing symptoms and even for the hoarding subtype (Olatunji et al., 2011). The findings remained significant after controlling for heightened anxiety, depression, and general fearfulness (Olatunji et al., 2005, 2011) and speak in favor of the hypothesis that the primarily ''protective function of disgust has been extended from physical to psychological contamination'' (Olatunji et al., 2011, p. 933). In conclusion, a possible hypersensitivity to disgust can not only be postulated for patients from the washing/contamination-subtype, but might be a general aspect of the illness.

As hypotheses, heightened scores were expected for patients in comparison to controls for the picture ratings for disgusting pictures in all dimensions (valence, coping and arousal), representing a more negative perception and emotional response. Based on the suggested biological hypersensitivity to disgust of OCD patients, we expected heightened neural activity in the insular cortex compared to controls for the contrast ''disgusting vs. neutral pictures.''

### MATERIALS AND METHODS

#### Study Procedures

Within the 1st week of hospitalization, patients underwent an fMRI scan, followed by a picture rating including the emotional dimensions valence, arousal, and coping of the pictures seen in the scanner. The controls were shown the pictures of their respective patients and also rated the pictures afterward. Clinical symptoms were assessed with the Yale-Brown Obsessive Compulsive Scale (Y-BOCS, Goodman et al., 1989; German version: Hand and Buettner-Westphal, 1991), the Symptom Checklist-90-R (SCL-90-R, Derogatis et al., 1977; German version: Glöckner-Rist and Stieglitz, 2012), the Depression-Anxiety-Stress Scales (Lovibond and Lovibond, 1995; German version: Nilges and Essau, 2015), and the Beck Depression Inventory II (BDI-II, Beck et al., 1996; German version: Hautzinger et al., 2009).

#### Participants

The sample consisted of 17 inpatients [six men and 11 women, mean age 43.5 years (SD = 10.7)], receiving an integrative treatment approach including cognitive-behavioral components at the Christian-Doppler University Hospital, Salzburg, Austria, as well as 17 healthy controls (HC) matched by age and gender. Patients were eligible to participate in the study if OCD was the main illness by clinical judgment based on ICD-10 and DSM-IV criteria and on the Structured Clinical Interview for DSM-IV Axis I disorders (SCID-I, First et al., 2002). Exclusion criteria consisted of neurological impairment and/or neurological diseases, acute psychosis, substance abuse, and/or suicidality. The mean Y-BOCS score of 26.7 (SD = 8.8) ranks the sample on the medium to the upper end of symptom severity. Comorbidities, as commonly found in OCD patients, included depression (eight patients), social phobia (two patients, in addition to depression) and one each from the schizophrenic spectrum, alcohol and substance abuse (currently abstinent), and posttraumatic stress disorder (PTSD). The mean BDI-II score of patients was 29 (SD = 9.4) and 1.2 (SD = 1.5, p < 0.001) for controls. All but one patient took some kind of antidepressant (mostly SSRI), seven of them in addition neuroleptics, three anticonvulsants, two benzodiazepine and one lithium. One patient also had to be medicated for high blood pressure, thyroid dysfunction and incontinence. The study was approved by the Ethics Commission Salzburg (Ethikkommission Land Salzburg, No. 415-E/1203/5-2012). Detailed information on the study was provided and written informed consent was obtained from all participants according to the Declaration of Helsinki.

#### Stimuli

For symptom provocation during the fMRI scan, patients as well as controls were shown colored pictures from four categories, namely individual OCD-related photos, standardized OCD-related photos from the Maudsley Obsessive-Compulsive Stimulus Set (MOCCS, Mataix-Cols et al., 2009), and disgusting and neutral pictures from the International Affective Pictures Set (IAPS, Lang et al., 2008). For details on the acquisition and selection process of the individual pictures see Viol et al. (2019). Examples of the disgusting pictures are depicted in **Figure 1**. Participants were instructed to passively view the pictures.

In the scanner, the pictures were displayed on an MRI compatible LCD screen by Conrac (distributed by Siemens) placed at the back of the MRI bore with a size of 18 inch and a resolution of 1,280 × 1,024 pixel. Participants were able to see the pictures on a mirror mounted directly above the participants' heads on the head coil in 145 mm distance. The original picture size was 750 × 750 pixel.

#### Picture Rating

Immediately after the fMRI scan, participants rated the pictures they have just seen in the scanner. Ratings were conducted computer-based with the E-Prime 2.0 presentation software<sup>1</sup> . First, the concept of valence and arousal as the two dimensions of emotions (Bradley et al., 1992) and coping (i.e., how well the participants felt that they could handle the situation) was explained to the patients, followed by a short introduction and training on the software. Two-sided t-tests were calculated for differences in groups and corrected for false discovery rates (FDR).

#### fMRI Data Acquisition

Images were acquired with a 3T Siemens TIM TRIO whole-body scanner (Siemens Symphony, Erlangen, Germany) with a 32-channel head coil. First, a high-resolution scan was acquired for anatomical referencing using a T1-weighted MPRAGE sequence (FoV: 256 mm, slice thickness: 1.0 mm, TR: 2,300 ms, flip angle: 9◦ , resolution: 1 × 1 × 1 mm). Functional images were obtained in two sessions with a short pause in between. Concerning the functional imaging, a total of 552 volumes were acquired using a T2<sup>∗</sup> -weighted gradient echo EPI with 36 slices (slice thickness: 3 mm, descending slice order, TR: 2,250 ms, TE: 30 ms, flip angle: 70◦ , FoV: 192 mm). The first six volumes of each functional session were discarded due to saturation effects (Sarty, 2007), leaving a total of 540 volumes. Stimuli were presented with the E-Prime 2.0 presentation software<sup>1</sup> as an event-related design in a pseudo-randomized order in two sessions (20 pictures of each category in each session). The pictures were shown for 4 s each, separated by a fixation cross; the inter-stimulus interval was 2 s. The resulting DICOM files were converted to 4D-NIfTI-files with the tools MRIConvert<sup>2</sup> and dcm2nii<sup>3</sup> .

#### Preprocessing

Preprocessing and statistical analysis were performed using the Statistical Parametric Mapping software package SPM12 (Wellcome Department of Cognitive Neurology, London, UK) implemented in Matlab (Mathworks, Inc., Natick, MA, USA, release 13a). Functional images were realigned to the first image, de-spiked with the AFNI 3d-despike function<sup>4</sup> , unwarped, corrected for geometric distortions using the field map of each participant, and slice time corrected.

The high resolution structural T1-weighted image of each participant was processed and normalized with the CAT12 toolbox<sup>5</sup> using default settings. Each structural image was segmented into gray matter, white matter and CSF, and denoised, then warped into MNI space by registration to the DARTEL template provided by the CAT12 toolbox via the high-dimensional DARTEL registration algorithm (Ashburner, 2007). Based on these steps, a skull-stripped version of each image in native space was created. To normalize functional images into MNI space, the functional images were coregistered to the skull stripped structural image and the parameters from the DARTEL registration were used to warp the functional images, which were resampled to 3 mm × 3 mm × 3 mm voxels and smoothed with a 6 mm FWHM Gaussian kernel. The quality of the preprocessing was checked using the tools BXH<sup>6</sup> and tsdiffana<sup>7</sup> .

#### Statistical Analysis

Since SPM uses a mass-univariate approach, the effects of the conditions were modeled for each voxel with a general linear model (Kiebel and Holmes, 2008). The movement parameters gained from the realignment procedure during preprocessing were used as regressors. A 2 × 2 analysis of variance (ANOVA) was calculated using the Multivariate and Repeated Measures

<sup>1</sup>http://www.pstnet.com

<sup>2</sup>https://lcni.uoregon.edu/downloads/mriconvert/mriconvert-and-mcverter

<sup>3</sup>http://www.mccauslandcenter.sc.edu/crnl/tools

<sup>4</sup>https://afni.nimh.nih.gov

<sup>5</sup>http://dbm.neuro.uni-jena.de/cat <sup>6</sup>https://wiki.biac.duke.edu/biac:xmlheader

<sup>7</sup>http://imaging.mrc-cbu.cam.ac.uk/imaging/DataDiagnostics

TABLE 1 | Arithmetic mean values of the picture rating for the dimensions valence, arousal and ability to cope.


<sup>∗</sup>p < 0.05. In brackets: SD. p: p-values of two-sided t-test with H0: mean (patients) = mean (controls), FDR-corrected.

(MRM)<sup>8</sup> toolbox for MATLAB provided by the University of Manchester. A within-subject factor (disgust and neutral pictures) was modeled along with a between-subject factor (patients and controls). Unless otherwise stated, p-values have been corrected for multiple comparisons (FWEPeak-correction).

A ROI analysis of the left and right insula was performed to further investigate the differences in activation between groups. In addition, the betas, i.e., the estimated β-parameters from the general linear model, were extracted for each participant. The values were extracted for the contrast ''Disgust vs. Neutral'' at the left insular cortex, using the voxel with maximal T-value (**Table 2**) at group level.

Note that the current study is part of a larger project including 4–5 scans during psychotherapy. The following sections report information and results relevant for the above-mentioned hypotheses only, i.e., data from the first fMRI-scans at the very beginning of the therapy, since the group differences should be greatest at this point.

## RESULTS

#### Picture Ratings

Patients scored significantly higher in arousal and assessed their ability to cope lower than controls for the disgusting pictures, but there was no difference in the rating of the valence (**Table 1**). The results are visualized in **Figure 2**.

Further investigation of the relationship between arousal and coping revealed a correlation of r = −0.99 for both patients

and controls; an ANOVA confirmed no effect of group (p = 0). Coping can, therefore, be fully explained by arousal (R <sup>2</sup> = 1).

#### Whole-Brain Analysis

A 2 × 2 ANOVA on whole-brain level with the betweensubject factor ''group'' (patients, controls) and the within-subject factor ''condition'' (disgust, neutral pictures) was performed to investigate the difference between groups and conditions. No interaction effect was found for p < 0.05 FWE-corrected on peak-level. When lowering the level of significance to

<sup>8</sup>https://www.click2go.umip.com/i/s\_w/mrm.html

p < 0.001 and using p < 0.05 FWE-corrected on cluster level, a cluster in the left precuneus/superior parietal lobe with a peak at [−24, −67, 40] became significant. The contrast estimates revealed a heightened activation in the disgust condition for the controls (**Figure 3**).

#### ROI-Analysis of the Insula Cortex

To specifically investigate the role of the insula in disgust processing, a ROI analysis of the insula cortex was performed for patients and controls for the contrast ''Disgust > Neutral'' (**Table 2**). Significant activation of the left and right insula was found for the controls for p < 0.05 FWE (cluster level, based on ppeak < 0.001). For patients, the activity was found in the left insula only and did not survive FWE correction on peak or cluster level. No significant activation was found for the opposite contrast (''Neutral > Disgust'') for neither controls nor patients.

Finally, the activity in the left insular cortex of patients and controls was further explored with the beta values, interpretable as the height of neural activity, of each participant. The result is presented in **Figure 4**. A two-sided t-test confirmed the assumption from **Table 2** that the activity in the left insular cortex of patients is indeed lower than that of controls (p = 0.002).

TABLE 2 | ROI analysis of the insula for the contrast "Disgust > Neutral" for patients and controls.


<sup>∗</sup>p < 0.0001 uncorrected with extent threshold = 10 voxels, ∗∗p < 0.05 FWECluster, ∗∗∗p < 0.05 FWEPeak.

FIGURE 4 | Beta-values of the left insular cortex in patients (left) and controls (right) for the contrast "Disgust vs. Neutral" reveal a lower neural activity, indicating no hyperactivity to disgust in obsessive-compulsive disorder (OCD) patients. 50% of the values are within the boxes, with the median marked as a horizontal line. The upper and lower 25% is indicated by the whiskers.

### DISCUSSION

#### OCD Affects Arousal and Ability to Cope Rather Than Emotional Valence of Disgust

The heightened sensitivity of OCD patients to disgusting pictures has been shown frequently in picture ratings. Our results of the picture ratings confirm this finding and allow to further differentiate the emotional dimension of this hypersensitivity. Notably, the valence of the disgusting pictures, i.e., how pleasant the pictures were rated, did not differ between groups. While the evaluation of the pictures was comparable between patients and controls, the reaction of patients was indeed significantly different: patients were more aroused and felt less able to cope with the depicted situations. In other words, OCD sufferers perceive disgust-related situations no different to healthy people, but become more agitated and feel less able to deal with the situation due to the heightened arousal. It seems like the hypersensitivity might actually be a problem of regulating arousal (**Figure 5**).

#### Group Differences in Neural Activity for Disgust

The differences in neural activation between patients and controls for disgusting stimuli compared to neutral pictures was assessed in two ways, first by an ANOVA on whole-brain level, i.e., assessing all voxels in the brain. Second, a ROI analysis was conducted for the insula cortex, which is known to be involved in disgust processing.

The whole-brain ANOVA did not reveal any different neural activity when the correction for multiple comparisons (FWE) was applied to each voxel. If the threshold was lowered to p < 0.001 per voxel and the FWE-correction was performed for clusters, heightened activation was found for the left precuneus

in controls for the disgust condition. The precuneus is known for several functions concerning general emotional processing, but not specific for disgust. It rather seems that the region is involved in emotion regulation, as it was linked to the intensity of emotions (Sato et al., 2015). Bestelmeyer et al. (2017) found activity in the precuneus for both valence and arousal, suggesting a modulating role. Linking these results with the heightened arousal reported by patients in the picture rating, one might suggest that the precuneus' activity is a neural correlate of the suppression of arousal in controls. Similar results have been reported by Ashworth et al. (2011), where patients with bulimia nervosa showed reduced activity in the precuneus when confronted with facial expressions of disgust and anger compared to controls. In conclusion, the role of the precuneus in emotion processing might be relevant for several psychiatric disorders but does not seem to be specific for disgust or OCD.

The ROI analysis of the insula for the difference between disgusting and neutral pictures confirmed the region's involvement in disgust processing. However, the group difference was not significant. An examination of the beta-values (interpretable as the neural activity at a specific voxel) shows increased activity in controls compared to patients. This is in opposition to our hypothesis of a hyperactivity in the insula in patients as an underlying mechanism for OCD. In other words, no basic neural hypersensitivity to disgust can be found in patients.

#### The Neural Reuse Hypothesis

What remains is the question of why the insular cortex is consistently reported as hyperactive in symptom-provoking conditions in OCD patients compared to controls, but not in the disgust-condition. As reported in detail in Viol et al. (2019), a significant group difference (p < 0.05, FWEPeakcorrected) was also found in this sample in the insula for the individual symptom-provoking OCD-stimuli vs. neutral pictures. Moreover, in the disgust condition, no abnormal activity was found in any of the regions commonly hyperactive in OCD, neither in the ACC, which is supposed to play the role of a conflict monitoring unit, nor in the frontal cortex, where executive functioning, impulse control and emotion regulation is assumed to take place. It can, therefore, be excluded that a general hypersensitivity of the insular cortex is responsible for heightened disgust-proneness in OCD patients. Still, its activation in symptom-provoking situations like standing in front of a clean toilet causes the neurons in the insula to fire more than in controls. It seems like a normally non-disgusting stimulus activates the insula in OCD patients, but not in controls. The patients ''abnormal'' insular response to symptom-provoking, but not to disgusting pictures, begs an explanation, which might be provided by the neural reuse hypothesis. The theory of neural reuse was originally developed in the context of brain development. The concept is based on the observation that a fundamental organizational principle of the brain seems to consist of acquiring already existing neural circuits to accomplish a new task. It is used as a possible explanation for how the brain is able to deal with tasks that it was originally, i.e., from an evolutionary point of view, not developed for. Such tasks include the acquisition of language, i.e., pathways supporting primate tool use have been extended to human language (Anderson, 2010, 2016; Glenberg and Gallese, 2012). Further studies also linked mathematical operations (Dehaene and Cohen, 2007) and emotional reactions to abstract stimuli like disgusting words (Ziegler et al., 2018) to this mechanism. By this, the theory is closely related to the idea of embodied cognition and mirror neurons (Gallese, 2008); a theoretical framework based on dynamic systems theory, and the reuse of common attractors was proposed by Candadai and Izquierdo (2018). Note that the idea differs from the concept of neural plasticity insofar as is does not (only) focus on lesions or loss of brain function, but suggests that new uses can be acquired to be used next to its primary purpose if a suitable function or structure for a particular new task is already existent in the brain. The recycling of a given system is therefore supposed to happen with only minor changes to the original structure, e.g., by establishing a functional connection to different regions, and without altering the original functionality (Anderson, 2010).

So far, neural reuse is used to explain a normal development, i.e., how the brain's structure and function are able to adapt to new tasks in the environment by searching (on a macroscopic level) its existing neural circuits that might fit to solve the problem. This search mechanism is yet unknown, but most likely based on a probabilistic evaluation of patterns (Lu, 2016). Naturally, this process is not error-save and might go wrong. With reference to OCD, a disease often triggered by stressful or traumatic life events (Real et al., 2011), this could mean that in search of a way to deal with this event, the brain erroneously comes to the conclusion that the disgust-mechanism is helpful, e.g., by providing a possibility to regain the feeling of control. In consequence, non-disgusting stimuli like the hands that have just been cleaned are still assessed as contaminated, i.e., these stimuli are now associated with the neural circuits for disgust processing. The aberrant focus on the disgust processing units of the brain might be governed or mediated by genetic and/or neurochemical abnormalities frequently found in OCD sufferers and their siblings (Hettema et al., 2001). Since neural reuse is thought to influence the neural plasticity on a local level (Hebbian learning, Anderson, 2016), OCD can manifest and become chronic.

The principle has recently been shown to be valid for the emotion of disgust insofar that the insular cortex is activated by reading disgust words—which themselves are of course not disgusting (Ziegler et al., 2018). Further evidence is provided by our finding of heightened values in the picture ratings and the involvement of the insula as a neural correlate of OCD, i.e., disgust does indeed play an important role in OCD. However, the missing group difference in the neural activation in the disgust-condition shows that the basic processing of disgust-related stimuli is not altered in OCD. One might, therefore, speculate that the insula—while functioning normally for disgust—is falsely recruited during the development of OCD and is now active in non-disgusting, but OCD-specific situations, too. This is supported by findings of Belin-Rauscent et al. (2016), who showed that individual vulnerability to compulsive behavior in rats is based on abnormalities in the anterior insula.

To conclude, the neural reuse hypothesis suggests that a primary neural activity (disgust processing) is reused by patients in situations that are not disgusting. On the one hand, this can explain why OCD-patients from the washing subtype experience their just extensively cleaned hands/items as still disgusting, and on the other hand, why disgust is also relevant for OCD patients from the other subtypes. For the latter, the fear of losing control and harming oneself and/or others lays behind the compulsions to check and control. Accompanying obsessive thoughts focus on the possibility of being aggressive, engaging in unwanted sexual behaviors, violating religious rules, or otherwise engaging in immoral behavior. Such morally unacceptable behavior elicits what is described as ''moral disgust,'' i.e., its original functionality has been expanded on the psychological as well as on the neural level. Last but not least, the theory might even explain the result of lower insular activity in patients than controls: by being recruited for OCD-specific stimuli, which are present in patients during considerable hours of the day, the region is highly occupied and might not have enough resources left for other tasks like normal disgust-processing.

#### Limitations

Some limitations have to be taken into consideration with regard to this study. First, patients were not medication-naïve. Even though psychotropic drugs are specifically designed to alter neural activity, it can be assumed that—if the patient still meets the criteria for OCD—the drug was not able to change the disease-specific activation to a non-clinical level. Second, comorbidities, especially with major depressive disorder, are common in OCD (Schiepek et al., 2011). In consequence, it cannot be excluded that some of the pictures also provoked altered neural activation due to a comorbidity in depression.

Concerning the interpretation of the results, one has to acknowledge the fact that the insula seems to be involved in a variety of other cognitive and emotional processes. With our research design, it cannot be fully excluded that its activity is due to e.g., the processing of fear since there was no such control condition available. Still, prior research and the missing correlation with the factor ''fear'' of the DASS (not reported) suggests that the insula is actually responding to disgust in this paradigm.

### Future Research

The results of this study can be fully integrated in the neural reuse hypothesis, but are not able to fully answer all questions concerned with this hypothesis. These should be focused in future research and investigate e.g., the identification of a general ''disgust processing system'' of the brain. While we were able to show that OCD-related stimuli recruit neural circuits of the insular cortex, one could speculate that other brain regions are also involved in the processing of disgust. To analyze these possible additional regions would be important in order to determine if the disgust-related activation in OCD is due to an abnormal connectivity with the insula only, or if other connections are altered, too. In this context, it is also not clear which regions are erroneously connected to the insula, i.e., how it is recruited. In addition, to further understand the etiology of OCD, it would be important to develop a theory on why the disgust system is reused, e.g., what problems or tasks existed in the environment of patients that required the brain to recruit the disgust circuits, and/or if there is a genetic disposition. With regard to psychotherapy, ideas on how can this concept can be used in the development of therapies for OCD should be investigated. Last but not least, considering the concept as a possible explanation for other psychiatric disorders—as discussed for social anxiety disorder by Varlet et al. (2014)—might improve understanding and treatment.

#### CONCLUSION

The study confirms prior findings on heightened reactivity to disgust in OCD patients, but only on the psychological level. In the fMRI scans, no difference was found in activation of the insula between patients and controls, suggesting a normal functioning of the basic disgust processing. However, the insular cortex, known to be relevant for disgust processing, is commonly activated during symptom-provoking conditions, suggesting an erroneous recruitment of neural disgust processing circuits. The finding is interpreted as a neural reuse ''gone wrong.'' The theory is especially appealing because it is based on an existing neural mechanism and goes in line with the economical/parsimonious aspect of evolution. Moreover, it provides a theoretical framework to integrate prior findings on the heightened sensitivity to disgust also for patients from subtypes other than washing.

#### ETHICS STATEMENT

This study was carried out in accordance with the Ethics Commission Salzburg (Ethikkommission Land Salzburg, No. 415-E/1203/5-2012) with written informed consent from all subjects. All subjects gave written informed consent in

#### REFERENCES


accordance with the Declaration of Helsinki. The protocol was approved by the Ethics Commission Salzburg.

#### AUTHOR CONTRIBUTIONS

KV analyzed the data and wrote the manuscript. BA and AK took the individual pictures with the patients at their homes, conducted the psychological tests and interviews and realized the picture ratings. MK set up the fMRI procedure and helped to analyze and interpret the data and advised on the statistical analysis. HS performed the statistical analysis of the picture ratings. E-MR realized the fMRI scans. SS-Y and LK prepared the scripts for fMRI analyses. BK-S and BS-S recruited the participants and gave information about the study. WA supervised the study. GS designed and supervised the study. All authors interpreted the results, contributed to manuscript revision, read, and approved the submitted version.


compulsive disorder. Psychiatry Res. 168, 238–241. doi: 10.1016/j.psychres. 2008.05.007


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Viol, Aas, Kastinger, Kronbichler, Schöller, Reiter, Said-Yürekli, Kronbichler, Kravanja-Spannberger, Stöger-Schmidinger, Aichhorn and Schiepek. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Habit Reversal Therapy in Obsessive Compulsive Related Disorders: A Systematic Review of the Evidence and CONSORT Evaluation of Randomized Controlled Trials

Melissa T. Lee<sup>1</sup> \*, Davis N. Mpavaenda1,2 and Naomi A. Fineberg1,2

<sup>1</sup> Hertfordshire Partnership University NHS Foundation Trust, Welwyn Garden City, United Kingdom, <sup>2</sup> University of Hertfordshire, Hatfield, United Kingdom

Background: Habit Reversal Therapy (HRT) has long been used in the treatment of Tourette Syndrome and Tic Disorders. It has more recently been used to treat Trichotillomania and skin picking behaviors, both considered as Obsessive Compulsive Related Disorders (OCRD).

#### Edited by:

Eric Hollander, Albert Einstein College of Medicine, United States

#### Reviewed by:

Ygor Arzeno Ferrão, Federal University of Health Sciences of Porto Alegre, Brazil Michael Van Ameringen, McMaster University, Canada

> \*Correspondence: Melissa T. Lee melissa.lee1@nhs.net

Received: 31 January 2019 Accepted: 01 April 2019 Published: 24 April 2019

#### Citation:

Lee MT, Mpavaenda DN and Fineberg NA (2019) Habit Reversal Therapy in Obsessive Compulsive Related Disorders: A Systematic Review of the Evidence and CONSORT Evaluation of Randomized Controlled Trials. Front. Behav. Neurosci. 13:79. doi: 10.3389/fnbeh.2019.00079 Objectives: This literature review sought to establish and quality assess the existing randomized controlled trial evidence supporting the use of HRT in the DSM-5 family of OCRDs.

Search Methods: EMBASE, PsycINFO, PubMed, and Cochrane databases were searched for key terms relating to each OCRD (as classified in the DSM-5), and HRT.

Selection Criteria: Titles and abstracts were screened, and any literature matching pre-specified criteria were then selected to be reviewed further. Of these, 8 Randomized Controlled Trials (RCT) relating to Trichotillomania, and 2 RCTs relating to Excoriation Disorder, were extracted and reviewed against the 2010 Consolidating Standards of Reporting Trials (CONSORT) statement.

Results: The review identified 10 RCTs of HRT, but these were limited to patients with a primary diagnosis of Trichotillomania or "excoriation behavior," only. There were some reports of the use of HRT in Tourette Syndrome or Tic Disorder with secondary OCD, but the OCD symptoms were not reliably reported on.

Conclusion: There is a gap in the current literature regarding the use of HRT in the DSM-5 OCRDs. In those RCTs that have been reported, the quality of study methodology was questionable as evaluated by CONSORT criteria. The implications of these findings are discussed, and suggestions are made for future research.

Keywords: habit reversal therapy, randomized controlled trial, obsessive compulsive and related disorders, HRT, RCT, habit

Obsessive-compulsive disorder (OCD) is a common, relatively treatment refractory neuropsychiatric disorder. One of its cardinal symptoms involves the urge-driven performance of compulsions i.e., stereotyped, repetitive motor and mental acts, usually designed to avert harmful consequences. Established treatments include medication with selective serotonin reuptake inhibitors (SSRI), and Cognitive Behavior Therapy (CBT) with exposure and response prevention (ERP). Although many cases are improved with these treatments, rates of incomplete recovery and treatment resistance to standard therapies are high: Approximately 40% patients fail to respond and 50% need further treatment (Fineberg et al., 2015).

ERP can be difficult for patients to undertake successfully, as the treatment requires facing feared situations for prolonged periods without engaging in compulsions whilst waiting for the compulsive urges to abate. Consequently, adherence is reported to be relatively low and premature discontinuation rates high (McDonald et al., 1988; Abramowitz et al., 2002). Delayed treatment is known to prolong ill health and reduce therapeutic gain (Dell'Osso et al., 2013). New, more highly efficacious and acceptable—treatment paradigms are therefore required to advance the clinical therapeutics of OCD.

Disorders other than OCD have been classified under the Obsessive Compulsive Related Disorders (OCRDs) grouping in the American Psychiatric Association (2013), Diagnostic and Statistical Manual of Mental Disorders (5th ed.) (DSM-5). These include Body Dysmorphic Disorder (BDD), Hoarding Disorder, Trichotillomania (hair-pulling disorder) and Excoriation Disorder. Compared with OCD, these OCRDs have received relatively little interventional analysis. Standard treatment approaches for BDD are similar to OCD and include SSRIs and CBT (Hong et al., 2018). Hoarding disorder is notably resistant to most forms of treatment (e.g., Ayers et al., 2014). In Trichotillomania and Excoriation Disorder, repetitive, stereotyped grooming acts represent key symptoms, and these OCRDs could otherwise be viewed as body-focussed habit disorders (Stein et al., 2006).

Meanwhile, advances are being made in understanding the neurobiological mechanisms underpinning OCD and the other OCRDs that may lead to the discovery and development of new therapeutic interventions (Fineberg et al., 2018). Many patients with OCRD describe their compulsions as being habitual in nature. Habits, otherwise known as "stimulus response behaviors," are relatively fixed responses that, through habit learning, automatically occur in response to a particular environmental trigger. In behavioral terms, they are defined as being insensitive to changes either in the "outcome value" of the behavior or the "environmental contingency." They are defined as "learned sequences of acts that have become automatic responses to specific cues, and are not functional in obtaining goals or end states" (Verplanken and Aarts, 1999, p. 104). In other words, once learned, habits (rather like compulsions) continue relatively unchanged, whether they are adaptive or not (Balleine and O'doherty, 2010). Over-reliance on the habit system has been hypothesized to play a role in generating compulsive symptoms in a range of OCRDs, such as OCD, trichotillomania (hair pulling disorder), excoriation (skin picking disorder), transient or motor tic disorder, and Tourette Syndrome (Chamberlain et al., 2007, 2009) and disorders of addiction (Voon et al., 2015; Gillan et al., 2016).

According to the "dual-system" theory, instrumental actions are normally supported both by a goal-directed system and a habit system, in dynamic balance (Balleine and O'doherty, 2010). According to this theory, the goal-directed system drives actions that are performed either to achieve desirable goals, or to avoid undesirable outcomes (Gillan et al., 2011). However, the habit system may take over once an action has been performed multiple times, exerting dominant control, so that the behavior is produced automatically without too much conscious effort. Although habit induction can be seen to lead to greater cognitive efficiency, it also leads to a loss of behavioral flexibility, and may thus contribute to the ongoing performance of stereotyped compulsive acts.

A rational next step, therefore, is to investigate whether treatments known to challenge habitual behavior are also effective in OCD and related disorders. The Habit Reversal Procedure was originally developed as a treatment for nervous habits and tics (Azrin and Nunn, 1973). Habit Reversal included several behavioral components that aim to help patients challenge habit performance, such as recording, awareness training, competingresponse practice, habit- control motivation, and generalizationtraining. Habit Reversal Therapy (HRT) has since been refined and developed as a multi-component behavioral intervention (Woods, 2001), and is mainly used for the treatment of Tourette Syndrome (Woods and Miltenberger, 1995) and Tic Disorders. Of note, approximately 50% of those with Tourette syndrome experience obsessive-compulsive behaviors or diagnosable OCD at some point in their lifetime (Leckman et al., 1995). Moreover, in the treatment of Tourette Syndrome, it has been suggested that it is often the symptoms of related or comorbid conditions, such as obsessive-compulsive symptoms, and not the tics themselves, that require most attention (Goodman et al., 2006). Given its success in treating Tourette Syndrome and Tic Disorder and the acknowledged clinical and neurobiological overlap between OCD and Tourette Syndrome and Tic Disorders (Fineberg et al., 2010), it is logical to speculate that HRT would be effective in some cases of OCRD. Indeed in some forms of OCD, particularly in those patients with symmetry/ordering symptoms, patients report premonitory urges very like those seen in tic disorders (Subira et al., 2015). These findings suggest a potential role for HRT in OCD with comorbid tics or where symmetry/ordering symptoms are present.

Whereas, the existing neuropsychological evidence implicates habit as a key mechanism in OCD (Gillan et al., 2015), HRT has so far mainly been investigated in other disorders characterized by habits. To date, there is very little published data on the effect of HRT on obsessive compulsive symptoms in OCD (reviewed in Coffey and Rapoport, 2010). HRT is one of the few treatments thought to be effective in Trichotillomania, together with SSRIs and other forms of behavioral therapy (Chamberlain et al., 2007). In Trichotillomania, compulsive hair pulling, affecting various parts of the body and resulting in noticeable hair loss, is associated with considerable shame and distress (Drysdale et al., 2009). The peak age at onset is 12–13 years, with the disorder often being chronic and difficult to treat (Diefenbach et al., 2000; Walsh and McDougle, 2001). Medical complications can arise in addition to the cosmetic and psychosocial consequences of the disorder, including infection, repetitive stress injury, and permanent loss of hair (Frey et al., 2005). Therefore, treatment studies should ideally investigate young people with trichotillomania as well as adults, and explore the effects of treatment on mental and physical health comorbidities.

A recent meta-analysis analysis (Bate et al., 2011) demonstrated a large effect of HRT from pre-assessment to final post-treatment assessments, in a combined group of disorders including Trichotillomania, Tourette Syndrome, nail biting, and stuttering, amongst other habitual behaviors. Research is continuing to support the use of HRT in Trichotillomania (e.g., Rahman et al., 2017), and has started to expand to look at its effectiveness in Excoriation Disorder, alongside a few other treatments thought be of benefit, such as SSRIs, glutamatergic agents and CBT (Lochner et al., 2017). Such research has used HRT in various modified forms. For example, Acceptance Enhanced Behavior Therapy (AEBT), in which Acceptance and Commitment Therapy (ACT) is combined with HRT, was used with some success in a case series of patients with Excoriation Disorder (Capriotti et al., 2015).

Given the increasing evidence suggesting the effectiveness of HRT for Trichotillomania, a disorder classified as an OCRD in the DSM-5, the question arises as to whether HRT would also work for other OCRDs, namely OCD, BDD, Hoarding Disorder and Excoriation Disorder. This literature review seeks to start to answer this question by establishing the research that has been conducted into the use of HRT for patients across the whole range of OCRDs and expands on this further by assessing the quality of any randomized controlled trials (RCTs) identified from this search using the standard Consolidated Standards of Reporting Trials (CONSORT) criteria (Moher et al., 2010).

### METHODOLOGY

#### Identification and Selection of Studies

A computer search was conducted using the following three databases: EMBASE, PsycInfo, and PubMed. Narrow searches were conducted using variable terms for "Habit Reversal Therapy," and the following OCRDs: OCD, BDD, Hoarding Disorder, Trichotillomania, and Excoriation. The Boolean operators "AND" and "OR" were used to combine these terms, and search in the title and abstract of available papers. There was no lower limit to the time period for publication and searches continued until March 2018. Please refer to **Appendix 1** in Supplementary Material for a full table of search terms used.

The titles and abstracts of papers were then screened for suitability, by looking for key words and phrases that included "habit reversal training," "habit reversal therapy," and the OCRDs. Exclusion criteria were those that did not have "habit reversal" in addition to an OCRD in their title and/or abstract, and those that were not available in English language. Any experimental studies were screened for the use of HRT within their methodology in some way, for example a combined therapeutic approach, or stating its use within a "behavior therapy," for example. Due to the number of "hits" for Tourette Syndrome and Tic Disorders with comorbid OCD, the decision was made to only include papers looking at these primary disorders that stated "Obsessive Compulsive Disorder" and exclude those that referred to "Obsessive Compulsive Symptoms." Where abstracts were not available, the full text was screened according to the same criteria.

A hand search was then conducted to screen references for any additional studies that may have been missed in this process. A separate search of the Cochrane Library was also performed. Conference abstracts, book chapters, treatment manuals, position papers, surveys, and acceptability trials were excluded from any results at any stage of the search. For a summary of this process, please refer to **Figure 1**.

Papers identified by the process outlined above were then screened for inclusion in the CONSORT evaluation. Only those studies identified as a randomized controlled study (RCT) from reading the title, abstract, or methodology, were included in this quality analysis.

### Quality Assessment

The quality of the included studies was assessed by the 25 item version of the CONSORT (Consolidating Standards of Reporting Trials) statement (Schulz et al., 2010), using the guidance published by Moher et al. (2010). The CONSORT statement is primarily utilized to assess RCTs, but it has been extended to cover other designs, such as non-inferiority and equivalence trials, and reporting of harm-related data (Boutron et al., 2008). The checklist, published in 1996 and revised in 2001, 2008, and 2010, comprises a set of guidelines that may be used to identify the strengths and weaknesses of clinical trials for both pharmacologic and non-pharmacologic treatments (Des Jarlais et al., 2004; Schulz et al., 2010). For example, in regard to study methodology, the checklist assesses whether a study has adequately reported the eligibility criteria for participants, has provided the precise details of the interventions intended for each group, and has provided justification (e.g., power analysis) for the obtained sample size. Failure to report these details will result in a lower level of CONSORT compliance and thus lower overall reporting quality.

All included studies were assessed for compliance with the 2010 guidelines of the CONSORT statement. To measure compliance, as advised within the CONSORT statement, a twopoint grading system was applied for each CONSORT criterion, where the first author (ML) gave a score of "0" if the item was not present at all, a "1" if the feature was partially present (i.e., some aspects of the CONSORT item were missing or unclear), and a "2" if the CONSORT item was present and clear. To demonstrate this scoring method, the CONSORT item 3 states: "Eligibility criteria for participants and the settings and locations where the data were collected." A score of 0 on this item would be given if the researchers noted that eligibility criteria were used but did not explain what these criteria were, and did not report the settings and locations where the data were collected; a score of "1" would be given if the researchers provided complete details

of the eligibility criteria (inclusion and exclusion criteria), but did not report the settings and locations where the data were collected (or vice versa); and a score of "2" would be given if the researchers provided clear descriptions of both the eligibility criteria used in the study and the setting and locations where the data were collected. In instances where the CONSORT item was not present due to inherent limitations of the study design (e.g., stating who was blinded after assignment to interventions), a score of "0" on that item was given. By systematically applying the CONSORT criteria to all relevant sections of each study, an overall summary of the study's quality as a clinical trial was produced. The evaluation method was independently checked for validity and consistency by one of the two co-authors, who repeated the exercise without knowledge of the original evaluation third author (DM). Where there was disagreement, both raters blindly re-rated the specific items, discussed the ratings, and reached an agreement.

#### RESULTS

For clarity, we present the results of the literature review according to each OCRD, followed by the results of the CONSORT evaluation of all the identified Randomized Controlled Trials (RCTs).

#### Literature Review

#### Habit Reversal Therapy in Trichotillomania

Overall, the literature search identified 30 case studies or series, 21 literature reviews, 1 systematic review/meta-analysis, 1 metaanalysis, 8 RCTs, and 12 other trials of HRT in hair pulling behavior and/or Trichotillomania. The 8 RCTS were each small in number of participants (after dropouts were considered, maximum N = 40, Rahman et al., 2017) and limited in several other aspects of design. Only 2 RCTs involved young people (Azrin et al., 1980; Rahman et al., 2017). They each found some evidence of benefit for HRT, mainly limited to the severity of hair pulling. Importantly, few studies reported using "Habit Reversal" as the defined experimental intervention; many termed the intervention as CBT but stated in the methodology that this contained components of HRT (see individual studies and metaanalyses below). Specifically, two RCTs stated that they used HRT on its own at some stage within the intervention (Azrin et al., 1980; Rahman et al., 2017). Two RCTs used HRT combined with another approach (Woods et al., 2006; Shareh, 2018), and another two used HRT as a component of an experimental intervention (Moritz and Rufer, 2011; Keuthen et al., 2012). The final two RCTs were designed to compare the effect of augmenting HRT with a pharmacological intervention (Ninan et al., 2000; Dougherty et al., 2006). The choice of control intervention (e.g., wait list; no treatment) was inadequate in many of the studies; notwithstanding, relatively high "placeboresponse" rates were found in some studies employing a form of neutral control (e.g., Azrin et al., 1980), emphasizing the importance of using an adequate comparison group to judge effect size. In addition, there was lack of consistency in the use of assessment instruments and some of the studies relied on a completer analysis, resulting in an increased risk of bias toward treatment success.

Please refer to **Table 1** for a list of the randomized controlled trials directly investigating the efficacy of HRT.

#### Individual Randomized Controlled Trials

The first published RCT of HRT for "hair pulling behavior" was conducted by Azrin et al. (1980). In this pioneering study, a mixture of 30 adults and 4 children were recruited through a newspaper advertisement for treatment of hair pulling, with no formal diagnosis of Trichotillomania required for participation. A coin flip was used to assign participants to the Habit Reversal (19 participants) or a Negative Practice Treatment (15 participants) group. The Negative Practice Treatment involved the participants standing in front of a mirror and acting out the motions of hairpulling without doing any damage at various time intervals. HRT was delivered in one session of approximately 2 h duration, with telephone contact over the next 2 or 3 days, with calls decreasing in frequency as the hair pulling episodes decreased. No formal measures of hair pulling behavior or Trichotillomania symptoms were used; participants used a selfreport chart of the number of occurrences or duration of hair pulling episodes, in addition to family or friend's self-report of observed frequency of hair pulling. Significant differences between the two groups were found in hair pulling behavior immediately after HRT and at each of the follow up periods, up to 3 months. For those in the HRT groups, hair pulling behavior reduced by 99% the first day after training, and 99% on the second day. The level of reduction (97–99.9%) was reported to continue for 4 weeks, and during the 4th month still reached 91%. This was compared to 58% reduction on the first day for the negative practice group, which produced 52–68% improvement during the 3-month follow-up.

In the study by Ninan et al. (2000), 23 adult participants with DSM-III-R Trichotillomania were randomized to receive CBT with HRT, Clomipramine, or pill placebo. Analysis indicated that those allocated to CBT underwent significantly greater change in severity (as measured by the Trichotillomania Severity Scale) and impairment (as measured by the Trichotillomania Impairment Scale), compared to both the Clomipramine and pill placebo groups at the 9-week end point. However, there was a high drop-out rate over the course of the study (4 of those receiving clomipramine, 2 receiving CBT, and 1 receiving placebo), reducing the sample size to 16 completers, on whom it appeared that the analysis was conducted. In addition, by omitting a psychological placebo treatment, the study was unable to adequately control for the nonspecific effects of therapist contact in the HRT group.

Dougherty et al. (2006) took a different approach, randomizing adult patients with DSM-IV Trichotillomania who had failed to respond to either 12 weeks of sertraline, or placebo, to 2 sessions of additional HRT. Those receiving HRT in combination with sertraline (N = 11) obtained better outcomes than those receiving HRT with placebo (N = 9), hinting at an added advantage for combining SSRI with HRT in trichotillomania. However, the design of the study means that it is difficult to draw conclusions on the effectiveness of HRT.


(Continued)

TABLE

1


Randomized

controlled

trials

of

HRT

in

trichotillomania.


Woods et al. (2006) randomized 25 Caucasian women with Trichotillomania to a combined therapy involving 10 sessions of Acceptance and Commitment Therapy and Habit Reversal Therapy (ACT/HRT) or a waiting list condition. Standardized outcome measures were used, and a significant group by time interaction was found, with the ACT/HRT group showing a significant decrease in symptom scores across time and superior efficacy on self-report measures of hair pulling severity and impairment on the NIMH clinician impairment. These reductions were maintained at 3 months follow up.

Notably, in the study by Moritz and Rufer (2011), obsessivecompulsive symptomatology was measured (using the Obsessive Compulsive Inventory-Revised (OCI-R; Foa et al., 2002) alongside hair pulling in approximately 31 adults with a selfreport diagnosis of Trichotillomania. The study compared "selfhelp decoupling" (authors stating that parts of decoupling may be considered a variant of HRT, as they both interfere at the motor level of functioning) with "self-help progressive muscle relaxation (PMR)." Using an intention to treat analysis, borderline significant advantage was found in favor of HRT in regards to hair pulling symptoms on the Massachusetts General Hospital-Hair-Pulling Scale (MGH-HPS) (p = 0.05), with medium to strong effect size. Decoupling was also associated with significant within-group improvements on measures of obsessive-compulsive symptoms (OCI-R; p = 0.04), hinting that HRT may have a therapeutic role in other OCRDs as well as trichotillomania.

In a sample of 33 adult participants with Trichotillomania, which excluded those with a history of treatment nonresponse, Keuthen et al. (2012) compared Dialectical Behavior Therapy Enhanced Cognitive Behavioral Treatment (DBT/CBT) to Minimal Attention Control (MAC). The DBT/CBT was stated to encompass "standard habit reversal training." The DBT/CBT group received 11 acute treatment sessions, followed by 4 maintenance treatment sessions over the following 3 months, with follow up assessment conducted at 3 and 6 months after the acute sessions ended. There were significant between group differences in hair pulling severity (MGH-HPS); NIMH Trichotillomania Severity Scale (NIMH-TSS); and impairment (NIMH Trichotillomania Impairment Scale (NIMH-TIS), as well as in measures of Affective Regulation Rating (ARR) for completers on baseline to week 11 change scores. During followup, although MGH-HPS scores worsened from post-treatment to 3- and 6-month follow-up, significant improvement compared to baseline on all TTM variables was still reported at 3- and 6-month follow up in the DBT/CBT completer group.

Rahman et al. (2017) randomized 40 children and adolescents with a primary diagnosis of Trichotillomania to a treatment as usual (TAU) control group or a course of 8 weekly sessions of HRT (based on the treatment protocol outlined by Woods, 2001). A large between group difference (d = 0.87) at the study endpoint was reported using measures of hair pulling symptoms (NIMH-TSS; MGH-HPS). A significantly greater number of responders was found in the HRT compared to the TAU group. Notably, however 52% of those randomized to the TAU group received no treatment over the course of the study, and the TAU treatments were not standardized, thereby potentially biasing the outcomes in favor of the HRT group. Many (at least 75%) of those classed as a "treatment responder" on HRT who completed a 1- and 3 month follow up assessment maintained a treatment response. However, only a small subsample completed the 3-month follow up, and it was suggested that booster sessions following acute HRT treatment should be provided to support any gains made.

Shareh (2018) randomized patients with DSM-5 trichotillomania to either 8 weekly sessions of a form of therapy involving "metacognitive methods" combined with habit reversal (MCT/HRT) or a wait list control group. The analysis focussed on the 29 completers and found significant differences between the groups on a variety of standardized outcome measures related to hair pulling severity (MGH-HPS; Y-BOCS-TM; self-monitoring) as well as depression (BDI-II), anxiety (BAI); self-esteem (RSES); and global assessment of functioning (GAF), from pre- to post-treatment. Due to the combined therapeutic approaches and the use of a wait list control only, one must be cautious about the conclusions drawn about the effect of HRT.

There were two further studies that were excluded in the process, but the reviewers felt were still important to report upon for completeness. The first, conducted by Flessner et al. (2008) conducted a pilot study in which 6 adult participants with either Trichotillomania or Chronic Skin Picking were allocated to either Acceptance and Commitment Therapy (ACT) followed by HRT, or HRT followed by ACT. Although the term "randomly" was used in the procedure, the "non-concurrent, multiple-baseline design" in the 6 participants involved two treatments that overlapped to such an extent that it was not possible for the reviewers to identify meaningful between group comparisons bearing on the efficacy of HRT.

The second study by Rogers et al. (2014) sought to address questions regarding a two-step model of care for Trichotillomania by only including those classed as "nonresponders." Although participants were "randomly assigned" to an immediate Step 1 condition, comprising "10 weeks of free access to StopPulling.com, consisting of assessment, intervention, and maintenance modules," or to waitlist, patients in both groups then had the option of accessing HRT, which itself was not randomized.

#### Meta-Analyses

In addition to the individual RCTs, two meta-analyses were identified by the literature search (Bloch et al., 2007; McGuire et al., 2014). Both appeared to demonstrate the benefits of HRT within the meta-analyses, but the authors drew attention to the serious limitations of the small number of studies that existed. For example, the lack of available RCTs that examine interventions targeted at youth with Trichotillomania. No single validated clinical instrument was used consistently across the research to measure severity and improvement of Trichotillomania symptoms. In addition, the meta-analyses indicated a lack of long-term follow up to establish the long-term durability of initial treatment gains, and lack of acknowledgment of comorbidities.

#### Habit Reversal Therapy in Excoriation Disorder

Only two RCTS were identified, alongside several case studies of HRT for skin-picking behavior. As the diagnosis of excoriation disorder was only defined in the last few years, neither RCT made a formal diagnosis of Excoriation Disorder, but instead relied on self-reports of skin picking.

Please refer to **Table 2** for a summary of the randomized controlled studies.

#### Individual Randomized Controlled Trials

Teng et al. (2006) randomized college aged students, 1 male and 24 females, to either 3 sessions of HRT delivered over approximately 3 weeks or a waiting-list control group. Due to dropouts during the course of the study, the final sample consisted of 19 females. Analysis of completers indicated significant benefits for the HRT group on self-reported measures of skin picking Photo rankings were also conducted, whereby those in the HRT group were ranked as being significantly more damaged in pre-treatment photos compared to post-treatment and follow-up. Although the control group also reported a significant decrease in skin picking between pre-treatment and follow-up, photo rankings of the skin picking showed no significant differences.

Moritz et al. (2012) randomized 70 predominantly female adult patients to Decoupling, deemed a "variant of HRT," vs. HRT. Both treatments were administered in the form of self-help treatment manuals. This therefore made it difficult to quantify the amount of treatment received. Those treated with HRT showed significantly greater improvement on the Modified 10-item Skin Picking Scale (p = 0.04) compared to those in the Decoupling treatment group on an intent to treat analysis. Similarly, analysis of the completers supported the significant findings (p = 0.03). However, it is worth noting that no follow up measures were taken, thus there was a lack of evidence for long term benefits of the intervention.

There were also 6 case studies/series identified and 3 literature reviews.

#### Habit Reversal Therapy in OCD

The majority of published studies reporting on HRT for OCD recruited patients with Tourette Syndrome/Tic disorders as the primary disorder and OCD as a comorbidity; these studies will be discussed in more detail below. A single RCT was identified but excluded on the basis of being a poster abstract only (Rojas and Gair, 2017). The abstract reported the outcome of a 12 session group intervention, which included habit reversal, on symptom scores in a pediatric OCD sample and stated that the group intervention was effective. The substantive peer-reviewed report could not be found. One literature review was identified (Coffey and Rapoport, 2010) which reviewed OCD and Tourette's Disorder, and spoke very briefly regarding habit-reversal being used in the treatment of Tics.

#### Habit Reversal Therapy in Other Disorders With OCD/OCRD as a Secondary/Comorbid Disorder

We found 3 studies of HRT for Tourette Syndrome/Tic Disorder, with OCD listed as a comorbidity. One was an RCT (Seragni


et al., 2018), another a pilot study (Woitecki and Döpfner, 2012), and the last a case study (Bryson et al., 2010). The full report of the pilot study was available in German only, so was excluded. However, it is worth noting that the available English translated abstract reported that positive results were found in the effect of HRT on comorbid symptoms, including OCD. Both the RCT and case study did not appear to measure OCD symptoms and so were excluded. It is also worth noting the study by Moritz and Rufer (2011), (**Table 1**), who used the OCI-R to measure comorbid obsessive-compulsive symptomatology in participants with trichotillomania and found a form of treatment including elements of HRT improved obsessive-compulsive symptoms on an intent to treat analysis.

We additionally screened a subsample of 15 identified literature reviews (Du et al., 2010; Jankovic and Kurlan, 2011; Robertson, 2012; Kurlan, 2014), to further ensure there were no overlooked RCTs on the effect of HRT on comorbid OCD symptoms. Within one of the literature reviews, a single case example was reported (Sulkowski et al., 2013) of a 15-year-old female with OCD and Trichotillomania treated using combined elements of ERP and HRT. Post treatment, her OCD symptoms reduced by 59% and Trichotillomania symptoms by 74%.

#### Habit Reversal Therapy in Hoarding Disorder and Body Dysmorphic Disorder

No published research into HRT and hoarding disorder, nor BDD was found.

#### CONSORT Evaluation

**Table 3** presents the CONSORT evaluation of all 10 RCTs identified through the literature search; 2 falling under the primary diagnosis of Excoriation (described as "skin picking"), 8 with the primary diagnosis of Trichotillomania. Each paper was assessed on 25 standard items. The two raters reached agreement on all ratings apart from items 20, 21, and 22 for one study (Shareh, 2018). For these three ratings, the lead author's (ML) ratings were used.

A comprehensive evaluation of all the limitations of the individual studies was beyond the scope of this paper; instead, an overview of the studies' main limitations is described. In general, the studies did report some level of background and explanation of rationale, research objectives, overviews of interventions, participant flow information, and study limitations, generalizability, and interpretation consistency. There was also reasonably thorough and adequate reporting of eligibility criteria of participants across the studies.

However, there were common weaknesses across many if not all the RCTS in identifying the study as an RCT, reporting the methods used for randomization, blinding, sample size determination, statistical analysis, recruitment, process, outcome evaluation, changes to the methodology, interim analysis, stopping guidelines, as well as reporting about who implemented the assessment or treatment, the similarities of comparator interventions, why the trial was ended or stopped, reporting of harms or unintended effects, registration number and name of trial registry, and where the full protocol can be accessed. Of these limitations, failures to identify the study as an RCT, report on power calculations, and report adverse effects/events are arguably critical.

## Areas for Quality Improvement in HRT Trial Design

#### Identification as an RCT

Although all the studies evaluated were identified as an RCT by the author, this was often difficult to establish. Only two studies reported that they were an RCT within the title (Keuthen et al., 2012; Rahman et al., 2017). One further study reported that it was a randomized controlled study within the abstract (Ninan et al., 2000), whilst the remainder of studies only reported this within the methodology.

#### Information on Randomization

Apart from two studies (Azrin et al., 1980; Moritz and Rufer, 2011), the studies were seriously deficient in providing information regarding the randomization process; the method, type, and mechanisms for randomization.

#### Blinding Procedures

Only Ninan et al. (2000) adequately fulfilled this criterion, stating who was blinded to the treatment condition at pre and posttreatment, and how blinding was maintained. Further, they stated the instances where blinded ratings were not conducted. Some RCTs partially fulfilled the criteria (Dougherty et al., 2006; Teng et al., 2006; Woods et al., 2006; Rahman et al., 2017), for example briefly stating that particular raters were blinded, but giving very little further information, if any. Those remaining did not reference blinding to any extent.

In regards to designing comparison interventions that were similar to the experimental treatment, only two studies fully fulfilled this criterion (Azrin et al., 1980; Moritz and Rufer, 2011) by clearing drawing comparisons between the conditions of the experiment. The remainder of studies did not fulfill this criteria at all.

#### Sampling Issues

We consistently found that studies did not justify their sample size. Rahman et al. (2017) presented a power analysis to determine the necessary sample size to observe a moderate or greater effect size. The only other study that partially fulfilled this criterion was Moritz and Rufer (2011) who stated that recruitment was stopped after a specified time period.

#### Changes to Method and Trial Outcomes

None of the studies reported any changes, or lack thereof to the methodology from that defined in their protocol. The CONSORT statement suggests that a change could be due to a "disappointing recruitment rate"; most studies reported some level of dropouts from their original samples that may have led to some change in their method but this was not reported.

The same was found for the item of changes to trial outcomes; no studies identified whether there had been any changes to the outcomes, including data collection or methods of analysis.

#### Why the Trial Was Ended or Stopped

Only Azrin et al. (1980) partially fulfilled this criterion.


Frontiers in Behavioral Neuroscience | www.frontiersin.org



#### Harms and Unintended Effects

Only Ninan et al. (2000) reported information relating to harms and unintended effects, and this was limited to the adverse effects of the pharmacotherapeutic condition, and the impact that this had on the results. Some studies, for example, Dougherty et al. (2006) reported within the methodology that should any side effects be experienced, participants were instructed to contact the study's physician investigator, but no numbers reflecting this were reported.

#### Other Information: Registration and Protocol

Only Keuthen et al. (2012) reported the registration number and name of the trial registry, and none of the studies provided information regarding where the full study protocol could be accessed and information regarding sources of funding and other support.

#### DISCUSSION

This systematic review was designed to establish the current research base relating to the use of HRT for patients suffering with OCRDs, and to evaluate the quality of the reporting in any RCTs identified using CONSORT criteria. A few systematic reviews have investigated HRT for individual OCRDS. For example, Lochner et al. (2017) reviewed the treatment options for skin-picking disorder. Although acknowledging the benefits of behavioral treatments, the authors drew attention to the sparse evidence base, and a need for consensus on symptom measures. Another review by Selles et al. (2016), drew similar conclusions, again referencing the lack of studies of treatments for excoriation. As far as we are aware, this is the first systematic review conducted specifically on the efficacy of HRT across all OCRDs, rather than the broader class of behavior therapies (e.g., CBT, ERP), which only sometimes includes HRT. A total of 10 RCTs were identified as incorporating HRT to treat an OCRD—specifically Trichotillomania and Excoriation. Four studies reported the exclusive use of HRT as the experimental intervention. Of the 10 RCTs, 6 of these had some form of credible control group for comparison (e.g., an alternative treatment, including a pharmacotherapeutic intervention), whereas 3 used a wait-list control group which is not acceptable. One RCT (Dougherty et al., 2006) studied HRT in treatment non-responders, so conclusions about the efficacy of HRT are limited.

In the RCTs identified, there was no clear evidence against the use of HRT; rather all the studies showed some evidence of a decrease in symptoms in the experimental groups receiving the HRT (regardless of the form in which this was delivered) relative to the control group. Thus, the findings broadly support the hypothesis that HRT could be an effective treatment intervention and warrant further investigation. However, no firm conclusions can as yet be drawn owing to the methodological limitations identified in this review.

Habit Reversal Therapy was originally developed as the Habit Reversal Procedure for treating nervous habits and tics. Thus far, the evidence hints that HRT may be an effective treatment for Excoriation and Trichotillomania and provides scope to expand

, 0; , 1; , 2; \*RCT Section of Study Evaluated Only.

research to explore the clinical efficacy of HRT across the full range of OCRDs, including OCD, BDD and hoarding disorder, in the form of well-designed RCTs. This approach would complement the emerging neurosciences research demonstrating biologically-based biases away from goal-directed control and toward habitual responding in the pathophysiology of OCD.

According to this research, anatomical overlaps have been demonstrated between the neural substrates of habit formation and the pathophysiology of OCD, converging in the corticothalamo-striatal neurocircuitry (Graybiel and Rauch, 2000; Burguiere et al., 2015). These observations led to a recent series of experiments investigating the role of habit learning in OCD, which demonstrated that patients with OCD show evidence of a shift away from goal-directed control and toward habitual responding, in paradigms variously testing slips of action during a rewarding game (Gillan et al., 2011), decision-making tendencies (Voon et al., 2015) and avoidance of electric shocks (Gillan et al., 2014). Further research by Gillan et al. (2015) using functional brain imaging of OCD patients whilst performing a habit learning task, reported dysfunctional hyper-activity in the caudate nucleus and medial orbitofrontal cortex. The caudate nucleus plays a key role in goal directed learning. Therefore, the finding that caudate hyperactivity was associated with selfreported urges to perform avoidance habits suggested that the habitual responding seen in the patients with OCD resulted from a primary failure in goal-directed control over actions. Together, these studies provide growing evidence of a selective deficit in goal-directed control over actions, resulting in biases toward performing habits, that may contribute to the symptomatology of OCD.

Habits are unlikely to completely explain the performance of compulsions, even in chronic states. However, by helping the individual to break the link between the stimulus (exposure to the cue to perform the compulsion) and response (the compulsion itself), HRT may theoretically be used alongside traditional CBT techniques such as exposure and response prevention as a method to extinguish compulsions more readily. Hypothetically, by weakening the habit, the individual may be able to exert greater instrumental control over the compulsive behavior, rendering it more amenable to conventional CBT techniques. This hypothesis could be tested using randomized controlled trial methodology comparing CBT with or without adjunctive HRT delivered as a prior intervention.

Throughout the review, we found evidence of benefit for "variants" of HRT, for example "movement decoupling" (Moritz and Rufer, 2011). The authors argue that HRT and decoupling may each "stop the dysfunctional movements by actively interfering at the motor level." Our work raises questions about the clinical importance of modifications that can be made to HRT, such as the added value of using a combination of HRT and another behavioral intervention.

The results of the CONSORT evaluation indicated that no studies thus far have filled a "good enough" standard against these criteria and highlight the need for improved trial design and reporting in the key areas of randomization, blinding, and sampling procedures. Some of these deficits have been highlighted in other evaluations of RCTs of CBT (King et al., 2017). However, it is important to note that research in this field can still be considered in its infancy, and the existing studies, have provided a vital starting point.

Key areas for improvement to aid future research were identified. We identified a concerning lack of research using children and/or adolescents, which represents a vital age range to target given the early onset for many of these disorders (Diefenbach et al., 2000; Walsh and McDougle, 2001). There is also a clear need for agreement on the primary outcome measures for many of the disorders assessed. For example, neither of the RCTs of skin picking used the same scale, and there was a reliance on self-monitoring in some form for both Excoriation and Trichotillomania. There was most consistency in the measurement of Trichotillomania, with the Massachusetts General Hospital Hairpulling Scale being used throughout the majority of the studies as one of many outcome measures. In addition, greater attention needs to be paid to those comorbid disorders that commonly occur with OCRDs and that may impact on treatment outcomes (Sulkowski et al., 2013). Evaluation of the impact of treatment on the medical complications and psychosocial consequences of these disorders is also important- for example in Trichotillomania (Frey et al., 2005) as well as social alienation and depression.

## LIMITATIONS OF THE REVIEW

We were unable to include articles unavailable in the English language. Although this did not appear to impact our review bar one paper (as acknowledged, Woitecki and Döpfner, 2012), it is possible that there is further work that was not identified using our method. In addition, by limiting the choice of databases searched, we may have missed some additional published research. However, review of the Cochrane database provided at least partial confirmation of the main search findings, and did not result in the identification of any further articles meeting the inclusion criteria. There is a further possibility that relevant research was not identified due to publication bias. For example, we became aware of a case report by Dillenburger (2006) that was not identified by our search and that used Habit Reversal Therapy as a successful treatment for OCD. It is possible that, as we screened the title and abstract only for key words and phrases, some research incorporating HRT was missed as reference to it was only made in the methodology section. In regards to the CONSORT evaluation, there were some discrepancies between the reviewers on the quality ratings, however, the approach taken to resolve this endeavored to ensure that all ratings were as consensual as possible.

#### CONCLUSION

So far, the evidence supporting the effectiveness of HRT in OCRDs is limited to a few studies of Trichotillomania and Excoriation. The existing studies are limited in terms of quality of study design and reporting. Our review highlights the need for well-designed studies to be conducted in the appropriate age groups across all the OCRDs. Such studies should measure key primary and secondary outcomes using well matched controls and blinded raters.

#### AUTHOR CONTRIBUTIONS

All three authors contributed to the paper searching, writing, analysis, and conclusions. ML and DM performed the CONSORT evaluation.

#### REFERENCES


#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbeh. 2019.00079/full#supplementary-material

interventions: the TREND statement. Am. J. Public Health 94, 361–366. doi: 10.2105/AJPH.94.3.361


tic disorders: an Italian randomized, single-blind pilot study. Minerva Pediatr. 70, 5–11. doi: 10.23736/S0026-4946.16.04344-9


∗ Indicates reference for randomized controlled trial included in CONSORT evaluation.

**Conflict of Interest Statement:** NF reports personal fees from OTSUKA, LUNDBECK, ABBOTT, SUN Pharma, TAYLOR AND FRANCIS, ELSEVIER; personal fees and non-financial support from RANZCP, WILEY; grants from NIHR, WELLCOME; grants and non-financial support from EU, ECNP, SHIRE; non-financial support from BAP, WHO, CINP, ISAD, RCPSYCH, INTERNATIONAL COLLEGE OF OC SPECTRUM DISORDERS, IFMAD, MHRA; other from OXFORD UNIVERSITY PRESS, all outside the submitted work.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling Editor declared a past co-authorship with one of the authors, NF.

Copyright © 2019 Lee, Mpavaenda and Fineberg. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Working Hypothesis for the Role of the Cerebellum in Impulsivity and Compulsivity

Marta Miquel <sup>1</sup> \*, Saleem M. Nicola2,3 , Isis Gil-Miravet <sup>1</sup> , Julian Guarque-Chabrera<sup>1</sup> and Aitor Sanchez-Hernandez <sup>1</sup>

<sup>1</sup>Área de Psicobiología, School of Health Science, Universitat Jaume I, Castellón de la Plana, Spain, <sup>2</sup>Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, United States, <sup>3</sup>Department of Psychiatry and Behavioral Sciences, Albert Einstein College of Medicine, Bronx, NY, United States

#### Edited by:

Margarita Moreno, University of Almería, Spain

#### Reviewed by:

Mark Stanton, University of Delaware, United States Trevor W. Robbins, University of Cambridge, United Kingdom Maria Teresa Colomina, University of Rovira i Virgili, Spain

\*Correspondence:

Marta Miquel miquel@uji.es orcid.org/0000-0001-9670-4210

> Received: 26 January 2019 Accepted: 23 April 2019 Published: 07 May 2019

#### Citation:

Miquel M, Nicola SM, Gil-Miravet I, Guarque-Chabrera J and Sanchez-Hernandez A (2019) A Working Hypothesis for the Role of the Cerebellum in Impulsivity and Compulsivity. Front. Behav. Neurosci. 13:99. doi: 10.3389/fnbeh.2019.00099 Growing evidence associates cerebellar abnormalities with several neuropsychiatric disorders in which compulsive symptomatology and impulsivity are part of the disease pattern. Symptomatology of autism, addiction, obsessive-compulsive (OCD), and attention deficit/hyperactivity (ADHD) disorders transcends the sphere of motor dysfunction and essentially entails integrative processes under control of prefrontalthalamic-cerebellar loops. Patients with brain lesions affecting the cortico-striatum thalamic circuitry and the cerebellum indeed exhibit compulsive symptoms. Specifically, lesions of the posterior cerebellar vermis cause affective dysregulation and deficits in executive function. These deficits may be due to impairment of one of the main functions of the cerebellum, implementation of forward internal models of the environment. Actions that are independent of internal models may not be guided by predictive relationships or a mental representation of the goal. In this review article, we explain how this deficit might affect executive functions. Additionally, regionalized cerebellar lesions have been demonstrated to impair other brain functions such as the emergence of habits and behavioral inhibition, which are also altered in compulsive disorders. Similar to the infralimbic cortex, clinical studies and research in animal models suggest that the cerebellum is not required for learning goal-directed behaviors, but it is critical for habit formation. Despite this accumulating data, the role of the cerebellum in compulsive symptomatology and impulsivity is still a matter of discussion. Overall, findings point to a modulatory function of the cerebellum in terminating or initiating actions through regulation of the prefrontal cortices. Specifically, the cerebellum may be crucial for restraining ongoing actions when environmental conditions change by adjusting prefrontal activity in response to the new external and internal stimuli, thereby promoting flexible behavioral control. We elaborate on this explanatory framework and propose a working hypothesis for the involvement of the cerebellum in compulsive and impulsive endophenotypes.

Keywords: cerebellum, compulsivity, addiction, impulsivity, prediction, habits

## INTRODUCTION

Compulsivity and impulsivity have been proposed as neurocognitive endophenotypes for a heterogeneous group of mental disorders (Dalley et al., 2011; Robbins et al., 2012) such as addiction, eating disorders, attention deficit/hyperactivity (ADHD), obsessive-compulsive (OCD), as well as other personality and neurodevelopmental disorders. Endophenotypes reflect underlying predisposing factors for the vulnerability to psychopathology (Miller and Rockstroh, 2013). With respect to compulsivity and impulsivity, both endophenotypes involve a failure in top-down control and response inhibition but they do not always coexist in the same disorder (Dalley et al., 2011; Robbins et al., 2012). Compulsivity is characterized by an over-engagement in behavioral or cognitive activities despite their countless negative consequences. Compulsive behavior is persistent and inappropriate to the context entailing a failure in terminating actions properly (Robbins et al., 2012). Impulsivity is the trend to showing a premature and poorly planned behavior inappropriate to the context (Moeller et al., 2001). It can be expressed at the behavioral level as impulsive actions (difficulty in stopping an ongoing response) but also as impulsive choices (failure in delaying gratification; Robinson et al., 2009; Dalley et al., 2011). Compulsivity and impulsivity may contribute in varying degrees to the grounds of the disorder, which tends to be more severe when both endophenotypes occur together (Fineberg et al., 2014).

It has been proposed that cortico-striatal-networks mediate motor and cognitive domains for each construct (Dalley et al., 2011). Essentially, the proposed network dysregulation to explain top-down control deficits comprises an imbalance between dorsal and ventral zones in behavioral control, with an under-activation of the dorsal frontal regions along with an over-activation of striatal zones (Fineberg et al., 2010). None of the accepted neuroanatomic models have included the cerebellum, even though much data shows its involvement in different forms of impulsivity and compulsivity. Here, we review these scattered but consistent findings and synthesize a working hypothesis for the cerebellum's contribution to impulsive and compulsive behavior.

#### THE CEREBELLUM: TOO MANY NEURONS JUST TO NOT WALK LIKE A ROBOT

The cerebellum includes 60 billion neurons, representing 80% of the total number of brain neurons (Azevedo et al., 2009; Barton, 2012; Lent et al., 2012). The reason why there are so many neurons in the cerebellum is one of the mysteries in brain evolution. Barton (2012) hypothesized that the cerebellum and cortico-cerebellar networks are fundamental components of the integrative brain systems enabling the prediction, organization, modeling and comprehension of complex sequences. This hypothesis is supported by evidence of cerebellar contributions to apparently dissociated brain functions that are altered in compulsive/impulsive disorders. These include Pavlovian conditioning (Pakaprot et al., 2009; Carbo-Gas et al., 2014a,b, 2017; Gao et al., 2016; Giovannucci et al., 2017); repetitive sequential learning (Wu et al., 2004; Doyon and Benali, 2005; Balsters and Ramnani, 2011); skill learning (Callu et al., 2007; De Bartolo et al., 2009); language (Leiner et al., 1993; Mariën et al., 2014; Verly et al., 2014); planning/prediction (Bastian, 2006; Bhanpuri et al., 2013); and social behavior (Kerney et al., 2017; Giocondo and Curcio, 2018; Hickey et al., 2018).

As established by tracing techniques, electrostimulation and optogenetics, the cerebellum appears to be closely connected to the functional loops that sustain compulsive and impulsive behavior (Ding et al., 2013; Herrera-Meza et al., 2014; Bostan and Strick, 2018). Thus, deep brain stimulation of the mediodorsal thalamic nuclei increases cFos expression in the deep cerebellar nuclei and the prefrontal cortex in rats (Moers-Hornikx et al., 2009). Additionally, cortical regulation of striatal activity can be modulated by the cerebellum (Moers-Hornikx et al., 2009). Furthermore, a direct dopaminergic VTA-cerebellar projection has also been demonstrated with detectable DA levels in the posterior lobules of the vermis (VII–X), the right and left hemispheres and the fastigial, interpositus and dentate nuclei (Glaser et al., 2006). More importantly, it has been shown that the cerebellar cortex may regulate dopamine release by several independent pathways. First, the cerebellum connects to the VTA through the reticulotegmental and pedunculopontine nuclei (Carbo-Gas et al., 2014b). Second, the cerebellum projects to the VTA through the mediodorsal and ventrolateral thalamus (Rogers et al., 2011). Finally, the deep cerebellar nuclei project directly to the VTA (Watabe-Uchida et al., 2012; Carta et al., 2019).

For some time now, it was proposed that the majority of mental disorders result from the dysregulation of normal central nervous system (CNS) development (Weinberger, 1987). Interestingly, the final cerebellar structure and functionality are developed postnatally, making cerebellar circuitry susceptible to alteration by external and internal factors at different developmental stages (Koziol et al., 2014). As an example, in terms of comparative anatomy, 50% of the cerebellum's adult weight is achieved at birth in primates, at 15 postnatal days in rats and after 1 year in humans (Howard, 1973; Watson et al., 2006). Moreover, the cerebellar growth asymptote is reached in rats at 400 postnatal days (Sullivan et al., 2006) and around the age of 4 years in children (Dobbing and Sands, 1973). Alterations of the cerebellum during very early periods of postnatal life (prenatal, and neonatal) are able to shape morphology and functions of the mature brain (Limperopoulos et al., 2014). Several studies suggest that cerebellar injuries during perinatal/postnatal stages are sufficient to bring alterations in distal cortical regions (for a review, see Wang et al., 2014). For example, early cerebellar damage has been associated with a reduction in the modulation of dopamine release in the medial prefrontal cortex as well as the reorganization of cerebello-cortical loops (Rogers et al., 2011, 2013). In rodent models, compensatory responses and changes in the activity of brain-related regions have been found to be much greater when cerebellar injuries occurred within the neonatal period (Lalonde and Strazielle, 2015). In children, prenatal and neonatal lesions of the cerebellum generate motor dysfunction (Stoodley and Limperopoulos, 2016) but also cognitive and emotional deficits such as increased anxiety and aggressive behavior (Watson et al., 2018); autistic-like symptomatology in language and social behavior as well as selective attention deficits (Steinlin et al., 2003; Schmahmann et al., 2007). Adult lesions produce a more limited cortical compensation (O'Donoghue et al., 1986). Likewise, alterations in plasticity genes such as neuregulin 1, N-Methyl-D-aspartate (NMDA) and GABA signaling genes, which results in disruptions of the normal CNS development, accompany prefrontal-cerebellar pathology in schizophrenia and autism (Andreasen, 1999; Nopoulos et al., 1999; Allen et al., 2004; Scott et al., 2009; Yeganeh-Doost et al., 2011; Edmonson et al., 2014; Koziol et al., 2014; Murphy et al., 2014; Shevelkin et al., 2014; Osipowicz et al., 2015). Additionally, numerous neuroimaging studies have found structural abnormalities and changes in connectivity in the cerebellum of addicted cohorts (Barrós-Loscertales et al., 2011; Yu et al., 2011; Bora et al., 2012; Ersche et al., 2012; Ding et al., 2013; Koehler et al., 2013; Segobin et al., 2014; Shen et al., 2018) and members of high-risk families that did not take drugs (Hill et al., 2011).

In summary, prefrontal-cerebellar physiopathology is common in the comorbid mental disorders in which compulsive and impulsive endophenotypes are present. However, prefrontalcerebellar alterations are not homogeneous and can affect different regions within these loops (for instance, dorsomedial prefrontal cortex vs. orbitofrontal cortex or the posterior vs. the anterior cerebellum), which might explain why prefrontalcerebellar dysfunction is also implicated in many mental disorders in which compulsivity and impulsivity are not always key symptoms. These include schizophrenia (Andreasen, 1999; He et al., 2018) depressive disorders (Yucel et al., 2013; Scheinost et al., 2018; Wang et al., 2018) as well as fear and anxiety disorders (Richter et al., 2005; Picó-Pérez et al., 2017). Therefore, dysfunctional prefrontal-cerebellar loops do not always result in compulsivity and impulsivity though they generate a failure in top-down and executive control that in turn can bring compulsive and impulsive symptoms.

### CEREBELLAR UNDERPINNINGS OF COMPULSIVITY

The compulsivity construct is far from unitary (Fineberg et al., 2014; Figee et al., 2016). Several dissociable dimensions of compulsivity have been proposed, including cognitive inflexibility, motor disinhibition, disadvantageous decisionmaking, attentional bias, impaired executive planning and bias toward habits. Deficits in inhibition of motor responses do not always coexist with impaired cognitive flexibility. For instance, OCD patients show both impaired motor inhibition and clear deficits in cognitive flexibility, whereas other compulsive disorders such as trichotillomania appear to be limited to impaired inhibition of motor behaviors (Chamberlain et al., 2005, 2006). Thus, the dimensions of the compulsivity construct seem to represent different clusters of brain functions mediated by neuroanatomically and neurochemically distinct components of cortico-subcortical circuitry (Hollander et al., 2016). Overall, compulsivity implicates a failure in top-down cortical control that causes behavioral disinhibition. The resulting behavioral deficits may additionally be due to over-activity in the basal ganglia, which promotes automatic and stereotyped behavioral repetition (Fineberg et al., 2010).

In the next section, we describe the evidence for cerebellar changes in patients with compulsive disorders, exploring the consequences of alterations in the cerebellum for several of the described dimensions of compulsivity.

### Structural Neuroimaging Findings in the Cerebellum of Subjects Suffering From Compulsive Disorders

Untreated polydrug abusers exhibited decreased gray matter (GM) in the cerebellum and frontoparietal cortices along with increased GM in basal ganglia (Ersche et al., 2011). Additionally, GM volume in Crus I of the cerebellum was described to be correlated with the severity of nicotine dependence (Shen et al., 2018). Similar results were found in other compulsive pathologies such as internet gamblers (Dong et al., 2012; Ding et al., 2013); OCD (Ersche et al., 2011) as well as genetic disorders including compulsive symptoms such as Prader-Willi syndrome (Ogura et al., 2011). By contrast, greater cerebellar GM volume has been reported in healthy, non-drug-abusing members of families at high risk for alcohol dependence as compared to members of control low-risk families (Hill et al., 2011).

Regarding functional connectivity, OCD patients exhibit stronger interconnectivity between the cerebellum and the basal ganglia than the control subjects but weaker interconnectivity with the prefrontal cortex (Vaghi et al., 2017). As the authors indicated, these results suggest less top-down control over the prefrontal cortex on the lower regions.

Therefore, the most common structural findings in the cerebellum of patients with compulsive disorders have been decreased GM volume in several regions of the cerebellum and increased basal ganglia-cerebellar connectivity (Barrós-Loscertales et al., 2011; Ersche et al., 2011; Ogura et al., 2011; Yu et al., 2011; Dong et al., 2012; Ding et al., 2013; Segobin et al., 2014; Vaghi et al., 2017; Shen et al., 2018).

### Compulsivity and Prediction

Perhaps surprisingly, compulsivity may be due in part to deficits in the brain's ability to make predictions. Some forms of decision-making require internal models of the environment that guide future choices using past outcomes (Blackwood et al., 2004). These internal representations, also called forward models, use memory to integrate predictions about the consequences of any given action with internal and external sensory inputs of the current state (Ito, 2008; Molinari et al., 2009). Individuals suffering from compulsive disorders demonstrate specific deficits in tasks in which behavioral outcome is regulated by internal models (mental representations of the world such as inferences). In a predictive-inference task, OCD patients do not consider the history of outcomes in order to regulate performance (Vaghi et al., 2017). They seem to have the capacity to establish the internal model but then the internal model fails to guide behavior. Vaghi et al. (2017) hypothesized that, in this case, actions become independent of internal models, leading to constant attempts to check the environment in order

to adjust behavior. According to this suggestion, compulsive behavior entails a dysregulation of the integrative brain sensorymotor mechanisms that allow the use of predictive relationships to plan ahead and control behavior on line.

Accordingly, a role for the cerebellum in compulsive behavior is suggested by the extensive literature indicating that one of the main functions of the cerebellum is to implement internal models (Ito, 1990, 1993, 2008). An internal model is similar to a mental model but implicit (Ito, 2008). A mental model is a schematic representation of reality that is used to explain the present events and predict the future (Johnson-Lairds, 1983). The prefrontal cortex acts as a controller to create and manipulate the mental representations of the world that are distributed throughout the sensorimotor cortices. The cerebellar internal model works as an implicit and thereby unconscious template of this mental sensorimotor representation of the world. Then, the cerebellum processes the current functional state using sensory, interoceptive, and proprioceptive information. If there is a match between the mental model and bottom-up information (as can occur in overlearned tasks), the next event can be predicted from the template (Wolpert et al., 1998; Ito, 2008; Leggio and Molinari, 2015). In case of a discrepancy between ''what I want to do'' (prefrontal cortex) and ''what is being done'' (sensory-motor responses), the cerebellum generates an error signal that is essential for updating the internal model and making behavioral adjustments on line (Fautrelle et al., 2011).

If cerebellar error signaling fails (e.g., after cerebellar damage or dysfunction), one would expect internal models to fail to update and therefore to be unable to influence behavioral adjustments. Consistent with this hypothesis, cerebellar lesions impair this predictive capacity in motor tasks such as reaching. Patients with lesions cannot generate anticipatory adjustments and fail to make ongoing corrections reaching objects (Manto et al., 1995; Chen et al., 2006; Bhanpuri et al., 2013). We propose to extend the hypothesis to compulsive behavior in that cerebellar impairment could affect the ability to terminate a wide range of ongoing behaviors when environmental contingencies change. Indeed, clinical reports of patients with cerebellar disease or lesions demonstrate the emergence of compulsive arm shaking, checking, washing, and stereotyped motor activities (Gonzalez and Philpot, 1998). Clinical studies have also suggested decision-making deficits after cerebellar injury (Cardoso et al., 2014).

In drug addiction, drug-related stimuli evoke drug memories and have the capacity to trigger craving and compulsive drug seeking (Shaham et al., 2003; Pickens et al., 2011). Importantly, neuroimaging studies of cue reactivity in drug addicts have consistently shown cerebellar activations when drug-related cues are presented (for a review, see Jasinska et al., 2014; Moulton et al., 2014; Miquel et al., 2016; and Moreno-Rius and Miquel, 2017). We used animal models to investigate the accurate location of the cerebellar area involved in these drug-cue associations (Carbo-Gas et al., 2014a,b, 2017). Our findings indicated that expression of cocaine-induced conditioned memory is accompanied by a selective increase in neural activity at the most external part of the granular cell layer in the posterior cerebellum (**Figure 1**). Such increase was only seen in animals that acquired the memory, but not in pseudoconditioned groups or in animals that, despite being trained in a contingent association, did not express the conditioned response towards cocaine-related cues (CS+). More importantly, this cerebellar activity appeared to be one of the correlates of the behavioral decision driven by the drug-related cue. Accordingly, when animals were confined in the presence of the CS+ with no opportunity to select other behavioral alternatives, cerebellar activity was normalized to control levels (Carbo-Gas et al., 2017). Recently, we proposed that the cerebellar cortex biases behavioral selection towards the context that predicts drug availability, and that this happens by generating predictions after presentation of the conditioned cue (Carbo-Gas et al., 2014b; Moreno-Rius and Miquel, 2017). We propose that the internal state modulates these predictions, increasing the probability of selecting the drug-associated context when the drug is absent in the body. In this way, the cerebellum, by activating drug-cue representations during abstinence, can contribute to compulsive drug seeking driven by both negative and positive reinforcement.

## Compulsivity, Habit Formation and Executive Behavioral Control

Classically, habits have been considered as overlearned, repetitive, sequential behaviors that are performed automatically and triggered by associated environmental signals (Graybiel, 2008). During acquisition of habits, there is a shift from goal-directed behavior regulated by an action-outcome process (R-O) to automatized responses triggered by the stimuli (S-R; Dickinson and Weiskrantz, 1985). Nevertheless, as Robbins and Costa (2017) have discussed recently, habits and skills are not equivalent processes. Habits refer to ''which stimuli elicit the behavior'' and do not necessarily involve overtraining. Like goal-directed behavior, habits are ''autonomous from the goal'' and thus outcome devaluation is unable to reduce the presence of habitual behavior. However, skills involve sequential learning that requires extended training though they may be goal-directed, and thereby still dependent on the outcome. Three dissociated but interconnected loops including different cortical and striatal regions have been proposed to underlie and control the establishment of habits: the limbic, associative and sensorimotor networks (Yin and Knowlton, 2006).

Neuroimaging studies of skill learning reported cerebellar deactivations during the automatic phase (Wu et al., 2004; Doyon and Benali, 2005; Balsters and Ramnani, 2011). Both the prefrontal cortex and cerebellum decrease their activity as sequential learning progresses. Then, if task demands increase, prefrontal cortex activity is engaged again but the cerebellum remains deactivated (Doyon and Benali, 2005). Electrophysiological recordings performed in the cerebellar cortex of rodents during motor learning showed similar findings. In these studies, the initial learning phase was characterized by high cerebellar cortical activity, which decreases with trials and repetition (de Zeeuw and Yeo, 2005; Garcia-Martinez et al., 2010). Thus, correlational research on the cerebellar contribution to motor learning suggested that the prefrontal cortex and cerebellum work in parallel during acquisition and progression of learning, but they are recruited differently when cognitive and motor demands grow. In accordance, non-invasive stimulation of the cerebellum supports the role of the cerebellum in the initial phase of motor learning (Darch et al., 2018). By contrast, hemicerebellectomy seems to delay the transition to response automatization rather than impair acquisition of sequential learning (Mandolesi et al., 2010).

Additional evidence strongly supports the contribution of the cerebellum to habits. During instrumental actions, goal-directed behavior (R-O) can compete with the stimulus-response automatic mechanism (S-R). When habitual behavior is established, the probability of responding for devalued outcomes increases (Adams and Dickinson, 1981). The ability to resolve and monitor the competition between habit and goal-directed processes depends on the engagement of inhibitory executive control (Watson et al., 2018). Contrary to the frontal pole, the cerebellum and other regions in the sensorimotor network, such as the premotor cortex, show greater activation when subjects respond to previously devalued outcomes, suggesting that they participate in the expression of S-R habits (Watson et al., 2018). Accordingly, in an elegant study, Liljeholm et al. (2015) demonstrated using functional imaging in humans that neuronal activity in the tail of the caudate/thalamus, the cerebellum and the lingual gyrus predicts insensitivity to devaluation. Participants with greater activity within these regions in the S-R relative to R-O conditions during the two first blocks of the instrumental learning phase responded to a greater proportion of trials with devaluated outcomes during the test phase. Of particular relevance is the fact that in Liljeholm's study the formation of habit did not require overtraining because R-O and S-R were trained separately as different experimental conditions for the same number of trials. Thus, activity within the caudate and cerebellum was not a function of practice or repetition, but rather it predicted the formation of a strong stimulusresponse association. This observation suggests a significant role not only of the basal ganglia but also the cerebellum in S-R habit formation.

Consistent with the human imaging studies, impaired ability to inhibit responding to the previously-rewarded but no-longercorrect stimulus (perseverative errors) has been observed both in a rodent model of autism in which cerebellar dysfunction is present (Dickson et al., 2010) and in hemicerebellectomized rats (De Bartolo et al., 2009). Moreover, a bilateral lesion in the interpositus nucleus of the cerebellum prevents rats from developing habits with overtraining (Callu et al., 2007). In these rats, behavior maintains the action-outcome features and transition to the automatic cue-response stage is not created.

Therefore, the cerebellum is not critical to learning goal-directed behaviors but appears to be required for habit and skill learning. Furthermore, lesion studies suggest that the integrity of the cerebellum is essential for the brain process underlying habit formation. Future research will ascertain whether the cerebellum is essential to habit formation or instead to the expression of habits.

Similar to the interpositus, the infralimbic prefrontal cortex has been demonstrated to be crucial for the establishment of habits in that repeated optogenetic inhibition of the infralimbic cortex disrupts habit formation (Smith and Graybiel, 2013). Impairment of the infralimbic cortex suppressed the shift away from goal-directed behavior to habitual reward seeking, even after substantial overtraining (Killcross and Coutureau, 2003; Miles et al., 2003; Smith and Graybiel, 2013).

It has been hypothesized that compulsive behavior may result from aberrant habit formation (Everitt and Robbins, 2005). In fact, patients suffering from compulsive disorders including OCD, drug addiction, and Tourette syndrome develop habits more easily than controls after reduced behavioral training (Gillan et al., 2011; Hogarth and Chase, 2011; Delorme et al., 2016; Ersche et al., 2016). According to one theory of drug addiction, compulsivity characterizes late stages of the disorder

(Everitt and Robbins, 2005; Koob and Volkow, 2010; Montigny et al., 2013). In the initial stages of drug intake, drug-related cues drive goal-directed behaviors towards contexts with drug availability. With extended drug experience, cue-action-outcome relationships can become over-consolidated, and drug-related cue/context can activate automatic behaviors (Everitt and Robbins, 2005).

The contribution of the cerebellum to the transition from recreational drug intake (goal-directed behavior) to compulsive habits is still unknown, although human and animal neuroimaging research supports a reorganization of the prefrontal-cerebellar network in addicted patients (Hester and Garavan, 2004; Bolla et al., 2005; Goldstein et al., 2007) and other primates with a history of cocaine self-administration (Porter et al., 2014). Altogether, these findings indicate that the downregulation in prefrontal cortices during addiction is accompanied by abnormal greater activity in the cerebellum when task demands increase. Importantly, the sensorimotor network, including brain regions underlying motor skills learning and action, increase their activity when drug-related cues are presented (Yalachkov et al., 2009). Thus, smokers showed higher activation than non-smokers in the right lateral cerebellum, the left premotor cortex, and the left superior parietal lobule during the presentation of smoking-related cues (Yalachkov et al., 2009). Additionally, smokers show more restrictive brain activity patterns than non-smokers during reward tasks. During a pattern-recognition task, a nonmonetary reward elicited activation only in the smokers' cerebellum. In non-smokers, the brain pattern was wider involving the striatum, prefrontal cortex and limbic cortices. Moreover, the presentation of a monetary reward was unable to activate the striatum in smokers as compared to nonsmokers (Martin-Sölch et al., 2001).

Recent findings from our laboratory indicate that the cerebellum can control learning-related activity (Gil-Miravet et al., 2018) and plasticity in the infralimbic cortex (unpublished results). Neurotoxic lesion of the posterior cerebellar cortex performed before conditioning increased cFos expression and mechanisms for synaptic stabilization in the infralimbic cortex. It is plausible that the cerebellar cortex could influence infralimbic function through disinhibition of the deep cerebellar nuclei. The cerebellar cortex exerts an inhibitory tonic control over the deep nuclei through GABAergic Purkinje axons (Gauck and Jaeger, 2000; **Figure 2**). In accordance, by reducing the synaptic function in Purkinje neurons it is possible to increase neuronal activity and PNN expression in the deep nuclei (Vazquez-Sanroman et al., 2015). Overall, our results suggest that the cerebellar cortex may regulate infralimbic activity in an inhibitory manner via inhibition of the deep cerebellar nuclei. In this way, cerebellar dysfunction might contribute to the establishment of drug-induced incentive habits by controlling activity and plasticity in the infralimbic cortex.

In summary, the findings described above highlight the cerebellar role in compulsivity. First, a dysfunctional prefrontalcerebellum network might mediate the inability to use internal models to regulate behavior. Second, the cerebellum is likely required for the formation and expression of habits. Third, in drug addicts and heavy drug users, impairment of executive functions has been repeatedly associated with a dysfunctional prefrontal-cerebellar pattern in which the cerebellum is overactive. Several years ago, we hypothesized that as the prefrontal cortex is downregulated by the repetition of drug experience, the cerebellum will increase its functional relevance, encouraging faster and automatic forms of control at the expense of behavioral flexibility (Miquel et al., 2009). So far, this hypothesis remains untested but it should be reformulated in light of the present evidence (see section ''A Working Hypothesis for the Role of the Cerebellum in Compulsivity and Impulsivity'').

### CEREBELLAR DYSFUNCTION IN IMPULSIVITY

Several dimensions contribute to motor and cognitive impulsivity including rapid decision making, intolerance to delays in reward delivery, as well as tendency to prematurely terminate response chains (Evenden, 1998). Separate cortico-striatal networks control different aspects of impulsivity (Eagle et al., 2008). The stop circuit involved in motor impulsivity comprises the right inferior frontal gyrus, anterior cingulate cortex, presupplementary and motor cortices, dorsal striatum (caudate/putamen), and subthalamic nucleus. Impulsive choices are triggered from the nucleus accumbens core, basolateral amygdala and orbitofrontal cortex. As with compulsivity, the cerebellum has been overlooked in the more influential anatomical models of impulsivity even though cerebellar dysfunction has repeatedly been linked to impulsive symptomatology (Mulder et al., 2008; Durston et al., 2011; de Zeeuw et al., 2013).

#### Structural Neuroimaging in the Cerebellum and Impulsivity Disorders

Dysregulation of the cerebellum, particularly of the cerebellar vermis, has been accepted as a potential etiological component of ADHD (Mulder et al., 2008; Durston et al., 2011; de Zeeuw et al., 2013; Pieterman et al., 2018). Earlier structural neuroimaging studies with children and adults with ADHD described reduced cerebellar volumes even after correction for total cerebral volume (Berquin et al., 1998; Mostofsky et al., 1998; Castellanos et al., 2001). Similar to ADHD patients, smaller cerebellar volume and reduced GM have been observed in preterm children with impulsive symptomatology (Matthews et al., 2018); compared with normal term infants, preterm children are more likely to show impulsive behavior, inattention, cognitive inflexibility, and meet a diagnosis of ADHD (Foulder-Hughes and Cooke, 2007; Farooqi et al., 2013; Morales et al., 2013; Pozzetti et al., 2014; Franz et al., 2018). Likely, the dynamic of cerebellar development in the last trimester of gestation makes the cerebellum more vulnerable to dysfunction in a preterm birth than other brain regions (Tran et al., 2017).

More recent studies have gone further in identifying the different dimensions of impulsivity related to specific structural cerebellar abnormalities. For example, greater GM volume in the right cerebellum was associated with higher motor impulsivity levels (Lee et al., 2011). Impulsivity is not always dysfunctional and indeed it can be observed in non-pathological individuals as a predisposition to premature and poorly planned responses (Moeller et al., 2001). Interestingly, GM abnormalities in the cerebellum only correlate with dysfunctional impulsivity since high impulsivity in normal subjects involved a different pattern of GM correlations (Hogarth, 2011).

Additionally, abnormal cerebellar connectivity patterns have been described in ADHD patients. In these patients, the cerebellum exhibits reduced connectivity with the prefrontal cortex (Wolf et al., 2009). Oldehinkel et al. (2016) investigated striatal connectivity in an extensive sample of subjects with a diagnosis of ADHD as well as in their healthy relatives. Hyperactivity, impulsivity and inattention were related to greater connectivity of the posterior putamen with the cerebellum and occipital cortex.

#### Impulsivity and Executive Function

Evidence for functional changes in the cerebellum of ADHD patients indicated attenuated cerebellar activity during the performance of executive tasks (Schulz et al., 2004; Valera et al., 2005). Neurofunctional models of ADHD have distinguished several subtypes of ADHD patients as a function of underlying brain pathways and primary functional deficits associated with them (Sonuga-Barke et al., 2008; Durston et al., 2011). The strongest evidence suggests an executive vs. reward-related dysfunction. In the first subgroup, patients show an impairment in behavioral inhibition including inattention (executive deficits). In the second one, the primary deficit was emotional/motivational and it was expressed as an aversion to delayed reward delivery (Sonuga-Barke et al., 2008; Durston et al., 2011). Inability to engage the cerebellum as well as prefrontal and parietal cortices during response inhibition tasks was found to be the hallmark for the subgroup with executive deficits (Stevens et al., 2018). Moreover, those patients with emotional and motivational-related deficits over-engaged the amygdala and ventral striatum during rewarded tasks with no change in prefrontal-cerebellar network.

Impulsivity is also present in bipolar disorder, the neuropsychopathology of which includes both executive and emotional-motivational deficits. In contrast to subjects diagnosed with ADHD, the difficulty in inhibiting a prepotent motor response in bipolar patients was accompanied by reduced striatal activity along with increased activation of the orbitofrontal cortex, amygdala and cerebellum (Fleck et al., 2011). Therefore, although the most common cerebellar correlate of behavioral disinhibition is reduced activity in the cerebellum, different pre-existing pathological conditions may constrain the type of brain pattern that will be observed during behavioral inhibition tasks.

In a recent genetic mouse model of ADHD (High-Active mice), downregulation of the prefrontal cortex was accompanied by hyperactivity in the granule cell layer of the cerebellar vermis during the performance of a high-speed rotarod task (Majdak et al., 2016). A low amphetamine dose normalized motor impulsivity symptom to control levels. However, amphetamine treatment reduced only cerebellar hyperactivity, leaving prefrontal downregulation unaltered. This finding points to the cerebellum as a therapeutic target for impulsive disorders similar to what has been suggested by studies using cognitive training in ADHD children (Hoekzema et al., 2010).

In drug abuse, impulsivity may act as a vulnerability factor to compulsive drug-seeking but also can be the result of repeated drug intake (Belin et al., 2008; Verdejo-García et al., 2008; Ersche et al., 2010; Hogarth, 2011; Whelan et al., 2012; Irimia et al., 2015). Cerebellar dysfunction has been proposed as one of the main factors to explain comorbidity between drug addiction and other impulsive disorders (Jasinska et al., 2014; Moulton et al., 2014; Miquel et al., 2016). Nevertheless, only a few studies have specifically investigated the cerebellar underpinnings of drug-related impulsivity. In alcoholic patients at different stages of remission, frontocerebellar dysfunction appears to be a key factor to predict and explain impulsive control deficits (Sullivan, 2003; Jung et al., 2014). Functional connectivity research demonstrated that anterior cingulate-cerebellar synchrony is degraded in alcoholics when responses have to be inhibited to avoid errors (Jung et al., 2014). Under uncertainty, alcoholics failed to activate the cerebellum, emitting more erroneous responses while compensatory activity was observed in the dorsal prefrontal and premotor cortices (Jung et al., 2014). Unlike alcoholics, adolescent cannabis users showed an increased correlation in the activity of the frontal-parietal-cerebellar network associated with poor inhibitory behavioral control in a Go/No-Go task (Behan et al., 2014). Greater correlation between the parietal cortex and cerebellum was also seen during resting state in cannabis users relative to control subjects. In this study, frontal-parietal-cerebellar hyper-connectivity did not compensate for performance as cannabis abuser committed more errors than the control group. Overall, despite the fact that both the type of drug and task conditions might be important factors for understanding the involvement of the cerebellum in drug-related impulsivity, aberrant cerebellar connectivity patterns are common to impulsive behavior in heavy drug users.

### Effects of Cerebellar Lesions on Impulsivity

Clinical reports on cerebellar diseases give support to the fundamental role of the cerebellum in modulating diverse motor, affective and cognitive domains. Beyond motor dysfunction, patients with lesions or disease affecting the posterior cerebellum showed difficulties in controlling their behavior and emotions, language deficits, and lack of concentration (Silveri et al., 1994; Schmahmann and Sherman, 1998; Kim et al., 2013; Tessier et al., 2015). The syndrome, which has been called ''the cerebellar cognitive-affective syndrome,'' is characterized by impairments in executive functions with disinhibited and inappropriate behavior, social aberrant behavior, personality changes, and language deficits (Schmahmann and Sherman, 1998).

In accordance with clinical observations, lesion studies in animals have established the relevance of the cerebellum for perseverative behavior and behavioral disinhibition (Bobée et al., 2000). Posterior vermis lesions result in a delay in behavioral inhibition during extinction trials (Callu et al., 2007). Animals that received vermis lesions when young showed perseverative behavior as adults, lack of attention to novel stimuli, and behavioral disinhibition (Bobée et al., 2000). Taken together, these results indicate that the cerebellum is a crucial component of the circuits controlling the inhibitory mechanisms for initiating actions.

### A WORKING HYPOTHESIS FOR THE ROLE OF THE CEREBELLUM IN COMPULSIVITY AND IMPULSIVITY

Although in many cases the evidence is incomplete and partial, a picture is beginning to emerge from research on the cerebellar contribution to compulsivity and impulsivity: pre- and postnatal developmental events can induce cerebellar dysfunction or alter cerebellar connectivity patterns, encouraging

basal ganglia-cerebellum connectivity while degrading prefrontal-cerebellum connections. Thus, it appears that a consequence of disrupting cerebellar function is an imbalance between dorsal (downregulation) and ventral (upregulation) influences on behavior, facilitating an over-reliance of ''Go'' brain mechanisms at the expense of ''No-Go'' inhibitory control, with actions becoming persistent and inappropriate to the context (**Figure 3**).

Overall, findings point to a modulatory function of the cerebellum in terminating or initiating actions through regulation of the prefrontal cortices. That is, the cerebellum may be crucial for restraining ongoing actions when environmental conditions change by adjusting prefrontal activity in response to the new external and internal constellation of stimuli, thereby promoting flexible behavioral control. Both electrical and non-invasive stimulation of cerebellar activity in animals and humans support a modulatory effect of the cerebellum on cortical activity (Forster and Blaha, 2003; Chen et al., 2014; Watson et al., 2014). It has been hypothesized that the cerebellar modulation consists of what has been called ''cerebellar brain inhibitory function'' (Darch et al., 2018). If this is the case, one should expect stimulation of cerebellar activity to improve prefrontal functionality and to reduce compulsive and impulsive behaviors (**Figure 2**). Notwithstanding, cerebellar stimulation can reduce or increase cortical activity as a function of the stimulation protocol (Casula et al., 2016) as well as the cortical population targeted (Watson et al., 2014). Moreover, cerebellar modulation involves subtle changes in synchronization of cortical firing more than global changes in neuronal activity (Watson et al., 2014). Also relevant is the fact that the cerebellum is not a functional unit and therefore it should not be expected that manipulations across different regions of the cerebellum should produce homogeneous effects either on behavior or on brain activity. For instance, stimulation of the cerebellar cortex should result in opposite effects to stimulation of deep cerebellar nuclei as they receive tonic inhibitory GABAergic control through Purkinje cells (Gauck and Jaeger, 2000).

A comprehensive understanding of the cerebellar function in compulsivity and impulsivity will require further research involving causative manipulation of the cerebellar activity and its connectivity since the majority of the current information comes from correlational research and clinical reports. For instance, it is known that impulsivity and disinhibition result from impairment of the cerebellar cortex, especially in the middle line (vermis; Silveri et al., 1994; Schmahmann and Sherman, 1998; Kim et al., 2013; Tessier et al., 2015). Thus, it should be possible to interfere with or mimic these effects by using pharmacogenetics tools as DREADDs (designer receptor exclusively activated by designer drugs) or optogenetics in paradigms such as Go/No-Go tasks and reward devaluation tests. These research tools could also be applied to drug-related compulsivity and impulsivity in animal models of addiction such as those proposed by Vanderschuren and Everitt (2004); Ahmed (2012) or Deroche-Gamonet and Piazza (2014). Importantly, the specific contribution of the cerebellum to drug addiction is an almost utterly uncharted field. The present model (**Figure 2**) predicts that by inhibiting activity in the cerebellar cortex impulsive and compulsive symptomatology would increase. On the contrary, the stimulation of the cerebellar cortex should improve behavioral inhibitory control in the above-mentioned paradigms and models. Opposite predictions may be made for the effects of direct manipulations in the deep cerebellar nuclei (DCN), as the DCN receives tonic inhibition from the cerebellar cortex

#### REFERENCES


(Gauck and Jaeger, 2000). If confirmed our expectations, the cerebellum would appear as the next therapeutic target for impulsive/compulsive disorders.

#### AUTHOR CONTRIBUTIONS

Several of the results revised here were obtained by IG-M, JG-C and AS-H as a part of their doctoral theses. They also critically reviewed the content of the manuscript. MM is responsible for the hypothesis and wrote the review. SN critically read and edited the review. All the authors approved the final version for publication.

### FUNDING

This research was supported by the following grants and fellowships: Universitat Jaume I (PREDOC2014/11); Universitat Jaume I (UJI-14I307.01/1); Ministerio de Economía y Competitividad (MINECO; PSI2015-68600-P); Plan Nacional de Drogas 2017 (PND-132400); and NIH grants (DA044761, DA019473, DA038412).

#### ACKNOWLEDGMENTS

We thank Chris de Zeeuw for his comments about several parts of the manuscript.


BDNF. Psychiatry Res. 194, 304–313. doi: 10.1016/j.pscychresns.2011. 05.006


Yucel, K., Nazarov, A., Taylor, V. H., Macdonald, K., Hall, G. B., and MacQueen, G. M. (2013). Cerebellar vermis volume in major depressive disorder. Brain Struct. Funct. 218, 851–858. doi: 10.1007/s00429-012- 0433-2

**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Miquel, Nicola, Gil-Miravet, Guarque-Chabrera and Sanchez-Hernandez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Increased Fear Memory and Glutamatergic Modulation in Compulsive Drinker Rats Selected by Schedule-Induced Polydipsia

Ángeles Prados-Pardo, Elena Martín-González, Santiago Mora, Ana Merchán, Pilar Flores and Margarita Moreno\*

Department of Psychology, Health Research Center, University of Almería, Campus de Excelencia Internacional Agroalimentario CeiA3, Almería, Spain

Compulsive behavior is observed in several neuropsychiatric disorders such as obsessive-compulsive disorder (OCD), anxiety, depression, phobia, and schizophrenia. Thus, compulsivity has been proposed as a transdiagnostic symptom with a highly variable pharmacological treatment. Recent evidence shows that glutamate pharmacotherapy may be of benefit in impaired inhibitory control. The purpose of the present study was: first, to test the comorbidity between compulsivity and other neuropsychiatric symptoms on different preclinical behavioral models; second, to assess the therapeutic potential of different glutamate modulators in a preclinical model of compulsivity. Long Evans rats were selected as either high (HD) or low (LD) drinkers corresponding with their water intake in schedule-induced polydipsia (SIP). We assessed compulsivity in LD and HD rats by marble burying test (MBT), depression by forced swimming test (FST), anxiety by elevated plus maze (EPM) and fear behavior by fear conditioning (FC) test. After that, we measured the effects of acute administration (i.p.) of glutamatergic drugs: N-Acetylcysteine (NAC; 25, 50, 100 and 200 mg/kg), memantine (3.1 and 6.2 mg/kg) and lamotrigine (15 and 30 mg/kg) on compulsive drinking on SIP. The results obtained showed a relation between high compulsive drinking on SIP and a higher number of marbles partially buried in MBT, as well as a higher percentage of freezing on the retrieval day of FC test. We did not detect any significant differences between LD and HD rats in FST, nor in EPM. The psychopharmacological study of glutamatergic drugs revealed that memantine and lamotrigine, at all doses tested, decreased compulsive water consumption in HD rats compared to LD rats on SIP. NAC did not produce any significant effect on SIP. These results indicate that the symptom clusters of different forms of compulsivity and phobia might be found in the compulsive phenotype of HD rats selected by SIP. The effects of memantine and lamotrigine in HD rats point towards a dysregulation in the glutamatergic signaling as a possible underlying mechanism in the vulnerability

#### Edited by:

Seth Davin Norrholm, Emory University School of Medicine, United States

#### Reviewed by:

Raül Andero, Autonomous University of Barcelona, Spain Thomas Seidenbecher, University of Münster, Germany

> \*Correspondence: Margarita Moreno mgmoreno@ual.es

Received: 31 January 2019 Accepted: 23 April 2019 Published: 07 May 2019

#### Citation:

Prados-Pardo Á, Martín-González E, Mora S, Merchan A, Flores P ´ and Moreno M (2019) Increased Fear Memory and Glutamatergic Modulation in Compulsive Drinker Rats Selected by Schedule-Induced Polydipsia. Front. Behav. Neurosci. 13:100. doi: 10.3389/fnbeh.2019.00100 to compulsive behavior on SIP. Further studies on SIP, could help to elucidate the therapeutic role of glutamatergic drugs as a pharmacological strategy on compulsive spectrum disorders.

Keywords: compulsivity, schedule-induced polydipsia, marble burying test, forced swimming test, elevated plus maze test, fear conditioning, glutamatergic modulators

### INTRODUCTION

Compulsivity has been defined as ''the performance of repetitive, unwanted and functionally impairing overt or covert behavior without adaptive function according to either rigid rules or as a means of avoiding perceived negative consequences'' (Fineberg et al., 2014). It is one of the principal symptoms in obsessive-compulsive disorder (OCD), that affects 2%–3% of the population and is considered as one of the ten leading neuropsychiatric disorders of disability (WHO, 2018). In the Diagnostic and Statistical Manual of Mental Disorders (5th edn), the obsessive-compulsive and related disorders family state that the course of OCD is often complicated by the co-occurrence of other disorders, including anxiety, specific phobia, depression, bipolar disorder, schizophrenia, and eating disorders as common comorbid pathologies (DSM-5; American Psychiatric Association, 2013). Indeed, compulsive behavior has been proposed as a trans-diagnostic symptom being comorbid especially with general anxiety disorders and depression (Gillan et al., 2017). For example, Torres et al. (2014, 2016) found that OCD patients, evaluated using the Dimensional Yale-Brown Obsessive–Compulsive Scale and Structured Clinical Interview for DSM-IV-TR Axis I Disorders, presented a lifetime prevalence of: 15.3% panic disorder (Torres et al., 2014), 56.4% major depression, 34.6% social phobia, 34.3% generalized anxiety disorder, and 31.4% specific phobia (Torres et al., 2016). Despite these studies, there are few experimental approaches in animals that have characterized the comorbidity with other altered pathological behaviors in preclinical models of compulsivity.

The clinical treatment of compulsivity in OCD patients has been focused on Serotonin reuptake inhibitors (SRIs), such as fluvoxamine, fluoxetine, sertraline, paroxetine and citalopram (reviewed in Fineberg and Gale, 2005). However, recent studies point out that up to 40% of patients do not respond successfully to SRIs treatment (Marinova et al., 2017). Recent studies suggest that glutamate-modulating drugs seem to have a beneficial effect in reducing compulsive symptoms in humans (Marinova et al., 2017) maybe because of its fundamental role in neuronal plasticity, learning, and memory (Javitt et al., 2011). Glutamate, the major excitatory neurotransmitter in the brain, is highly implicated in the cortico-striatal-thalamic circuit (Ting and Feng, 2011), the proposed neuroanatomical basis in compulsive deficit (reviewed in Menzies et al., 2008; Fineberg et al., 2010); which present a rich glutamatergic receptor density (Monaghan et al., 1985). A dysregulation of glutamatergic signaling in the corticostriatal circuitry has been suggested in OCD, with reduced glutamatergic concentrations in the anterior cingulate cortex, as well as overactivity of glutamatergic signaling in the striatum and orbitofrontal cortex (Pittenger et al., 2011; Ting and Feng, 2011; Milad and Rauch, 2012).

Preclinical and clinical data have shown evidence that glutamatergic drugs could be a promising potential benefit in compulsive disorders. The N-Acetylcysteine (NAC), glutathione (GSH) precursor and a cell-permeable antioxidant, decrease the synaptic glutamate release (Moran et al., 2005). In clinical studies, NAC treatment has been shown to be effective in SRI-resistant OCD patients (Lafleur et al., 2006). Chronic treatment of NAC in OCD patients, 10–12 weeks, reduced the Yale-Brown Obsessive-Compulsive Scale (Y-BOCS; Afshar et al., 2012; Paydary et al., 2016). Moreover, it has also shown to improve symptomatology in other psychiatric syndromes, including depression, bipolar disorder, suicidality, and self-injurious behavior (Pittenger et al., 2005; Price et al., 2009; Niciu et al., 2014). In a preclinical study using an acute administration of 100 mg/kg of NAC reduced ethanol self-administration and ethanol-seeking behavior (Lebourgeois et al., 2018). Furthermore, memantine (MEM), an uncompetitive N-Methyl-D-aspartate (NMDA) receptor antagonist, that is currently employed in the treatment of Alzheimer disease (Reisberg et al., 2003) has also shown a beneficial effect in compulsivity. MEM reduce glutamate release through inhibition of voltage-dependent calcium channel and protein kinase C (Lu et al., 2010). In OCD patients, MEM reduced the Y-BOCS scores after chronic treatment with MEM (Ghaleiha et al., 2013; Haghighi et al., 2013). Preclinical studies showed that acute administration of 25 mg/kg MEM suppressed ethanol self-administration in non-dependent rats and decreased by half the one of post-dependent rats during acute withdrawal (Alaux-Cantin et al., 2015). Besides, the administration of MEM (10 mg/kg) and amantadine, another uncompetitive NMDA receptor antagonists (30 mg/kg), significantly inhibited compulsive marble burying in mice (Egashira et al., 2008). Moreover, the combination of MEM and fluoxetine reduced scratching behavior, considered as an effective model for studying compulsive behavior (Wald et al., 2009). Lamotrigine (LAM) is an established anticonvulsant drug, with antiepileptic activity due to the inhibition of the voltage-sensitive neuronal membrane sodium channels, inhibition of the excitatory amino acids release such as glutamate and aspartate, and blockade of the calcium-channel (Cheung et al., 1992; Xie et al., 1995; Cunningham and Jones, 2000; Prabhavalkar et al., 2015). A clinical study with chronic treatment with LAM evidenced a decrease in Y-BOCS scores in OCD patients, in addition to the Hamilton Rating Scale for Depression scores, the Clinical Global Impression-Improvement scores and the obsession and compulsion subscales (Bruno et al., 2012; Khalkhali et al., 2016). Besides, preclinical research showed that 15 and 30 mg/kg acute treatment of LAM significantly reduced immobility in the forced swimming test (FST; Li et al., 2010). However, there is insufficient preclinical research on the therapeutic role of these glutamate release modulators on reducing compulsive behaviors.

Schedule-induced polydipsia (SIP), a model of compulsive behavior (Moreno and Flores, 2012), is characterized by the development of an adjunctive behavior of repetitive drinking in food-deprived animals which are exposed to intermittent food-reinforcement schedules (Falk, 1961, 1966). An analogous phenomenon, called psychogenic polydipsia, which involves compulsive non-regulatory fluid consumption, is observed in 6%–20% of psychiatric patients (Evenson et al., 1987; de Leon et al., 1994, 2002; Dundas et al., 2007; Iftene et al., 2013). SIP is considered an animal model of compulsive drinking effective for studying the compulsive phenotype and modeling different psychopathologies related to compulsive spectrum disorders (Moreno and Flores, 2012; Hawken and Beninger, 2014; Belin-Rauscent et al., 2016). The individual differences observed on SIP acquisition support the selection of high compulsive drinking rats (HD) vs. low drinker rats (LD). In our laboratory, we have found consistent differences between these two populations in the inhibitory response deficit. Thus, HD rats selected by SIP have shown increased perseverative–compulsive responses under extinction conditions on the 5-Choice Serial Reaction Time task (5- CSRT; Moreno et al., 2012); impulsive decision making on the delay-discounting task (Cardona et al., 2011); less latent inhibition effect, considered as a behavioral model of schizophrenia, and augmented behavioral inflexibility in a spatial reversal learning task, characteristic in OCD patients (Navarro et al., 2017). Thus, HD and LD rats selected by SIP has shown consistent behavioral differences among different behavioral paradigms. Otherwise, SIP is considered a good model for researching the psychopharmacology of the compulsive phenotype (Platt et al., 2008; Moreno and Flores, 2012; Rodriguez et al., 2017). Indeed, studies on SIP revealed the efficacy of antipsychotic (haloperidol, clozapine, and pimozide) and antidepressant (fluoxetine) drugs in reducing SIP water intake (Snodgrass and Allen, 1989; Didriksen et al., 1993; Mittleman et al., 1994; Hogg and Dalvi, 2004; Dwyer et al., 2010). In HD rats selected by SIP, citalopram and the serotonin 5- HT2A/<sup>C</sup> receptor agonist DOI reduced compulsive drinking (Navarro et al., 2017). Moreover, a recent study has revealed that HD rats showed cortical reduced serotonin 5-HT2A receptor binding and increased serotonin and reduced glutamate efflux compared to LD rats (Mora et al., 2018). Therefore, the study of comorbid altered behaviors and the effect of glutamatergic drugs in compulsive HD rats selected by SIP could help for a better characterization of the compulsive endophenotype and explore new possible pharmacological targets for its treatment.

According to the previous clinical data, in the present study, first, we have explored the presence of other altered behaviors, including other forms of compulsivity and typical comorbid symptoms, such as depression, general anxiety and pathological fear disorder in the high compulsive drinker rats HD selected by SIP. The animal models selected to achieve this goal has been: the marble burying test (MBT) as a assay of compulsive-like behavior (Egashira et al., 2008; de Brouwer and Wolmarans, 2018); the FST developed in Porsolt et al. (1977) as an animal model of depression that assess learned helplessness; the elevated plus maze test (EPM) as a behavioral measure of anxiety for rodents (Pellow et al., 1985); and finally, the fear conditioning (FC) to test aversive learning considered as a behavioral paradigm that model specific phobias (Berardi et al., 2012). Furthermore, as a second goal, we assessed the efficacy of different glutamatergic drugs in reducing compulsive drinking on SIP. We explored the dose-response effects of acute administration of NAC, MEM, and LAM in reducing compulsive drinking on SIP. The results are discussed regarding the contributions of the characterization of comorbid altered behaviors in the compulsive phenotype rat population HD selected by SIP and the implication of the glutamatergic modulators as a new pharmacological strategy for compulsive neuropsychiatric disorders.

## MATERIALS AND METHODS

### Subjects

A total of 16 male Long Evans rats (Janvier Labs, Le Genest-Saint-Isle, France) weighing between 250–350 g at the start of the experiments were used in the present study. The animals were housed four rats per cage (50 × 15 × 25 cm) at 22◦C, with a 12:12-h light-dark cycle (lights off at 08:00 h) and food and water provided ad libitum. Before SIP training and after 10 days of habituation, rats were gradually reduced to 85% of their free-feeding body weight through controlled feeding, and their body weights were maintained at this level of deprivation throughout the experiments. Food was provided daily 30 min after each experimental session. All testing was performed between 9:00 and 15:00 h. All the procedures were conducted following the Spanish Royal Decree 53/2013 on the protection of experimental animals, the European Community Directive (2010/63/EU) for animal experiments and approved by the University of Almería Animal Research Committee.

#### SIP Procedure

A complete description of the SIP procedure has been previously described (Moreno and Flores, 2012). First, over two successive days, we assessed the amount of water consumed by each rat in 60 min (baseline). Unlimited access to a bottle of water was provided (100 ml), and 60 food reward pellets were placed together (45 mg of dustless pellets; catalog number 259901-PE-45/50T TSE Systems, Germany). After one session of habituation to the SIP chambers (35 × 25 × 34 cm), the animals were exposed to a fixed time 60-s (FT-60s) schedule of food reward pellet presentation for 60-min sessions. During each SIP session, a bottle of water (100 ml) was positioned opposite the food-magazine in the SIP chamber, the amount of water intake was recorded at the end of the test session. The licking behavior to the bottle of water was detected when the animal touches the metal drinking tube (spout) of the bottle. The spout is connected to the metal grid of the SIP chamber, where the animal stands, by an electronic circuit with a low current, less than ten microAmp, inappreciable to the animal. When the rat touches the water spout of the bottle, this closes the circuit, producing a 50 ms pulse, which registers a lick. The scheduling and recording of the experimental events are controlled using a computer and the commercial software Med PC (Cibertec SA, Madrid, Spain). For each rat, we recorded the following measures: the total amount of water (milliliters) removed from the bottle, the total number of licks to the bottle, and the total entries to the food magazine. After 20 daily sessions, the animals were separated into two specific populations, HD and LD, according to whether their rates of drinking (average for each animal over the last five sessions) were above or below the group median, respectively (the number of animals in each group of LD and HD rats was n = 8).

#### Experimental Design

The order of the behavioral assessment and drug testing are summarized in **Figure 1**.

#### Experiment 1

#### **Behavioral Assessment**

We examined the presence of other altered behaviors considered as comorbid symptoms for compulsivity in high compulsive animals selected by SIP. We assessed compulsive-like behavior on MBT (Taylor et al., 2017), depressive-like behavior on FST (Yan et al., 2010), anxiety-like behavior on EPM (Pellow et al., 1985) and specific phobia behavior on FC (Berardi et al., 2012) in LD and HD rats selected by SIP (n per group = 8). The screening in each test commenced at least 1 week after the previous one.

#### Experiment 2

#### **Glutamatergic Drugs**

The behavioral effects of acute systemic administration of different glutamatergic drugs were tested in both groups of LD and HD rats in SIP (n per group = 8). We explored the effects of acute intraperitoneal injections (i.p.) of NAC (25, 50, 100 and 200 mg/kg), MEM (3.1 and 6.2 mg/kg) and LAM (15 and 30 mg/kg) in LD and HD rats in SIP. The drug doses, the injection time of 60 min before behavioral testing, were selected based on previous experiments (Li et al., 2010; Réus et al., 2010; Lebourgeois et al., 2018). All animals received drugs according to a fully randomized Latin-square design, separated by a minimum of 72 h between drug test sessions. There was a wash-out period of 1 week between each drug tested (animals continued performing SIP sessions during this week). The experimental sessions were led on Tuesdays and Fridays, and baseline testing was accomplished on Mondays and Thursdays. On Wednesdays, animals performed SIP procedure, but the results were not analyzed.

#### **Behavioral Assessment**

MBT began placing the rat into a corner of the cage containing nine marbles, being careful to place the rat on bedding as far from marbles as possible. Animals were allowed to remain in the cage undisturbed for 30 min. Rats were returned to its home cage after test completion, taking extreme care not to move or dislodge the marbles in the process of removing the subject from the cage. The number of marbles partially and completely buried was counted by two observers blinded to the experimental groups. We found a great concordance between observers. A marble was scored as partially buried if two-thirds of its surface area is covered by bedding and completely buried if all the surface area is covered by bedding (Angoa-Pérez et al., 2013).

FST was performed in a plastic cylinder containing 20 cm in diameter and 40 cm in height water temperature was 23–25◦C, and the depth of water was set to prevent animals from touching the bottom. Rats swam in the cylinder for 2 min. The time

each animal spent immobile during the last min of the test was counted by two observers blinded to the experimental groups. We found a great concordance between observers. Immobility was defined as floating or absolute lack of motion (i.e., the absence of all movements except those required to maintain balance; Dong et al., 2018).

For EPM rats were placed at the junction of the four arms of the maze, facing an open arm, and entries/duration in each arm was recorded by a video-tracking system and observer simultaneously for 10 min. We found a good concordance in data collected with both methods. An increase in open arm activity (duration and/or entries) reflects anti-anxiety behavior (Walf and Frye, 2007).

FC started placing the rat into a novel set of cages with a shock grid floor capable of delivering foot-shock where, after 3 min exploration period, they received three pairings of a 10 s light (82 lx) with a shock (0.5 mA during 1 s). The light-shock trials were delivered after a 3-min acclimation time, the inter-lights intervals were 1 min, and the rats remained in the chambers for an additional minute after the last shock. Next day rats were allowed a 3 min exploration period after which they were presented with 22 lights (10 s, 82 lx, 1 min inter-lights interval) in the absence of a foot shock (Simone and McCormick, 2017). The freezing time was counted by the Video Freeze Software (Med PC) which detected changes at the pixel level from one video frame to the next. Hence, data can reflect the total time animals spent in motionless during the session, the percentage of time motionless and the number of freezing episodes.

#### Drugs

We explored the effects of acute intraperitoneal injections (i.p.) of NAC (25, 50, 100 and 200 mg/kg; Lebourgeois et al., 2018), MEM (3.1 and 6.2 mg/kg; Li et al., 2010) and LAM (15 and 30 mg/kg; Réus et al., 2010) in LD and HD rats in SIP. NAC [(2R)-2-(Acetylamino)-3-mercapto propanamide] and MEM [3, 5-Dimethyl-tricyclo (3.3.1.13, 7) decan-1-amine hydrochloride] were dissolved in 0.9% saline. LAM [6-(2, 3-Dichlorophenyl)- 1, 2, 4-triazine-3, 5-diamine] was suspended in 1% Tween-80 in 0.9% saline. All drugs were purchased from Sigma-Aldrich (Madrid, Spain). The injection volumes were 1 ml/kg for all drugs. For all drug solutions, the final pH was adjusted to approximately 6.4 using 0.1 M NaOH, and they were aliquoted after preparation and frozen at −80◦C before use.

#### Data Analyses

Behavioral data on SIP acquisition were analyzed using two-way repeated-measure analysis of variance (ANOVA), with ''group'' (LD and HD) as the between-subject factor and ''sessions'' (20 sessions) as the within-subject factor. The differences on the MBT, FST, EPM, and FC of the behavioral assessment in LD and HD were studied using Student's t-test (T-test). When appropriate, the effect size of the group differences was calculated using Cohen's d (d; mean difference divided by pooled standard deviation). The differences on FC blocks and the effects of the different drugs in LD and HD on SIP were analyzed using two-way repeated-measure ANOVA, with group (LD and HD) as the between-subject factor and ''percentage of freezing'' (percentage of time spent on freezing during the different blocks of the retrieval day) or ''drug'' (different doses of drug and vehicle) as the repeated within-subject factor. When appropriate, the effect size of the group differences was calculated using eta-squared (η 2 ). Post hoc comparisons were performed using the Newman-Keuls test. Statistical significance was set at p < 0.05. All analyses were computed using Statistica software (version 6.0).

## RESULTS

### LD and HD Selected by SIP

The mean water intake and licks in LD and HD during the acquisition and maintenance of SIP is shown in **Figures 2A,B**. In the experimental phase, the mean water intake over the last 5 days of SIP was 4.3 ± 0.6 and 11.2 ± 1.9 ml for LD and HD, respectively (**Figure 2C**). The number of licks also showed SIP acquisition. The mean total licks averaged across the last 5 days of SIP were 885.1 ± 202.9 and 2742.9 ± 536.9 for LD and HD, respectively (**Figure 2D**).

ANOVA revealed significant differences in water intake according to the interaction between SIP acquisition sessions and LD vs. HD (SIP session effect: F(19,266) = 11.759, p < 0.001; group effect: F(1,14) = 10.332, p < 0.01; interaction SIP session × group effect: F(19,266) = 2.58, p < 0.001). This difference was also confirmed by the significant interaction observed in the total number of licks (SIP session effect: F(19,266) = 11.890, p < 0.001; group effect: F(1,14) = 13.647, p < 0.01; interaction SIP session × group effect: F(19,266) = 3.38, p < 0.001). Post hoc analysis indicated significant differences between the LD and HD animals in the water intake at session 6 (p < 0.01) onwards. Furthermore, animals in the HD group significantly increased their consumption of water from session 4 (p < 0.05) compared to session 1. Differences between the LD and HD groups in the number of total licks at session 6 (p < 0.05) were also observed, and HD rats increased their number of licks from session 5 (p < 0.001) compared to session 1. We also found significant differences in the number of magazine entries according to the interaction between SIP acquisition sessions and LD vs. HD (session × group effect: F(19,266) = 2.124; p < 0.01; session effect: F(19,266) = 4.515, p < 0.001; group effect: F(1,14) = 5.577, p < 0.05). Differences between the LD and HD groups in the number magazine entries at session 11 (p < 0.001 were also observed, and HD rats increased their number of magazine entries from session 6 (p < 0.05) compared to session 1.

### Experiment 1

## Behavioral Assessment

### **Marble Burying Test**

The number of marbles partially (2/3) and completely buried by LD and HD rats on MBT are shown in **Figure 3A**. T-test and the effect sizes by Cohen's d showed that HD rats had a significantly increased number of marbles partially (2/3) buried compared to LD rats (df = 14; T-test = −2.22; p < 0.05; d = 1.186). There was no significant effect on the number of marbles completely buried between LD and HD rats (df = 14; T-test = 1.14; p = 0.27).

#### **Forced Swimming Test**

The percentage of immobile time of LD and HD rats on FST are shown in **Figure 3B**. T-test showed no significant difference in the percentage of immobile time between LD and HD rats (df = 14; T-test = 0.35; p = 0.72).

#### **Elevated Plus Maze Test**

The time LD and HD rats spent on the open arm before changing to the other, and the number of entries in the open arm on EPM are shown in **Figures 3C,D**. T-test showed that there was no significant difference in the mean time and the number of entries in the open arms between LD and HD rats (df = 14; T-test = −0.09; p = 0.92; df = 14; T-test = 0.86; p = 0.40). The mean time LD and HD rats spent on the closed arm before changing to the other was 1.53 ± 0.35 and 1.83 ± 0.39, respectively. The mean number of entries in the closed arm on EPM was 9.38 ± 0.67 for LD rats and 8.88 ± 1.24 for HD rats. T-test showed that there was no significant difference in the mean time and the number of entries in the closed arms between LD and HD rats (df = 14; T-test = −0.60; p = 0.56; df = 14; T-test = 0.38; p = 0.71). The mean time LD and HD rats spent on one arm before changing to another one was 1.53 ± 0.06 and 1.70 ± 0.08, respectively. The mean number of entries in open and closed arms was 18.50 ± 1.14 for LD rats and 16.38 ± 1.23 for HD rats. T-test showed that there was no significant difference in the mean time and the number of entries in open and closed arms between LD and HD rats (df = 14; T-test = −1.85; p = 0.08; df = 14; Ttest = 1.35; p = 0.20).

#### **Fear Conditioning**

The percentage of freezing time of LD and HD rats on FC during the acquisition day, the percentage of freezing time during the contextual fear test and the cued fear test at the retrieval day, as well as the percentage of freezing during the different blocks of trials on the retrieval day, is shown in **Figures 3E–H**. No significant differences were found in the percentage of freezing time spent by LD and HD rats during the acquisition day (df = 14; T-test = −0.45; p = 0.65), nor in the contextual fear test on the retrieval day (df = 14; T-test = −1.51; p = 0.15). However, T-test and effect sizes by Cohen's d revealed a significant increase in the percentage of freezing time spent by HD compared to LD rats during the cue presentation on retrieval day (df = 14; T-test = −3.12; p < 0.01; d = 1.67). The analyses of the 4 blocks of trials on the retrieval day by ANOVA and η 2 revealed that both, LD and HD rats, significantly reduced the percentage of freezing time in the different blocks of the retrieval day (Trial effect: F(3,42) = 36.64; p < 0.001;

time LD and HD rats spent on forced swimming test (FST), (C) mean number of entries by LD and HD rats on the open arms in elevated plus maze test (EPM), (D) seconds spent by LD and HD rats on the open arms in EPM, (E) percentage of freezing LD and HD rats exhibited during fear acquisition day, (F) percentage of freezing LD and HD rats exhibited during contextual fear test on retrieval day, (G) percentage of freezing LD and HD rats exhibited during cued fear test on retrieval day, and (H) percentage of freezing LD and HD rats exhibited during the four blocks of time (6 min per block) at cued fear test on retrieval day of fear conditioning procedure (FC). Data are expressed as the means ± SEM. <sup>∗</sup>p < 0.05; ∗∗p < 0.01; ∗∗∗p < 0.001 indicate significant differences between groups.

η <sup>2</sup> = 0.931); whether the significant increased percentage of freezing time spent by HD compared to LD rats was maintained through the four blocks of trials on the retrieval day (group effect: F(1,14) = 9.73; p < 0.01; η <sup>2</sup> = 0.933). No significant differences were observed by group × trial interaction (F(3,42) = 0.27; p = 0.84).

#### Experiment 2

#### Glutamatergic Drugs

#### **N-Acetylcysteine**

The effects of NAC on water intake and licks in SIP are shown in **Figures 4A,B**, and the number of magazine entries after NAC administration are shown in **Table 1**. ANOVA showed that NAC did not induce significant differences in water intake (group × drug interaction, F(4,56) = 0.63, p = 0.64; group effect, F(1,14) = 109.15, p < 0.001; drug effect, F(4,56) = 0.38, p = 0.82), total licks (group × drug interaction, F(4,56) = 0.57, p = 0.68; group effect, F(1,14) = 111.89, p < 0.001 drug effect, F(4,56) = 0.90, p = 0.47), and magazine entries (group × drug interaction, F(4,56) = 0.28, p = 0.89; group effect, F(1,14) = 8.41, p < 0.05; drug effect, F(4,56) = 0.51, p = 0.73).

#### **Memantine**

The effects of MEM on water intake and total licks in SIP are shown in **Figures 4C,D**. Effects of MEM on magazine entries are depicted in **Table 1**. MEM significantly reduced compulsive TABLE 1 | Effects of N-Acetylcysteine (NAC), memantine (MEM) and lamotrigine (LAM) on total magazine entries in low drinkers (LD, n = 8) and high drinkers (HD, n = 8) rats on schedule-induced polydipsia (SIP).


Data are expressed as the means ± SEM. <sup>∗</sup>p < 0.05 indicate significant differences vs. vehicle administration in the same group of rats.

water intake in HD rats compared to LD rats (group × drug interaction, F(2,28) = 4.51, p < 0.05; group effect, F(1,14) = 24.05, p < 0.001; drug effect, F(2,28) = 8.42, p < 0.01; η <sup>2</sup> = 0.930). Post hoc analyses revealed that MEM reduced dose-dependent water intake in HD rats at both doses: 3.1 (p < 0.05) and 6.2 mg/kg (p < 0.001) compared with vehicle in the same group. MEM did not affect water intake in LD rats. The comparison between LD and HD revealed a dose dependent reduction of the significant differences in water intake disappearing at the highest dose (vehicle, p = 0.0001; 3.1 mg/kg, p = 0.041; 6.2, p = 0.572). Moreover, MEM also significantly reduced the total licks in HD rats compared with the LD group (group × drug interaction, F(2,28) = 6.04, p < 0.01; group effect, F(1,14) = 16.96, p < 0.05; drug effect, F(2,28) = 5.50, p < 0.01; η <sup>2</sup> = 0.730). Post hoc comparison confirmed a decrease in the total licks in the HD group at the highest dose used 6.2 mg/kg (p < 0.001) compared with vehicle in the same group. Differences between LD and HD remained significant at all doses tested. MEM administration did not affect the number of magazine entries in both groups of rats (group × drug interaction: F(2,28) = 2.663; p = 0.087; drug effect: F(2,28) = 2.507; p = 0.099; group effect: F(1,14) = 1.569; p = 0.23).

#### **Lamotrigine**

The effects of LAM on water intake and total licks in SIP are shown in **Figures 4E,F**. The effects of LAM on magazine entries in SIP are shown in **Table 1**. LAM significantly reduced compulsive water intake in HD rats compared to LD rats (group × drug interaction: F(2,28) = 11.396, p < 0.0002; group effect: F(1,14) = 5.187, p < 0.05; drug effect: F(2,28) = 3.532, p < 0.05; η <sup>2</sup> = 0.882). Post hoc analyses revealed that LAM reduced dose-dependent water intake in HD rats at both doses: 15 (p < 0.05) and 30 mg/kg (p < 0.01) compared with vehicle in the same group. LAM reversed the significant differences on water intake between LD and HD rats on SIP (vehicle, p = 0.008; 15 mg/kg p = 0.16; 30 mg/kg, p = 0.914). LAM did not affect water intake in LD rats. Moreover, LAM also significantly reduced the total licks in HD rats compared with the LD group (group × drug interaction, F(2,28) = 11.40, p < 0.001; group effect, F(1,14) = 5.18, p < 0.05; drug effect, F(2,28) = 3.53, p < 0.05; η <sup>2</sup> = 0.870). Post hoc comparison showed a dose dependent decrease in the total licks in the HD group at both doses used 15 mg/kg (p < 0.05) and 30 mg/kg (p < 0.001) compared with vehicle in the same group. The comparison between LD and HD revealed a dose dependent reduction of the significant differences in the number of licks disappearing at the highest dose (vehicle, p = 0.0001; 15 mg/kg p = 0.005; 30 mg/kg, p = 0.86). LAM administration reduced magazine entries in both groups of rats (group × drug interaction: F(2,28) = 3.61, p < 0.05; group effect: F(1,14) = 0.19, p = 0.67; drug effect: F(2,28) = 4.65, p < 0.05; 0.931). Post hoc analyses revealed a decrease in magazine entries in HD rats only at the highest dose tested 30 mg/kg (p < 0.05) compared with vehicle and with the LD group.

#### DISCUSSION

The present study investigated the presence of possible comorbid symptoms (compulsive, depressive, anxious and fear behavior) in animals selected by high compulsive drinking behavior on SIP, HD rats. Moreover, we investigated the therapeutic potential of glutamatergic drugs for reducing compulsive drinking behavior in HD rats on SIP. The findings showed that HD rats, characterized by excessive and persistent compulsive drinking on SIP, also exhibited a compulsive behavior on MBT by a higher number of marbles partially buried (2/3) compared to LD rats. Besides, compulsive HD rats selected by SIP had an increased fear behavior profile on FC, showed by a higher percentage of freezing time in the first block of the retrieval day as well as across the following blocks, compared to LD rats. These differences between HD and LD rats might not be attributed to individual differences in reactivity to novelty. HD rats selected by SIP did not differ in spontaneous locomotor reactivity to novelty compared with LD rats (Moreno et al., 2012). Moreover, in the present study, no significant differences were found in the number of magazine entries, considered as a control measure of motor activity or motivational behavior (Navarro et al., 2015), between HD and LD on SIP.

The acute administration of glutamatergic drugs revealed that MEM and LAM reduced, in a dose-dependent manner, compulsive intake in HD rats on SIP, and did not affect LD behavior. Hence, the observed effect cannot be considered as a compensatory behavior by the use of these treatments. Moreover, we discard other possible side effects, as in previous studies the selected doses of NAC, MEM, and LAM did not affect locomotor activity in rats (Li et al., 2010; Réus et al., 2010; Lebourgeois et al., 2018). However, NAC administration did not selectively affect compulsive intake in SIP, as LD and HD kept significant differences at all doses administrated.

#### Assessment of Comorbid Behaviors on Compulsive HD Rats

HD rats selected by SIP showed comorbidity with compulsive behavior on MBT, by a significantly increased number of marbles partially buried compared to LD rats. Previous studies have found that HD rats selected by SIP showed other behavioral compulsivity forms such as compulsive lever pressing, during the pre-training phase to assess latent inhibition (Navarro et al., 2017), proposed as an OCD model (Joel and Avisar, 2001); and behavioral inflexibility in a spatial reversal task (Navarro et al., 2017). In contrast, other studies on rats with high levels of grooming, considered as a compulsive-like behavior, have shown a reduced number of marbles buried in MBT, showing a negative correlation between these factors (Reimer et al., 2015). The reason for these contradictory results could be due to the fact that compulsivity is not a unitary phenomenon and can be expressed by different forms (Fineberg et al., 2018).

The assessment of depressive behavior revealed that LD and HD rats selected by SIP did not exhibit any differences in depressive-like behavior measured on FST. The compulsive HD rats might not have depression signs as a comorbid behavior. Nevertheless, other preclinical studies have shown associations between depressive and compulsive behavior in the same individuals. For example, the administration of 8- OH-DPAT, a 5-HT1<sup>A</sup> agonist, proposed as an OCD model (Yadin et al., 1991), increased the immobility time on FST (Sela et al., 2010). Moreover, the administration of the purinergic receptor P2R antagonist [pyridoxalphosphate-6-azophenyl-2', 4'-disulfonic acid tetrasodium salt (PPADS)] in Swiss mice, reduced depressive-like behavior in the FST, as well as compulsive-like behavior in MBT (Pereira et al., 2013). The effect of antidepressants on addictions, considered as compulsive disorders, has created some controversy. On the one hand, some preclinical studies have demonstrated reductions in alcohol addiction subsequently to the administration of different 5-HT receptors agonists (Naranjo et al., 1986; Higley et al., 1998; Martijena et al., 2005). On the other hand, the possibility that antidepressant treatment might increase susceptibility to alcoholism has been overlooked (Alén et al., 2013, 2014). Moreover, several clinical studies have shown that pathological gambling, associated with elevated compulsivity, frequently co-occurs with major depression (Cunningham-Williams and Cottler, 2001; Baer et al., 2015; Redden et al., 2015; Agarwal et al., 2016; Grant et al., 2016; Rickelt et al., 2016). More research is needed to clarify the relation between depressive and compulsive behavior.

Anxiety behavior measured by EPM did not show any significant differences between HD and LD rats selected by SIP. Nevertheless, we have replicated the results published in 2008 by López-Grancha, in which there were no differences in the EPM between LD and HD rats selected by SIP (López-Grancha et al., 2008). Moreover, animals with distinct levels of self-grooming emission, considered as a compulsive-like behavior, did not differ in the exploration of the EPM (Reimer et al., 2015). In contrast, a previous study has shown that an increased compulsive behavior in the MBT has also been accompanied by increased anxiety response in the EPM and open-field test in the same animals (Mitra et al., 2016). These contradictory results posit the relevance of the study on individual differences, using populations more prone to a behavioral deficit. Self-grooming and MBT might be evaluating different kinds of compulsivity, as well as anxiety is also a neuropsychological domain that could be expressed by different symptoms (reviewed in Ströhle et al., 2018). For instance, compulsive drinkers HD rats selected by SIP did not differ in anxiety-like behavior assessed using EPM to LD rats, while they differed in anxiety-like behavior measured by freezing time on the retrieval day in FC.

The assessment of fear behavior by FC revealed that HD rats selected by SIP showed a significantly augmented percentage of freezing time compared to LD rats during cued-fear memory on the retrieval day. Thus, HD and LD rats had no differences in the percentage of freezing time on the acquisition day, nor in the exploration period when exposed to the fear context on the retrieval day. Previous findings in our laboratory, have shown that under extinction conditions, HD rats had a greater increase in perseverative responses, considered as compulsive behavior, compared to LD rats on 5-CSRT (Moreno et al., 2012). Moreover, HD rats have shown increased c-Fos activity in the basolateral amygdala compared with LD rats (Merchán et al., 2019). The basolateral amygdala, as an essential structure in the neural system for FC (Phillips and LeDoux, 1992; Vazdarjanova and McGaugh, 1998), is highly implicated in cued-related fear memories and not essential for contextual FC (reviewed in Curzon et al., 2009). HD animals selected by SIP might be a convenient phenotype to study the neuronal basis of individual differences in habit formation under extinction conditions. Thus, in HD rats, a possible alteration in the basolateral amygdala might underlie the observed increased cued-fear memory on FC that possibly also affect the vulnerability to develop compulsive behaviors. In this sense, clinical studies demonstrated that OCD patients continued to exhibit a differential skin conductance response to the conditioned stimuli in the extinction phase of fear conditioned computer task, while control participants extinguished fear (Geller et al., 2017). Translational neuroscience studying fear could help us to better understand brain circuitry underlying fear behavior, although the translation of animal model results into the clinic is limited and more research is needed (Flores et al., 2018).

### Effects of Glutamatergic Drugs on Compulsive Rats on SIP

The administration of NAC (25, 50, 100 and 200 mg/kg) revealed no significant differences in the water intake nor LD, nor in HD rats on SIP. Conversely, previous research has demonstrated that NAC (90 mg/kg), chronically and systemically administered, resulted in significant reductions of compulsive binge eating in a rodent model (Hurley et al., 2016). NAC systemically administrated has been demonstrated to abolish the recovery of compulsive cocaine-seeking behavior in a rodent model through augmenting the glutamate/cystine antiporter activity and reestablishing the concentration of extracellular glutamate in the nucleus accumbens (Baker et al., 2003a,b). Moreover, the acute administration of NAC at 100 mg/kg reduced motivation, seeking and relapse to self-administration of ethanol in rats (Lebourgeois et al., 2018). However, acute injections of NAC (0, 30, 60, or 120 mg/kg) did not have any result on self-administration of methamphetamine in rats (Charntikov et al., 2018). Some clinical studies have suggested the possible therapeutic role of NAC in OCD patients, showing a reduction in the scores of the Y-BOCS after treatment with NAC during 10 and 12 weeks respectively (Afshar et al., 2012; Paydary et al., 2016).

The acute systemic administration of MEM, 3.1 and 6.2 mg/kg, decreased compulsive drinking in HD rats on SIP, compared to LD rats that remain unaffected. Hence, these results could not be considered as a general effect on rats exposed to SIP, pointing towards the neuropsychopharmacological effects of MEM might be involved in the vulnerability to compulsive non-regulatory drinking on SIP. In contrast, previous studies, have found that acute administration of MEM at 5 and 25 mg/kg in mice, did not affect water intake on SIP, but revealed a reduction in regulatory drinking (Escher et al., 2006). Although in this study, mice were not selected according to the rate of compulsive drinking. However, in the same study MEM have been found as a useful treatment for reducing compulsive alcohol intake, the administration of MEM 10 and 25 mg/kg significantly reduced alcohol drinking in mice on SIP (Escher et al., 2006). Moreover, findings revealed that acute administration of 10 mg/kg MEM significantly inhibited compulsive behavior in MBT without affecting locomotor activity in mice (Egashira et al., 2008). Furthermore, acute administration of 25 mg/kg MEM blocked ethanol self-administration in non-dependent rats, as well as it decreased by half the one of post-dependent rats during acute withdrawal (Alaux-Cantin et al., 2015). Otherwise, compulsive lever pressing, proposed as an OCD model (Joel and Avisar, 2001), was not affected by an NMDA antagonist (MK 801), while an NMDA partial agonist (D-cycloserine) decreased this behavior (Albelda et al., 2010). In this sense, the present results also contrast with the no effect found after ketamine administration in HD and LD rats on SIP (Martín-González et al., 2018). Though both ketamine and MEM typify the same kind of drugs, they diverge in voltage dependence and blocking kinetics (Danysz and Parsons, 1998). In human studies, MEM showed a therapeutic role in obsessive-compulsive patients, by reducing the Y-BOCS scores after chronic treatment with MEM during 8 weeks (Ghaleiha et al., 2013) and 12 weeks (Stewart et al., 2010; Haghighi et al., 2013). Other study investigating MEM augmentation of risperidone treatment in children with autism spectrum disorders revealed that the group receiving MEM showed significant improvements in the subscales: irritability, stereotypic behavior, and hyperactivity of the Aberrant Behavior Checklist-Community (Ghaleiha et al., 2013).

Our data showed that the administration of LAM, 15 and 30 mg/kg, significantly decreased compulsive water drinking in HD rats, compared to LD rats, on SIP. There are few preclinical studies on the behavioral effects of LAM, most of them related to as an anti-depressant like effect. The acute administration of LAM at 16 and 32 mg/kg of LAM induced a reduction in immobility time in the FST (Prica et al., 2008). Similarly, LAM at 15 and 30 mg/kg significantly reduced immobility in the FST (Li et al., 2010). In human studies, have evidenced that 16 weeks of treatment with LAM in obsessive-compulsive patients significantly reduced the Y-BOCS scores, as well as the Hamilton Rating Scale for Depression scores and the Clinical Global Impression-Severity scores (Bruno et al., 2012). More recently, two other studies using adjunctive treatment of LAM in addition to SRIs treatment led in treatment-resistant OCD patients during 8 and 12 weeks respectively, revealed a greater reduction in total YBOCS scores in LAM group (Hussain et al., 2015; Khalkhali et al., 2016).

Collectively, the beneficial effects of MEM and LAM administration in reducing compulsive drinking in HD rats on SIP suggest a therapeutic role for glutamate inhibition, antagonizing NMDA receptor or blocking calcium and sodium channels in pre-synaptic terminals. In contrast, the lack of effect of NAC in compulsive intake in HD rats on SIP posits the idea of the possible relevance of the differential effect by the specific stimulation of the presynaptic terminal. These results support the possible dysregulation in glutamatergic signal previously observed, in which HD rats selected by SIP showed a decreased basal level of glutamate in the medial prefrontal cortex (mPFC), restored by serotonin 5-HT2A/<sup>C</sup> agonist DOI (Mora et al., 2018). Moreover, the effects of glutamatergic drugs MEM and LAM suggest a possible modulatory role in the neuroanatomic and neurochemical alterations observed in dopamine D<sup>2</sup> receptors and 5-HT2A receptors in HD rats selected by SIP (Pellón et al., 2011; Moreno et al., 2012; Mora et al., 2018).

Preclinical studies on compulsivity, using the dopamine D<sup>2</sup> and D<sup>3</sup> receptor agonist quinpirole (QNP) in rats (Szechtman et al., 1998), have also evidenced a dysregulation by an increased glutamate release in the subtantia nigra and a lower extracellular concentration in the nucleus accumbens (Abarca et al., 1995; Krügel et al., 2004; Escobar et al., 2015). Therefore, the proposed underlying mechanism in compulsivity of the QNP-OCD model was associated with decreased dopaminergic and glutamatergic neurotransmission in the mPFC to the nucleus accumbens, pointing toward a loss of executive control (Escobar et al., 2015). Furthermore, NMDA dependent glutamate neurotransmission in the cortico-striatal circuitry seems to play a central role by the functional interaction with serotonin and dopamine receptors in executive response control and compulsivity measured by the 5-CSRT (reviewed in Carli and Invernizzi, 2014). In example, the local infusions of NMDA receptor antagonist 3-((R)-2 carboxypiperazin-4-yl)-propyl-L-phosphonic acid ((R)-CPP) in the mPFC and also in the infralimbic cortex impaired accuracy and increased premature and perseverative responding, raising glutamate, dopamine, and GABA release in the dorsomedial striatum (Pozzi et al., 2011; Murphy et al., 2011; Agnoli et al., 2013). Similarly, in OCD patients, a dysregulation of glutamatergic signaling in the cortico-striatal circuitry has been suggested, with decreased concentrations of glutamate in the anterior cingulate cortex, accompanied by overactivity of the glutamate signaling in the striatum and orbitofrontal cortex (Pittenger et al., 2011; Ting and Feng, 2011; Milad and Rauch, 2012). Other authors proposed that the beneficial effect of MEM in OCD patients could be mediated by functional disconnection of the hippocampus with critical frontal regions (Vlˇcek et al., 2018), by its effect on decreasing glutamate level in the hippocampus (Glodzik et al., 2009). Finally, we could hypothesize that according to these results, a possible explanation under the differences in compulsive HD rats selected by SIP might be an altered function of glutamatergic NMDA receptors that affect firing in cortical neurons in mPFC and affect glutamatergic, as well as dopaminergic and serotoninergic signal in the striatum.

### CONCLUSION

The exploration of other possible comorbid behaviors in compulsive HD rats selected by SIP indicated a relation with another form of compulsivity, measured by marble burying, and an increased vulnerability to cued fear behavior showed by an increased percentage of freezing time on FC compared to LD rats. No differences were found in the assessment of the depressive behavior on FST, nor in anxious behavior on EPM, replicating previous results from our laboratory (López-Grancha et al., 2008). The acute administration of glutamatergic drugs on SIP revealed that MEM and LAM dose-dependently and selectively decreased compulsive intake in HD rats, and did not affect LD on SIP. However, NAC did not affect compulsive drinking on SIP. These differences might be due to the specific action of the drugs on the presynaptic terminal. Further studies might disentangle the specific implication of the fear learning component and the dysregulation in glutamatergic neurotransmission, and its relation with the dopamine D2/<sup>3</sup> and serotonergic 5-HT2A receptors, in the mechanisms of vulnerability to compulsive behavior in HD rats on SIP.

### REFERENCES


#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of ''the Spanish Royal Decree 53/2013 on the protection of experimental animals, the European Community Directive (2010/63/EU) for animal experiments.'' The protocol was approved by the University of Almería Animal Research Committee.

### AUTHOR CONTRIBUTIONS

MM and PF designed research. ÁP-P, EM-G, SM and AM performed research. ÁP-P and EM-G analyzed data. ÁP-P and MM wrote the manuscript with the help of the other authors.

### FUNDING

This work was supported by grants: Ministerio de Economía, Industria y Competitividad; Ministerio de Ciencia Innovación y Universidades from the Gobierno de España (Spanish Governement) and Fondo Europeo de Desarrollo Regional (Grants number MINECO-FEDER PSI2015-70037-R; MICINN-FEDER PGC2018-099117-B-C21).


treatment. Int. J. Neuropsychopharmacol. 21, 42–58. doi: 10.1093/ijnp/ pyx088


disorder (OCD). Behav. Neurosci. 112, 1475–1485. doi: 10.1037//0735-7044. 112.6.1475


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Prados-Pardo, Martín-González, Mora, Merchán, Flores and Moreno. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Impulsivity Derived From the Dark Side: Neurocircuits That Contribute to Negative Urgency

#### Eric P. Zorrilla<sup>1</sup> \* and George F. Koob<sup>2</sup>

<sup>1</sup>Department of Neuroscience, The Scripps Research Institute, La Jolla, CA, United States, <sup>2</sup>Neurobiology of Addiction Section, Intramural Research Program, National Institute on Drug Abuse, Baltimore, MD, United States

Negative urgency is a unique dimension of impulsivity that involves acting rashly when in extreme distress and impairments in inhibitory control. It has been hypothesized to derive from stress that is related to negative emotional states that are experienced during the withdrawal/negative affect stage of the addiction cycle. Classically, a transition to compulsive drug use prevents or relieves negative emotional states that result from abstinence or stressful environmental circumstances. Recent work suggests that this shift to the "dark side" is also implicated in impulsive use that derives from negative urgency. Stress and anxious, depressed, and irritable mood have high comorbidity with addiction. They may trigger bouts of drug seeking in humans via both negative reinforcement and negative urgency. The neurocircuitry that has been identified in the "dark side" of addiction involves key neuropeptides in the central extended amygdala, including corticotropin-releasing factor. The present review article summarizes empirical and conceptual advances in the field to understand the role of the "dark side" in driving the risky and detrimental substance use that is associated with negative urgency in addiction.

#### Edited by:

David Belin, University of Cambridge, United Kingdom

#### Reviewed by:

Erin Jane Campbell, Florey Institute of Neuroscience and Mental Health, Australia Sharon Morein-Zamir, Anglia Ruskin University, United Kingdom

#### \*Correspondence:

Eric P. Zorrilla ezorrilla@scripps.edu

Received: 03 January 2019 Accepted: 05 June 2019 Published: 25 June 2019

#### Citation:

Zorrilla EP and Koob GF (2019) Impulsivity Derived From the Dark Side: Neurocircuits That Contribute to Negative Urgency. Front. Behav. Neurosci. 13:136. doi: 10.3389/fnbeh.2019.00136 Keywords: negative urgency, impulsivity, compulsive drug use, negative affect, withdrawal, substance or alcohol use disorder, orbitofrontal cortex, extended amygdale

#### IMPULSIVITY, COMPULSIVITY, AND NEGATIVE URGENCY

Impulsivity and compulsivity play central roles in both lay and scientific conceptualizations of addiction (Jentsch and Taylor, 1999; Koob and Le Moal, 2008; Economidou et al., 2009; Kwako et al., 2019). Behavior that is impulsive, from the Latin impuls- (''tending to impel or driven onward''), is defined as an action that is instigated suddenly without forethought of potential consequences. An impulsive behavior may be experienced as acting instinctively upon emotions or involuntary impulses and can be followed by regret, guilt, or shame. Externally, impulsivity may appear as acting hastily, capriciously, or on whims and prioritizing immediate gains vs. later outcomes. Behavior that is compulsive, from the Latin compuls- (''driven or forced''), is defined as an action that results from or is related to an irresistible urge, whereby irresistibility can be operationalized as behavior that persists despite aversive or incorrect outcomes. A compulsive behavior is often experienced as outside of one's control and, even during its performance, can be intrusive, unwanted, and ego-dystonic. From a historical perspective, the constructs of impulsivity and compulsivity have received increasing cross-cultural attention recently, much focused on their role in addictive behavior. The word ''impulsive'' exploded into use in Western countries generally during the later 19th century and preceded the emergence of the word ''compulsivity'' during the later 20th century (Michel et al., 2011; **Figure 1**). Many compulsive behaviors are commonly described (**Table 1**), but ''compulsive drug use'' and ''compulsive drinking'' are among the most frequent, constituting ∼1.3% of the written use of ''compulsive'' in the modern English language (**Table 1**). Here, we propose that the even more modern construct of negative urgency (Cyders and Smith, 2008) may play an integrative role that bridges impulsivity with compulsivity in the ''dark side'' of addiction.

### DEFINITION AND MEASUREMENT OF NEGATIVE URGENCY

Conceived as an emotion-based trait, negative urgency refers to acting rashly when in extreme distress and involves impairments in inhibitory control (Cyders and Smith, 2008). In people, negative urgency has been measured via the eponymous subscale of the Urgency, Premeditation (lack of), Perseverance (lack of), Sensation Seeking, Positive Urgency, Impulsive Behavior Scale (UPSS-P; Whiteside and Lynam, 2003; Lynam et al., 2006; Cyders and Smith, 2008). The 12 self-report Negative Urgency items, distinct from the more recently developed Positive Urgency subscale, quantify the tendency to act rashly during negative affective states. Exemplar items that refer to actions during negative mood include the following: (i) When I feel bad, I will often do things I later regret in order to make myself feel better now; (ii) I often make matters worse because I act without thinking when I am upset; and (iii) Sometimes when I feel bad, I can't seem to stop what I am doing even though it is making me feel worse. Although the first and second of these items are consistent with the

TABLE 1 | Relative n-gram frequency (vs. "compulsive") in the English Corpus of Google Books.


Note: searches were performed in May 2018 with http://books.google.com/ngrams as [(term)/compulsive]. The n-gram dataset was generated in July 2012 by Google (v20120701). <sup>1</sup> Includes compulsive shopping, compulsive buying. <sup>2</sup> Includes compulsive drug, compulsive substance, compulsive smoking, compulsive nicotine, compulsive tobacco, compulsive marijuana, compulsive caffeine, compulsive cocaine, compulsive opiate, compulsive heroin, compulsive morphine, compulsive psychostimulant, compulsive amphetamine, compulsive methamphetamine. <sup>3</sup> Includes compulsive drinking, compulsive alcohol, compulsive ethanol. <sup>4</sup> Includes compulsive washing, compulsive hand washing, compulsive cleaning. <sup>5</sup> Includes compulsive grooming, compulsive hair, compulsive nail. <sup>6</sup> Includes compulsive Internet, compulsive video game.

reviewed definition of impulsive behavior, the third is also consistent with compulsivity (i.e., persistent ''irresistibly'' despite negative outcomes).

Other drug use-relevant items that have been identified by factor analysis to also load on Negative Urgency include the following: (i) I have trouble controlling my impulses; (ii) I have trouble resisting my cravings (for food, cigarettes, etc.); and (iii) It is hard for me to resist acting on my feelings. Each of these items is potentially consistent with either impulsivity or compulsivity, depending on their perseveration in the face of actual negative or incorrect outcomes. Thus, although Negative Urgency is typically viewed as a dimension of impulsive behavior, several of the items that measure it may also detect a predisposition for or changes in compulsive behavior (i.e., irresistible, viscerally driven behavior with loss of control despite negative consequences).

In early work, the Negative Urgency subscale showed good internal consistency and construct validity (Whiteside and Lynam, 2003). Subsequently, the UPSS-P has been adopted for a short form (Cyders et al., 2014b) and for children (Zapolski et al., 2011) and has been translated to many languages (Van der Linden et al., 2006; Kämpfe and Mitte, 2009; Keye et al., 2009; Verdejo-García et al., 2010; Billieux et al., 2012; Candido et al., 2012; Lim and Lee, 2014; D'Orta et al., 2015; Fossati et al., 2016; Poprawa, 2016; Shokri and Sanaeeour, 2016; Bteich et al., 2017; Sediyama et al., 2017; Bousardt et al., 2018).

### NEUROCIRCUITRY OF ADDICTION: VIEW FROM THE DARK SIDE

Preclinical research in animal models and imaging studies in humans have provided critical insights into the pathological behavior that characterizes addiction. Convergent results show that individuals with addiction undergo progressive functional and even structural disruptions of brain regions that subserve normal processes of incentive salience, habits, emotional regulation, stress, and executive function (Robbins and Everitt, 1999; Everitt and Robbins, 2005; Koob and Volkow, 2010, 2016; Goldstein and Volkow, 2011; Belin et al., 2013). Heuristically, drug addiction has been conceptualized as a cycle of three stages. Each stage reflects basic neurocircuitry that is involved in aberrant motivation, and each stage is predominantly linked to a functional domain and brain functional networks that interact with each other (**Figure 1**). The binge/intoxication stage, via the neurocircuitry of the basal ganglia, reflects the rewarding, incentive salience, and pathological habit effects of drugs. The withdrawal/negative affect stage, via the extended amygdala and other regions (e.g., lateral habenula), reflects the loss of reward and motivation and the enhanced sensitivity and recruitment of brain stress systems, leading to negative emotional symptoms, such as dysphoria, anhedonia, and irritability (**Figure 1**). The preoccupation/anticipation (''craving'') stage, via neurocircuitry of the prefrontal cortex (PFC), reflects deficits in executive function, including impulsivity and the loss of control over drug taking.

The neurocircuitry and neuropharmacology of the withdrawal/negative affect stage of the addiction cycle is built on the opponent-process, affective dysregulation model of addiction (Koob and Bloom, 1988; Koob and Le Moal, 2005, 2008; Koob and Zorrilla, 2010; Zorrilla et al., 2013, 2014; George et al., 2014; Koob et al., 2014), an extension of opponent-process theory (Solomon and Corbit, 1974; see ''Glossary'' section for definitions of relevant terms in the model). The ''dark side'' affective dysregulation hypothesis posits that drugs of abuse initially activate brain circuits that elicit pleasurable emotional states (''a-process''). To restore emotional homeostasis, however, counterregulatory, opponent processes (''b-process'') follow that decrease mood and increase negative emotional states, such as vigilance and tension.

The affective dysregulation model extends opponentprocess theory by further proposing that hysteresis develops with repeated cycles of intoxication/withdrawal whereby the negative opponent process (b-process) initiates earlier, to a greater degree and more persistently than the rewarding a-process. With repeated exposure to the substance of abuse, the opponent process eventually predominates. A greater quantity and frequency of use of the substance is then needed to restore euthymia. When the substance is not used, negative emotional symptoms of withdrawal develop, such as irritability, anxiety, dysphoria, and subjective feelings of need. This deficit emotional state or hyperkatifeia (Shurman et al., 2010) is putatively dissociable from somatic withdrawal signs and can sensitize and persist with repeated substance use. This model accounts for protracted abstinence symptoms, in which hypohedonia, negative emotional behavior, hyperarousability, and greater behavioral responses to stress may be seen despite months of sustained abstinence. Under this conceptualization of the ''dark side'' of addiction (Koob and Le Moal, 2001, 2005, 2008), substance use is compulsively escalated or renewed (in relapse) via negative reinforcement mechanisms because it transiently prevents or relieves the emotional sequalae of the withdrawal/negative affect stage. Thus, this allostatic model of brain motivational systems views addiction as a cycle of increasing dysregulation of brain reward/anti-reward mechanisms that yield negative emotional states and thereby promote compulsive drug use via a different source of motivation, namely negative reinforcement (Koob and Bloom, 1988).

The counteradaptive b-processes that initially function to restore emotional homeostasis but ultimately lead to allostasis are proposed to reflect two mechanisms: within-system downregulation of brain reward/incentive motivational systems and between-system recruitment/activation of stress/aversion neurocircuitry (Koob and Le Moal, 2008). With regard to within-system neuroadaptations, one prominent hypothesis is that dopamine and opioid peptide reward/incentive motivational systems become compromised, which is evident during both acute withdrawal and protracted abstinence. The argument is that a decrease in dopamine and opioid peptide function leads to lower motivation for non-drug-related stimuli and greater sensitivity to cues that are associated with the abused drug (i.e., increase in incentive salience; Melis et al., 2005). Supporting this hypothesis, animal models of alcohol withdrawal show a decrease in activity of the mesolimbic dopamine system and a decrease in serotonergic neurotransmission in the nucleus accumbens (Koob, 2015). Strong support for a compromised mesolimbic dopamine system is seen in both animal studies and human imaging studies (Ashok et al., 2017). Other proposed within-system neuroadaptations may include changes in gene transcription and receptor transduction mechanisms in the connections of the mesolimbic dopamine system, including the nucleus accumbens (Feng and Nestler, 2013).

With regard to between-system neuroadaptations that underlie the withdrawal/negative affect stage, neurocircuits and neurochemical systems that subserve arousal-stress putatively become engaged in a homeostatic attempt to counter the ongoing presence of the perturbing drug and restore normal emotional function. The extended amygdala (Heimer and Alheid, 1991) may represent a common neuroanatomical substrate that integrates brain arousal-stress and hedonic processing systems to produce the proposed between-system adaptations. The extended amygdala, a distinct entity in the basal forebrain (Alheid and Heimer, 1988), is composed of several basal forebrain structures, including the central nucleus of the amygdala, bed nucleus of the stria terminalis, sublenticular substantia innominata, and a transition zone in the medial nucleus accumbens (e.g., shell; Heimer and Alheid, 1991). The extended amygdala is innervated by many limbic structures, including the basolateral amygdala and hippocampus, and sends major projections to the medial ventral pallidum and lateral hypothalamus. It thus bridges classic limbic structures with the extrapyramidal motor system. Accordingly, the extended amygdala is thought to play important roles in both fear conditioning (Le Doux, 2000) and emotional aspects of pain processing (Neugebauer et al., 2004).

Consistent with the posited between-system engagement of brain arousal-stress systems, chronic exposure to all major drugs with dependence or abuse potential leads to the dysregulation of corticotropin-releasing factor (CRF) brain stress systems, including those in the hypothalamic-pituitary-adrenal axis and extended amygdala. Thus, acute withdrawal from chronic drug exposure yields higher levels of adrenocorticotropic hormone, corticosterone, and extracellular amygdala CRF (Koob, 2008). In addition to CRF, dynorphin, vasopressin, norepinephrine, hypocretin (orexin), and substance P may also contribute to negative emotional states during drug withdrawal. Thus, multiple neuropeptides and neurotransmitters underlie the activation of pro-stress, pro-negative emotional state neurocircuitry and form the neurochemical bases for hedonic opponent processes (Koob, 2015). In opposition, anti-stress circuitry, including neuropeptide Y (NPY), nociceptin, and endocannabinoid systems (Koob, 2015), normally buffer the aforementioned pro-stress actions. Thus, the activation of pro-stress neuromodulators coupled with an inadequate anti-stress response may yield negative emotional states that drive negative reinforcement.

### NEGATIVE URGENCY AND ADDICTION: TOBACCO, ALCOHOL, COCAINE, PATHOLOGICAL GAMBLING, AND FOOD

Negative affect, negative urgency (VanderVeen et al., 2016), and attentional (''decreased ability to focus'') and motor (''acting without thinking'') impulsivity are comorbid in alcohol and drug addiction (Billieux et al., 2007; Stautz and Cooper, 2013). Indeed, negative urgency may be an endophenotype for alcohol and tobacco addiction and other clinical disorders (Cyders and Smith, 2008; VanderVeen et al., 2016). Negative urgency has been particularly associated with negative emotional states (VanderVeen et al., 2016). Negative urgency is viewed as contributing to addictive behavior under this affective dysregulation, negative reinforcement framework because: (1) negative emotional episodes (i.e., irritability, anxiety, and depression) that are associated with the withdrawal/negative affect stage may represent or give rise to intense urges (e.g., to attempt to self-medicate) that are moderated or mediated by negative urgency; and (2) the impairment of executive control that is associated with negative urgency may decrease the capacity to resist urges to pursue substance use in the preoccupation/anticipation stage such that behavior occurs both rapidly without forethought of potential harm (impulsively) and also, with disease progression, despite actual negative or incorrect consequences (compulsively).

#### Tobacco

Consistent with a negative reinforcement model, smoking to relieve negative mood is a common self-reported motivation for smokers (Doran et al., 2009). Expectancies of smoking's effects on emotion are linked to the broad construct of impulsivity (Doran et al., 2007a,b, 2011). Previous work has hypothesized a unique role for the negative urgency aspect of impulsivity specifically in predicting levels of nicotine dependence. Indeed, a meta-analysis that explored the relationships between impulsivity-related traits and both smoking status and nicotine dependence severity found that negative urgency's relationship to the severity of nicotine dependence was the second strongest one identified (Kale et al., 2018). One model, termed the Acquired Preparedness model, of addictive behaviors, posits that individuals are differentially prepared to acquire high-risk expectancies of smoking's effects based on individual differences in personality. These include predispositions to act impulsively when in extreme negative mood states (Negative Urgency) or positive mood states (Positive Urgency; Smith and Anderson, 2001). The model proposes that individuals who act out in response to extreme emotional states are more likely to perceive smoking (or similar behaviors) as reinforcing compared with individuals who do not act out in response to extreme emotional states. They then form expectancies for the positive and negative reinforcement that would result from smoking. For positive reinforcement with positive urgency, individuals may learn that smoking is enjoyable and pleasurable, an expectancy that would lead to smoking when in a good mood and positive reinforcement from smoking. For negative reinforcement with negative urgency, individuals may instead learn that smoking will lessen their negative emotional state. This expectancy may lead them to smoke to alleviate such states and promote negative reinforcement processes.

Consistent with the Acquired Preparedness model, positive affect expectancies for smoking in 139 adult college-aged smokers were found to mediate the relationship between positive urgency and the degree of nicotine dependence symptoms (Spillane et al., 2013). Conversely, the study also found an indirect association between negative urgency and the expectancy that smoking would reduce negative affect whereby affect relief expectancies for smoking mediated the relationship between negative affect and levels of nicotine dependence (Spillane et al., 2013). More specifically, negative urgency explained the variance of negative affect reduction expectancies, which in turn accounted for the variance in smoking (Spillane et al., 2013). In an earlier study (Spillane et al., 2010), negative urgency did not predict the level of dependence. However, both studies included subjects with low-to-moderate nicotine dependence, and the authors hypothesized that negative urgency may manifest in other domains of smoking behavior. For example, high-negative-urgency individuals are predicted to be at greater risk for relapsing in response to the negative emotional states that are associated with quitting.

Consistent with this observation, a study of smokers with high anxiety sensitivity found that these subjects tended to show negative urgency, which in turn was associated with greater expectations that negative reinforcement (i.e., affect relief) would result from smoking or abstaining from smoking (Guillot et al., 2014). Others showed that the expectancy that smoking would reduce negative affect mediated the relationship between negative urgency and the level of nicotine dependence in college-aged smokers (Spillane et al., 2013). The authors argued that treatments that target the fear of anxiety symptoms and the tendency to act impulsively in response to negative affect may be particularly efficacious in promoting smoking cessation in smokers with high anxiety sensitivity (Guillot et al., 2014). Examples include interoceptive exposure, distress tolerance skills training, and mindfulness training.

#### Alcohol

Data also support a role for negative urgency in alcohol use disorders (AUDs). One study examined within-person relationships among specific emotions, alcohol intoxication, and acute dependence symptoms and between-person effects of urgency vs. self-control (i.e., premeditation and perseverance). Consistent with the above-reviewed negative reinforcement model (Koob and Bloom, 1988; Baker et al., 2004; Ahmed and Koob, 2005), the authors found that sadness and anxiety were each directly associated with dependence symptoms and that, in some participants, daytime anxiety was positively associated with subsequent alcohol intoxication (Simons et al., 2010). Relevant to the present discourse, the relationship between anxiety and intoxication was significant only for individuals who were higher in negative urgency or lower in positive urgency (Simons et al., 2010).

A study of 215 undergraduate students who indicated a history of deliberate self-harm tested the hypothesis that negative urgency accounts for the effects of affective lability and self-control on self-harm, problematic alcohol consumption, and eating problems. Accordingly, structural equation modeling found that negative urgency was significantly associated with several self-harming measures, problematic alcohol use, and eating problems (Dir et al., 2013). Negative urgency was also directly associated with affective lability and inversely associated with self-control (Dir et al., 2013). The authors asserted that negative urgency was unique in being the only impulsivity-related trait in their study that increased the risk of self-harm and problematic eating or alcohol use (Dir et al., 2013).

Another study explored the way in which negative urgency was related to emotional experience and alcohol-seeking behaviors in 34 community-dwelling, alcohol-using adults. The volunteers were tested in two counterbalanced sessions of intravenous alcohol self-administration: one with negative mood induction and one without (VanderVeen et al., 2016). Negative urgency predicted greater mood changes in response to negative mood induction and greater alcohol craving before and after an alcohol prime selectively in the negative mood condition (and not in the neutral mood condition). The subjects also had higher blood alcohol levels and more alcohol seeking. Thus, the results suggest that negative urgency could amplify alcohol self-administration via an increase in emotional reactivity to negative events and an increase in alcohol craving in response to an initial alcohol exposure (VanderVeen et al., 2016).

A study of 194 college students investigated whether the relationship between negative urgency and drinking behavior could be explained by positive and negative alcohol outcome expectancies and drinking motive. Path analysis showed indirect relationships between negative urgency and alcohol use via alcohol outcome expectancies and affect enhancement motives (Anthenien et al., 2017). The authors hypothesized that individuals who present high negative urgency may drink alcohol to reverse their emotional distress by enhancing their positive affect (enhancing positive alcohol outcome expectancies; Anthenien et al., 2017).

Negative urgency has also been linked to the higher prevalence of comorbid AUD in adults with attentiondeficit/hyperactivity disorder (ADHD; Daurio et al., 2018). In a study of 794 adult subjects, different components of impulsivity were tested to understand the relationship between ADHD symptoms and the severity of alcohol dependence. Negative and positive urgency mediated the relationship between alcohol dependence severity and overall adult ADHD symptoms, including hyperactivity/restlessness and problems with self-concept (Daurio et al., 2018). The authors suggested that negative and positive urgency account for more of the relationship between alcohol dependence severity and ADHD symptoms than do other common measures of impulsivity, such as sensation seeking, the lack of premeditation, and the lack of perseverance.

A study of 273 young-adult Australians examined the contributions of several different facets of impulsivity to problematic alcohol use. Negative urgency, like many of the other measures of impulsivity, was directly related to greater alcohol intake, binge drinking, and alcohol-related problems. When simultaneously considered as regression predictors, however, only negative urgency and the lack of premeditation emerged as unique predictors of binge drinking behavior. Similarly, only negative urgency and positive urgency were unique predictors of alcohol-related problems. Both effects of negative and positive urgency were similarly present in college-attending vs. non-college-attending participants. The authors concluded that interventions that are tailored to target impulsive responding in response to extreme negative or positive emotional states are needed to mitigate binge drinking and adverse drinking outcomes in young adults (Tran et al., 2018).

A study of 675 community-dwelling adults in the Rockland Project used structural equation modeling path analysis to evaluate the mediating vs. moderating roles of urgency in the relationship between depression and problematic alcohol or cannabis use. Negative urgency, not positive urgency, was a unique mediator of the relationships between depressive symptoms and both problematic alcohol use and problematic cannabis use. Additionally, negative urgency moderated the relationship between depressive symptoms and problematic cannabis use. Specifically, at low levels of negative urgency, depressive symptoms predicted less problematic cannabis use, whereas at high levels of negative urgency, depressive symptoms predicted greater cannabis use. The authors concluded that despite being statistically correlated with each another, negative and positive urgency had distinct influences on the relationship between depressive symptoms and alcohol and cannabis use, with negative urgency having unique predictive significance (Um et al., 2019a).

A recent study sought to identify core functional domains that are associated with AUD in a large, diverse sample of 454 volunteers. Factor analysis identified three intercorrelated functional dimensions—impaired executive control, negative emotionality, and dysphoria-associated incentive salience—each of which discriminated participants with vs. without AUD. Negative urgency was 50% higher in participants with AUD than those without AUD and loaded strongly on the impaired inhibitory control factor and, less so, on the negative emotionality factor (Kwako et al., 2019).

#### Cocaine

For studies of cocaine addiction, evidence indicates an interaction between negative urgency and cognitive performance. In a study that compared cocaine-dependent individuals and pathological gamblers with regard to cognitive performance and impulsivity, cocaine-dependent individuals had higher scores on UPPS-P Negative Urgency and poorer working memory performance (Albein-Urios et al., 2013). Both groups presented an increase in positive urgency and impairments in Stroop inhibition vs. healthy controls. The peak amount of cocaine use was inversely correlated with working memory and response inhibition performance (Albein-Urios et al., 2013). The authors argued that cocainespecific elevations of negative urgency and deficits in working memory function may reflect ''cocaine neurotoxicity'' (Albein-Urios et al., 2013). This putative link to neurotoxicity is consistent with studies in both humans and animal models that showed cocaine-induced deficits in working memory and PFC dysfunction (George et al., 2007; Tomasi et al., 2007), the latter of which may impair the control over impulsive or compulsive substance use.

Cocaine-dependent patients with personality disorders had greater negative urgency, an increase in borderline beliefs, a decrease in inhibition and attention regulation, and a decrease in temporal pole gray matter volume compared with cocainedependent patients without personality disorders (Albein-Urios et al., 2013). The authors argued that patients with comorbid personality disorders present an increase in negative urgency and impairments in executive control, in addition to executive function deficits and impulsive traits that are seen in cocaine dependence (Albein-Urios et al., 2013). In a study that employed trait impulsivity measures and decision-making tasks in cocainedependent subjects and healthy controls, negative urgency distinguished individuals with cocaine dependence from healthy controls (Torres et al., 2013).

### Pathological Gambling

Data also support a role for negative urgency in pathological gambling. In a case-control study of 30 pathological gamblers and their respective controls, negative and positive urgency showed the greatest effect sizes of all impulsivity dimensions that discriminated groups (Michalczuk et al., 2011). Many other studies have observed increases in negative urgency in subjects with pathological gambling (Torres et al., 2013; Yan et al., 2016; Mick et al., 2017; Shakeel et al., 2019). A meta-analysis review that involved a total of 2,134 gamblers and 5,321 controls found that negative urgency had the greatest effect size of all impulsivity constructs (Maclaren et al., 2011). In a study of 1,002 Canadian adults, negative urgency, unique among impulsivity constructs, prospectively predicted subsequent problem gambling in both men and women (Farstad et al., 2015). Of the impulsivity constructs, negative urgency also uniquely predicted at-risk gambling (Yan et al., 2016), suggesting that it may be an early risk factor in the progression to addictive behavior. Negative urgency especially increased in a subgroup of pathological gamblers who presented high impairments in executive function and greater overall psychopathology (Mallorquí-Bagué et al., 2018b).

Particularly striking in Torres et al. (2013), negative urgency covaried uniquely with gambling overpathologization, supporting the hypothesis that negative urgency is a sign of overpathologization in addictive processes. Consistent with this hypothesis and supporting a trans-diagnostic role for negative urgency, studies have found that negative urgency is associated with not only the severity of gambling disorder (Savvidou et al., 2017; Steward et al., 2017) but also with cocaine dependence, tobacco addiction, and alcohol addiction (Albein-Urios et al., 2013; Spillane et al., 2013; VanderVeen et al., 2016; Daurio et al., 2018). Thus, negative urgency may be involved in the overpathologization of addiction in general, suggesting a common neurocircuitry basis. Also consistent with this trans-diagnostic perspective, negative urgency was higher in gamblers who smoked regularly vs. those who did not (Boothby et al., 2017). Negative urgency also longitudinally predicted worse treatment response in gamblers who received outpatient cognitive-behavior therapy, predicting low treatment compliance and greater relapse during treatment (Mallorquí-Bagué et al., 2018a; Mestre-Bach et al., 2019).

#### Food

Negative urgency, in addition to other facets of impulsivity, is also high in disorders that involve the compulsive intake of palatable food, including food addiction as defined by the Yale Food Addiction Scale, and disorders with binge eating, including bulimia nervosa (Murphy et al., 2014; Ceccarini et al., 2015; Meule et al., 2015, 2017a,b; Pivarunas and Conner, 2015; de Vries and Meule, 2016; VanderBroek-Stice et al., 2017; Rose et al., 2018). Negative urgency is related more strongly to binge eating in patients with bulimia nervosa and adolescents with uncontrolled eating than to other aspects of impulsivity, such as sensation seeking, lack of planning, or lack of persistence (Fischer et al., 2008; Pearson et al., 2014; Booth et al., 2018). Furthermore, negative urgency was shown to predict binge eating symptoms prospectively from elementary school to middle school and in college students (Pearson et al., 2015) and adults (Farstad et al., 2015). Negative urgency was also associated with more frequent snacking in adolescents (Smith and Cyders, 2016; Coumans et al., 2018). A path analysis study of 315 patients with eating disorders on the binge spectrum found that low self-directedness and emotional regulation were broadly associated with eating psychopathology, whereas negative urgency was uniquely associated with food addiction (Wolz et al., 2017). From a treatment standpoint, a multicomponent behavior therapy intervention in obese adolescents reduced body mass index relative to the degree to which it reduced negative urgency (Delgado-Rico et al., 2012). From a comorbidity perspective, negative urgency also predicted alcohol use problems in women with disordered eating (Fischer et al., 2007).

### Summary and Future Directions

In summary, clinical and community-based findings support the hypothesis that negative urgency plays a transdiagnostic role in addiction. It is increased across alcohol, tobacco, cocaine, gambling, and food addictive disorders, especially with regard to problematic and hazardous use. In many studies, negative urgency was associated with measures of addiction severity and comorbid addiction or other psychopathology. Several findings indicate a greater or unique role for negative urgency compared with other recognized constructs of impulsivity, including positive urgency. Negative urgency was associated with negative emotions, impairments in executive control, and substance use in many studies, but more work is needed to clarify its mediating vs. moderating role in the relationships among these constructs and the degree to which it operates as an antecedent risk factor vs. emerging consequence of addiction. Initial work suggests that negative urgency may be an early risk factor, but it may also be a useful target for treatment or predictor or indicator of treatment response, areas that warrant further study. The etiology of antecedent individual differences in negative urgency also remains to be determined. Studies have shown that negative urgency is a relevant construct in both men and women (Smith and Cyders, 2016). No consistent gender differences have been reported at the level of negative urgency (Smith and Cyders, 2016), but negative urgency may still play differential roles between genders for specific behaviors (Davis-Becker et al., 2014). Moreover, levels of negative urgency differ according to pubertal status (i.e., increasing with puberty; Gunn and Smith, 2010; Pearson et al., 2010). Thus, the role of negative urgency in known gender and developmental differences in addiction vulnerability remain to be more fully explored. Finally, new work that uses ecological momentary assessment or similar approaches is needed to test the model that is proposed herein that impulsive and eventually compulsive substance seeking and use more often follow abstinence- or environment-associated distress in people who are high in negative urgency.

### NEUROCIRCUITRY IMPLICATED IN NEGATIVE URGENCY AND ADDICTION

Neurobiological data on negative urgency are limited to date, but negative urgency has been hypothesized to reflect impairments in the ''top-down'' cortical control over both basal ganglia and extended amygdala function (**Figure 2**). This topic was very recently reviewed in detail (Um et al., 2019b), so we only briefly discuss key findings here. Most, if not all, of the data are derived from human imaging studies. Deficient top-down control has been hypothesized to reflect a loss of control over pathological habits that involve basal ganglia and extended amygdala processing (Robbins and Everitt, 1999; Everitt and Robbins, 2005; George et al., 2007; Belin et al., 2013) and greater attention to, incentive

FIGURE 2 | Negative urgency circuitry in the neurobiology of addiction. Simplified inter-relationships are shown between higher-order cortical regions [green shading: orbitofrontal cortex (OFC) and compartments of the prefrontal cortex (PFC), including the anterior cingulate cortex (ACC), dorsolateral PFC (dPFC), ventrolateral PFC (vPFC) and ventromedial PFC (vmPFC)] that regulate activity of the extended amygdala (red shading: central nucleus of the amygdala, bed nucleus of the stria terminalis, portion of nucleus accumbens shell) and basal ganglia (blue shading: including nucleus accumbens core, dorsal striatum, globus pallidus). Negative urgency is hypothesized to reflect a vulnerability to extreme negative affect that impairs the efficacy of higher-order inhibitory control from such regions as the OFC and ventromedial PFC over drug-taking and drug-seeking behaviors that are subserved by the extended amygdala and basal ganglia. Negative urgency is also hypothesized to reflect alterations of cortico-amygdalar and cortico-striatal modulation by the insular cortex (representing interoceptive state and context) and other prefrontal cortical regions, including the prelimbic cortex and ACC. Note that regions are illustrated heuristically and are not intended to be neuroanatomically precise. Reprinted from Koob et al. (2013), with permission from Elsevier.

salience of, or cognitive resource interference from emotionevoking stimuli. Consequently, in the presence of negative emotion, there is a reduction of inhibitory control over potentially detrimental actions and habits, the latter reflecting increased behavioral control by the dorsolateral striatum (Everitt and Robbins, 2005; Belin and Everitt, 2008; Belin et al., 2013; Giuliano et al., 2019). These biases putatively reflect alterations of the structure, function, or connectivity of orbitofrontal cortex (OFC)/ventromedial prefrontal cortex (vmPFC) projections to the basal ganglia and extended amygdala (Cyders and Smith, 2008; Robbins et al., 2012; Smith and Cyders, 2016; **Figure 2**).

Supporting this hypothesis, trait urgency was related to the amplitude of resting-state low-frequency fluctuations in the lateral OFC and vmPFC in healthy volunteers (Zhao et al., 2017). Trait negative urgency also predicted an increase in activation of the vmPFC in response to an alcohol odor cue in social drinkers and mediated the association between activation of the vmPFC and alcohol craving and problematic drinking (Cyders et al., 2014a). Similarly, negative urgency predicted greater OFC and amygdala activation in response to negative visual stimuli and mediated the relationship between activation and risky behavior (Cyders et al., 2015). Additionally, negative urgency was also associated with resting-state and inhibitory task-related activation (Go/No-Go or gambling tasks) in other structures that subserve self-regulation and decision making under risk. These included the dorsolateral and ventrolateral PFC, anterior insula, and anterior and posterior cingulate (Clark et al., 2008; Xue et al., 2010; Hoptman et al., 2014; Chester et al., 2016; Zhao et al., 2017). Greater insula activation prospectively predicted real-world substance use in subjects who were high in negative urgency (Chester et al., 2016). Negative urgency also predicted greater increases in medial PFC (mPFC) activity during anticipation in a delayed incentive task (Weiland et al., 2014). Finally, in patients with schizophrenia, urgency was associated with a reduction of cortical thickness in such structures as the vmPFC, orbitofrontal and inferior frontal gyri, and rostral anterior cingulate cortex (Hoptman et al., 2014). As discussed above, cocaine-dependent subjects with comorbid personality disorders had more negative urgency, more intense borderline beliefs, inferior response inhibition and attention regulation, and less gray matter in the right temporal pole compared with cocaine-dependent individuals without these comorbidities (Albein-Urios et al., 2013). Others have shown a wide distribution of lower gray matter volume in the PFC, insula, and amygdala-temporal lobe in cocaine-dependent subjects compared with healthy controls, with a correlation between greater trait impulsivity (i.e., a lack of premeditation and negative urgency) and lower gray matter volume in the left inferior/middle frontal gyrus in cocaine-dependent individuals (Moreno-López et al., 2012).

Neurochemically, negative urgency may reflect deficient 5-hydroxytryptamine (5-HT) and lower dopamine activity in the OFC and vmPFC (Floresco and Tse, 2007; Cyders and Smith, 2008), leading to less inhibition of both basal gangliaand extended amygdala-subserved impulses. Accordingly, a composite polygenic 5-HT score predicted alcohol problems via an increase in negative urgency and not via other measures of impulsivity (Carver et al., 2011; Wang and Chassin, 2018). Evidence of lower 5-HT activity and responsiveness is likewise seen in bulimia nervosa with regard to the presence and severity of binge symptoms (Cyders and Smith, 2008). Genetic variation of the GABRA2 subunit, which encodes the γ-aminobutyric acid (GABA) receptor α2 subunit and is related to alcohol use problems via urgency, has also been linked to alterations of insula activation responses (Villafuerte et al., 2012) and dorsolateral PFC GABA concentrations (Boy et al., 2011).

Traditionally, negative urgency has been conceptualized as a stable dispositional antecedent that potentiates responses to extreme situational distress (e.g., Engel et al., 2007; Fischer et al., 2018). Consistent with this view, negative urgency predicted greater subsequent mood changes, alcohol cue-induced craving, and intravenous alcohol self-administration after negative mood induction (VanderVeen et al., 2016) and an increase in negative affect and relative reinforcing value after laboratory stressors (Owens et al., 2018). From the ''dark side'' perspective, negative urgency is viewed as a trait that increases the likelihood of relapsing during opponent-process- or life stress-associated distress (Ahmed and Koob, 2005).

Negative urgency, however, is not strictly trait-like. Situational factors can impact inhibitory signals from the OFC/vmPFC to the amygdala (Silbersweig et al., 2007). Relevant to drug addiction, both cues (positive and negative), the onset of negative emotional states, and any combination thereof can trigger the reinstatement of drug seeking. Accordingly, an increase in substance cue reactivity is seen in the hypothesized negative urgency network (i.e., basal ganglia, amygdala, OFC, cingulate cortex, vmPFC, dorsolateral PFC, and anterior insula) in people with substance use disorders and predicts relapse (Heinz et al., 2009; Goudriaan et al., 2010; Engelmann et al., 2012; Mainz et al., 2012; Jasinska et al., 2014).

Moreover, environmental history can elicit enduring changes in urgency. For example, childhood abuse persistently increased amygdala activation and reduced prefrontal cortical control over amygdalar responses (Teicher et al., 2003) and is well documented to predispose individuals to drug addiction (Choi et al., 2017). Furthermore, effective psychological interventions for obesity and gambling disorder have been reported to reduce negative urgency (Delgado-Rico et al., 2012; Garcia-Caballero et al., 2018).

### NEGATIVE URGENCY AND THE DARK SIDE OF ADDICTION

A novel hypothesis that is proposed herein is that opponentprocess adaptations to alcohol, tobacco, and other substances of abuse may also increase both negative affect and negative urgency. Decreases in striatal dopamine D<sup>2</sup> receptor availability in alcohol-addicted subjects were correlated with lower glucose metabolism in frontal cortical regions that underlie inhibitory control, such as the dorsolateral and anterior cingulate cortices (Volkow et al., 2007). Importantly, these relationships are not seen in non-AUD controls, consistent with the hypothesis that they reflect circuitry changes. Perhaps accordingly, negative urgency also predicted greater caudate responses to alcohol-related images in alcohol-dependent individuals (Chester et al., 2016). Similarly, in pathological gamblers, higher negative urgency was correlated with lower striatal D<sup>2</sup> receptor availability, indexed by [11C]-raclopride binding potential (Clark et al., 2012).

As noted above, the increase in substance cue reactivity that is seen in prefrontal-basal ganglia and prefrontal-amygdala circuits in subjects with substance use disorders predicts relapse and may reflect adaptations within negative urgency circuits. These circuits may involve not only reward processing, as is often interpreted (e.g., Stice et al., 2015; Schulte et al., 2016; Winter et al., 2017), but also stress processing (Koob and Schulkin, 2018). This perspective is reinforced by the finding that central nucleus of the amygdala activation maintains habitual, drug-seeking behavior via dorsolateral striatum activation (Murray et al., 2015). From a ''dark side'' perspective, negative urgency forms another pathway for impulsivity deficits to continue throughout the addiction cycle, not being simply limited to positive urgency, reward, and basal ganglia function.

### ANIMAL MODEL OF NEGATIVE URGENCY?

In an intriguing attempt to bridge human self-reports and behavioral measures of negative urgency to animal models, some groups have attempted to back-translate negative urgency concepts to rats. One group showed that human subjects who scored high in negative urgency presented more behavioral responding and greater frustration following unexpected reward omission in a monetary-based task vs. subjects who scored lower in negative urgency (Gipson et al., 2012). Similarly, they found that rats exhibited an increase in operant responses for intravenous amphetamine or sucrose pellets after unexpected reward omission. The results suggested that impulsive behavior that is engendered by the unexpected omission of reward may represent a valid behavioral model of negative urgency that can be linked to substance abuse (Gipson et al., 2012). In a follow-up study that used the same reward omission task to determine the neurochemical bases of the reward omission effect in this model of negative urgency, contingent responding was higher following the omission of an expected reward than following the delivery of an expected reward (Yates et al., 2015). Dopamine and 5-HT uptake were measured in individual rats using synaptosomes that were prepared from the nucleus accumbens, dorsal striatum, mPFC, and OFC. The Vmax values for the dopamine transporter in the nucleus accumbens and serotonin transporter in the OFC were positively correlated with negative urgency scores, suggesting that mood-based impulsivity (i.e., negative urgency) is associated with greater dopamine transporter function in the nucleus accumbens and serotonin transporter function in the OFC (Yates et al., 2015). Similarly, Cifani et al. (2009) performed a series of studies and found that the frustrative nonreward of being placed in a context where a previously available preferred food could be seen and smelled but no longer eaten elicited behavioral and neuroendocrine signs of stress and subsequently greater palatable food self-administration and binge eating (Cifani et al., 2009; Piccoli et al., 2012) via mechanisms that involved extended amygdala CRF activation (Micioni Di Bonaventura et al., 2014; Pucci et al., 2016). These results highlight the power of the ecologically valid challenges of reward omission and frustrative nonreward to drive an increase in use. The models also are consistent with the increasing trend for preclinical models to recapitulate diagnostic and translationally-meaningful symptoms of substance use disorders (Belin-Rauscent et al., 2016). To develop the models further within a negative urgency framework, remaining to be shown is that the use is ''driven forward'' in an impulsive manner, meaning: (i) rapidly/suddenly; (ii) with prioritization of immediate vs. later outcomes (e.g., in a delayed-discounting task framework; Herman et al., 2018); and (iii) in a risky fashion, without apparent behavioral consideration given to possible negative outcomes. Ultimately, the latter aspect may progress to compulsive behavior, in which use commences not only hastily and without forethought of possible negative outcomes but even despite recognized and experienced negative outcomes (e.g., if drug-reinforced responses are instead punished or presented in other similar approach-avoidance conflict frameworks; Pelloux et al., 2007; Jonkman et al., 2012).

Economidou et al. (2009) have previously shown that antecedent high impulsivity, defined by a 5-choice serial reaction time task, predicted cocaine relapse after punished responding. Similarly, Belin et al. (2008), Ansquer et al. (2014), and Murray et al. (2014) showed that high impulsivity in the 5-choice serial reaction time task prospectively predicted the development of compulsive cocaine self-administration and compulsive adjunctive behavior, in a manner mitigated by the norepinephrine transporter inhibitor atomoxetine. Here, we propose that a negative mood-driven measure of urgent impulsivity (e.g., greater reward omission-induced shifts toward immediate vs. later rewards or the greater reward omission-enhancement of substance seeking/use even in an exposed or otherwise risky context) may similarly predict the development of compulsive drug use or relapse.

## CONCLUSIONS

The transition from drug use to addiction is accompanied by the downregulation of brain reward circuitry and the enhancement of ''antireward''/stress circuitry that involves an opponentprocess ''dark side'' of emotional dysregulation. Negative urgency may reflect impairments in ''top-down'' cortical-tobasal ganglia and cortical-to-extended amygdala processing, leading to a reduction of inhibitory control over potentially detrimental actions. Such impairments may yield heightened ''bottom-up'' basal ganglia and extended amygdala processing, leading to greater attention to incentive salience, pathological habits, or cognitive resource interference from emotion-evoking stimuli. Limited data to date, almost exclusively in humans, reflect alterations of the structure, function, or connectivity of orbitofrontal/ventromedial prefrontal cortical–basal ganglia and extended amygdala projections and support the hypothesis that the resulting negative urgency facilitates the transition to compulsive drug seeking and may even help maintain compulsive drug seeking. The further development of animal models and human experiments to better study the most critical aspects of negative urgency and identify molecular loading on specific neurochemical circuits that convey vulnerability and resilience to compulsive drug seeking via negative urgency are charges for the future.

### GLOSSARY

#### a-process

The a-process represents the initial positive hedonic or mood impact of a stimulus. In summation with the subsequent, opponent, counter-regulatory b-process, it yields a net affective stimulus (state). An individual who still experiences a positive hedonic mood state from a drug of abuse is hypothesized to retain a predominant a-process and experience positive reinforcement when using a substance of abuse (Koob and Le Moal, 2001).

#### Allostasis

The process of achieving stability through change (Koob and Le Moal, 2001).

#### Allostatic Load

The cost to the brain and body of the deviation, accumulating over time, and reflecting in many cases pathological states and accumulation of damage (Koob and Le Moal, 2001).

#### Allostatic State

A state of chronic deviation of the regulatory system from its normal (homeostatic) operating level (Koob and Le Moal, 2001).

### Antireward

A concept based on the hypothesis that there are brain systems whose function is to limit reward when triggered by excessive activity in the reward system. Both within-system and betweensystem neuroadaptations may underlie antireward adaptations in addiction.

#### Between-System Neuroadaptation

A between-system neuroadaptation is defined as a circuitry change, in which one circuit (i.e., stress or anti-reward circuits) becomes activated by another circuit (i.e., the reward circuit; Koob, 2004).

#### b-process

The b-process represents the counter-regulatory, opponentprocess response to the initial activating a-process. In summation with the prior, initial a-process, it yields a net affective stimulus (state). A homeostatic b-process that simply balances the activational process (a-process) would restore the initial emotional equilibrium set point. In the affective dysregulation, allostasis model, however, the b-process does not balance the activational process but rather shows residual hysteresis with repeated engagement. This creates a progressively greater allostatic state and a persistent, net negative affective stimulus when drug use stops, experienced as withdrawal and, later, protracted abstinence. An individual in this allostatic state is hypothesized to experience negative reinforcement, with partial relief from the negative state when using a substance of abuse (Koob and Le Moal, 2001).

#### Compulsivity

A behavioral predisposition to experience and act upon irresistible urges. Compulsive behaviors are often experienced as outside of one's control, intrusive, and unwanted. They can be operationally defined as perseverative responding in the face of adverse consequences or in the face of incorrect outcomes (Koob, 2014).

#### Impulsivity

A behavioral predisposition toward rapid, unplanned reactions to stimuli without regard to the possible negative consequences of these reactions to the self or others. Operationally, impulsivity is often measured as a bias toward immediate smaller vs. delayed larger rewards and the inability to inhibit or alter a course of action (Koob, 2014).

#### Incentive Salience

Through the process of conditioning, previously neutral stimuli become linked to natural or drug reinforcers and acquire the ability to engender or increase the motivation to seek the reinforcer (Koob and Le Moal, 2006).

#### Negative Urgency

The behavioral predisposition to act rashly and impulsively when in extreme distress (Cyders and Smith, 2008).

#### Positive Urgency

The behavioral predisposition to act rashly and impulsively when in an extremely positive mood state, such as euphoria (Cyders and Smith, 2008).

#### Within-System Neuroadaptation

Within-system neuroadaptations are the process by which the primary cellular response element to the drug adapts to neutralize the drug's effects. Thus, within-system neuroadaptations occur within the neurocircuits and neurochemical systems that are initially engaged by the rewarding substance to elicit its rewarding effects, or a-process

### REFERENCES


(Koob, 2004). Persistence of these opposing effects after the drug disappears produces adaptation.

## AUTHOR CONTRIBUTIONS

EZ and GK contributed to writing the manuscript equally.

### FUNDING

Financial support for this work was provided by the Pearson Center for Alcoholism and Addiction Research and National Institute on Alcohol Abuse and Alcoholism grant AA006420. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

### ACKNOWLEDGMENTS

We thank Dr Mark Gold and two anonymous reviewers for helpful comments and Michael Arends for assistance with manuscript preparation.


among men and women aged 50+ years. Int. Psychogeriatr. 29, 359–372. doi: 10.1017/s1041610216001800


predict fifth grade alcohol consumption. Psychol. Addict. Behav. 24, 617–627. doi: 10.1037/a0020334


induced binge-like palatable food consumption in female rats with a history of food restriction. J. Neurosci. 34, 11316–11324. doi: 10.1523/JNEUROSCI.1854- 14.2014


UPPS impulsive behavior scale. Exp. Clin. Psychopharmacol. 11, 210–217. doi: 10.1037/1064-1297.11.3.210


**Conflict of Interest Statement**: EZ and GK are inventors on a patent filed for CRF1 receptor antagonists (USPTO application no. 2010/0249138).

Copyright © 2019 Zorrilla and Koob. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Corrigendum: Impulsivity Derived From the Dark Side: Neurocircuits That Contribute to Negative Urgency

Eric P. Zorrilla<sup>1</sup> \* and George F. Koob<sup>2</sup>

<sup>1</sup> Department of Neuroscience, The Scripps Research Institute, La Jolla, CA, United States, <sup>2</sup> Neurobiology of Addiction Section, Intramural Research Program, National Institute on Drug Abuse, Baltimore, MD, United States

Keywords: negative urgency, impulsivity, compulsive drug use, negative affect, withdrawal, substance or alcohol use disorder, orbitofrontal cortex, extended amygdale

#### **A Corrigendum on**

**Impulsivity Derived From the Dark Side: Neurocircuits That Contribute to Negative Urgency** by Zorrilla, E. P., and Koob, G. F. (2019). Front. Behav. Neurosci. 13:136. doi: 10.3389/fnbeh.2019.00136

In the original article there was an error. The reference for Um et al. (2019) was incorrectly cited.

A correction has been made to the section **Negative Urgency and Addiction: Tobacco, Alcohol, Cocaine, Pathological Gambling**, **and Food** subsection **Alcohol,** paragraph seven:

"A study of 675 community-dwelling adults in the Rockland Project used structural equation modeling path analysis to evaluate the mediating vs. moderating roles of urgency in the relationship between depression and problematic alcohol or cannabis use. Negative urgency, not positive urgency, was a unique mediator of the relationships between depressive symptoms and both problematic alcohol use and problematic cannabis use. Additionally, negative urgency moderated the relationship between depressive symptoms and problematic cannabis use. Specifically, at low levels of negative urgency, depressive symptoms predicted less problematic cannabis use, whereas at high levels of negative urgency, depressive symptoms predicted greater cannabis use. The authors concluded that despite being statistically correlated with each another, negative and positive urgency had distinct influences on the relationship between depressive symptoms and alcohol and cannabis use, with negative urgency having unique predictive significance (Um et al., 2019a)."

Additionally, a correction has been made to the section **Neurocircuitry Implicated in Negative Urgency and Addiction**, paragraph one:

"Neurobiological data on negative urgency are limited to date, but negative urgency has been hypothesized to reflect impairments in the "top-down" cortical control over both basal ganglia and extended amygdala function (Figure 2). This topic was very recently reviewed in detail (Um et al., 2019b) so we only briefly discuss key findings here. Most, if not all, of the data are derived from human imaging studies. Deficient top-down control has been hypothesized to reflect a loss of control over pathological habits that involve basal ganglia and extended amygdala processing (Robbins and Everitt, 1999; Everitt and Robbins, 2005; George et al., 2007; Belin et al., 2013) and greater attention to, incentive salience of, or cognitive resource interference from emotion-evoking stimuli. Consequently, in the presence of negative emotion, there is a reduction of inhibitory control over potentially detrimental actions and habits, the latter reflecting increased behavioral control by the dorsolateral striatum (Everitt and Robbins, 2005; Belin and Everitt, 2008; Belin et al., 2013; Giuliano et al., 2019). These biases putatively reflect alterations of the structure, function, or connectivity of orbitofrontal cortex (OFC)/ventromedial

Edited and reviewed by: Liana Fattore, Italian National Research Council (CNR), Italy

> \*Correspondence: Eric P. Zorrilla ezorrilla@scripps.edu

#### Specialty section:

This article was submitted to Motivation and Reward, a section of the journal Frontiers in Behavioral Neuroscience

> Received: 09 July 2019 Accepted: 06 August 2019 Published: 27 August 2019

#### Citation:

Zorrilla EP and Koob GF (2019) Corrigendum: Impulsivity Derived From the Dark Side: Neurocircuits That Contribute to Negative Urgency. Front. Behav. Neurosci. 13:188. doi: 10.3389/fnbeh.2019.00188 prefrontal cortex (vmPFC) projections to the basal ganglia and extended amygdala (Cyders and Smith, 2008; Robbins et al., 2012; Smith and Cyders, 2016; Figure 2)."

## REFERENCES


The authors apologize for this error and state that this does not change the scientific conclusions of the article in any way. The original article has been updated.


Copyright © 2019 Zorrilla and Koob. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Systematic Review on the Influences of Neurotoxicological Xenobiotic Compounds on Inhibitory Control

Cristian Perez-Fernandez, Pilar Flores and Fernando Sánchez-Santed\*

Department of Psychology and Health Research Center, University of Almería, Almería, Spain

Background: Impulsive and compulsive traits represent a variety of maladaptive behaviors defined by the difficulties to stop an improper response and the control of a repeated behavioral pattern without sensitivity to changing contingencies, respectively. Otherwise, human beings are continuously exposed to plenty neurotoxicological agents which have been systematically linked to attentional, learning, and memory dysfunctions, both preclinical and clinical studies. Interestingly, the link between both impulsive and compulsive behaviors and the exposure to the most important xenobiotic compounds have been extensively developed; although the information has been rarely summarized. For this, the present systematic review schedule and analyze in depth the most important works relating different subtypes of the above-mentioned behaviors with 4 of the most important xenobiotic compounds: Lead (Pb), Methylmercury (MeHg), Polychlorinated biphenyls (PCB), and Organophosphates (OP) in both preclinical and clinical models.

#### Edited by:

Trevor W. Robbins, University of Cambridge, United Kingdom

#### Reviewed by:

Jessica Lang, RWTH Aachen Universität, Germany Paul A. Eubig, University of Georgia, United States

> \*Correspondence: Fernando Sánchez-Santed fsanchez@ual.es

Received: 31 January 2019 Accepted: 06 June 2019 Published: 04 July 2019

#### Citation:

Perez-Fernandez C, Flores P and Sánchez-Santed F (2019) A Systematic Review on the Influences of Neurotoxicological Xenobiotic Compounds on Inhibitory Control. Front. Behav. Neurosci. 13:139. doi: 10.3389/fnbeh.2019.00139 Methods: Systematic search strategy on PubMed databases was developed, and the most important information was structured both in text and in separate tables based on rigorous methodological quality assessment.

Results: For Lead, Methylmercury, Polychlorinated biphenyls and organophosphates, a total of 44 (31 preclinical), 34 (21), 38 (23), and 30 (17) studies were accepted for systematic synthesis, respectively. All the compounds showed an important empirical support on their role in the modulation of impulsive and, in lesser degree, compulsive traits, stronger and more solid in animal models with inconclusive results in humans in some cases (i.e., MeHg). However, preclinical and clinical studies have systematically focused on different subtypes of the above-mentioned behaviors, as well as impulsive choice or habit conformations have been rarely studied.

Discussion: The strong empirical support in preclinical studies contrasts with the lack of connection between preclinical and clinical models, as well as the different methodologies used. Further research should be focused on dissipate these differences as well as deeply study impulsive choice, decision making, risk taking, and cognitive flexibility, both in experimental animals and humans.

Keywords: impulsivity, compulsivity, inhibitory control, lead, methylmercury, organophosphates, polychlorinated biphenyls

## INTRODUCTION

#### Rationale

Impulsivity is the inability to control a poorly predefined, risky and (often) inappropriate behavior in a specific context. At the same time, compulsivity is the persistence in an aimless, excessive and rigid action which could be considered maladaptive in changing contingencies (Moreno and Flores, 2012; Robbins et al., 2012; Fineberg et al., 2014). Both impulsivity and compulsivity are multidimensional constructs composed of a few distinguishable sub-domains within each endophenotype (Dalley et al., 2011; Robbins et al., 2012; Bevilacqua and Goldman, 2013; Fineberg et al., 2014; Dalley and Robbins, 2017).

Impulsive-related behaviors can be more accurately categorized based on functional cognitive tests on motor impulsivity (impulsive action), impulsive decision-making, impulsive choice and poor reflection rate. Compulsive-related sub-domains can be categorized into flexibility to contingency, attentional shifting, attentional bias/disengagement, and habit formation (Fineberg et al., 2014). In this way -albeit part of the same behavioral dimension- every single sub-domain can be individually studied and is affected in varying degrees in every single related disorder. Empirical data therefore supports both their inter-dependence and their independence (Fineberg et al., 2014).

The main neurocognitive tasks designed for impulsive choice assessment in preclinical research are the delayed discounting task, probabilistic, and temporal discounting tasks. Impulsive action is often evaluated with the 5-choice serial reaction time task (5C-SRTT, premature responding), differential reinforcement to low/high rates (DRL/H, prepotent responding), go no go task (GNGT, prepotent responding), stop signal task (SST, inability to stop initiated response), the simple reaction time task (SRTT) as well as different non-standardized operant schedules with fixed intervals (D'Amour-Horvat and Leyton, 2014).

Compulsivity is also assessed with 5C-SRTT (perseveration), scheduled-induced polydipsia (SIP, adjacent repetitive behavior), delayed alternation task (DAT, along with working memory component), marble burying task (MBT, along with anxiety), the trail making test (TMT, along with attention) as well as operant schedules by including both reversal and extinction phases (inflexibility to shifting contingencies) (Moritz et al., 2009; Thomas et al., 2009; Izquierdo and Jentsch, 2012; Angoa-Pérez et al., 2013; Navarro et al., 2017).

In clinical research, several both questionnaire-based and neurocognitive tasks are commonly used in order to assess these traits. Attending to impulsive choice and action analyzed with neurobehavioral tools, all the paradigms indicated for preclinical models are also used in humans, with the continuous performance test (CPT) as an alternative to the 5C-SRTT. Related functions such as risk taking are usually assessed with Iowa Gambling Task (IGT) (D'Amour-Horvat and Leyton, 2014).

Otherwise, compulsive perseveration and inflexibility is often assessed with shifting contingencies (reversal and extinction) with Wisconsin sorting card task (WCST) or DAT. The most important clinical scales and questionnaires for different impulsive and compulsive traits go from specific scales such as the Barratt impulsivity scale (BIS) or brief symptoms inventory (BSI) to general batteries such as the NEPSY or parent/teacherreferred indexes such as the Conners' rating scale or the Swanson, Nolan and Pelham scale (SNAP-IV) (Ahmad and Warriner, 2001; Reise et al., 2013; Crameri et al., 2016).

Different pathologies (psychiatric, neuropsychological, neurodevelopmental and neurodegenerative) have impulsive and/or compulsive traits as part of their central features, and they comprise the category of impulsivity/compulsivity spectrum disorders (Skodol and Oldham, 1996). Some of the most important related pathologies include obsessive-compulsive disorder (OCD), personality disorders, frontal lesions following vascular disease or traumatic brain injury, autism spectrum disorder (ASD), attention deficit/hyperactivity disorder (ADHD), idiopathic Parkinson's disorder, and Alzheimer's disease [Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR); American Psychiatric Association, 2000].

Altered top-down regulation following mismatched fronto-striatal pathways, dysregulation of the basal ganglia and limbic regions, as well as the bottom-up influences of the monoaminergic system have been proposed as the physiological substrate of both impulsive and compulsive behaviors (Dalley et al., 2011; Fineberg et al., 2014; Dalley and Robbins, 2017).

On the one hand, the impulsive-related circuit has been linked to the anterior and inferior cingulate, ventromedial/lateral prefrontal cortex, medial/lateral orbitofrontal cortex, premotor cortex, hippocampus, infra/prelimbic cortices, ventral/dorsal striatum, and certain sub-thalamic areas (Dalley et al., 2011; Fineberg et al., 2014, 2018; Dalley and Robbins, 2017). Noradrenergic, serotonergic and dopaminergic systems are the most widely associated with impulsivity in an area/sub-domaindependent manner, although gamma aminobutyric acid (GABA), glutamatergic and even endocannabinoid systems also play a significant role (Moreno et al., 2010, 2012; Dalley et al., 2011; D'Amour-Horvat and Leyton, 2014; Fineberg et al., 2014; Dalley and Robbins, 2017; Isherwood et al., 2017; Merchán et al., 2017; Dellu-Hagedorn et al., 2018).

On the other hand, compulsivity has been related to the dysregulation of both direct (ventral) and indirect (dorsal) cortico-striato-thalamo-cortical pathways (van den Heuvel et al., 2016; Fineberg et al., 2018), with the orbitofrontal cortex, the dorsal/ventral striatum as well as limbic regions such as hippocampus as essential structures (Moreno and Flores, 2012; Fineberg et al., 2014, 2018). Other areas such as the dorsolateral/lateral/ventromedial prefrontal cortex, the supplementary motor area, and the premotor cortex were also linked to these traits (Fineberg et al., 2014; Grant and Kim, 2014; Dalley and Robbins, 2017). Like for impulsivity, different neurotransmitter systems have been associated to compulsive behaviors. Cortico-striatal glutamatergic projection pathways in the regulation of GABA activity at dorsal striatum (van den Heuvel et al., 2016), 5-HT at the orbitofrontal cortex in relation to cognitive flexibility and dopamine in different frontal structures for flexibility and habit learning (Fineberg et al., 2018) have been proposed. The implication of the cholinergic system in compulsive polydipsia was also observed (Martín-González et al., 2018; Mora et al., 2018)

The involvement of genetics in impulsivity is set at around 20–60%, with some variability between subdomains and with interesting particularities related to age and gender, also depending on the assessment method (self-report scales vs. neurobehavioral measures) (Bezdjian et al., 2011; Bevilacqua and Goldman, 2013; Fineberg et al., 2014). Compulsivity heritability rates show little evidence of significant gene influence (Fineberg et al., 2014), although OCD epidemiological twin studies suggested an estimate of between 27 and 47% heritability in the adult population, with higher rates in children (≈ 55% variance) (Pauls, 2010), similar to other studies (Monzani et al., 2014).

Besides individual modulator of impulsive or compulsive behaviors, in the last decades also environmental modulators have been extensively studied. Specifically, researcher laid the focus to xenobiotic compounds, both natural and artificially produced. Lead (Pb) is a heavy metal widely used in multiple industrial and commercial products such as paints and cosmetics. It can be found in both organic and inorganic forms (Rana et al., 2018) and modulates different neurotransmitter systems such as GABA, dopamine, acetylcholine and glutamate with direct effects on N-methyl-D-aspartate (NMDA) ion channels as well as alters the regulation of intracellular calcium release (Mason et al., 2014; Chibowska et al., 2016).

In addition, methylmercury (MeHg) has an important industrial use and derived products (i.e., burning coal), although the most important sources of MeHg exposure for humans are fish and crustacean's consumption (Mergler et al., 2007; Li et al., 2014; Goodrich et al., 2016). Its neurotoxicological profile has been linked to different processes such as mitochondrial dysfunction, microtubule alterations, oxidative stress, intracellular calcium release, and concentration and lipid peroxidation (Karri et al., 2016; Zhang and Van Gestel, 2017). Some of these process apparently mediated by glutamatergic system dysfunction (Karri et al., 2016).

Furthermore, PCBs represent a heterogenous chemical group composed by more than 200 congeners (i.e., PCB 153, 126) and industrial mixes (i.e., Aroclor 1242, 1248, 1254 and 1260) characterized by strong persistence in the environment (Ribas-Fito et al., 2001). The main sources of exposure in humans go from fish and animal fats consumption to industrial and commercial products. The neurotoxicological effects of this family of xenobiotics highly depend on the planarity of the specific congener attending to the capacity to bind the aryl hydrocarbon receptor (planar or coplanar vs. non-planar/coplanar PCBs). This depends on the disposition of the chlorine atoms at the molecule. Interestingly, these congeners with low affinity to aryl hydrocarbon receptor also have a toxicological profile by interacting with other proteins (Fischer et al., 1998). All these compounds can induce negative effects on health by multitude molecular mechanisms such as intracellular activity mismatching (second messengers, calcium dependent processes, kinases activity) as well as different neurotransmitter systems such as dopamine, serotonin, GABA, amongst others (Choksi et al., 1997; Tilson and Kodavanti, 1998; Boix and Cauli, 2012).

Finally, OP compounds are a wide range of pesticides commonly used for agricultural, public/residential and industrial purposes. Their main mechanism of action is the irreversible inhibition of different classes of cholinesterase (ChE), particularly acetylcholinesterase (AChE) at the CNS (Fukuto, 1990). However, alternative targets such as direct impacts on other neurotransmitter systems than cholinergic system (5HT, Dopamine and Endocannabinoid systems, amongst others), other proteins into the cholinergic system (such as nicotinic and muscarinic receptors, amongst others), mitochondrial function alterations, oxidative stress, and lipid peroxidation have been proposed (Akbel et al., 2018). From all the OP compounds found in the current market, Chlorpyrifos (CPF) is the most widely used the last decades (Eaton et al., 2008).

The well-documented negative effects on health and cognitive functioning following exposure to these agents is gaining interest in the last decade due to their widespread use and environmental persistence (Safe, 1994; Burns et al., 2013; Sánchez-Santed et al., 2016; Abreu-Villaça and Levin, 2017). Added to this, all these compounds have a stronger toxicological profile when exposure occurs during the development stage, where several research groups have been focusing their interest on in the last decades (i.e., Ribas-Fito et al., 2001; Hu et al., 2006; Myers et al., 2015; Hertz-Picciotto et al., 2018). However, and to the best of our knowledge, there are no systematic reviews specifically focusing on the relations between these hazardous compounds and impulsive and compulsive outcomes, neither human nor preclinical models. Only few systematic and classical reviews partially touched this issue, mostly focused on impulsive traits and subtypes in ADHD, ASD patients, and suicide behaviors (London et al., 2005; de Cock et al., 2012; Freire and Koifman, 2013; Polanska et al., 2013; González-Alzaga et al., 2014; Daneshparvar et al., 2016).

#### Objectives

The main objective of the present review was to systematically analyze, schedule, and critically study the different works focused (both as a central or secondary role) on some of the 4 xenobiotic compounds described above in relation to the different impulsive and/or compulsive subtypes in both preclinical and clinical fields. The four agents were selected as they are the most widely used hazardous xenobiotic compounds of the last decades and come from different sources of exposure in the environment.

#### Research Question

Attending to the rationale and the main objectives proposed, our main questions are based on 2 different aspects: (1) Does the exposure to the different xenobiotic compounds actually increase impulsive and/or compulsive behaviors in both human and animal models? and (2) Are both preclinical and clinical fields working in the same direction with a translational perspective or are they completely unrelated?

### SELECTION PROCEDURES AND SEARCH STRATEGY

#### Study Design

The present manuscript represents a systematic review of the most important empirical works published in English in relation to the exposure to different hazardous agents and their effects on impulsive and/or compulsive outcomes. Two different selection checklists (one for preclinical and another for human studies) were design following general Cochrane's guidelines for studies acceptance. After initial evaluation, once a study successfully fulfilled essential criteria to get included into the present review, a deep quality assessment protocol was undergone. Both final selection and quality assessment was done by two of the authors individually. All discrepancies were solved in a single meeting by analyzing the affected studies and discussing the different points of view.

#### Participants, Interventions, Comparators

The participants of the different experiments were: Humans (children, adolescents and adults), monkeys (Macaca fascicularis, Rhesus monkeys and Squirrel monkeys), rats (Sprague Dawley, Wistar, Long Evans, amongst others), and Mice. Interventions were varied and are systematically described into the different tables (**Supplementary Tables 1–4**).

Comparators in preclinical studies were animals exposed to different vehicles in the same fashion that the experimental ones. In the case of the clinical studies, most of the studies did not include acceptable comparators (i.e., longitudinal studies which correlated xenobiotic concentration with neurobehavioral performance), although some works used control groups with non to little exposed participants (non-randomized controlled studies).

## Systematic Review Protocol, Search Strategy, Data Sources, Study Selection

The systematic review was as follows: It was exclusively done by using PubMed as database. No date limit was set in order to select a study. For animal studies, we limited the search to murines and non-primate humans. Reviews, meta-reviews, systematic reviews, letters to editor and single cases were discarded. Only works in English were chosen.

The following words: Pb, Methylmercury, MeHg, Polychlorinated Biphenyls, PCBs, Organophosphates, Chlorpyrifos were individually mixed with the following functions and tasks: Inhibitory control, impulsivity, impulsive choice, impulsive action, motor impulsivity, decision-making, risk-taking, premature, discounting, Go/NoGo, stop signal, continuous performance, compulsivity, compulsive, flexibility, inflexibility, perseverative, perseveration, extinction, 5C-SRTT, reversal learning, marble burying, alternation, stroop and trail making. We noticed that early works used operant schedules with no standard nomenclature; we added the concept "operant." The concept "lead" was not used due to the extremely high result rate, as expected.

The selection protocol was based on the reading of the abstracts and titles. When the abstract included some behavioral outcome in relation to our compounds, we included the study in the next phase. After this, duplicated works were discarded. All the studies were then deeply analyzed with respect to their methodological quality. The study had to have one impulsive and/or compulsive sub-trait as an essential or secondary scope of the paper, always in relation with one or more of the compounds studied.

In some cases, such behaviors included here as impulsive or compulsive outcomes were not explicitly considered liked that by the authors, but a more general inhibitory control performance or even mixed with learning or attentional functioning. Those studies which fulfilled a minimum of methodological quality were introduced into the final analyses. Their main characteristics are systematically described into the different tables. Only a few studies were recruited parallel to the main PubMed searching, based on their relevance in relation to our scope. They were acquired from other referenced papers. The last day of searching was the 14th January 2019. Finally, the quality of all the studies was deeply analyzed.

### Systematic Study of the Internal Validity (Quality) of Each Selected Study

The systematic analysis of the quality of all the included studies was conducted based on two models which generated a global quality index: (1) the method's section analysis as described in the Strengthening the Reporting Studies in Epidemiology (STROBE) statement checklist (von Elm et al., 2008), and (2) the specific analysis of the six main bias sources as referred by the Cochrane's guidelines.

Briefly, STROBE statement checklist is composed by a total of 22 (9 in methods section) individual and independent subsections (items), previously used in other systematic reviews focused on neurotoxicological effects of different xenobiotic compounds on behavior (i.e., González-Alzaga et al., 2014). Although it was designed for observational studies, the adaptation for experimental studies such as the preclinical works analyzed here was basically needed for the STROBE's subsection referred to participants and study design (Item 6).

However, and as pointed out in the Cochrane's guidelines, bias control is the most important factor in order to ensure proper quality in both randomized and non-randomized studies. Although STROBE statement checklist also gives importance to this, bias control score is only 1 out of the 9 items at the method's analysis. In order to solve this, we eliminated this and made a deep analysis of each of the six bias sources from Cochrane: Selection bias (random sequence generation and allocation concealment), performance bias (blinding of participants and personnel), detection bias (blinding of outcomes assessment), attrition bias (incomplete outcome data), and reporting bias (selective reporting). This gave us a score range from 0 to 6 (one point per bias control). The STROBE gave us a score range from 0 to 8 (from the 8 remaining items of methods section after removing bias subsection).

For total quality score, we simply added these two rates, generating a range from 0 (null quality) to 14 (very high quality). That is to say, the initial relative contribution of bias control to the STROBE (1/9, 11%) turned to almost 4 times more relevance following this assessment protocol (6/14, 43%). For the total quality score, ranges were from 0 to 3, 4 to 5, 6 to 7, 8 to 9, 10 to 12, 13 to 14 for low (L), medium-low (ML), medium (M), medium-high (MH), high (H) and very high quality, respectively. Added to this, sample size of each study was also included, with a "+" symbol when the sample size was appropriate (≥8), "–" when inappropriate and "?" when the number of animals/group was not explicit, or we were not able to find this information.

#### RESULTS

### Flow Diagram of Systematic Searching Protocol and Studies Selection

The flow diagram following the whole output from the beginning to the last selection is displayed in the **Figure 1**. Our first screening of total results drove to 2,551 studies. Once we read the abstracts, sample was to 456 possible candidates, letting 188 original studies after duplicates were discarded. Following this, a deep analysis focused on the study of the description of the behavioral task described on each paper reduced the sample to 99 papers. However, we found more studies based on parallel searching by analyzing the references of the above-mentioned works. The final number of analyzed studies was 129.

#### Internal Validity and General Quality

Sample size in preclinical studies was generally acceptable except for most of the studies with monkeys (i.e., Bushnell and Bowman, 1979a,b; Rice et al., 1979; Cory-Slechta et al., 1981; Levin et al., 1987; Levin and Bowman, 1988; Newland et al., 1994; amongst others). To this, around 15% of the studies used low to very low animals per group. All clinical studies used a proper sample size, in some cases with very large sample (i.e., Zhang et al., 2009; Sagiv et al., 2010, 2012; Hong et al., 2015; Joo and Roh, 2016).

Although bias checking was the same for both clinical and preclinical studies, slight differences in the score criteria of some items in the STROBE invalidate the proper comparisons between fields (i.e., a high-quality clinical study does not mean that it is better done than a medium-high quality preclinical work). On this way, we were severer in the penalization of the lack of explicit randomization in preclinical studies, something which affected to the item related to participants and study design (item 6) and, in some circumstances, the item associated with the proper explanation of how study size was arrived at (item 10). Furthermore, the control of confounding variables (item 7) was common and properly done in most of the clinical studies (the control of the main confounding variables such as age, education level, etc.). However, we set motor and/or motivational outcomes as the essential control assessment in preclinical works, something rarely implemented in most of the studies.

For preclinical studies (89), the lowest valuated work was Rice et al. (1979), with 5 points out of 14, meanwhile 3 studies were classified as high quality with a general valuation of 12 out of 14 (Lilienthal et al., 1990; Garavan et al., 2000; Maurissen et al., 2000). We found neither very high nor low works. The 18% of the studies were categorized as medium-low, meanwhile the 27% and the 32% were labeled as medium and mediumhigh, respectively. Finally, the remaining studies (24%) were classified as high quality. Otherwise, the worst valuated paper across clinical studies (39) was Sánchez Lizardi et al. (2008). On the other hand, two studies obtained the highest rate with 11 out of 14 (Marks et al., 2010; Wesseling et al., 2010). Only one study showed low-quality as mentioned above, meanwhile the 10%, the 13% and the 59% were classified as medium-low, medium and medium-high quality, respectively. The remaining 15% were labeled as high-quality.

#### Main Findings

#### Natural Xenobiotic Compounds (The Case of Two Ubiquitous Environmental Contaminants) **Lead**

A total of 31 preclinical studies were included in relation to both impulsivity and compulsivity traits (**Supplementary Table 1a**). From these, 4 (13%) were classified as high, 5 (16%) as mediumhigh, 12 (39%) as medium and 10 (32%) as medium-low quality. Attending to sample size, 15 (48%) used low to very low animals per group. Otherwise, at clinical level, a total of 13 studies were accepted for review, with 2 (15%) labeled as high, 9 (69%) as medium-high, 1 (8%) as medium and 1 (8%) as medium-low quality (**Supplementary Table 1b**). All of them with a proper sample size.

#### **Impulsivity and Lead**

Only one out of 11 works (9%) was categorized as medium-high (Cory-Slechta et al., 1981), 6 (55%) as medium (Cory-Slechta and Thompson, 1979; Rice and Gilbert, 1985; Cory-Slechta et al., 1996, 1998, 2002; Brockel and Cory-Slechta, 1998; Stangle et al., 2007), and 4 (36%) as medium-low quality (Rice et al., 1979; Angell and Weiss, 1982; Cory-Slechta et al., 1985; Rice, 1992).

Despite some early research did not find increased responding rates following Pb exposure (Rice and Gilbert, 1985), other such as Rice (1992) found increased impulsive action in adult monkeys exposed from PND1 by inefficient responding pattern in a DRL task, particularly at later stages. Similarly, Cory-Slechta et al. (1981) observed that rats chronically exposed to Pb from weaning increased the total number of responses, shortening their duration in a modified DRL-like paradigm and showing insensitivity to external auditory facilitation.

Cory-Slechta and Thompson (1979) early found increased responding rate on a fixed interval schedule of reinforcement in rats exposed to 50–300 ppm/day from weaning. This was also observed following 25 ppm by shortening inter response times in a later study (Cory-Slechta et al., 1985). However, effects after weaning have not been systematically observed in rats (Angell and Weiss, 1982). Otherwise, early post-natal exposure increased response-rate in rats, which can be added to the increased response-rate and response-burst pattern observed in Rice et al. (1979) in monkeys exposed from PND1.

These early post-natal exposure effects were also observed in other fixed interval reinforcement paradigms, showing greater rates of responding per minute (Cory-Slechta et al., 1998) and lower post-reinforcement pauses (Cory-Slechta et al., 2002) in

exposed male rats. Moreover, the literature supports that the dopamine system, particularly in the ventral and dorsomedial striatum, is implicated in the fixed interval altered behavior following Pb exposure. These findings are like the increased impulsivity and inefficient behavioral response patterns (Multiple fixed ratio waiting-forward schedule of reinforcement) observed by Brockel and Cory-Slechta (1998) in rats. This dopamine implication on the regulation of impulsive traits on Pb exposure is not surprising (Cory-Slechta et al., 1996).

Attending to the studies carried out with humans (10), 2 of them (20%) were labeled as high (Ethier et al., 2015 and Nigg et al., 2010), 7 (70%) as medium-high (Stewart et al., 2003, 2006; Nicolescu et al., 2010; Plusquellec et al., 2010; Boucher et al., 2012a,b; Hong et al., 2015) and 1 as medium quality (Lee et al., 2018).

Although Ethier et al. (2015) found that gestational but not concurrent exposure (exposure at the time of testing) to Pb was linked to greater commission error rates (visuospatial attentionshift paradigm), other studies found that higher Pb levels in both gestational and post-natal measures also increase impulsive action rates in a GNGT (Boucher et al., 2012a). This behavioral alteration following both developmental stages was also found in a CPT (Stewart et al., 2003). In contrast, Boucher et al. (2012b) found that postnatal but not gestational Pb exposure levels were associated with hyperactivity/impulsivity type ADHD symptomatology, supported by both previous and recent studies (Nigg et al., 2010; Lee et al., 2018).

Added to this, Hong et al. (2015) found a positive correlation between postnatal Pb exposure levels and impulsivity rates (both questionnaire and neurobehavioral-based) in children. These postnatal-linked findings were also supported by the results from Nicolescu et al. (2010) in pre-adolescent children and Stewart et al. (2006) in children. Finally, low levels of postnatal (but not gestational) exposure positively correlated with greater parent-reported impulsive behavior in Inuit children (Plusquellec et al., 2010).

#### **Compulsivity and Lead**

In animal studies, 21 works analyzed the relation of Pb exposure with flexibility and perseveration outcomes. 4 of them (19%) were labeled as high (Hastings et al., 1984; Cohn and Cory-Slechta, 1993, 1994; Garavan et al., 2000), 4 (19%) as medium-high (Bushnell and Bowman, 1979a; Rice and Gilbert, 1990a; Hilson and Strupp, 1997; Lim et al., 2005), 6 (29%) as medium (Levin and Bowman, 1986, 1988; Rice and Karpinski, 1988; Newland et al., 1994; Cory-Slechta et al., 2002; Chang et al., 2014) and 7 (33%) as medium-low (Bushnell and Bowman, 1979b; Rice et al., 1979; Rice, 1985, 1990, 1992; Gilbert and Rice, 1987; Rice and Gilbert, 1990b).

Gestational Pb exposure in monkeys was linked to insensitivity to shifting contingencies by perseverative responding even 5–6 years after exposure (Newland et al., 1994). Earlier works such as those of Bushnell and Bowman (1979a) as well as Rice et al. (1979) also found increased inflexibility (increased responding at time out periods) and impaired learning after Pb exposure in different monkeys in a serie of discriminative reversal learning tasks, effects with longterm persistence (Bushnell and Bowman, 1979b). Rice (1985) also found greater inflexibility rates on non-spatial discrimination tasks (color, form, and with the influence of irrelevant cues) in chronically exposed from PND1 monkeys. In addition, Gilbert and Rice (1987) as well as Rice and Gilbert (1990a) observed that the introduction of irrelevant cues increased such impairments, also with primates.

Moreover, concurrent exposure appeared to be more critical than early exposure in inflexibility-related behaviors, although early exposure exacerbated such alterations. Following this, Rice (1990) found similar alterations in spatial discrimination paradigms in monkeys, with both relevant and irrelevant cues. Interestingly, although all groups were impaired following the introduction of novel irrelevant cues, only animals exposed from birth to onwards were affected on the normal task. However, some examples of preserved flexibility following Pb exposure were found, specifically in rats (i.e., Hastings et al., 1984).

Similar perseveration/lack of inhibition/effects of irrelevant cues were found following a DAT in monkeys exposed to Pb from PND1 (Rice and Karpinski, 1988). However, these taskrelated alterations were insensitive to age of exposure (Rice and Gilbert, 1990b). These results agree with those previously reported by Levin and colleagues, where chronically exposed monkeys but with scheduled high pulses of Pb for the first year of life systematically performed the DAT with higher rates of perseveration (Levin and Bowman, 1986). Interestingly, this was not found when chronic exposure did not consist of high pulse dosage stages (Levin and Bowman, 1988). In addition, other studies failed to find such alterations in DAT in monkeys when using larger dosages (Rice, 1992).

Once again, dopamine system seems to play an essential role in the regulation of these behaviors (Levin et al., 1987), as well as in the regulation of other compulsive traits in rats (Chang et al., 2014). Furthermore, the glutamatergic system was linked to specific modulations of perseverative responding in exposed rats from weaning (Cohn and Cory-Slechta, 1993, 1994). However, other authors proposed that these altered patterns resulting from the reversal task could be not due to impaired inhibition or inflexibility because abnormal behavior in exposed animals was found at late stages after reversal. After controlling the different post-reversal stages, they concluded that other cognitive functions (attentional, learning, or associative) rather than inflexibility could be at the basis of Pb-related impairments (Rice, 1985; Hilson and Strupp, 1997; Garavan et al., 2000).

Attending to the clinical field, we found only 4 acceptable studies for this scope, 3 of them (75%) labeled as mediumhigh (Stiles and Bellinger, 1992; Surkan et al., 2007; Nicolescu et al., 2010) and the remaining (25%) as medium-low quality (Evans et al., 1994).

Stiles and Bellinger (1992) found that both early (2 years old) and late (10 years old) postnatal Pb levels positively correlated with greater perseverative outcomes in children at the age of 10 assessed with the WCST and the California Verbal Learning Test for Children. Similarly, Evans et al. (1994) found that postnatal exposure levels were associated with both learning and flexibility impairments in children (twins), as indicated by a greater number of errors during acquisition of a visual discrimination task and a reversal condition. Furthermore, an increased number of perseverative responses (WCST) was associated with lower level of Pb exposure (5–10 ug/dL) in children (Surkan et al., 2007). Finally, Nicolescu et al. (2010) also found increased compulsive responses, as shown by altered flexibility rates in exposed children.

#### **Methyl-Mercury (MeHg)**

The systematic analysis of the studies of this section is shown in **Supplementary Tables 2a,b**. A total of 21 preclinical studies were included for analysis. From these, 5 (24%) were categorized as high, 10 (48%) as medium-high and 6 (28%) as medium quality. 7 (33%) used low to very low sample size. On the other hand, a total of 13 clinical studies were included for the final analysis, 3 (23%) high, 8 (62%) medium-high and 2 (15%) medium quality, with proper sample size in all cases.

#### **Impulsivity and MeHg**

Nine out of those 21 works specifically studied impulsive outcomes. From these, 2 studies (22%) were labeled as high (Paletz et al., 2006; Boomhower and Newland, 2016), 5 (56%) as medium-high (Rasmussen and Newland, 2001; Reed and Newland, 2007, 2009; Reed et al., 2008; Sable et al., 2009) and 2 (22%) as medium quality.

Increased impulsive choice (delayed discounting task) in adulthood was linked to postnatal MeHg exposure during the entire adolescent period in mice (Boomhower and Newland, 2016). In terms of motor impulsivity, as Paletz et al. (2006) observed that gestational/perinatal MeHg exposure increased responding rate as well as triggered an inefficient behavioral pattern in rats at early stages of a DRL task. This impulsive action pattern was also observed under progressive rate escalation at low response demands requirement. Moreover, Reed et al. (2008) found sensitization to reinforce magnitude in exposed rats.

Furthermore, Reed and Newland (2007) also introduced an external stimulus in order to generate a "clock" control of fixed interval operant behavior, and still found that the high exposure group showed generally high rates of impulsivity on both clocked and un-clocked components. This exposure protocol also triggered long-term DA system sensitivity, effects that were found independently of DA 1/2 receptor regulation. The importance of the DA system in the regulation of impulsive/compulsive traits following MeHg exposure has strong empirical support (Newland et al., 2008). For instance, Reed and Newland (2009) found altered DA system in differential reinforcement to both clocked and un-clocked components increased sensitivity to cocaine administration following gestational/perinatal MeHg exposure.

Otherwise, a hypo-sensitive GABAergic system in MeHg exposed rats was also found after pentobarbital challenge as well as an increase in impulsive action by aging (Newland and Rasmussen, 2000; Rasmussen and Newland, 2001). Similar alteration of the DA system was also found in other works (Sable et al., 2009). Whilst the latter authors did not find altered DRL performance in rats exposed to MeHg (both gestational and postnatal), the co-exposure of MeHg with amphetamine was less disruptive in MeHg groups in comparison with control rats.

Attending to human studies, all works except 1 (12) did study impulsivity-linked variables in relation to MeHg exposure. From these, 2 (17%) were classified as high (Ethier et al., 2015; van Wijngaarden et al., 2017), 8 (66%) as medium-high (Stewart et al., 2003, 2005, 2006; Yokoo et al., 2003; Nicolescu et al., 2010; Plusquellec et al., 2010; Boucher et al., 2012a,b) and 2 (17%) as medium quality (Debes et al., 2006; Lee et al., 2018).

All these preclinical findings on impulsive action have also been observed in children using a DRL task following gestational exposure to MeHg by earning less money whilst producing faster responses (Stewart et al., 2006). However, other studies found that gestational exposure could be more linked with attentional deficits (based on teacher reports) than hyperactive/impulsive traits (Boucher et al., 2012b). In an earlier study, Yokoo et al. (2003) also found a positive correlation between levels of exposure to MeHg and commission errors in an attentional task in adult population.

In a recent study, Lee et al. (2018) explored the effects of postnatal exposure on different types of ADHD (attentional and hyperactive/impulsive) in children and observed significant positive correlation between Hg levels and parental reports of hyperactive/impulsive traits. Interestingly, a recent metaanalysis conducted by Yoshimasu et al. (2014) summarized the relation between Hg (organic and inorganic forms) with both ADHD and ASD. Authors found moderate negative correlations between environmental exposure to such agents and both developmental pathologies.

However, the relation between MeHg gestational exposure levels and increased impulsive action has not been systematically found in humans (Stewart et al., 2003, 2005), as well as for postnatal (Nicolescu et al., 2010) or both gestational and postnatal (Debes et al., 2006; Plusquellec et al., 2010; Ethier et al., 2015; van Wijngaarden et al., 2017).

#### **Compulsivity and MeHg**

At preclinical level, 14 out of 21 studies did include compulsive outcomes in relation to MeHg exposure. From these, 3 (21%) were high (Doré et al., 2001; Goulet et al., 2003; Boomhower and Newland, 2017), 6 (43%) medium-high (Goldey et al., 1994; Newland et al., 2004; Widholm et al., 2004; Weiss et al., 2005; Onishchenko et al., 2007; Reed and Newland, 2007) and 5 (36%) were medium quality (Gilbert et al., 1993; Newland et al., 1994, 2013; Reed et al., 2006; Paletz et al., 2007).

An early study by Newland et al. (1994) found long-term insensitivity to changing contingencies in monkeys following gestational MeHg exposure. Reed et al. (2006) found that gestational/perinatal exposure to MeHg increased perseverative responding on a spatial discrimination task in rats at early reversal phases. Like this, inflexibility was also observed in exposed mice following reversal stages in place-learning paradigms (Onishchenko et al., 2007). Furthermore, Paletz et al. (2007) also found these alterations in rats in both spatial and visual discrimination schedules at early reversal stages and in a dose-dependent manner.

Interestingly, other studies have found that the transition from DRH to DRL increased inflexibility/perseveration in MeHg exposed (gestational/perinatal) in a dose-dependent manner (Newland et al., 2013). Similarly, Reed and Newland (2007) also found increased perseveration (high response rates at reinforce omission stage) in both clocked and un-clocked fixed interval components in high exposed rats. Interestingly, aging appears to be an important factor in MeHg effects on cognitive function and, specially, in perseveration (Newland et al., 1994, 2004; Paletz et al., 2007).

Performance of exposed rats on delayed/alternation task has also been explored. On this way, MeHg exposure for short periods during gestation was linked with a significant decrease in DAT performance without affecting motor activity in mice (Doré et al., 2001). Moreover, other studies found that both perinatal and concurrent exposure have an impact on DAT performance, leading exposed animals to appropriately unlearn the task, particularly at longer delays (Weiss et al., 2005). DAT alterations were also observed in rats exposed during gestational and perinatal stages, along with perseveration in a spatial reversal learning task (Widholm et al., 2004).

However, an early study conducted by Gilbert et al. (1993) found that adult monkeys exposed to this agent throughout pregnancy performed even better than their control counterparts. A similar lack of effects was found following both gestational (Goldey et al., 1994) and gestational-postnatal exposures in rats (Goulet et al., 2003). Finally, Boomhower and Newland (2017) recently found increased "perseverative" errors at the second reversal, thus attentional/associative influences could fit better than inflexibility in this case.

Unfortunately, the effects MeHg on compulsive behavior have rarely been studied, with 3 main works with both high (Philibert et al., 2008; van Wijngaarden et al., 2017) and medium-high quality labeling (Nicolescu et al., 2010). Both Philibert and van Wijngaardeen's studies found significant negative influences of postnatal MeHg exposure in compulsive traits (the obsessive/compulsive sub scale of the BSI in adult women and errors from intra/extra dimensional shift in children, respectively). Otherwise, Nicolescu's study did find no association between inflexibility rates and postnatal Hg levels, like in the case of impulsive outcomes above mentioned.

### Artificial Xenobiotic Compounds

#### **PCBs**

The systematic analysis of the studies in this section is shown in **Supplementary Tables 3a,b**. A total of 23 preclinical studies were included, 8 of them (35%) classified as high, 8 (35%) as mediumhigh, 2 (8%) as medium and 5 (22%) as medium-low quality. Two of them used low to very low sample size per group. Otherwise, a total of 15 clinical studies were included, categorized as high (1, 7%), medium-high (13, 86%), medium quality (1, 7%), all of them with proper sample size.

#### **Impulsivity and PCBs**

With regard to animal studies, a total of 15 works studied impulsive, 5 (33%) labeled as high (Lilienthal et al., 1990; Holene et al., 1998; Rice and Hayward, 1998; Berger et al., 2001; Lombardo et al., 2015; Monaikul et al., 2017), 6 (40%) as mediumhigh (Rice and Hayward, 1999b; Sable et al., 2006, 2009; Johansen et al., 2011; Meyer et al., 2015), 2 (13%) as medium (Rice, 1997, 1998) and 2 (13%) as medium-low (Rice and Hayward, 1999a; Johansen et al., 2014).

Lilienthal et al. (1990) found increased response rates between reinforcements (continuous reinforcement schedule test) in rats exposed to high dietary doses of PCBs, both gestational and postnatal. Following this, Rice (1997) found reduced inter response time and pauses on a multiple fixed interval schedule of reinforcement in postnatal exposed monkeys. Similarly, Rice (1998) also observed an altered learning pattern on a DRL task in postnatal exposed monkeys, with lower inter response time and reinforces gained, as well as hyperactivity.

In addition, both gestational and postnatal exposure to PCB increased reinforced responses, decreased response duration and shorter inter response time during DRH schedule in rats (Sable et al., 2006). Sable et al. (2009) found increased motor impulsivity (ratio of reinforced/non-reinforced responses) following PCB developmental exposure in DRL with larger effects for males. Added to this, a recent study conducted by this same group also found increased impulsive action and shorter inter response time in PCB exposed rats using the same exposure protocol, but with female rats (Meyer et al., 2015).

Otherwise, Holene et al. (1998) found that male rats postnatal exposed to both PCB-153 and 126 congeners were more impulsive and hyperactive as well as less efficient in reward collecting, with greater burst responding for PCB-153 exposed animals. Increased motor impulsivity and hyperactivity following this congener was also found in female hypertensive rats (Johansen et al., 2014). However, PCB-52 congener but not 153 or 180 decreased performance (variable schedule of reinforcement) in postnatal exposed adolescent rats (Johansen et al., 2011). In fact, this hyperactive and impulsive action pattern observed was also found following the mix Aroclor 1248 exposure during adolescence/early adulthood in adult male rats (Berger et al., 2001; Lombardo et al., 2015).

Finally, PCB negative effects on impulsive traits have not been always proved. On this way, postnatal PCB exposure had little effect on a progressive fixed ratio schedule (Rice and Hayward, 1999a). Related to this, Rice and Hayward (1998, 1999b) did not find significant alterations in both fixed interval/ratio and DRL or both multiple random intervals and progressive ratio reinforcement schedules following both gestational and early postnatal exposure to PCB-126 in rats. This lack of influence on DRL was supported by Sable et al. (2006), which involved perinatal PCB exposure, and other more recent works such as Monaikul et al. (2017), which involved adolescent PCB exposure.

In human studies, a total of 14 studies analyzed impulsive outcomes in relation to different PCB congeners, 1 (7%) of them was labeled as high (Ethier et al., 2015) and the remaining 13 (93%) as medium-high (Jacobson and Jacobson, 2003; Stewart et al., 2003, 2005, 2006; Vreugdenhil et al., 2004; Plusquellec et al., 2010; Sagiv et al., 2010, 2012; Boucher et al., 2012a,b; Verner et al., 2015; Behforooz et al., 2017).

Stewart et al. (2003) found that higher rates of gestational exposure to PCBs were related to impaired response inhibition (adapted vigilance task) in children, linked with a significant reduction in the splenius of the corpus callosum volume. These authors also observed similar effects at older ages in the same cohort of children (Stewart et al., 2005, 2006). Both gestational and postnatal PCB levels were also linked to clinical symptomatology of ADHD, specifically impulsivity and hyperactivity outcomes, albeit gestational exposure played a more solid role (Verner et al., 2015).

Furthermore, Jacobson and Jacobson (2003) found that gestational exposure to PCB influences on impulsivity was modulated by breastfeeding time. Authors observed that exposed pre-adolescent children made more commission errors in a CPT (thus showing general impulsivity), but only when they had received <6 weeks of breast milk feeding. This protective effect of breastfeeding against PCB related alterations has also recently been observed in a Spanish cohort (Forns et al., 2012), although the authors did not find significant relationships between PCB levels and impulsive traits (CPT).

Finally, Sagiv and colleagues also linked gestational PCB exposure with ADHD-like symptomatology in children, although this seems to be more strongly associated with attentional traits than the inhibitory control dimension, particularly in boys (Sagiv et al., 2010, 2012). Similar results were found in a recent study conducted by Behforooz et al. (2017). The results of this study revealed a positive correlation between PCB exposure rates and altered attention in young male adults, with no significant effects regarding impulsive domains (Conner's CPT-II), like other previous studies (Ethier et al., 2015). This lack of association between PCB exposure levels and impulsive behavior has also been found elsewhere (Plusquellec et al., 2010; Boucher et al., 2012a,b).

#### **Compulsivity and PCBs**

A total of 13 preclinical studies analyzed compulsive outcomes. From these, 4 (31%) were classified as high (Holene et al., 1998; Rice, 1999; Zahalka et al., 2001; Monaikul et al., 2017), 5 (38%) as medium-high (Schantz et al., 1989, 1995, 1996; Berger et al., 2001; Sable et al., 2006), 1 (8%) as medium (Rice, 1998), and 3 (23%) as medium-low quality (Bowman et al., 1978; Rice and Hayward, 1997; Widholm et al., 2001).

Early preclinical studies found that monkeys following gestational/early postnatal exposure to Aroclor 1248 increased perseverative and inflexible behavior patterns on spatial, color reversal tasks and shifting stages in probabilistic tasks, but also general hyperactivation (Bowman et al., 1978). Rice and Hayward (1997) found increased long-term inflexibility (non-spatial discrimination reversal tasks) and altered DAT acquisition in monkeys given postnatal PCBs. More recently, high rates of perseveration during the extinction (DRL task) were also observed in high PCB exposed rats during development (Sable et al., 2006).

Interestingly, Holene et al. (1998) also found this during extinction phases of a fixed interval schedule of reinforcement but following into the first two postnatal weeks. In addition, continuous gestational/postnatal exposure until weaning to Aroclor 1254 increased perseverative responses in exposed male rats at the first reversal series, whilst exposed females showed this at later stages (Widholm et al., 2001). In this case, responding in females could be strongly linked to attention/learning deficits than compulsivity.

Contrary, exposure to Aroclor 1248 mix enhanced performance on spatial reversal tasks with shape as the irrelevant stimuli (Schantz et al., 1996). Like this, Aroclor 1248 mix did not induce specific alterations on extinction phase following fixed interval schedule of reinforce, even though it generally increased impulsive rates (Berger et al., 2001). Following this, 1016 compound also impaired learning of spatial tasks without affecting reversal (Schantz et al., 1996), like later research in spatial tasks (Rice and Hayward, 1998) and intermittent schedules (Rice and Hayward, 1999b). Moreover, some studies found improved perseverative rates following PCB exposure (Monaikul et al., 2017). Finally, lack of effects on perseveration performance (DAT) was also observed in rats following perinatal (gestational and postnatal) exposure to PCB-126 congener and others (Rice, 1999; Zahalka et al., 2001).

Once again, human studies are less prominent. On this way, only 2 studies were found in relation to compulsive traits, labeled as medium-high (Jacobson and Jacobson, 2003) and medium quality (Fimm et al., 2017). Following impulsive patterns previously described, Jacobson and Jacobson (2003) found that gestational exposure to PCBs in relation to breastfeeding increased compulsive responses, as assessed by WCST in children. However, Fimm et al. (2017) did not find significant altered flexibility or attentional domains in relation to PCB exposure in adults.

#### **Organophosphates**

The systematic analysis of the studies in this section is displayed in **Supplementary Tables 4a,b**. A total of 17 preclinical studies were included in the present systematic review. From these, 4 (24%) were labeled as high, 7 (41%) as medium-high and 6 (35%) as medium quality, two of them with low to very low sample size. At the same time, 13 studies with humans were included, 2 of them (15%) classified as high, 4 (31%) as medium-high, 2 (15%) as medium, 4 (31%) as medium-low and 1 (8%) as low quality, all of them with acceptable sample size.

#### **Impulsivity and OPs**

At a preclinical level, 8 studies analyzed the influences of OP exposure on impulsivity, both choice and action subtypes. From these, 5 (63%) were labeled as medium-high (Cardona et al., 2011; López-Granero et al., 2013, 2014; Terry et al., 2014; Peris-Sampedro et al., 2016) and the remaining 3 (37%) as medium quality (Cardona et al., 2006; Middlemore-Risher et al., 2010; Montes de Oca et al., 2013).

Cardona et al. (2006) found increased long-term impulsive choice after acute exposure to high doses of CPF in adult rats. The high compulsive rats (split by SIP) had larger rates of impulsivity and altered decision-making. Follow up study also found both short and longer-term increased impulsive choice in high-compulsive CPF-exposed rats (Cardona et al., 2011). However, this same high acute dose was not able to induce either short or long-term impulsive performance in wistar rats, regardless of whether parathion (15 mg/kg) nor Diisopropylfluorophosphate (1.5 mg/kg) OP compounds (López-Granero et al., 2014) were used. However, López-Granero et al. (2013) found impaired impulsive choice associated with low CPF chronic dietary exposure for 31 consecutive weeks, followed by an increase in AChE read-trough variant expression in exposed animals.

Beyond choice, impulsive action was also studied in relation with OP exposure. On this way, Middlemore-Risher et al. (2010) found that chronic moderate doses of CPF for 14- or 30-days increased premature responding (5-CSRTT), with higher rates in the 30-day condition. Further, washout periods revealed even greater impulsive responding in exposed rats in general. Terry et al. (2014) used the OP agent Diisopropylfluorophosphate for 30 consecutive days in adult wistar rats. Authors found that increments in the inter trial interval generally increased premature responding. Interestingly, this effect was attenuated in exposed animals compared control rats during exposure, with the opposite effect during washout periods. However, such effects of OP compounds on impulsive action have not always been observed (Montes de Oca et al., 2013; Peris-Sampedro et al., 2016).

Based on human studies, 9 works analyzed impulsive outcomes in relation with OP exposure. From these, 2 (22%) were classified as high (Marks et al., 2010; Wesseling et al., 2010), 3 (33%) as medium-high (Ruckart et al., 2004; Kofman et al., 2006; Fortenberry et al., 2014), 1 (11%) as medium (Suarez-Lopez et al., 2017) and 3 (33%) as medium-low quality (Zhang et al., 2009; Mackenzie Ross et al., 2010; Joo and Roh, 2016).

Particularly with exposure in children, symptoms derived from high doses of OP agents were linked to increased motor impulsivity assessed with the NEPSY (Kofman et al., 2006). Other authors found alterations in parental-reported impulsive control (the Pediatric Environmental Neurobehavioral Battery and the Personality Inventory for Children) in exposed children (Ruckart et al., 2004). Furthermore, Suarez-Lopez et al. (2017) recently reported poorer inhibitory control (NEPSY-II) in children who suffered environmental exposure during specific months of the year (e.g., Mother's Day flowers harvest). Authors observed than these effects increased when approaching the critical time of exposure, with girls showing the worst performance. Added to this, ADHD symptomatology was linked to OP metabolite levels at gestational and, in lesser degree, postnatal urinary samples (Marks et al., 2010), eminently with the attentional dimension. Finally, other studies also found no relation between gestational exposure and ADHD symptomatology in children (Fortenberry et al., 2014).

Studies conducted in the adult population linked OP exposure to both impulsivity traits, with special emphasis on suicidal behaviors. Suicide ideation is a complex behavior in which impulsivity traits play an important role. Zhang et al. (2009) found that people who suffered chronic exposure (storage of OP compounds at home) increased the percentage of suicidal ideation in comparison with non-storers. Similarly, both selfreported depressive symptoms and suicidal traits were associated with OP exposure, but not with Carbamate agents (Wesseling et al., 2010). Moreover, higher number of OP exposure symptoms have recently been linked to suicide attempts, impulsivity (BIS), and aggressive behaviors (Lyu et al., 2018, published as letter to editor). Finally, recent study conducted by Joo and Roh (2016) found larger rates of suicidal ideation and depression in farmers, although specific xenobiotic agents were not discussed.

#### **Compulsivity and OPs**

A total of 15 preclinical studies analyzed compulsive outcomes following OP exposure. From these, 4 (27%) were categorized as high (McDonald et al., 1988; Maurissen et al., 2000; Sánchez-Santed et al., 2004; Timofeeva et al., 2008), 5 (33%) as mediumhigh (Cardona et al., 2011; Terry et al., 2014; Savy et al., 2015, 2018; Peris-Sampedro et al., 2016) and 6 (40%) as medium quality (Raffaele et al., 1990; Cardona et al., 2006; Middlemore-Risher et al., 2010; Chen et al., 2012; Terry et al., 2012; Montes de Oca et al., 2013).

Cardona et al. (2006) found that the single, high dose of CPF previously indicated also increased compulsive behavior in high drinking rats compared with a high drinking control group. A similar observation was reported in follow up studies, although this increased compulsive trait was found irrespective of high/low drinker status (Cardona et al., 2011), apparently with an important GABAergic influence. Further, the high, acute dose of CPF previously described in Cardona's studies was also linked to a long-term increased perseveration (5-CSRTT) in rats (Montes de Oca et al., 2013). Such perseverative responding rates were ameliorated following amphetamine challenge, thus indicating an important dopaminergic role on its regulation. Authors also found increased dopamine tone in the hippocampus but not the striatum, as well as decreased levels of both GABA and glutamate in the striatum.

Similarly, increased perseverative responses following CPF exposure were found in the previously described Middlemore-Risher's study, but only during washout periods in the longer exposure protocol and subtle decreases during exposure. Like what was found with impulsive responding, Peris-Sampedro et al. (2016) reported that CPF also blocked the basal higher perseverative responding rate of the APOE-4 mice. Interestingly, Terry et al. (2014) did not find altered perseverative responding but inflexibility during time-out periods following Diisopropylfluorophosphate exposure. These authors also found this pattern in the reversal stage of a water maze paradigm following adolescent exposure (Terry et al., 2012), both studied conducted with wistar rats.

In other terms, MBT has been scarcely studied in relation to OP exposure. In this regard, Savy et al. (2015) found that sub-chronic 5-day exposure to low doses of both CPF and Diazinon decreased compulsive-like behavior (MBT) in adult rats for a further week in the case of Diazinon, which is related to 5-HT transporter downregulation in both the frontal cortex and hippocampus. Interestingly, Diazinon also produced similar effects at longer exposure periods (Savy et al., 2018).

Following this, early studies also found a decreased switching capacity (increased perseveration) linked to exposure to other types of OP such as Diisopropylfluorophosphate and Disulfoton in alternation tasks, along with a possible link with decreased muscarinic binding at different brain areas (McDonald et al., 1988). In a similar vein, Chen et al. (2012) found altered DAT performance, with a strong increase of lose-shift errors in male mice exposed during gestation, particularly at longer delays and along with decreased cells number at both hippocampal and frontal structures. However, these types of effects have not been systematically observed following low, sub-chronic exposure during development (Maurissen et al., 2000; Timofeeva et al., 2008) or high, acute doses of CPF during adulthood (also Paraoxon) in DAT (Sánchez-Santed et al., 2004).

Finally, to the best of our knowledge, studies of compulsivity traits related to OP exposure in human population are sparse. A total of 5 studies were included, 1 (20%) categorized as medium-high (Ismail et al., 2017), another (20%) as medium (Mittal et al., 2011), 2 (40%) as medium-low (Mackenzie Ross et al., 2010; Rohlman et al., 2016) and 1 (20%) as low quality (Sánchez Lizardi et al., 2008).

In this regard, higher rates of perseveration assessed by WSCT were positively correlated with OP metabolite levels in 7-year old children (Sánchez Lizardi et al., 2008). In later childhood, Mackenzie Ross et al. (2010) found altered flexibility (CALCAP choice, trails B and Stroop tests) in the exposed group compared with a non-exposed group. Furthermore, TMT alterations were also linked to postnatal OP exposure, albeit authors found no effects in reversal tasks (Rohlman et al., 2016). This was also observed in both TMT and verbal fluency test following high accidental exposure (Mittal et al., 2011). Authors linked this to a general alteration of blood flow patterns in exposed participants, particularly affecting males in the occipital areas of the right hemisphere. Furthermore, Trail Making Test alterations following pesticide exposure have not been systematically found (Ismail et al., 2017).

### DISCUSSION

The present manuscript summarizes the most important empirical works published in English and focused on the relationships between the exposure to 4 xenobiotic compounds commonly found in human environments (Pb, MeHg, PCBs, and OPs) with the different subtypes of impulsive (choice and action) and compulsive (perseveration) behaviors. This is, to the best of our knowledge, the first time that these agents are systematically analyzed in relation to their specific properties in the regulation of impulsive and compulsive sub-traits, both human and non-human research and into a single manuscript.

Most of the systematic and classical reviews focused on ADHD and ASD, with an important role of impulsive traits (de Cock et al., 2012; Polanska et al., 2013; Daneshparvar et al., 2016). In relation to OP and other pesticides, different reviews on suicide behaviors were also done (London et al., 2005; Freire and Koifman, 2013). Briefly, these studies globally concluded that the exposure to the different agents is linked to increased ADHD symptomatology, also hyperactive/impulsive type. However, attentional alterations were more commonly found and MeHg influences seems to be weaker and less studied, with lead as the best analyzed by far. Compulsive traits have been basically not explored and arranged in relation to the exposure to these agents.

### Main Findings

#### Impulsivity and Xenobiotic Compounds

The systematic analysis of the most important empirical studies on the different neurotoxicological agents shows similar effects amongst them. On this way, the strong point throughout all compounds is a clear link between both gestational and postnatal exposure with increased impulsive action as well as general hyperactivation.

Pb was early linked to specific increase of both activity and improper response inhibition. Most of the preclinical studies clearly found this negative relation between exposure and impulsive action. Larger impulsive rates were systematically found throughout the different quality ranges, with only counted exceptions (Angell and Weiss, 1982 -post weaning exposed; Rice and Gilbert, 1985). This hyperactivity and motor impulsive patterns were early described by non-human primates works in Rice's studies (i.e., Rice et al., 1979; Rice, 1992) and researching with murines represented by Cory-Slechta's works (i.e., Cory-Slechta and Thompson, 1979; Cory-Slechta et al., 1981). The use of specifically designed fixed intervals/ratios, progressive ratios and DRL schedules following low to middle chronic exposures to Pb during postnatal development systematically triggered a long-term prepotent responding pattern. However, most of the preclinical studies were done with low to very low sample size, strongly limiting further generalizations.

Interestingly, this strong relation between postnatal exposure rates and impulsive action increase was systematically found in human studies, both children and young adults (i.e., Nicolescu et al., 2010). Furthermore, all human studies analyzed here found significant increased rates of impulsive behavior in relation with exposure degree, thus methodological quality should not be an important factor on this association. This can also be found in some pervious reviews on ADHD patients such as Daneshparvar et al. (2016), albeit attention deficits were stronger associated with Pb exposure.

Related to MeHg, increased impulsivity rates in preclinical models have been systematically observed independently the quality of the study, with some exceptions at the medium-high range (i.e., Sable et al., 2009). Both gestational and early postnatal exposure increased impulsive action rates both non-primate humans and, eminently, rats and mice models. In this case, few studies found an additive effect of gestational-early postnatal continuous exposure to MeHg in mismatched performance in different type of operant schedules (i.e., Paletz et al., 2006; Reed and Newland, 2007).

Clinical research on MeHg influences on impulsive action triggered inconclusive, contrary results. Only in counted cases, authors replicated the increased impulsivity found in preclinical studies following gestational (i.e., Stewart et al., 2006) and postnatal (i.e., Yokoo et al., 2003; Lee et al., 2018). However, many works had negative results on this issue (i.e., Stewart et al., 2003, 2005). Attending to their relative quality, both high classified studies (Ethier et al., 2015; van Wijngaarden et al., 2017) did find no significant influences of MeHg exposure on impulsive traits and only 2 out of 8 medium-high quality studies (Yokoo et al., 2003; Stewart et al., 2006) observed these influences. This inconsistence and lack of enough empirical research was also declared in previous-made reviews focused on ADHD symptomatology (i.e., Polanska et al., 2013).

Albeit confusing, these data are not unexpected, due to one of the most common exposure source for humans comes from fish and crustaceous consumption along with other industrial origins such as burning coal. Thus, the negative effects on cognition derived from MeHg exposure in humans is probably masked by other "positive for health" molecules present into fish, disconnecting both clinical and preclinical results. Some authors have proposed than some of these molecular targets could be the Omega-3 fatty acids present into some fish varieties, Selenium levels found into the seafood (eminently for inorganic mercury) and/or vitamin E interactions. However, all these candidate chemicals showed contradictory (Omega-3 fatty acids) or insufficient empirical support (Se and vitamin E) (Mergler et al., 2007). Interestingly, Omega-3 fatty acids diet supplementation has been linked to reduced cognitive and motor impulsivity (BIS) (Conklin et al., 2007) as well as improved impulsive profile in ADHD children (Derbyshire, 2017). Furthermore, these effects could be mediated by the regulation of the 5-HT system (Patrick and Ames, 2015). Attending to the data summarized here, increased impulsive rates following MeHg exposure in humans has not been systematically evidenced.

Furthermore, several studies have found increased premature responding pattern as well as general hyperactivation following PCB exposure in different paradigms (i.e., Lilienthal et al., 1990; Rice, 1997; Berger et al., 2001). Attending to quality, most of the high and medium-high quality works shows this profile with one exception at each category (Rice and Hayward, 1998, 1999b, respectively). Both gestational and early postnatal developmental stages are especially sensitive to PCBs, but postnatal influence has stronger empirical support attending to 8 out of 15 are studies with exclusive postnatal exposure and the rest are continuous gestational/postnatal exposure.

These preclinical studies have their translation meaning in different works on humans where increased commission errors and general impaired response inhibition were linked to both gestational and postnatal exposure (Stewart et al., 2003, 2005, 2006), also attentional performance (i.e., Sagiv et al., 2010, 2012). When analyzed by methodological quality, the only high-rated study did not find this relation following both gestational and postnatal exposure, but altered attentional performance was found (Ethier et al., 2015). Furthermore, 5 out of the 13 studies categorized as medium quality found non to little relation between exposure rates and impulsive outcomes, without developmental exposure stage influence. Interesting, some cohorts like Stewart's and Jacobson's are consistent in this negative relationship. This contradictory information could be due to strong differences in population, assessment tools and/or PCB congener.

Added to this, some works linked CPF and other OP agent's exposure with increased impulsive choice and action by premature responding (i.e., Cardona et al., 2006; Middlemore-Risher et al., 2010; Terry et al., 2014). However, an important disparity exists with no effects in some other works (i.e., Montes de Oca et al., 2013). Nevertheless, this could be explained by the inconsistency between researching groups by doing completely different exposure patterns, both in terms of dosage and time of exposure.

Attending to quality level, impulsive choice rates have been systematically increased by OP exposure in both mediumhigh (Cardona et al., 2011; López-Granero et al., 2013) and medium quality (Cardona et al., 2006), albeit exceptions have been also noticed (López-Granero et al., 2014). In relation with impulsive action, only one medium-high cataloged study (Terry et al., 2014) found decreased rates during exposure but increase premature responding during wash-out periods. This was also been found in lower-rated studies such as (Middlemore-Risher et al., 2010). That is to say, the apparent consistency in the increased discounting following OP exposure contrasts with the contradictory information from motor impulsivity.

Following this, increased impulsive rates in children have been linked to OP levels, both by neurocognitive task and questionnaire-based protocols (i.e., Ruckart et al., 2004; Kofman et al., 2006). Furthermore, such agents have been linked to suicide behaviors as well as suicidal ideation (i.e., Zhang et al., 2009; Wesseling et al., 2010), as previous summarized in other reviews (i.e., London et al., 2005). Albeit depressive and other mood mismatching could be at the basis of this extreme behavioral pattern, impulsive reasoning and execution seems to be an important variable in its development. Attending to the preclinical empirical information explained here, this idea gains support. Quality assessment demonstrated that from the 2 high quality studies, only one (Wesseling et al., 2010) found this negative relationship but specifically on suicide ideation, while this is slightly increased in medium-high studies (2 out of 3, 67%) and in all the studies lower labeled (4). Thus, the apparent clear link between OP exposure and impulsive traits in humans is moderated when methodological procedures are improved.

#### Compulsivity and Xenobiotic Compounds

Several studies have systematically associated Pb exposure with a perseverative responding pattern following multitude of reversal tasks, both spatial and non-spatial, both relevant and irrelevant associated stimuli (i.e., Bushnell and Bowman, 1979a; Newland et al., 1994). This inflexibility or insensitivity to changing contingencies was also found when changing rules happened even in a continuous fashion (i.e., DAT paradigm). Like previously found for impulsive action, postnatal exposure has been systematically associated with such impairments, albeit some works defended the stronger influence of concurrent over early exposure (i.e., Gilbert and Rice, 1987).

However, when deeply study the quality of each methodology, 2 out of the 4 better classified studies did not find this relation (Hastings et al., 1984; Garavan et al., 2000), and the other two included glutamatergic drugs into their analysis with different results (Cohn and Cory-Slechta, 1993, 1994). However, this negative association is systematically observed throughout the remaining quality categories, with some exceptions (Levin and Bowman, 1988).

Some studies such as Hilson and Strupp (1997), Rice (1985) or Garavan et al. (2000) discriminate between pure perseveration and altered learning after a change of contingencies. They concluded that the alteration at late but not early stages in a reversal condition could be more associated with a learning disturbance than with a compulsive-like pattern. This kind of specification when studying compulsive traits is essential in order to control possible learning, higher-order influences on our conclusions. This could be also added to the common lack of separation between hyperactivity and impulsive action in animal models.

Although generalization is limited attending to the number of the studies done, increased rates of compulsive traits have been found throughout all the four human studies analyzed, independently of their relative quality (Stiles and Bellinger, 1992; Evans et al., 1994; Surkan et al., 2007; Nicolescu et al., 2010).

Similar data were found with Pb exposure were linked to MeHg; developmental exposure to MeHg decreases sensitivity to changing contingencies as well as a strengthens perseveration by an inflexible pattern in exposed animals, both with and without spatial component (i.e., Reed et al., 2006). This happened it does not matter whether it took place in reversal or extinction conditions. Otherwise, DAT and other alternation tasks showed contradictory results. Attending to quality influence, the 3 highest classified works found opposite results in the relation of MeHg exposure with perseveration and inflexibility rates, with two studies showing negative influences (Doré et al., 2001; Boomhower and Newland, 2017) and one without this effect (Goulet et al., 2003). Meanwhile, medium-high studies showed a clear negative influence of MeHg most cases (83%) with counted

exceptions (Goldey et al., 1994). All this without an apparent dosage or developmental stage influence.

This apparent consistency contrasts with the almost nonexistent empirical work in humans, where from 3 works, 2 found significant negative influences of postnatal MeHg exposure (Philibert et al., 2008; van Wijngaarden et al., 2017). On this way, the presumably negative effects of MeHg exposure on compulsive behavior have not enough empirical support in humans.

Different preclinical and clinical studies have linked PCB exposure with increased perseverative and inflexible patterns. However, only one out of three high and two out of 7 medium-high quality preclinical studies (29%) have shown this effect (Holene et al., 1998; Sable et al., 2006). The lowest classified studies showed this negative effect on perseveration and flexibility (Rice and Hayward, 1997; Widholm et al., 2001). This pattern was found in multiple tasks with reversal stages with or without spatial components, external control stimuli (relevant or irrelevant) influences and in DAT-like tests or extinction phases. However, most of the analyzed studies (7 out of 11) did not find any effect on compulsive perseveration or, in some cases, did not specifically describe perseverative responding in DAT.

All this must be added to the lack of empirical study of the influences of PCB exposure on human cognitive flexibility and perseveration, attending to the limited, contradictory results that currently exist (Jacobson and Jacobson, 2003; Fimm et al., 2017). This is presumably due to the important methodological differences amongst studies. Taking all together, the increase of compulsive behaviors in animals is, at least, not robust and in humans basically not explored.

Like this, CPF and other OP compounds increased perseverative responding in different paradigms such as 5C-SRTT or SIP, where susceptible basal compulsive animals were even more inflexible following CPF exposure (i.e., Cardona et al., 2011). Only 1 out of the 4 high quality studies found increased rates of perseverative behavior (McDonald et al., 1988). Otherwise, from the 5 medium-high quality works, two of the studies showed increased rates (Cardona et al., 2011; Terry et al., 2014) and other two reduced rates (Savy et al., 2015, 2018). However, it seems that OP alters specific components of compulsivity more related to perseveration (SIP, 5C-SRTT) than other paradigms strongly affected by other variables such as anxiety and working memory (MBT and DAT).

Finally, the influences of the different OP compounds exposure on human have been little studied, with only a few representatives which exert, in some cases, opposite information. The higher classified study (Ismail et al., 2017) did not show such influence of OP exposure on compulsive traits, meanwhile the remaining (4), lower-categorized works found this negative influences with different assessment methodologies.

#### Neurodevelopmental Exposure Stages and Impulsive/Compulsive Traits

From all the studies summarized here, most of them exposed animals or took biological samples during development. From these, the ≈50%, 100%, ≈80% and the 18% of the preclinical studies on Pb, MeHg, PCBs, and OPs started their exposure protocol at a pre-gestational, gestational or early postnatal stage. Otherwise, all the clinical studies of Pb, MeHg, and PCBs exposure worked with children and used gestational and/or postnatal exposure measures. The exception of the OPs in the clinical field (≈40%) is reasonable attending to their main use as pesticides, being applicators the main population of exposure. Otherwise, the low developmental rate in preclinical research is surprising as developmental OP exposure to low or very low doses is the most research field in the neurotoxicology from, at least, the early 2000s' (i.e., Slotkin and colleagues' extensive studies).

### Limitations of the Present Systematic Review

The present manuscript has faced several difficulties based on 3 different aspects: (1) searching strategy, (2) evolution of impulsivity/compulsivity constructs definition, (3) standardization and specificities of some of the behavioral paradigms.

Briefly, Boolean searching strategy was found inefficient in order to achieve all the titles we included into the review. In fact, an important amount of studies (some of them with a capital importance for our scope) were included thanks to the referenced works by the initially analyzed manuscripts. Attending to this, we cannot discard some important studies could be out of the analysis. This was also related to the misused of standardized paradigms but different operant manipulations, eminently in early studies. Some studies did not explicitly discuss their behavioral outcomes in terms of impulsivity or compulsivity, but a general inhibitory control functioning or, in some cases, defining a clear impulsive pattern with attentional/hyperactivation mismatching. The lack of these concepts (impulsivity, compulsivity, DAT, 5C-SRTT, amongst others) make the idea of access to all the published studies a hard issue. Finally, some paradigms are known to have an important component from other cognitive functions such as working memory (DAT), attention (TMT and Stroop) and anxiety (MBT) and the real meaning of the main outcomes still under intense discussion (Moritz et al., 2009; Thomas et al., 2009; Angoa-Pérez et al., 2013). In this regard, some of the outputs discussed here as compulsivity could be discussed in other terms.

### CONCLUSIONS AND FUTURE GUIDELINES

However, the present systematic review lets to conclude that Pb, MeHg, PCBs, and OPs have the capacity to induce specific alterations in mammalian (mice, rats, monkeys, and humans) in their capacity to control improper behaviors, behold preponderant responses (motor impulsivity), and adapt to new contingencies (inflexibility/perseveration). However, there are strong limitations which are relatively shared amongst such compounds: (1) the important empirical support of the increased impulsive action and compulsive perseveration following Pb exposure is weakened by the improper sample size used by an important part of the preclinical studies. (2) The lack of parallel analysis of motor activity and/or motivation in preclinical models do not let us to discard these factors from the main variables (impulsivity or simply hyperactivation?). (3) The substantial empirical support in relation to the effects of MeHg exposure in preclinical models contrasts with the contradictory findings from human studies, probably due to the exposure source (i.e., positive effects on health from other substances present in fish such as omega-3 fatty acids). (4) Impulsive choice has rarely been explored both animals and humans. (5) Compulsive behavior study is almost non-existent and unspecific in humans. 6) There is a general lack of linking between human studies and preclinical models. (7) There is an apparent lack of the influences of developmental exposure to OP on impulsive and compulsive traits.

To our criteria, it is essential to implement delayed discounting-like tasks and other related paradigms with decisionmaking as well as risk taking outcomes in preclinical works following those different compounds exposure. Motor and motivational parallel outcomes must be also analyzed in order to avoid uncontrolled influences on the main impulsive/compulsive behaviors in preclinical studies. It is also surprising the little number of developmental OP exposure effects on impulsive and compulsive behaviors, attending to the vast research on both gestational and postnatal exposure in relation with other cognitive functions. Specific studies on this issue should be conducted. Finally, human studies should take

#### REFERENCES


advantage on compulsive traits, both by questionnaire-based and neurocognitive tasks in relation to the level of exposure, following similar and adapted tasks previously observed as very sensitive in non-primate humans.

### AUTHOR CONTRIBUTIONS

All authors contributed to the present manuscript. CP-F made the searching protocol, analyzed the different studies, wrote the first version of the manuscript and applied all the necessary changes. PF and FS-S supervised the whole procedure as well as re-structured and improved general lines until get the current status of the manuscript.

#### FUNDING

The study was supported by a grant from the Ministerio de Economía y Competitividad, Spanish Government and the Fondo Europeo de Desarrollo Regional (MINECO-FEDER) [Grant numbers: PSI2015-70037-R and PSI2017-86847-C2-1-R-MINECO-FEDER].

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbeh. 2019.00139/full#supplementary-material


and animal studies. J. Toxicol. Environ. Healtht B, Crit. Rev. 16, 127–283. doi: 10.1080/10937404.2013.783383


produces changes in activity level and stimulus control in outbred male Wistar Kyoto rats. Behav. Brain Func. 7:18. doi: 10.1186/1744-9081-7-18


alternation impairment in obsessive–compulsive disorder (OCD). Psychiatry Res. 170, 66–69. doi: 10.1016/j.psychres.2008.09.003


and the effects of amphetamine. Toxicol. Appl. Pharmacol. 51, 503–513. doi: 10.1016/0041-008X(79)90375-2


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Perez-Fernandez, Flores and Sánchez-Santed. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Genetic Model of Impulsivity, Vulnerability to Drug Abuse and Schizophrenia-Relevant Symptoms With Translational Potential: The Roman High- vs. Low-Avoidance Rats

#### Osvaldo Giorgi <sup>1</sup> \*, Maria G. Corda<sup>1</sup> and Alberto Fernández-Teruel <sup>2</sup> \*

<sup>1</sup>Department of Life and Environmental Sciences, University of Cagliari, Cagliari, Italy, <sup>2</sup>Medical Psychology Unit, Department of Psychiatry and Forensic Medicine, School of Medicine, Institute of Neurosciences, Universitat Autónoma de Barcelona, Barcelona, Spain

#### Edited by:

Margarita Moreno, University of Almería, Spain

#### Reviewed by:

Almira Vazdarjanova, Augusta University, United States Marta Miquel, University of Jaume I, Spain

\*Correspondence:

Osvaldo Giorgi giorgi@unica.it Alberto Fernández-Teruel albert.fernandez.teruel@uab.cat

Received: 31 January 2019 Accepted: 18 June 2019 Published: 05 July 2019

#### Citation:

Giorgi O, Corda MG and Fernández-Teruel A (2019) A Genetic Model of Impulsivity, Vulnerability to Drug Abuse and Schizophrenia-Relevant Symptoms With Translational Potential: The Roman High- vs. Low-Avoidance Rats. Front. Behav. Neurosci. 13:145. doi: 10.3389/fnbeh.2019.00145 The bidirectional selective breeding of Roman high- (RHA) and low-avoidance (RLA) rats for respectively rapid vs. poor acquisition of active avoidant behavior has generated two lines/strains that differ markedly in terms of emotional reactivity, with RHA rats being less fearful than their RLA counterparts. Many other behavioral traits have been segregated along the selection procedure; thus, compared with their RLA counterparts, RHA rats behave as proactive copers in the face of aversive conditions, display a robust sensation/novelty seeking (SNS) profile, and show high impulsivity and an innate preference for natural and drug rewards. Impulsivity is a multifaceted behavioral trait and is generally defined as a tendency to express actions that are poorly conceived, premature, highly risky or inappropriate to the situation, that frequently lead to unpleasant consequences. High levels of impulsivity are associated with several neuropsychiatric conditions including attention-deficit hyperactivity disorder, obsessivecompulsive disorder, schizophrenia, and drug addiction. Herein, we review the behavioral and neurochemical differences between RHA and RLA rats and survey evidence that RHA rats represent a valid genetic model, with face, construct, and predictive validity, to investigate the neural underpinnings of behavioral disinhibition, novelty seeking, impulsivity, vulnerability to drug addiction as well as deficits in attentional processes, cognitive impairments and other schizophrenia-relevant traits.

Keywords: Roman high- and low-avoidance rats, genetic animal model, impulsivity, schizophrenia-relevant symptoms, sensation/novelty seeking, vulnerability to drug abuse, mesoaccumbens and mesocortical dopaminergic pathways

### INTRODUCTION

Impulsivity is a multifaceted trait that involves premature responses or actions without foresight, often leading to adverse or maladaptive consequences. Impaired impulse control and deficits in attention are characteristic, and in some cases core aspects, of several neuropsychiatric conditions, such as attention-deficit hyperactivity disorder, obsessivecompulsive disorder, antisocial behavior, mania, schizophrenia, and drug addiction (Dalley et al., 2011; Jupp and Dalley, 2014; Hoptman, 2015; Hayward et al., 2016; Chase et al., 2017; Dalley and Robbins, 2017).

Research on the pathophysiological substrate of impulsivity has benefited from the availability of both, psychometrical and experimental measures of this multifaceted construct combined with functional magnetic resonance imaging (fMRI) studies in humans, and valid models to assess impulsive behavior either as a trait or a state in animals.

An example of animal model of impulsivity is represented by the Roman high- (RHA) and low-avoidance (RLA) lines/strains of rats. These rats are bidirectionally selected and bred for their rapid (RHA) vs. extremely poor (RLA) ability to acquire the two-way active avoidance response in a shuttle box. More than four decades of research have characterized RHA rats as less fearful, anxious and stress-sensitive than their RLA counterparts (Steimer and Driscoll, 2003; Driscoll et al., 2009; Río-Álamos et al., 2017a,b). Remarkably, compared with RLAs, RHA rats display an impulsive phenotype, as shown by their response profiles in the delayed response latency 20 (DRL-20), schedule-induced polydipsia, and 5-choice serial reaction time (5CSRT) tasks (Zeier et al., 1978; Moreno et al., 2010; Merchán et al., 2019; see ''The Roman Rats: A Genetic Model of Differential Anxiety, Novelty Seeking, Impulsivity, Attention Deficits and Associated Traits'' section). Moreover, relative to RLAs, RHA rats display several impairments in schizophrenia-relevant phenotypes, such as prepulse inhibition (PPI; Oliveras et al., 2015; Río-Álamos et al., 2019), latent inhibition (LI; Fernández-Teruel et al., 2006), spatial working memory (Oliveras et al., 2015) and reversal learning (Río-Álamos et al., 2019) as well as a trend towards a reduction of some types of social behavior (Coppens et al., 2012; Del Rio et al., 2014; see ''RHA Rats as a Model of Deficits in Attentional Processes, Cognitive Impairments and Other Features Relevant for Schizophrenia Research'' section). These behavioral profiles are consistent with findings of a reduced volume and function of the medial prefrontal cortex (mPFCx), hippocampus (HC) and amygdala (AMY) of RHA vs. RLA rats (Río-Álamos et al., 2017a, 2019). Finally, as it would be expected from a model with translational value, compared with RLAs, RHA rats also show more intense behavioral sensitization following the repeated administration of psychostimulants (Corda et al., 2005; Giorgi et al., 2005a, 2007) and enhanced vulnerability to drug abuse/addiction (Fattore et al., 2009), which are associated with a more robust mesolimbic dopaminergic tone (Giorgi et al., 1997, 2003, 2005b; Piras et al., 2003; Lecca et al., 2004; see ''The Roman Rats as a Genetic Model of Vulnerability to Drug Addiction'' section).

In recent years, it has become apparent that rather than a form of self-medication secondary to typical schizophrenia symptoms or medication side effects, substance abuse comorbidity is an independent primary disease symptom in schizophrenia (Chambers et al., 2001). Nevertheless, the limited availability of animal models of substance abuse comorbidity in psychotic disorders based on genetic selection is an important drawback in preclinical research on the impact of gene-environment interactions on such conditions. Here, we will survey experimental evidence supporting the view that the RHA rat line/strain may be used as a reliable animal model of genetically determined vulnerability to substance use comorbidity in schizophrenia in terms of face, construct, and predictive validity. Implications of these findings for investigations on the etiology and treatment of Impulsive Compulsive Spectrum Disorders are considered in the present review article.

### TESTS AND MODELS OF THE IMPULSIVITY TRAIT AND IMPULSIVE ACTIONS/RESPONSES

Although preclinical and clinical studies have led to relevant advances in our knowledge of some psycho- and neuro-biological mechanisms underlying impulsivity and its involvement in several psychiatric disorders (Jupp and Dalley, 2014; Chase et al., 2017; Dalley and Robbins, 2017), the currently available treatments for impulsivity and impulsivity-related disorders have limited efficacy and application. Thus, rigorously validated animal models, which allow manipulation of experimental conditions that is not possible in humans, are critical to enable progress in our understanding of the psychological, neurobiological and genetic basis of deficits of impulse control and for the development of innovative therapies (Dalley et al., 2011; Hayward et al., 2016).

Of note, the assessment of the validity of animal models ought to address the three dimensions of face, construct, and predictive validity. Face validity refers to a phenomenological similarity between the model and the disorder being modeled. Construct validity implies that the model has a sound theoretical rationale and heuristic potential, that is, it can aid in discovering novel characteristics of the modeled condition. The concept of predictive validity implies that manipulations known to influence the pathological state (i.e., clinically effective treatments) should have similar effects in the model (Willner, 1984).

An important aspect of animal models involves the development of tests, tasks and/or measures that allow the evaluation of behavioral responses reflecting the human trait that one intends to model. In this regard, for example, the DRL-20, stop signal, delay discounting, scheduleinduced polydipsia, 5C-SRT, and 5-choice continuous performance (5C-CP) tasks allow measuring impulsivity, impulsive responses and attention in laboratory rodents (Hayward et al., 2016; Dalley and Robbins, 2017). In addition, some of these models have the advantage of being completely translatable to humans (e.g., the stop signal paradigm, delay discounting, 5C-SRT and 5C-CP tests are used in both rodents and humans; Hayward et al., 2016; Dalley and Robbins, 2017).

Another significant aspect concerning animal analogs is the strategy employed to model impulsivity either from an ''etiologyfocused'' or a ''behavior to biology'' perspective. Genetically engineered rats or mice and brain lesion-based models constitute experimental strategies for modeling impulsivity from an etiology-focused perspective. Notably, the etiological hypotheses are the main constraints of these models, which begin by assuming a given pathophysiological mechanism and then investigate the resulting phenotype (Jupp et al., 2013; Hayward et al., 2016). Conversely, the ''behavior to biology'' models use a different strategy, which starts by recreating or identifying an impulsive phenotype and then follows by investigating its neuropsychological underpinnings. Some of these preclinical models select groups of animals from a heterogeneous population on the basis of the different characteristics of a behavioral trait or behavioral response related with impulsivity. The Lister-Hooded rats selected for excessive premature (i.e., impulsive) responses in the 5C-SRT test (Robinson et al., 2009; Hayward et al., 2016; Dalley and Robbins, 2017) and the Wistar rats stratified for extreme values of schedule-induced polydipsia (Merchán et al., 2019, and references therein) are examples of these models of impulsive behavior. Other ''behavior to biology'' models use selective bidirectional breeding for a given phenotype and then systematically evaluate the underlying neurobiological mechanisms and associated traits. The latter are especially relevant since one would expect from a valid impulsivity model that such associated traits were related to the selected trait: for example, rats with an impulsive phenotype should be more vulnerable to addictive drug taking, compulsive drug seeking and relapse. Examples of these models are the low- and high-impulsive inbred rats (Belin et al., 2016), and the RHA and RLA rats (Escorihuela et al., 1999; Moreno et al., 2010; Tournier et al., 2013) that are the main focus of this review article.

### COPING STYLE, BEHAVIORAL FLEXIBILITY, BEHAVIORAL INHIBITION AND IMPULSIVITY

The concept of coping style arises mainly from the field of research on stress and relates to how an organism deals, or tries to deal, with a threat. In general, it refers to consistent response profiles in the face of challenges that are similar across various situations and stable over time (Coppens et al., 2010). However, coping styles also extend their influence to non-threatening or non-aversive situations, such as appetitive conditions or tasks that involve some kind of conflict between actions that lead to worse consequences (e.g., a fast instrumental response that leads to an immediately available small reward) and actions that lead to a better outcome (e.g., a slow, delayed instrumental response that leads to a big reward). Two different styles of coping behavior, proactive and reactive, are considered in coping research. Typically, rodents with a proactive coping style show impulsive (i.e., premature and poorly conceived) responding profiles, display impaired behavioral flexibility (i.e., the ability to adapt responses to changing environmental demands) and attention, and are often novelty/incentive seekers (Coppens et al., 2010). Numerous studies indicate that the responses that characterize the proactive coping style are predominantly driven by internal mechanisms or routines (i.e., they are not much sensitive to changes in the environment), whereas reactive coping animals are much more dependent on the environment (Coppens et al., 2010). This difference in the sensitivity to changes in environmental conditions critically influences behavioral flexibility, which is generally higher in subjects characterized by a reactive coping style. For example, proactive coping RHA rats -that are also impulsive- are impaired at solving a spatial reversal learning task compared with their reactive coping RLA counterparts (Escorihuela et al., 1995; Río-Álamos et al., 2019). Thus, proactive coping styles in experimental animals and in humans are frequently positively associated with impulsivity, behavioral (cognitive) inflexibility and also with deficits in behavioral inhibition (for a review, see Coppens et al., 2010).

Behavioral inhibition is a trait related to the reactive coping style and anxiety. In this context, anxiety is operationally defined as the increased susceptibility to signals of punishment, non-reward and novelty, that leads to the expression of reactive coping responses, such as freezing or risk assessment in the face of a conflict between incompatible goals/responses (Gray and McNaughton, 2000). Conversely, proactive coping individuals tend to display disinhibited and impulsive responses/behavior under the same conditions (Gray and McNaughton, 2000; Avila and Torrubia, 2006; Coppens et al., 2010).

Notably, the RHA and RLA rat lines/strains display differential impulsivity and profound divergences in associated traits, such as coping style, behavioral inhibition, behavioral/cognitive flexibility, attention, cognitive ability (e.g., spatial learning, working memory) and novelty seeking, as described in the following section (Driscoll and Bättig, 1982; Driscoll et al., 1998, 2009).

### THE ROMAN RATS: A GENETIC MODEL OF DIFFERENTIAL ANXIETY, NOVELTY SEEKING, IMPULSIVITY, ATTENTION DEFICITS AND ASSOCIATED TRAITS

#### Behavioral Inhibition and Anxiety

The Roman rat lines were established in Rome in the 1960s through bidirectional selection and outbreeding of Wistar rats showing very high (RHA) vs. extremely low (RLA) rates of acquisition of the two-way active avoidance response (Bignami, 1965, see ''Introduction'' section). Subsequently, the Swiss sublines RHA/Verh and RLA/Verh were derived from the original lines (Driscoll and Bättig, 1982) and other outbred sublines were established in the USA (Satinder, 1975), France (Willig et al., 1991; Castanon et al., 1995) and Cagliari, Italy (Giorgi et al., 2005b). In addition, two inbred Roman strains, RHA-I and RLA-I, generated through brother/sister mating of the respective Swiss sublines, were established in Barcelona and Germany (Schwegler et al., 1997; Driscoll et al., 1998; Escorihuela et al., 1999). It was soon noticed that the selection for two-way avoidance acquisition did not lead to rat lines primarily differing in some general process related to ''learning ability.'' In contrast, the Roman lines differ in the emotional responses displayed in stress or anxiety paradigms (reviewed by Driscoll and Bättig, 1982; Driscoll et al., 2009). Accordingly, there is ample experimental evidence showing that two-way active avoidance learning is blunted by fear/anxiety (Gray, 1980; Fernández-Teruel et al., 1991b; Gray and McNaughton, 2000; Vicens-Costa et al., 2011; LeDoux, 2015; LeDoux et al., 2017; Fernández-Teruel and Tobeña, 2018). In fact, RHA rats learn faster and more effectively than their RLA counterparts to avoid the electric shock in the two-way active avoidance test, whereas RLA rats perform better than RHA rats in some operant and spatial memory tasks (Zeier et al., 1978; Nil and Bättig, 1981; Driscoll and Bättig, 1982; Willig et al., 1991).

In parallel, early studies on emotional aspects showed that RLA rats were more sensitive to classical Pavlovian aversive conditioning than their RHA counterparts (Imada, 1972) and displayed more robust hormonal responses (i.e., increments in the plasma concentrations of corticosterone, prolactin, and ACTH) to several types of acute psychological and physical stressors (Gentsch et al., 1981, 1982). These neuroendocrine findings have been replicated in many subsequent studies (Castanon et al., 1992; Castanon and Mormède, 1994; Steimer et al., 1998; Steimer and Driscoll, 2003; Carrasco et al., 2008; Díaz-Morán et al., 2012; Río-Álamos et al., 2017a,b).

Anxiety response profiles are markedly different between RHA and RLA rats, as evidenced in experimental procedures involving unconditioned or conditioned conflict. Among the former, some of the earliest demonstrations suggesting between-line differences in anxiety were the increased defecation and reduced exploratory activity observed in RLA rats in the open field test (Gentsch et al., 1981, 1982) and their avoidance of the illuminated center of the hexagonal tunnel maze (Driscoll and Bättig, 1982; Martin et al., 1982). These results have been corroborated in numerous studies (Fernández-Teruel et al., 1991a, 1992a,b, 1994, 1997, 2002b; Escorihuela et al., 1997, 1999; Steimer and Driscoll, 2003; Estanislau et al., 2013; Tapias-Espinosa et al., 2018) and extended to other novelty-based, unconditioned conflict conditions, such as the hyponeophagia test, the light/dark box, the elevated plus-maze and the elevated zero-maze, in which RLA rats consistently display higher levels of behavioral inhibition and anxiety responses, which include avoidance of open/illuminated spaces, freezing, reduced exploratory activity and increased self-grooming (Ferré et al., 1995; Steimer et al., 1998; Escorihuela et al., 1999; Fernández-Teruel et al., 2002b; Steimer and Driscoll, 2003; Estanislau et al., 2013; Río-Alamos et al., 2015; Río-Álamos et al., 2017a,b).

Moreover, in procedures involving Pavlovian aversive conditioning or conditioned approach-avoidance conflict, RLA rats display, compared to their RHA counterparts, increased conditioned freezing (Imada, 1972; López-Aumatell et al., 2009a; Martínez-Membrives et al., 2015), enhanced shockinduced suppression of drinking (Ferré et al., 1995; Corda et al., 2018) and more pronounced fear-potentiated acoustic startle (Schwegler et al., 1997; Yilmazer-Hanke et al., 2002; López-Aumatell et al., 2009a,b). The enhanced behavioral inhibition of RLA rats is also observed in instrumental procedures of frustrative reward omission (i.e., extinction) or reward down-shift (Torres et al., 2005; Rosas et al., 2007; Gómez et al., 2009b; Coppens et al., 2012, 2013; Manzo et al., 2014b). Finally, another example of the reactive (i.e., behavioral inhibition) vs. proactive coping styles in the Roman lines/strains is that, in the forced swimming test, an experimental model that is extensively used for the screening of antidepressant drugs (Porsolt et al., 1977), RLA rats display fewer active movements aimed at escaping from the water cylinder (i.e., struggling) and more immobility than RHA rats (Piras et al., 2010, 2014; Díaz-Morán et al., 2012). Moreover, either the subacute or chronic treatment with the antidepressants desipramine, chlorimipramine or fluoxetine significantly decrease immobility in RLA, but not RHA rats (Piras et al., 2010, 2014), supporting the predictive validity of the model.

Notably, it has been reported that GABA-related anxiolytic drugs decrease behavioral inhibition (or anxiety) of RLA, but not RHA rats, in some of the above mentioned conflict-based tests (Martin et al., 1982; Driscoll and Stübi, 1985; Fernández-Teruel et al., 1991c; Steimer and Driscoll, 2003; Torres et al., 2007), whereas the anxiogenic drug pentylenetetrazol, a blocker of the chloride channel of the GABA-A receptor, enhances behavioral inhibition (i.e., anxiety) in the Vogel's conflict test predominantly in the RLA line (Corda et al., 2018). Of relevance in this context is the finding that the function of the GABA-A receptor (assessed by means of the GABA-stimulated <sup>36</sup>Cl<sup>−</sup> uptake and GABA-stimulated [3H]-Zolpidem binding) is reduced in the whole cerebral cortex of RLA vs. RHA rats (Giorgi et al., 1994; Bentareha et al., 1998), which may contribute to the divergent behavioral inhibition and emotional responses that distinguish RHA from RLA rats.

Gray and McNaughton (2000) and McNaughton and Corr (2004) have proposed that behavioral inhibition, a main expression of cognitive anxiety, is a consequence of the activation of the ''Behavioral Inhibition System,'' whose neuroanatomical substrate is the septo-hippocampal system (SHS). The activity of the SHS, in close interplay with the AMY and related circuitry, seems to be responsible for the appearance of some species-specific inhibitory responses, such as freezing or risk-assessment, in the face of situations in which there is a conflict between incompatible goals/responses or uncertainty (Gray and McNaughton, 2000). That is, the activation of the SHS, as well as the AMY in more specific situations, leads to responses that characterize states of anxiety and/or fear (Gray and McNaughton, 2000; LeDoux, 2015; see also Fernández-Teruel and Tobeña, 2018). Activation of the central GABAergic system by anxiolytic benzodiazepines and other GABA-mimetic drugs decreases the activity of the SHS, and thus reduces behavioral inhibition in conflict conditions (Gray and McNaughton, 2000). It is noteworthy that, compared with their RHA counterparts, RLA rats display a more pronounced neural activation of the HC and AMY, as reflected by increased c-fos expression, when exposed to different novelty-based conflict situations (Meyza et al., 2009). Accordingly, the volume of the HC and AMY is larger (Río-Álamos et al., 2017a), and the neuronal density is higher in both brain areas of RLA vs. RHA rats (Garcia-Falgueras et al., 2012; Gómez et al., 2009c). In addition, the activity of the phospholipase C signaling cascade is higher in the HC of RLA vs. RHA rats (Sallés et al., 2001).

Collectively, the experimental findings reviewed above support the view that RHA rats are less anxious and more behaviorally disinhibited than RLA rats and other rat strains/stocks used as external controls, such as Wistar rats or the ''National Institutes of Health Heterogeneous Rat Stock'' (NIH-HS) rats (Gómez et al., 2009a; López-Aumatell et al., 2009b; Díaz-Morán et al., 2012; Estanislau et al., 2013; Martínez-Membrives et al., 2015).

#### Sensation Seeking and Impulsivity

As mentioned in ''Coping Style, Behavioral Flexibility, Behavioral Inhibition and Impulsivity'' section, the proactive coping style of RHA rats is associated with behavioral disinhibition and novelty/sensation seeking. In this regard, the relative behavioral disinhibition of RHA vs. RLA rats is also apparent in tests measuring the preference for novelty which involves some degree of ''approach-avoidance'' conflict and are considered to capture more specifically the sensation/novelty seeking (SNS) trait. Such tests include the novel object exploration test, the ''Y'' maze, familiar vs. unfamiliar spaces, which include novel objects and novel/unknown spatial distributions (Pisula, 2003), and the hole-board test with unfamiliar objects under the holes (File and Wardill, 1975). In these tests, compared with the RLA strain and, in some measures/tests, also with the outbred NIH-HS strain, RHA rats consistently display more intense exploration of the novel/unfamiliar objects or spaces (Fernández-Teruel et al., 1992b, 1997; Escorihuela et al., 1999; Pisula, 2003; Estanislau et al., 2013; Manzo et al., 2014a; Cuenya et al., 2016).

The SNS trait in humans, cats and rats is characterized by high levels of exploratory and risk-taking behaviors and is commonly associated with impulsiveness and a tendency to experiment with drugs of abuse (Bardo et al., 1996; Siegel, 1997). SNS cats, for example, show increased exploratory behavior and lowered efficiency in bar pressing responses during an inhibitory DRL task (i.e., an appetitive instrumental conditioning paradigm in which reward is obtained only following low response rates; Saxton et al., 1987a,b). Importantly, a similar behavior has been reported in RHA rats (Zeier et al., 1978).

Regarding the neurophysiological basis of SNS, an interesting early finding is that, like SNS humans and cats, RHA rats are visual evoked potential augmenters at the brain cortical level, whereas RLA rats (as well as the Wistar rats used for comparison) are visual evoked potential reducers (Siegel and Driscoll, 1996; Siegel et al., 1996; Siegel, 1997).

Studies in humans have shown that behavioral disinhibition is a component of sensation seeking and that this trait is moderately correlated with impulsivity. Thus, although sensation seeking and impulsivity are different traits, they show a significant positive association. This finding has prompted the development of composite psychometrical measures of ''impulsive sensation seeking,'' using psychometrical instruments that combine both traits into a single construct (Zuckerman, 1996; Roberti, 2004; Magid et al., 2007).

The RHA rats encompass many characteristics that are consistent with a model of ''impulsive sensation seeking.'' Part of the evidence reviewed above, such as the deficit of RHA rats in the DRL20 paradigm, in which they are less able to inhibit irrelevant responses than their RLA counterparts (Zeier et al., 1978), or the behavior of RHA vs. RLA rats in some learning tasks in several types of mazes (Nil and Bättig, 1981; Driscoll and Bättig, 1982; Willig et al., 1991), had already suggested that deficits of impulse control in RHA rats could play a role in their deficient performance in those tasks. Accordingly, it has been reported that inhibitory control is impaired in RHAs compared with RLA rats, as shown by an increased number of premature responses made by the former in the 5-CSRT task (Moreno et al., 2010). In a delay discounting task, RHA rats also display enhanced impulsive choice and prefer the immediate and smaller reward as opposed to a delayed larger reinforcement (Moreno et al., 2010). In addition, in the procedure of schedule-induced polydipsia, the RHA strain shows stronger acquisition of the adjunctive responses, and a higher asymptotic response level of both water intake and licking responses than RLA rats (Moreno et al., 2010; see also Klein et al., 2014; Merchán et al., 2019).

Consistent with the contention that they represent an impulsive phenotype, RHA rats display a higher frequency of lever pressing than their RLA counterparts during the acquisition phase in an unpredictable operant conditioning paradigm and during the extinction phase upon appetitive instrumental conditioning (Gómez et al., 2009a; Moreno et al., 2010; Coppens et al., 2012, 2013; Klein et al., 2014). Moreover, RHA rats are more perseverant than RLAs at pressing the active lever during the extinction phase of an intravenous cocaine self-administration program (Fattore et al., 2009; see ''The Roman Rats as a Genetic Model of Vulnerability to Drug Addiction'' section). Thus, the RHA phenotype includes characteristics of both the SNS and impulsivity traits.

The absence of strain-dependent differences in home cage activity (Meyza et al., 2009; Estanislau et al., 2013) as well as the finding that both Roman lines display similar lever press responding during the acquisition of an appetitive operant task under a fixed ratio-1 (Coppens et al., 2013), argues against the possibility that the distinct impulsive responding of the two lines/strains is due to differences in motor activity.

Notably, although RHA rats show a more impulsive behavioral profile than their RLA counterparts, it is clear that they are capable of displaying response inhibition under threatening conditions; thus, RHA rats show high levels of conditioned freezing behavior, either when exposed to a cue (López-Aumatell et al., 2009a) or a conditioned context during Pavlovian or operant aversive conditioning (López-Aumatell et al., 2009a,b; Díaz-Morán et al., 2012; Martínez-Membrives et al., 2015).

A large body of experimental evidence has established that the mesolimbic dopamine (DA) system is a key component of the neural circuitry mediating brain reward (Wise, 2002). This system consists of DA neurons in the ventral tegmental area (VTA) of the mesencephalon that project to forebrain regions such as the nucleus accumbens (Acb), mPFCx, AMY, and HC. Activation of the mesolimbic DA system plays a pivotal role in sensation seeking behavior (Bardo et al., 1996) and in the reinforcing effects of drugs of abuse (Wise, 2002; Pierce and Kumaresan, 2006; see ''The Roman Rats as a Genetic Model of Vulnerability to Drug Addiction'' section).

It has been shown that the density of D2 DA auto-receptors (D2-autoR) is lower in the substantia nigra/VTA of RHA vs. RLA rats (Tournier et al., 2013), consistent with reports relating SNS with low midbrain DA D2-autoR density in humans (Zald et al., 2008). Moreover, the density of postsynaptic DA D2/D3 receptors is lower in the striatum and nucleus accumbens (Acb) of RHA vs. RLA rats and it has been shown that striatal DA D2 receptor availability is inversely correlated with novelty seeking behavior of Roman rats in the hole-board test (Tournier et al., 2013). Importantly, sensation seeking in humans has also been negatively associated with striatal D2 receptor availability (Gjedde et al., 2010). Taken together, the above mentioned findings and the effects of dopamine-mimetic psychostimulants on striatal D2 receptor availability and on the extracellular concentrations of DA in the terminal fields of the mesolimbic dopaminergic projections support the view that the functional tone of the mesolimbic dopaminergic system is more intense in impulsive sensation seekers such as RHAs than RLA rats (Giorgi et al., 2007; Tournier et al., 2013).

In studies using rats selected for impulsivity, i.e., behaviorally separated in extreme ''high impulsive'' (HI) vs. ''low impulsive'' (LI) groups according to their levels of premature responding in the 5-CSRT task, it was found that HI animals had decreased D2/D3 receptor availability in the Acb, and D2/D3 receptors negatively predicted impulsive behavior (Dalley et al., 2007; see also Barlow et al., 2018). This is consistent with the aforementioned findings in the Roman rats and with a similar D2/D3 receptor profile found in ADHD patients (Buckholtz et al., 2010; Ghahremani et al., 2012).

Finally, a recent fMRI study in a large sample of healthy and distressed humans has shown that the activity of the left ventrolateral prefrontal cortex and bilateral ventral striatum, including the Acb, during an ''uncertain reward expectancy'' paradigm is positively related with impulsive sensation seeking (Chase et al., 2017). This is also in line with the proposal of an enhanced functional tone of the mesolimbic dopaminergic pathway in impulsive sensation seekers like RHA rats (Giorgi et al., 2007).

To sum up, RHA rats show considerable similarities with impulsive and sensation seeking humans regarding SNS, impulsivity, visual evoked potentials, and the underlying mesolimbic DA transmission-related parameters. These findings highlight the face and construct validity of RHA rats as a genetic model of impulsivity and SNS.

### RHA RATS AS A MODEL OF DEFICITS IN ATTENTIONAL PROCESSES, COGNITIVE IMPAIRMENTS AND OTHER FEATURES RELEVANT FOR SCHIZOPHRENIA RESEARCH

As mentioned previously, a proactive coping style is often associated with impairments in attention and cognitive flexibility, as well as heightened impulsivity (for a review see Coppens et al., 2010). These traits have in turn been associated with schizophrenia in patients (Brown et al., 2018; Ho et al., 2018). Thus, using both psychometric impulsivity scales and the delay discounting task (a measure of action planning and impulse control in which higher rates of reward discounting, due to the preference of smaller and immediate rewards over delayed and larger ones, reflect enhanced impulsivity), it has been shown that impulsivity measures are positively associated in a consistent manner with schizophrenia, schizoaffective disorder, and schizophrenia risk in unaffected biological relatives of patients (Ahn et al., 2011; Brown et al., 2018; Ho et al., 2018). In addition, the presence of heightened impulsivity in schizophrenia is paralleled by impairments in working memory (Ahn et al., 2011; Brown et al., 2018).

Therefore, we considered of interest to investigate whether RHA rats would also display cognitive and attentional impairments and other schizophrenia-relevant phenotypes. In this context, more than two decades ago it was demonstrated that there were marked differences in spatial cognitive abilities between the Roman rat lines. Thus, spatial reference learning (i.e., place learning) in the Morris water maze was impaired in RHA rats (Driscoll et al., 1995; Escorihuela et al., 1995). Moreover, RHA rats also were impaired in a reversal place task, suggesting a blunted behavioral/cognitive flexibility relative to their RLA counterparts (Escorihuela et al., 1995; Fernández-Teruel et al., 1997). These early findings have been replicated and extended by showing that RHAs also exhibit spatial working memory deficits, as compared with both RLA rats and the genetically heterogeneous NIH-HS rats (Willig et al., 1991; Aguilar et al., 2002; Oliveras et al., 2015, 2016).

It is noteworthy in this context that the impairments in working memory of schizophrenic patients may be due, at least in part, to alterations in cortical GABAergic inhibitory transmission. Accordingly, it has been shown that, in individuals with schizophrenia, a deficiency in brain derived neurotrophic factor- (BDNF-) mediated signaling combined with decreased levels of the GABA synthetizing enzyme, GAD67, results in reduced inhibitory neurotransmission of GABA neurons (i.e., basket cells) in the dorsolateral prefrontal cortex (Lewis et al., 2005; Kimoto et al., 2014). This deficiency in GABA-mediated transmission leads to a decreased inhibition of pyramidal neurons that have been hypothesized to contribute to altered gamma oscillations and impaired working memory in schizophrenia (Lewis et al., 2005). In view of the above findings it would be worthwhile to evaluate GABAergic transmission in the mPFCx (corresponding to the dorsolateral frontal cortex of the human brain) of RHA and RLA rats, since our previous results concerning GABA-stimulated <sup>35</sup>Cl<sup>−</sup> uptake were performed in the whole cerebral cortex (Giorgi et al., 1994; Bentareha et al., 1998; see ''The Roman Rats: A Genetic Model of Differential Anxiety, Novelty Seeking, Impulsivity, Attention Deficits and Associated Traits'' section).

LI and PPI of the startle response are attention-related processes that are impaired in schizophrenia and other diseases, such as bipolar disorder (Gray et al., 1991; Swerdlow et al., 1994, 1996; Cromwell et al., 2008; Lubow and Weiner, 2010; Kohl et al., 2013). The procedures used to evaluate LI and PPI are similar in rodents and humans, and the performances in both paradigms are currently considered as endophenotypes of schizophrenia (Gray et al., 1991; Powell and Miyakawa, 2006; Lubow and Weiner, 2010; Jones et al., 2011; Swerdlow and Light, 2016). Notably, RHA rats display deficient LI of the two-way active avoidance response compared with Sprague-Dawley rats (Fernández-Teruel et al., 2006). Likewise, compared with RLAs, RHA rats show impaired LI of the fear-potentiated startle (Esnal et al., 2016).

In line with these strain-related differences in LI, we have shown in a series of studies that RHA rats display clear and consistent deficits of PPI of the acoustic startle response compared with RLAs (Del Rio et al., 2014; Oliveras et al., 2015, 2016, 2017; Río-Alamos et al., 2015; Río-Álamos et al., 2017a,b, 2019; Tapias-Espinosa et al., 2018, 2019) and that PPI deficits predict spatial working memory impairments (Oliveras et al., 2015). Altogether, the above mentioned cognitive and attentional profiles of RHA vs. RLA rats suggest that the former may represent a model of schizophrenia-relevant features. This is further supported by the increased mesolimbic/mesostriatal dopaminergic functional tone of RHA rats (Giorgi et al., 2007; Tournier et al., 2013), their enhanced sensitivity to the locomotor and DA releasing effects of acute and chronic psychostimulants (Corda et al., 2005; Giorgi et al., 2005a,b, 2007; Guitart-Masip et al., 2008), and by the finding that, relative to RLA rats, RHAs exhibit some impairments in social behavior (Coppens et al., 2012; Del Rio et al., 2014).

It is noteworthy that PPI of the acoustic startle is further impaired by the non-selective DA D1/D2 receptor agonist apomorphine and improved by the DA D2/D3 receptor antagonist haloperidol in RHAs but not RLA rats (Oliveras et al., 2017).

Two additional lines of evidence support the view that RHA rats may be considered as a putative model of schizophreniarelevant symptoms/features. Firstly, social isolation rearing induces deleterious effects on PPI, anxiety, and spatial reference memory only in RHA rats (Oliveras et al., 2016). Second, in Sprague-Dawley rats, bilateral neonatal lesions of the ventral hippocampus (NVHLs), a neurodevelopmental model of schizophrenia, result in enhanced locomotor activation and stereotypies in adult rats upon the acute administration of amphetamine (Lipska and Weinberger, 2000). Moreover, in adult NVHL rats, a challenge with amphetamine elicits an augmented increase in DA output in the core (AcbCo) and, at the same time, an attenuated dopaminergic response in the shell (AcbSh) of the Acb, as compared with shams (Corda et al., 2006). Importantly, the effects of the amphetamine challenge in this neurodevelopmental model of schizophrenia are closely reminiscent of those observed in psychostimulant-sensitized RHA rats (Giorgi et al., 2005b, 2007; see also ''The Roman Rats as a Genetic Model of Vulnerability to Drug Addiction'' section).

Notably, an interesting recent development is the finding that, compared with their RLA counterparts, RHA rats display a severe disruption of the dimeric metabotropic glutamate-2 (mGlu2)/5- HT2A receptor complex consisting in a dramatic reduction of the cortical, hippocampal and striatal density of mGlu2 receptors, associated with an increase of 5-HT2A receptor density in the frontal cortex (Klein et al., 2014; Fomsgaard et al., 2018). This molecular profile is reminiscent of what has been found in the cortex of drug-free schizophrenic patients (González-Maeso et al., 2008). Accordingly, it has been shown that the dimeric mGlu2/5-HT2A receptor complex is critically involved in the pharmacologic effects of atypical antipsychotics; thus, clozapine does not decrease locomotion in rodents with a disruption of the dimeric mGlu2/5-HT2A receptor complex (González-Maeso et al., 2008; Kurita et al., 2012). Therefore, the alterations observed in this complex in RHA rats may at least partly account for some of the phenotypical schizophrenia-related traits observed in this strain and the almost complete lack of enhancing effect of clozapine on either the baseline or MK-801-impaired PPI performance (Oliveras et al., 2017).

On the other hand, serotonin transmission has also been related also to impulsivity (Dalley and Robbins, 2017). Thus, in RHA rats, but not RLAs, there is a positive correlation (r = 0.94, p = 0.02) between 5-HT2A receptor density and premature responses in the 5-CSRT task (Klein et al., 2014). This is consistent with findings from studies in rodents and humans in which the pharmacological blockade of the 5-HT2A receptor produces reductions of several types of impulsive responses (Klein et al., 2014; Dalley and Robbins, 2017). These findings suggest that 5-HT2A receptors are involved in impulsivity and, more specifically, that they may facilitate at least some forms of impulsive behavior (reviewed by Klein et al., 2014; see also Dalley and Robbins, 2017).

Altogether, the above reviewed behavioral, pharmacological, molecular and neuroanatomical profiles point to RHA rats as a model exhibiting several phenotypical characteristics relevant for impulsivity, sensation seeking, and schizophrenia research. These phenotypes include attentional and cognitive impairments, deficits in social behavior, premature responding and waiting impulsivity, enhanced mesolimbic dopaminergic activity and schizophrenia-like alterations of cortical mGlu2 and 5-HT2A receptors.

Finally, it has recently been shown that, compared with their RLA counterparts, RHA rats exhibit significant reductions of the volumes of the mPFCx and HC [measured by means of magnetic resonance imaging (MRI)] as well as dramatically enlarged lateral ventricles (Río-Álamos et al., 2017a, 2019). In addition, PPI-elicited neural activity of the prefrontal cortex (as measured by c-fos activation) is lower in RHA than RLA rats (Tapias-Espinosa et al., 2019). These neuroanatomical and functional alterations are reminiscent of those observed in schizophrenic patients and provide further support to the contention that RHA rats may constitute a genetic model of schizophrenia-relevant features with face, construct, and predictive validity (reviewed by Río-Álamos et al., 2019).

### THE ROMAN RATS AS A GENETIC MODEL OF VULNERABILITY TO DRUG ADDICTION

A combination of environmental and genetic factors underlies the vulnerability to develop a drug addiction (Nestler, 2001, 2000; Kreek et al., 2005a). Accordingly, interacting with the direct effects of abused drugs, environmental and genetic factors may induce the transition from casual, intermittent drug use to regular intake, the progression from abuse to compulsive drug seeking and taking (i.e., addiction), and the propensity for repeated relapse even after a long-term drug-free state (Kreek et al., 2005a,b; Everitt and Robbins, 2016).

In this section, we examine experimental evidence supporting the view that the Roman lines represent a valid model to investigate how and to what extent the genetic background influences the neural and behavioral traits involved in the major features of the individual vulnerability to addiction, including high sensitivity to the acute effects of drugs of abuse, the development of sensitization upon repeated administration, and the propensity to self-administer addictive drugs.

Early work in the 1980s–90s provided evidence suggesting that the behavioral patterns distinguishing the Roman rat lines may be mediated, at least in part, by differences in the functional properties of the mesolimbic and mesocortical dopaminergic projections. Thus, it was shown that: (i) various stressors activated the mesocortical dopaminergic pathway in RHAs but not RLA rats (D'Angio et al., 1988; Giorgi et al., 2003); (ii) RHA rats showed a faster turnover rate of DA in the caudate nucleus (Driscoll et al., 1990) and displayed more intense stereotypies and yawning than RLA rats upon acute challenge with apomorphine (Driscoll et al., 1985; Gimenez-Llort et al., 2005; Sanna et al., 2013); and (iii) the density of DA D1 receptors in the Acb was higher in RHA vs. RLA rats (Giorgi et al., 1994; Guitart-Masip et al., 2006).

The mesolimbic dopaminergic pathway has long been considered to play a major role in mediating the reinforcing effects of psychostimulants, opiates, nicotine, ethanol, and cannabinoids (Pierce and Kumaresan, 2006). Thus, the abovementioned differences between the Roman lines, together with differences in sensation seeking and impulsive behavior, prompted us to investigate how drugs of abuse affect behavior and central DA function in both rat lines and whether there are differences in addiction liability between them. In this context, and in line with the hypothesis that RHA rats would be more responsive to drugs of abuse than their RLA counterparts, we found that acutely administered morphine and the psychostimulants cocaine or amphetamine elicit larger increments in DA output, as measured by brain microdialysis, in the AcbSh than the AcbCo of RHA rats, whereas such difference is not present in RLA rats (Lecca et al., 2004). Moreover, the three drugs induce a higher increase of locomotor activity in the former line (Giorgi et al., 1997, 2005a,b; Piras et al., 2003; Lecca et al., 2004; reviewed by Giorgi et al., 2007).

The line-related differences in the effects of acute morphine and psychostimulants on the mesolimbic dopaminergic transmission suggested that the Roman lines also may differ in the responsiveness of their neural circuits of reward to other appetitive stimuli. Accordingly, compared with RLA rats, RHAs have been shown to display higher preference for ethanol and larger increases in DA output in the AcbSh after acute ethanol intake (Fernández-Teruel et al., 2002a; Manzo et al., 2012, 2014a; Corda et al., 2014). Furthermore, RHA rats also display higher preference for saccharin solutions (Razafimanalina et al., 1996; Fernández-Teruel et al., 2002a) and palatable food (Giorgi et al., in preparation) and show more intense sexual motivation than their RLA counterparts (Sanna et al., 2017, 2019). Of note, palatable food intake and sexual activity are associated with a larger increment in accumbal DA output in RHA than RLA rats (Giorgi et al., in preparation; Sanna et al., 2017, 2019).

The above findings prompted us to study whether the Roman lines also differ in the propensity to develop behavioral sensitization upon repeated administration of drugs of abuse. Behavioral sensitization is characterized by a progressive increase in the intensity of locomotor activation and stereotypes upon repeated exposure to a constant dose of a drug, such as psychostimulants and opiates, or by a greater response on re-challenge with a lower dose of drug than used in the initial chronic-intermittent exposure (Segal and Kuczenski, 1987; Pierce and Kalivas, 1997; Vanderschuren and Kalivas, 2000). This increased response is believed to reflect long lasting adaptations in neural circuits involved in motivation and reward (Nestler, 2001; Robinson and Berridge, 2001; Everitt and Wolf, 2002; Li et al., 2004). Accordingly, repeated exposure to psychostimulants enhances the incentive motivational properties of addictive drugs, facilitates drug self-administration (Horger et al., 1990; Piazza et al., 1990; Mendrek et al., 1998; Lorrain et al., 2000), and potentiates the responses to conditioned rewards (Shippenberg and Heidbreder, 1995; Taylor and Horger, 1999; for a review, see Giorgi et al., 2007). The locomotor effects observed following the repeated administration of psychostimulants or morphine consistently indicate that the Roman rat lines differ markedly in the susceptibility to develop behavioral sensitization (Piras et al., 2003; Corda et al., 2005; Giorgi et al., 2005a,b, 2007): RHA rats displayed a significant increment in locomotor activity after a challenge dose of each drug given several days/weeks upon completion of the repeated drug treatment whereas no enhanced locomotor activity in response to the challenge drug dose was observed in RLA rats (Piras et al., 2003; Corda et al., 2005; Giorgi et al., 2005a, 2007; Guitart-Masip et al., 2008; Tournier et al., 2013).

Moreover, brain microdialysis assays revealed that the challenge with cocaine, amphetamine or morphine elicited larger increases in DA output in the AcbCo of RHA than RLA rats that had respectively received repeated doses of cocaine, amphetamine or morphine (Piras et al., 2003; Giorgi et al., 2005b, 2007). In parallel, and only in RHA rats, the drug challenges induced an attenuated dopaminergic response in the AcbSh. It is remarkable that the above mentioned repeated drug treatment schedules, which induced behavioral sensitization only in RHA rats, produced concomitant neural adaptations in the AcbCo and AcbSh exclusively in this rat line (Giorgi et al., 2007).

Behavioral and neurochemical sensitization to abused drugs is thought to play a key role in several addiction-related features, such as compulsive drug-seeking and the long-lasting vulnerability to relapse (see Fattore et al., 2009 and references therein). Therefore, we considered of interest to investigate the susceptibility of RHA and RLA rats to develop drug self-administration. To this aim, we assessed the acquisition, maintenance, extinction, reinstatement and reacquisition of intravenous self-administration of cocaine in RHA and RLA rats in an operant FR-1 task (Fattore et al., 2009). It was found that RHA rats displayed substantially more frequent lever-press operant responding than their RLA counterparts during all the phases of cocaine self-administration.

The persistently higher frequency of active lever responding of RHA vs. RLA rats during the maintenance phase of cocaine self-administration could not be attributed to a generalized rate effect in RHA rats since the inactive lever responding was not increased relative to RLA rats. Furthermore, extinction tests were conducted in the absence of cocaine reinforcement, and yet cocaine-seeking responses at the active lever in RHA rats were more frequent relative to RLAs and required almost twice as many test sessions to reach extinction criteria (Fattore et al., 2009).

After 3 weeks of withdrawal, when RHA and RLA rats had extinguished to similar low response levels, non-contingently administered cocaine injections induced significantly more frequent responses at the active lever in RHAs relative to RLA rats, whereas saline injections were without effect in both lines. Collectively, these results suggest that RHA rats are more susceptible than RLAs to relapse to cocaine seeking, whether induced by exposure to the cocaine-associated context (i.e., extinction), or to a priming injection of cocaine (i.e., reinstatement). In addition, these findings support the contention that, in RHA rats, behavioral sensitization may be involved in relapse-related processes (Fattore et al., 2009). Importantly, the AcbCo has been implicated in the reinstatement of cocaine-seeking induced by the administration of a priming dose of the drug or by exposure to drug-related cues (Ito et al., 2004; Hollander and Carelli, 2007). This experimental evidence is consistent with the view that the AcbCo and its afferent glutamatergic projections originating in the mPFCx play a key role in cocaine-induced relapse (McFarland et al., 2003). Accordingly, as mentioned above, the functional tone of the dopaminergic transmission is markedly increased in the AcbCo of RHA rats behaviorally sensitized to cocaine (Giorgi et al., 2007).

Besides the central role of the mesolimbic dopaminergic projection to the Acb, other brain areas and neural circuits may be involved in the susceptibility of RHA rats to develop cocaine seeking and taking upon chronic exposure to the same drug. For instance, a dysfunction of the frontocortical projections to limbic nuclei, which is thought to be a consequence of long-term exposure to drugs of abuse, may lead to a deficient inhibition of inappropriate/irrelevant responses during operant extinction or in the presence of drug-paired stimuli (Jentsch and Taylor, 1999). Accordingly, in RHA rats behaviorally sensitized following chronic treatment with cocaine, but not in their RLA counterparts, the dopaminergic transmission in the mPFCx is markedly reduced (Giorgi et al., in preparation). Furthermore, compared with their RLA counterparts, RHA rats show a smaller frontocortical volume (Roberti, 2004; Río-Álamos et al., 2019) and a reduced activation of the mPFCx upon exposure to PPI testing and a variety of novelty-related situations (Meyza et al., 2009; Río-Álamos et al., 2019; Tapias-Espinosa et al., 2019).

Substance use comorbidity is a frequent event in many psychiatric disorders and is particularly prevalent in schizophrenic populations (Selzer and Lieberman, 1993). Thus, dual diagnosis in schizophrenia is associated with a remarkably high prevalence of cocaine, amphetamine, alcohol, cannabis, and nicotine use (DeQuardo et al., 1994; Buckley, 1998; Dalack et al., 1998; Dixon, 1999).

A widely held neurobiological explanation for substance use comorbidity in schizophrenia is the self-medication hypothesis, which postulates that patients use addictive drugs to relieve aversive disease symptoms or the adverse side effects of medication. Hence, this hypothesis posits that susceptibility to drug abuse is a reaction to the psychotic disorder or medication side effects, and thus represents a secondary symptom (Buckley, 1998; Dalack et al., 1998; Krystal et al., 1999). In contrast, experimental evidence accumulated in the last years supports the view that the pathologic substrate of schizophrenia may contribute to the susceptibility to addiction by facilitating the functional activity of the neural circuitry that mediates positive reinforcement (Chambers et al., 2001). Two key lines of evidence support this primary addiction hypothesis: (i) the putative neuropathology underlying schizophrenia involves alterations in neural circuits that regulate positive reinforcement, incentive motivation, novelty seeking, behavioral inhibition, and addictive behavior as well as schizophrenia-relevant attentional/cognitive impairments (Chambers et al., 2001 and references therein); and (ii) experimental manipulations that model neuropathologic and behavioral aspects of schizophrenia in animals (e.g., the NVHL model, Lipska and Weinberger, 2000) also facilitate positive reinforcement and the incentive motivational effects of rewarding stimuli (Chambers and Self, 2002). An implicit feature of this hypothesis is that both the schizophrenia syndrome and vulnerability to addiction are primary disease symptoms, each directly caused by common neuropathologic substrates.

Most important, the results reviewed herein show that RHA rats include behavioral and neurochemical traits related with both, schizophrenia and addiction. Therefore, RHA rats may represent a promising model of substance use comorbidity with face, construct, and predictive validity.

#### AN INTEGRATED PERSPECTIVE

Collectively, the findings reviewed herein provide a detailed account of the major traits that distinguish RHA from RLA rats, from coping style, impulsivity, and behavioral inhibition

increased density of 5-HT2AR and the markedly decreased density of mGlu2R in RHA vs. RLA rats is consistent with the possibility of a hyper-functional glutamatergic cortical system that would in turn lead to an increased functional tone of the mesolimbic dopaminergic system and a decreased functional tone of the meso-cortical dopaminergic system in RHA rats. Collectivelly, the lowered function of the mPFC, the HC, and the AMY, along with the increased functional tone of the mesolimbic dopaminergic system of RHA rats favor behavioral disinhibition, impulsive actions/responses, attentional/cognitive deficits and vulnerability to drug addiction. Modified from Probst and van Eimeren (2013). Abbreviations: mPFC, medial prefrontal cortex; HC, hippocampus; AMY, amygdala; Acb, nucleus accumbens; VTA, ventral tegmental area; mGlu2R, metabotropic glutamate type 2 receptors; 5-HT2AR, serotonin type 2A receptors; D2R, dopamine type 2 receptors; D1R, dopamine type 1 receptors.

through attention and cognitive ability to novelty seeking, drug seeking, mesocorticolimbic dopaminergic transmission, and schizophrenia-relevant behaviors.

A hypothetical integrative model of the relationships among the main traits of the particular neurobehavioral profiles of the Roman rats is shown in **Figure 1**. We acknowledge that the proposed model is a simplification and does not take into account the role of other neurotransmitters that may be involved in the neurobehavioral traits that distinguish the Roman lines. It is hypothesized that the lower activity and volume of the mPFC, the HC, and the AMY, together with the higher density of 5-HT2A receptors and the dramatically decreased density of mGlu2 receptors in the frontal cortex of RHA vs. RLA rats may determine a hyper-functional glutamatergic cortical system in RHA rats (Meyza et al., 2009; Klein et al., 2014; Río-Álamos et al., 2017a, 2019; Wood et al., 2017; Tapias-Espinosa et al., 2019). This in turn would elicit an increment in the functional tone of the dopaminergic VTA neurons projecting to the Acb (Elert, 2014) and a decrease in the functional tone of the mesocortical dopaminergic system of RHA rats (Dalley et al., 2007; Giorgi et al., 2007; Probst and van Eimeren, 2013; Tournier et al., 2013; Elert, 2014; Klein et al., 2014; Dalley and Robbins, 2017; Fomsgaard et al., 2018). In addition, the decreased density of DA D2 receptors together with the increased density in DA D1 receptors in the limbic system of RHAs relative to RLA rats (Giorgi et al., 1994, 2007; Guitart-Masip et al., 2006) may also contribute to the more robust functional tone of the mesolimbic dopaminergic system of the former line. It is proposed that this integrated neural circuitry underlies novelty seeking (Tournier et al., 2013), impaired impulse control (Dalley et al., 2007), vulnerability to drug sensitization and abuse (Giorgi et al., 2007; Fattore et al., 2009; Tournier et al., 2013) and schizophreniarelevant attentional/cognitive impairments (Wakabayashi et al., 2015; Río-Álamos et al., 2019; Tapias-Espinosa et al., 2019; for a review see Swerdlow and Light, 2016). Hence, compared

#### REFERENCES


with RLA and heterogeneous/outbred rats, RHAs appear to be unique in that they include all the phenotypes mentioned above, and this makes this rat strain a promising heuristic tool to investigate relationships among these traits and their underlying neurobiological/genetic bases.

Finally, according to the primary addiction hypothesis, the pathophysiologic underpinnings of schizophrenia facilitate the vulnerability to substance use disorder by potentiating the functional activity of the neural circuitry that mediates positive reinforcement thereby leading to the high prevalence of substance use comorbidity in schizophrenics. Hence, the distinct neurobehavioral profile of RHA rats makes this strain a valid model of dual diagnosis schizophrenia.

#### AUTHOR CONTRIBUTIONS

All authors contributed to the bibliographic research, were involved in writing the manuscript, and approved its final version.

#### FUNDING

The authors and part of the work reviewed herein are supported, or have received support from grants ARS (Autonomous Region of Sardinia, L.R. 7/2007, Project Code No. CRP-59842) to OG and PSI2017-82257-P to AF-T, OG and MC. AF-T is the recipient of the ''ICREA-Academia 2013'' prize.

#### ACKNOWLEDGMENTS

The authors wish to thank Dr Peter Driscoll and all the co-workers quoted in the list of references for their invaluable and continuing help throughout the long Roman rat story. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.


Roman rat lines. Physiol. Behav. 51, 1235–1241. doi: 10.1016/0031-9384(92) 90314-r


behavior among the NIH heterogeneous stock and the Roman high- and low-avoidance rats. Neurosci. Res. 77, 187–201. doi: 10.1016/j.neures.2013.09. 012


the 5-HT2A promoter region in the roman low- (RLA-I) and High- (RHA-I) avoidance rat strains. Mol. Neurobiol. 55, 1998–2012. doi: 10.1007/s12035-017- 0457-y


low-avoidance rats. Neuroreport 14, 2433–2438. doi: 10.1097/00001756- 200312190-00029


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Giorgi, Corda and Fernández-Teruel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Bed Nucleus of the Stria Terminalis, Homeostatic Satiety, and Compulsions: What Can We Learn From Polydipsia?

Tomek J. Banasikowski1,2 and Emily R. Hawken1,2 \*

<sup>1</sup> Department of Psychiatry, Queen's University, Kingston, ON, Canada, <sup>2</sup> Providence Care Hospital, Kingston, ON, Canada

A compulsive phenotype characterizes several neuropsychiatric illnesses – including but not limited to – schizophrenia and obsessive compulsive disorder. Because of its perceived etiological heterogeneity, it is challenging to disentangle the specific neurophysiology that precipitates compulsive behaving. Using polydipsia (or nonregulatory water drinking), we describe candidate neural substrates of compulsivity. We further postulate that aberrant neuroplasticity within cortically projecting structures [i.e., the bed nucleus of the stria terminalis (BNST)] and circuits that encode homeostatic emotions (thirst, hunger, satiety, etc.) underlie compulsive drinking. By transducing an inaccurate signal that fails to represent true homeostatic state, cortical structures cannot select appropriate and adaptive actions. Additionally, augmented dopamine (DA) reactivity in striatal projections to and from the frontal cortex contribute to aberrant homeostatic signal propagation that ultimately biases cortex-dependent behavioral selection. Responding becomes rigid and corresponds with both erroneous, inflexible encoding in both bottom-up structures and in top-down pathways. How aberrant neuroplasticity in circuits that encode homeostatic emotion result in the genesis and maintenance of compulsive behaviors needs further investigation.

Keywords: obsessive compulsive disorder, schizophrenia, dopamine, BNST, orbitofrontal cortex

#### INTRODUCTION

Compulsivity can be a dominant and debilitating clinical feature of several psychiatric conditions including obsessive-compulsive disorder (OCD), schizophrenia, substance abuse, and other obsessive-compulsive spectrum disorders (OCSD) (Hariprasad et al., 1980; Berman et al., 1995; Everitt and Robbins, 2005). Compulsions are repetitive, apparent, and purposeful behaviors that are performed according to certain rules or in a stereotyped fashion. When expressed, compulsions are time-consuming, cause significant distress, and interfere in both function and quality of daily life (American Psychiatric Association, 2013). Despite an emergent body of research, identifying the neurophysiology of compulsivity is challenging and the underlying neural mechanisms of compulsivity remain speculative.

Behavioral heterogeneity and symptom dimensionality observed across compulsive spectrum disorders complicates the search for a neurological trace of compulsivity. For instance, the "classical" compulsive behaviors seen in OCD are characterized by excessive repetition of

#### Edited by:

Trevor W. Robbins, University of Cambridge, United Kingdom

#### Reviewed by:

Ingo Willuhn, Netherlands Institute for Neuroscience (KNAW), Netherlands Tara Arbab, Netherlands Institute for Neuroscience, Netherlands, in collaboration with reviewer IW Marco Venniro, National Institute on Drug Abuse (NIDA), United States

#### \*Correspondence:

Emily R. Hawken hawken@queensu.ca; erhawken@gmail.com

Received: 06 March 2019 Accepted: 12 July 2019 Published: 01 August 2019

#### Citation:

Banasikowski TJ and Hawken ER (2019) The Bed Nucleus of the Stria Terminalis, Homeostatic Satiety, and Compulsions: What Can We Learn From Polydipsia? Front. Behav. Neurosci. 13:170. doi: 10.3389/fnbeh.2019.00170

intentional "normal" behaviors and/or mental acts in an attempt to soothe discomfort brought on by obsessions, the accompanying diagnostic feature of OCD. However, not all observed compulsions captured by this definition are associated with an OCD diagnosis. Primary polydipsia, or excessive, nonregulatory water drinking, is just one example of the clinical heterogeneity found in OCSD, a behavior that shares some of its core features with other psychiatric diagnoses including schizophrenia. By isolating and identifying common neurological substrates across diagnoses marked by compulsive behaving (including primary polydipsia associated with schizophrenia and OCD) we can highlight unique neurological features specific to compulsivity in psychiatric illness.

To further characterize the neuropathophysiology of psychiatric disease states, we need appropriate animal models. Some preclinical models of compulsivity have been enormously useful in distilling discrete mechanisms and neural representations of pathological behavior (Everitt and Robbins, 2005; Szechtman et al., 2017). Among the currently available animal models, schedule-induced polydipsia (SIP) is recognized as the most robust and replicable preclinical model of compulsivity (Woods et al., 1993; Moreno and Flores, 2012; Gardner Gregory, 2018). Excessive water drinking (polydipsia) occurs experimentally when hungry animals are exposed to intermittent/scheduled access to food and unlimited access to water (see **Figure 1**). In this protocol, some animals will drink themselves into a water intoxicated state mimicking primary polydipsia. SIP is an ideal animal model to study compulsivity specifically because the behavior is both ethological and ecological and can be induced across species, including humans (Schuster and Woods, 1966; Falk, 1969; Kachanoff et al., 1973; Dale, 1979).

Decades of research using the SIP protocol enables a full exploration of the hypothesis presented here, that disordered neuroplasticity underlying essential homeostatic emotions within limbic and cortical structures contributes to the compulsive phenotype observed in polydipsia. Because compulsive drinking (expressed both as SIP in an OCD-like animal model and also as primary polydipsia associated with schizophrenia) can be induced in humans as well as rodents, we further speculate that identified neural substrates [including the bed nucleus of the stria terminalis (BNST)] can apply to psychiatric populations, i.e., those with OCSD. Thus, herein, we synthesize the current understanding of the neurophysiology of compulsivity through the lens of polydipsia.

#### HOMEOSTATIC EMOTIONS DRIVE APPROPRIATE ACTION SELECTION FOR ADAPTIVE BEHAVING

Observationally, polydipsia is simply the overconsumption of fluid, and most often, water. Polydipsia can occur for several physiological reasons, for instance, in response to exaggerated thirst that results from disease states like diabetes insipidus. However, primary polydipsia or "psychogenic" polydipsia is nonphysiologic and non-regulatory in nature, i.e., no underlying physiological condition can explain the individual's need to overdrink. Polydipsia is maladaptive as over-drinking individuals seek and consume water despite the risk of severe life-threatening consequences (Hawken et al., 2009). Additionally, psychogenic polydipsia most commonly co-occurs with disordered thinking associated with a psychotic state. While some patients report drinking in response to excessive thirst, more report drinking excessively to feel better, to cleanse or purify themselves, or to appease/soothe other obsessions and/or delusions (Illowsky and Kirch, 1988; Millson et al., 1992). Observationally, in overdrinking patients normal thirst and satiety signals are no longer tied to homeostatic fluid regulation.

Maintaining adequate hydration is essential to an animal's survival. Although most of our everyday drinking behavior is social, prandial, or habitual in nature and not in response to thirst, multiple fail-safe systems are in place to ensure that fluids lost are quickly replenished (McKinley et al., 2019). One such system is our brain's ability to accurately recognize the feeling or emotion of "thirst" and behave appropriately (drink) to satiation. Most studies regulating thirst indicate that normal drinking behavior is generally preparatory, and to a lesser extent consummatory, in nature (Konorski, 1967; Egan et al., 2003; Saker et al., 2016). In other words, the brain quickly adjusts behavior in anticipation of falling dangerously far from homeostasis by initiating actions to maintain or restore fluid balance. Within the preparatory/consummatory framework, normal ingestive drinking behavior is therefore tightly regulated by a combination of incentive and homeostatic cues making the drive to drink both introceptively driven yet also perpetuated by anxiogenic situations that are extroceptively mediated. Drinking when thirsty is both pleasurable and rewarding, and extinction of thirst activates dopamine (DA) pathways in the brain (Ettenberg and Camp, 1986). Thus, many of the same brain regions that are activated in decision making, reward, stress, fear, and anxiety also mediate water seeking and intake (Egan et al., 2003; Shin and Liberzon, 2010). Such a coordinated response by multiple interconnected circuits across the brain ensures specific goaldirected behaviors are engaged to maintain fluid homeostasis to promote survival (Allen et al., 2019).

Being entirely subjective, thirst can be categorized along with the undeniable physiological need to eat, breath, and sleep as an essential homeostatic "emotion." This primal and essential emotion arises from interoceptive cues compiled by sensory circumventricular organs that sense dramatic shifts in water and energy homeostasis (McKinley et al., 2019; Zimmerman et al., 2019). From these interfaces, interoceptive/homeostatic information is projected onto the cortex though polysynaptic relays including structures in the limbic system like the BNST (McKinley et al., 2019). Cortical structures decode the ascending homeostatic information to motivate appropriate behavior (to drink) and restore physiological thresholds, or quench one's thirst (Craig, 2003). Interestingly, satiety or satiation of thirst is likely more than a lack of thirst or removal of a thirsty state, as distinct cortical regions are activated in each condition (McKinley et al., 2019). Allen et al. (2019) recently reported thirsty and sated states produce separate but proximal patterns of neuronal activation across several structures in the rodent

brain. Such a brain-wide and independent representation of drinking behavior likely reflects the fact that thirst-satiety signals are evolutionarily necessary to prevent the threat of excessive hydration. Indeed, the satiety signal is so important that swallowing water is perceived first before any systemic absorption of water (McKinley, 2009). Thus satiety, like thirst, is itself an essential homeostatic emotion, with both playing pivotal roles in preventing the dangerous physiological consequences of unregulated non-homeostatic drinking.

While thirst and satiety are likely represented in various iterations across the brain, imaging studies in humans infused intravenously with hypertonic saline to induce thirst identify key structures involved in the conscious detection or "feeling" of thirst. Both positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) show that the anterior cingulate (ACC) and insular cortex are activated in subjects made thirsty, and that satiation of thirst by drinking quickly extinguished that activity (Denton et al., 1999a,b; de Araujo et al., 2003; Egan et al., 2003). Upon wetting the mouth, activation of orbitofrontal cortex (OFC) occurs, a feature also extinguished with satiety. If subjects are asked to overdrink, a reactivation of insular cortex is seen, along with activation of the amygdala and periaqueductal gray regions – regions implicated in the "valence" networks and associated emotions including stress, fear, and anxiety (Saker et al., 2016). Despite widespread activity in several cortical and other brain regions correlating with thirst scores and subsequent satiety, conscious feelings of "thirst" and "satiety" and other homeostatic emotions arise from a yet to be identified aggregate or pattern of circuit/network activity (Allen et al., 2019).

Nevertheless, cortical regions including the ACC, insular, and OFC cortices have been implicated in many homeostatic emotions beyond thirst, including hunger and deep pain, reflecting an adverse condition within the body that requires immediate behavioral response (Craig, 2003). Generally, the ACC is activated by occurrences of adverse conditions that require decision making as to appropriate strategic responding (Gehring and Taylor, 2004). Accordingly, lesions of the ACC in humans, primates, and rats results in discernable apathy, or indifference and a lack of motivation to rectify an adverse condition (Eslinger and Damasio, 1985; Johansen et al., 2001). Conversely, a role for the insular cortex may be to identify

the specific homeostatic perturbation and tag it with the appropriate emotional label – for instance, the insular cortex detects dehydration and translates it as thirst. Lastly the OFC, a cortical structure interconnected with ACC and insula, provides the hedonic component to the behavioral response of drinking to signal that water in the mouth is pleasant if thirsty but less so when not thirsty. Thus, sensory signals conveying the homeostatic state of the individual will reach the OFC, from the bottom-up, and be interpreted according to their biological significance to dictate appropriate action selection. In this proposed schema (McKinley, 2009), the cortical generation of the emotion of thirst would involve activations of neurons within the ACC for motivational intensity (urgency), the insula for homeostatic specificity (e.g., thirst, hunger, and satiety), and the orbito-frontal region for decision to initiate or maintain fluid seeking and consummatory behavior. However, as a clear neural "seat" of homeostatic emotions remains unknown, we can only speculate how homeostatic signals stop eliciting appropriate and adaptive behavioral repertoires in polydipsia.

### PSYCHOGENIC POLYDIPSIA IN SCHIZOPHRENIA

In primary or "psychogenic" polydipsia, homeostatic emotions somehow become disassociated from appropriate behavioral selection. A person with polydipsia may either feel thirsty but is unable to encode satiety to stop drinking behavior or is not drinking in response to thirst in the first place. Whether compulsive over-drinking results from an altered perception of thirst and/or satiety is unconfirmed (Goldman et al., 1996). However, in response to infusion of hypertonic saline, thirst ratings in compulsive water drinkers are higher than in normal drinkers, and remain elevated following drinking episodes. Conversely, drinking rapidly abolishes thirst emotion in noncompulsive drinkers (Thompson et al., 1991). To understand how or when a disconnect between homeostatic emotional drivestates and goal-directed behaviors occur, we must examine the context in which polydipsic behavior manifests.

The significant incidence of psychogenic polydipsia in patients with severe chronic psychiatric illness (de Leon et al., 1994) highlights central traits key to resolving the neurophysiology of compulsive overdrinking. While not exclusive to mental illness (Stuart et al., 1980), polydipsia is detected in over 20% of longterm institutionalized patients with a diagnosis of schizophrenia (Blum et al., 1983; Kirch et al., 1985; de Leon et al., 1996; Mercier-Guidez and Loas, 2000). Additionally, patients with polydipsia associated with schizophrenia are also more likely to be male and to compulsively smoke (de Leon et al., 1994; Mercier-Guidez and Loas, 2000). Furthermore, episodes of polydipsiainduced water intoxication in schizophrenia are correlated with a greater severity of psychotic illness, characterized by earlier onset of psychosis, severe symptom breakthrough in periods of illness stability, higher rates of alcohol abuse, lower levels of global functioning, and longer institutionalized living (Poirier et al., 2010). From a mechanistic perspective, severe polydipsia may represent an identifiable subgroup that exhibits more neurological impairments than those without compulsive drinking tendencies (Illowsky and Kirch, 1988).

Neurodegenerative processes may underlie some of the illness progression observed in schizophrenia (Lieberman, 1999) and may contribute to the incidence of polydipsia in this population. In first episode schizophrenia, structural alterations are present indicating whole brain and hippocampal volume reductions (Steen et al., 2006; Vita et al., 2006). Importantly, some structural changes progress in a subgroup of patients during the course of illness suggesting that polydipsia could be a behavioral expression of neurodegeneration (DeLisi et al., 2004). In support of this hypothesis, primary polydipsia is significantly associated with a chronic course of psychotic illness (de Leon et al., 1996). Accordingly, when polydipsia develops it typically onsets 5–15 years following the original psychiatric diagnosis (Vieweg V. et al., 1984; Vieweg W.V. et al., 1984). With disease progression in schizophrenia, stereotypical behaviors also become increasing common and can include compulsive smoking, odd grooming patterns, pacing, and other repetitive motor actions (Arieti, 1974; Luchins et al., 1992; Alexander et al., 1993; Tracy et al., 1996; Morrens et al., 2006). Polydipsia is typically clustered into this stereotypical or "bizarre behaviors" category. Thus, some posit that stereotypies and other ritualistic mannerisms (including polydipsia) co-occurring in schizophrenia constitute a coherent group of symptoms that are mediated by common (across diagnoses) neurological abnormalities (Luchins, 1990).

### POLYDIPSIA, SCHIZOPHRENIA, AND OCD: IS THERE NEUROBIOLOGICAL OVERLAP?

In part due to polydipsia's temporal association with other repetitive behaviors (grooming, pacing, and ruminating), some suggest that polydipsia is related to the specific stereotypies and compulsions observed in OCSD or OCD itself (Deas-Nesmith and Brewerton, 1992; Shutty et al., 1995; Subramanian et al., 2017). However, the incidence of co-occurring polydipsia in OCD is far less evident than the association of polydipsia with schizophrenia. Nevertheless, rates of obsessive-compulsive symptoms in psychotic illness vary greatly and are estimated to range between 2.5 and 64% (de Haan et al., 2013; Grover et al., 2017). Differential diagnoses pose challenges for both clinicians and investigators to distinguish between compulsions and stereotypic behavior (Berrios, 1989). Incidentally, a substantial proportion (up to 37.5%) of those with schizophrenia also have a diagnosis of OCD (Lysaker et al., 2000; Nechmad et al., 2003; de Haan et al., 2013; Grover et al., 2017). The high incidence rate of schizophrenia with concurrent OCD symptomology resulted in the formulation of a new clinical entity for the dual diagnosis termed "schizo-obsessive disorder" (Hwang and Opler, 1994; Scotti-Muzzi and Saide, 2017). Therefore, comparing the neurophysiology of compulsions associated with schizophrenia with OCD could identify common disease-specific mechanisms of compulsive behavior.

Indirect lines of evidence from imaging studies support a loss of brain matter across brain structures common to

TABLE 1 | Gray and white matter volume changes reported across the brain in polydipsia associated with schizophrenia, schizophrenia, and obsessive compulsive disorder (OCD).


NC, no change. <sup>∗</sup>Negative correlation with illness symptoms.

polydipsia, schizophrenia, and OCD (**Table 1**). While limited, imaging studies of patients with polydipsia associated with schizophrenia show a global loss of brain matter suggested by the presence of significantly enlarged ventricles as compared to those without polydipsia (Emsley et al., 1995; Leadbetter et al., 1999). While schizophrenia itself is not associated with ventricular enlargement (Ebdrup et al., 2010), individuals with OCD and poor insight also exhibit bilateral ventricular enlargement (Luxenberg et al., 1988; Okasha et al., 2000). Gray and white matter loss throughout fronto-cortico-striatal and limbic regions known to be dysfunctional in both schizophrenia and OCD could account for the observed ventricular enlargement (Harrison et al., 2009; Fornito et al., 2013; Jung et al., 2013; Huang et al., 2018).

In line with a view that reduced functional neuroplasticity in key networks contributes to OCSD, structural volume loss across fronto-cortico-striatal loops is consistently reported in both schizophrenia and OCD. In schizophrenia, reduced OFC and ACC volumes are linked with greater severity of formal thought disorder, low levels of insight, and a longer duration of the illness (Goldstein et al., 1999; Gur et al., 2000; Wilke et al., 2001; Shapleske et al., 2002; Kawasaki et al., 2004; Riffkin et al., 2005; Shad et al., 2006, Shad et al., 2007; Tregellas et al., 2007; Bellani and Brambilla, 2008; Nakamura et al., 2008; Schobel et al., 2009a). Resting state connectivity studies also correlate checking behavior in OCD with alterations in frontal regions, specifically the OFC and ACC (Harrison et al., 2013). Patients with polydipsia associated with schizophrenia additionally exhibit volume reductions in the left insular cortex (Nagashima et al., 2012) as do individuals with OCD and exaggerated checking compulsions (Pujol et al., 2004; van den Heuvel et al., 2009; Alvarenga et al., 2012). In striatal structures, reduced volumes are reported in the caudate nucleus in both treatment naïve, first-episode patients with schizophrenia and in OCD (Pujol et al., 2004; Crespo-Facorro et al., 2007; Atmaca et al., 2008; Rotge et al., 2009; van den Heuvel et al., 2009; Ebdrup et al., 2010). Additionally, in treatment refractory OCD, amygdalar volumes are negatively correlated with symptom "harm/checking" scores, illness duration, and symptom-severity (Pujol et al., 2004; Atmaca et al., 2008; Boedhoe et al., 2017; Fouche et al., 2017; Reess et al., 2018). Thus, reduced functional capacity of fronto-cortico-striatal and limbic structures likely contribute to the development of specific maladaptive behaviors associated with each disease state. However, how compulsive drinking (polydipsia) reflects deficits in neuroplasticity within pathologically common structures cannot be determined from available human imaging studies. Further insight into the neural plastic mechanisms regulating polydipsia can be gained from pre-clinical animal models of compulsive drinking.

### DOPAMINE DRIVES POLYDIPSIA IN HUMANS AND RODENTS

Cumulative evidence suggests that schedule-induced polydipsia (SIP) is a valid animal model of compulsive water drinking (Woods et al., 1993; Moreno and Flores, 2012; Gardner Gregory, 2018). Good disease models show a similar pattern of symptoms to the disease being modeled, have measures that can be objectively quantified, and are both reproducible and robust (Geyer and Markou, 1994). In rodents, excessive drinking in the SIP protocol was first observed (Falk, 1961) when a hungry animal consumed excessive amounts of water during predictable but intermittent food access (i.e., scheduled; **Figure 1**). Like primary polydipsia, compulsive drinking induced

by the SIP protocol is non-homeostatic in that animals engage in drinking behavior in the face of excessive overhydration (Falk, 1971). Like compulsions, SIP behavior is excessive, ritualistic, and maladaptive as animals can drink themselves into a dilutional water-toxic state that results in death (Hawken et al., 2013b). Furthermore, not all psychiatric patients nor all animals exposed to the SIP paradigm develop compulsive drinking. Such face validity suggests SIP is a powerful pre-clinical tool to help us understand how neurological, environmental, and genetic factors trigger or contribute to this compulsive phenotype (Rosenzweig-Lipson et al., 2007; Toscano et al., 2008; Moreno and Flores, 2012).

As the compulsion to overdrink in humans is postulated to be a behavioral manifestation of schizophrenia, some of the discrete neurobiology underlying the expression of polydipsia can be inferred from the neuropathology of schizophrenia itself. Currently, the etiology of schizophrenia continues to be debated, although it is generally conceptualized as a complex neurodevelopmental disorder with features of neurodegeneration and risk factors (both genetic and environmental) predicting its onset (Lieberman, 1999; Lewis and Lieberman, 2000; Tandon et al., 2008a,b). Historically, increased striatal DA function and impaired prefrontal cortical activity are the two most robust neuropathological features associated with schizophrenia's core positive, negative, and cognitive symptoms (Lewis and Anderson, 1995; Laruelle and Abi-Dargham, 1999). Accordingly, DA is identified as one of the key neuromodulators dysregulated in schizophrenia.

In its original inception, the DA hypothesis proposed that schizophrenia results from DA overactivity within subcortical striatal structures (van Rossum, 1966). Evidence for hyperdopaminergia in schizophrenia was initially observed in chronic users of psychostimulants, where excessive use of DA-augmenting drugs induced psychotic states (Bell, 1965; Angrist and Gershon, 1970). Concurrently, antipsychotic drugs developed to alleviate psychosis did so through selectively blocking the DA D2 receptors (Seeman et al., 1975). Much later, increased dopaminergic activity in schizophrenia was confirmed by imaging studies. In patients with schizophrenia, displacement of DA at D2 receptors by amphetamine (AMPH) induced DA release is increased in the striatum (Laruelle et al., 1996; Abi-Dargham et al., 1998; Laruelle and Abi-Dargham, 1999). Additionally, estimated striatal DA release following AMPH correlates with the severity of AMPH-exacerbated psychotic symptoms (Laruelle et al., 1999). However, striatal hyperdopaminergia fails to fully explain the negative and cognitive symptoms that also characterize schizophrenia. Thus, in a revised DA hypothesis, hyperdopaminergia, or increased reactivity of the mesolimbic DA system, is now primarily linked to the positive, psychotic features of schizophrenia, i.e., hallucinations and delusions (Carlsson, 1974) while a hypodopaminergia of the PFC may underlie observed cognitive dysfunction (Slifstein et al., 2015).

Psychogenic polydipsia in schizophrenia is temporally tied to psychosis, arguably a state of striatal hyperdopaminergia, where patients' increased drinking behavior parallels psychotic exacerbation and is normalized when psychosis remits (Raskind et al., 1975; Zubenko et al., 1984). This suggests a role for elevated striatal DA function in primary polydipsia associated with schizophrenia. In animals, an ideally fluctuating amount of striatal DA is necessary for compulsive drinking in SIP to develop. Fixed, short intervals of food presentation to a food-restricted animal stimulates necessary DA activity (Cousins et al., 1999) with more DA released in the nucleus accumbens (NAc) following food consumption when animals are hungry versus when they are sated (Ahn and Phillips, 1999). In a hungry rat, DA levels increase in the NAc along-side elevated drinking over the course of several SIP trials (Hooks et al., 1994; Weissenborn et al., 1996; Moreno and Flores, 2012). This implies that an appropriately reactive DA system is necessary for SIP. Disrupting the integrity of the short-latency DA signal with acutely administered DA agonists and antagonists prior to each SIP session consistently prevents SIP development (see **Table 2**) (Mittleman et al., 1994; Flores and Pellón, 1995; Escher et al., 2006; López-Grancha et al., 2008). Furthermore, increased D2-like and decreased D1-like receptor binding throughout the NAc, medial prefrontal cortex, amygdala, and the ventral tagmental area (VTA) are associated with SIP expression (Pellón et al., 2011). Therefore, increased dopaminergic tone and/or reactivity throughout the mesolimbic and mesocortical networks may lead to aberrant reward-related learning (Banasikowski et al., 2012) that facilitates the compulsive water consumption as seen in primary polydipsia.

By inducing subtle but functionally significant augmentation of the DA system, compulsive drinking can be enhanced. We assessed the ability of amphetamine sensitization (subchronic AMPH), NMDA hypofunction (subchronic MK-801), and social isolation (from weaning) models of schizophrenia-like symptoms to augment striatal DA and "mimic" a schizophrenia-like state in rodents. Evidence suggests that repeated AMPH and MK-801 permanently increases DA transmission along the rat VTA-NAc pathway (Hall et al., 1998; Jentsch et al., 1998) (for review see Svensson, 2000; Lodge and Grace, 2008, 2012; Beninger et al., 2009). Chronic exposure to NMDA induces a loss of GABAergic transmission to disinhibit DA neuron population activity of midbrain DA neurons (Floresco et al., 2003; Lodge and Grace, 2006). Cortical parvalbumin-positive GABA interneurons affected by chronic exposure to NMDA receptor antagonists are similarly affected by isolation rearing, a developmental model of schizophrenia (Jones et al., 1990; Wilkinson et al., 1994; Hall et al., 1998; Heidbreder et al., 2000; Miura et al., 2002; Fabricius et al., 2010, 2011; Han et al., 2011; Powell et al., 2012). As predicted, all models that increase striatal DA reactivity also significantly increased compulsive water drinking expressed as SIP (Hawken et al., 2011, 2013a; Hawken and Beninger, 2014).

Collectively, both animal and human data suggest that those with augmented, but intact, DA systems are prone to develop compulsive polydipsic behavior. In SIP, the temporal contiguity of the DA signal evoked by food stimuli precipitates a nonspecific locomotor/approach behavior to an available drinking spout and facilitates acquisition of SIP (Jacobs and Farel, 1971; Vaccarino et al., 1989; Blackburn et al., 1992; Wise, 2004; Alcaro and Panksepp, 2011). Rats with a "compulsive phenotype" may have a more reactive DA system (e.g., a larger food-evoked

TABLE 2 | Interventions and effects on compulsive behaving induced by the schedule-induced polydipsia (SIP) protocol.


Frontiers in Behavioral Neuroscience | www.frontiersin.org

#### TABLE 2 | Continued


#### TABLE 2 | Continued


#### TABLE 2 | Continued


DA signal and subsequent locomotor activation) and thus an increased sensitivity to the "activating" properties of food and other stimuli associated with schedules inside and outside the SIP-protocol. While overall homecage drinking may not differ between rats with SIP and those without, compulsive phenotypes may demonstrate a pattern of adjunctive drinking to any predictable (scheduled) environmental cues (Hawken et al., 2013b). In schizophrenia and particularly during psychotic episodes, individuals experience an exaggerated striatal DA response when presented with reinforcing stimuli, like meals, initiating generalized behavior and possibly drinking, if the appropriate stimuli (access to water) are available. Thus, the acutely psychotic and hospitalized individual could be prone to stereotypical/ritualistic behaviors due to augmented DA reactivity and the scheduled routines of institutionalized living. How DA specifically modulates circuits and systems central to OCSD to generate compulsive actions, however, must be studied through investigating SIP as an animal model of OCD.

### SCHEDULE-INDUCED POLYDIPSIA AS A MODEL OF OBSESSIVE COMPULSIVE DISORDER

Schedule-induced polydipsia is recognized as a valid model of compulsive behaving (i.e., OCD) in part due to the ability of serotonin reuptake inhibitors (SRIs) to disrupt SIP development and expression (**Table 2**). The ameliorative effects of SRIs on OCD symptoms were originally sufficient to assume serotonin (5-HT) dysfunction to be at the neurophysiological core of OCD symptoms (DeVeaugh-Geiss et al., 1989) for review see, (Barr et al., 1992). However, SRI monotherapy often fails (McDougle et al., 1994) and a subset of patients treatment-refractory to SRIs have benefited from adjunctive therapies that include DA antagonists (e.g., antipsychotics; for review see Koo et al., 2010).

Evidence to support a role for DA in OCD and other compulsive behaviors is reported. Notably, pharmacologically increasing DA neurotransmission with DA agonists exacerbates

compulsivity traits and behaviors in both animal models (Szechtman et al., 1998) and susceptible humans (Frye and Arnold, 1981; Rosse et al., 1993; Kotsopoulos and Spivak, 2001). Evidence for modified synaptic DA activity (via dopamine transporter binding) in the striatum is also found in patients with OCD (van der Wee et al., 2004; Hesse et al., 2005). In SIP-prone rats, differences in the binding affinity of DA D1-like/D2-like receptors exist in limbic and cortical circuits (amygdala, VTA, NAc, and medial prefrontal cortex [mPFC]) (Pellón et al., 2011). Furthermore, pharmacological pretreatment with either DA agonists/antagonists, serotonergic or monoamine modulators, reduce SIP behaviors (i.e., drinking and licking; see **Table 2**) (Moreno and Flores, 2012; Navarro et al., 2015; Rodriguez et al., 2017; Sukhanov et al., 2019). High drinking rats also show elevated serotonergic activity in the amygdala (Moreno et al., 2012) and reduced 5-HT2A receptor binding in the mPFC (Mora et al., 2018) with additional evidence for changes in DA activity in the PFC, NAc, and amygdala (Moreno et al., 2012). Evidence supports a clear role for 5-HT, DA, and potentially 5-HT-DA interactions in the pathophysiology of compulsive phenotypes.

The DA hypothesis in OCSD is based on region-specific DA dysfunction within cortico-limbic-striato-thalamic-cortical (CLSTC) loops (Modell et al., 1989). For instance, enhanced or attenuated DA reactivity in some neurocircuits may change the weight of circuits to bias behavior toward habitual and compulsive responding (Nelson and Killcross, 2006; Ott and Nieder, 2019). Habits result from behavior performed frequently with an unchanging outcome and once established, are generally less flexible to future changes in predicted/expected outcomes (Balleine and Dickinson, 1998; Gillan et al., 2014). The medial parts of the striatum are necessary for goal-directed, outcomesensitive behaviors but as responding becomes habitual, neuronal control gradually shifts to more lateral parts of the striatum critical for behavioral habits (Yin et al., 2004, 2006). In these regions, DA modulates the acquisition of stimulus-outcome, action-outcome and stimulus-response associations (Yin et al., 2004, 2006; Clarke et al., 2008; Xue et al., 2008; Belin et al., 2009). Flexible and appropriate behavioral responses or "cognitive control" via the frontal cortex is also heavily influenced by dopaminergic neuromodulation (Ott and Nieder, 2019). Thus, aberrant DA functioning in striatal and cortico-striatal loops that include the OFC and mPFC are believed to promote maladaptive or excessive habit formation like that observed in psychiatric disorders (Robbins et al., 2012; Gillan et al., 2016).

Accordingly, a dysfunctional balance between goal-directed and habit behavior with an over reliance on habitual circuitry may precipitate the compulsive phenotype (Gillan et al., 2011; Gremel et al., 2016). Technically, SIP is classified as an "adjunctive" behavior along with other behaviors reliably produced by schedules (for reviews see Flory, 1971; Roper, 1978; Singer et al., 1982). Adjunctive or displacement behaviors are considered a separate behavioral class outside of those produced by classical operant or Pavlovian learning paradigms, and are incentive in nature (Breland and Breland, 1961). Attempts to re-classify SIP as partially goal-directed in nature highlights the elements of the behavior that overlap with operant mechanisms (Killeen and Pellón, 2013). SIP expression is contingent on multiple pairings of salient stimuli in a defined context (Falk, 1966, 1967; Flory, 1971; Lashley and Rosellini, 1980) suggesting acquisition and expression of excessive drinking engages classical learning processes (Hawken and Beninger, 2014). Incidentally, initial SIP expression is likely the result of normal goal-directed learning that over time, can precede an over reliance on habitual responding observed in compulsivity (Hawken and Beninger, 2014) (for review see Gillan and Robbins, 2014).

The most commonly reported brain abnormality in OCD and OCSD is dysregulation of the neural feedback circuit that involves both goal-directed and habit circuits, including frontal, limbic, and striatal structures (Saxena et al., 1998; Graybiel and Rauch, 2000; Kopell and Greenberg, 2008). As in humans, compulsive-like animals may have an inherent predisposition to form habits and compulsive behavior due to imbalanced striatal circuits (Graybiel and Rauch, 2000; Gillan et al., 2011). In SIP, we demonstrated that animals who use predominately striatal-learning strategies (those that rely on the integrity of the striatum – specifically the DLS) (Packard and McGaugh, 1996) to learn the location of a food pellet in a Y-maze drink significantly more water when subsequently exposed to the SIP paradigm (Gregory et al., 2015). In addition, spine density in DLS neurons was found to increase following SIP demonstrating that plasticity in brain regions central to habit formation may contribute to SIP (Íbias et al., 2015). Furthermore, early in SIP training we found significant increases in neuronal activation in the mPFC and OFC regions. The increase in immediate early gene (IEG) FosB/1FosB was most pronounced in animals that demonstrated both SIP and striatal-learning tendencies. Repeated activation of OFC-ventral striatal pathway in mice produced grooming sensitization over days further supporting cortical-striatal involvement in compulsive behaving (Ahmari et al., 2013). Additionally, SIP-prone rats exhibited more c-Fos activity in the OFC and basolateral amygdala compared to non-compulsive drinking animals (Merchán et al., 2018). This is in line with human studies that show increased metabolic activity of the striatum and OFC in OCD patients and during symptom provocation (Breiter and Rauch, 1996; Saxena et al., 1999). However, the identity of the discrete mechanism(s) in the OFC/subcortical regions responsible for selecting appropriate behavior via goal-oriented or habit circuit recruitment in humans is not fully known.

### THE BED NUCLEUS OF THE STRIA TERMINALIS (BNST) IN HOMEOSTATIC EMOTION AND COMPULSIVE BEHAVING

Current effective treatments for OCDS may provide further insight into putative neurophysiology. Deep brain stimulation of the BNST has yielded sustained symptom relief in obsessions and compulsions in otherwise refractory OCD (Nuttin et al., 1999, 2013; Raymaekers et al., 2017; Winter et al., 2018). A collection of several sexually dimorphic interconnected nuclei, the BNST is part of the extended amygdala with extensive bi-directional connectivity with the CLSTC-loop and beyond

(Prewitt and Herman, 1998; McDonald et al., 1999; Dong et al., 2001; Hasue and Shammah-Lagnado, 2002; Dong and Swanson, 2003, 2004b, 2006a,b,c; Rodaros et al., 2007; Li and Kirouac, 2008; Avery et al., 2014; Gregory et al., 2019). Functionally, the BNST filters and/or integrates multiple ascending modalities, mapping with adequate resolution, interoceptive information onto motivational systems for adaptive physiological and behavioral outcomes (Jennings et al., 2013; Kim et al., 2013; Lebow and Chen, 2016). Preclinical work compliments studies in humans: we have demonstrated a role for the oval subregion of the BNST (ovBNST) in compulsive behaviors including addiction, compulsive sucrose-seeking, and finally, in SIP (Krawczyk et al., 2013; Gardner Gregory, 2018; Maracle et al., 2019). Accordingly, electrical stimulation of the BNST is reported to suppress SIP in rodents (van Kuyck et al., 2008). Together, these findings highlight an emergent player, the BNST, in compulsive neurophysiology.

The BNST is fast becoming a relevant region of interest in stress-related psychiatric illness because of its role in emotion processing (Avery et al., 2016; Lebow and Chen, 2016). OCD is "stress responsive," with stressful events precipitating OCD onset and symptoms (Toro et al., 1992). Nuclei within the BNST are bi-directionally connected to several nuclei in both the hypothalamus and the amygdala (Dong et al., 2001; Dong and Swanson, 2004a,b; Jennings et al., 2013), two structures implicated in stress and anxiety, respectively. A role for the cortico-amygdalar circuitry in OCD is emerging (Simon et al., 2014), however, how the hypothalamic-pituitary-adrenal (HPA) axis, a key region involved in stress-reactivity, and stress itself impacts OCD pathophysiology is understudied. Along the HPAaxis in rodents, adrenalectomy hastens the emergence of SIP and exogenous corticosterone reverses the effect (**Table 2**; Devenport, 1978). In intact rats, corticosterone administration also inhibits SIP (Tang et al., 1984; Mittleman et al., 1988b). Furthermore, anxiogenic stimuli (e.g., foot shock) that increase corticosterone also suppress SIP acquisition (Brett et al., 1982). However, foot shock effects on SIP are dose-dependent, where low shock intensity augments SIP and high voltage prevents SIP (Galantowicz and King, 1975). Together, these findings indicate stress reactivity may modulate SIP and/or compulsivity. Given its role in processing classic emotional stimuli and events (e.g., stress and anxiety), we further postulate that the BNST encodes homeostatic emotions, specifically hunger and satiety, to guide appropriate adaptive responding.

Indeed, we recently identified a neural mechanism within the ovBNST that may represent the homeostatic emotion of hungersatiety (**Figure 2**; Hawken et al., 2019). In order to establish compulsive drinking through the SIP protocol, animals must first be hungry, a drive-state typically induced through food restriction (**Figure 1**). In sated (fed ad libitum) male rats, lowfrequency stimulation of ovBNST GABAergic synapses produces increases in inhibitory postsynaptic currents and promotes long-term potentiation (iLTP) in the majority of neurons recorded. Following acute food-restriction (24 h), however, GABAergic plasticity toggles to long-term depression (iLTD) via endocannabinoid dependent mechanisms (**Figure 2**; Hawken et al., 2019). In this framework, the novel endocannabinoid

receptor GPR55 and its ligand, LPI, mediate a hunger-satiety signal (iLTP) while the classic endocannabinoid CB1 receptors and their ligand, 2-AG, promote a hunger-state (iLTD). This effect is plastic as after a brief refeeding window, iLTP is reinstated. In SIP-prone rats, bi-directional inhibitory plasticity in ovBNST neurons is lost, unable to toggle between iLTP and iLTD, as synapses become stuck transmitting a "hunger" signal well after being refed (**Figure 2**; Gardner Gregory, 2018). In this case, CB1 receptors and 2-AG drive reduced GABA transmission in the synapse and iLTD predominates. A loss of synaptic plasticity may be reflected in SIP studies that confirm SIP-prone rats continue to excessively drink despite correcting for their caloric deficit (Falk, 1971). Thus, synapses within the ovBNST driven by hunger and satiety cues lose the ability to correctly encode representative homeostatic emotion. Future research is needed to explore the consequences of inflexible GABAergic synapses in the ovBNST in the bottom-up development of complex behaviors like compulsivity.

A role for the BNST in compulsivity, however, is gleaned by the structural and functional connectivity of its nuclei within the CLSTC-loops implicated in compulsivity. Locally, the ovBNST subregion sends projections to the lateral hypothalamus with efferents and afferents to regions of the amygdala (Dong et al., 2001). The ovBNST also is bi-directionally connected to the fusiform (Dong et al., 2001), another nucleus within the BNST complex thought to receive projections from the frontal cortex

(Lebow and Chen, 2016). Top-down, structural and functional projections from the frontal cortex (i.e., OFC/PFC/insula) to the BNST have been identified in humans, non-human primates, and rodents (Reynolds and Zahm, 2005; Fox et al., 2010; Motzkin et al., 2015). Bottom-up, pathways from the BNST to the frontal cortex are likely multi-synaptic and indirect via the amygdala, striatum, or other areas. For instance, the BNST in humans and rodents is both structurally and functionally connected to the NAc (Dong and Swanson, 2004b; Wood and Swann, 2005; Avery et al., 2014). Both the BNST and the NAc have prominent roles in compulsive drug use (Koob and Volkow, 2010, 2016; Krawczyk et al., 2013). Consistent with a distinct role in "valence surveillance" (Lebow and Chen, 2016), we postulate that the BNST, in part, contributes bottom-up afferent interoceptive information regarding homeostatic satiety via direct and indirect connections with the cortex and goaldirected and habit circuits to represent and convey homeostatic emotions of hunger and hunger-satiety.

As a candidate for encoding homeostatic emotions, the extended amygdala (i.e., the BNST) acts as an integrative hub to detect and signal exteroceptive and interoceptive shifts in the body. Subsequently, the BNST transduces homeostatic information onto circuits that assess contexts and invigorate behaviors to correct any physiological imbalance (eating when hungry, drinking when thirsty or escape/avoidance of threat stimulus). Thus, inaccurate neural representation of interoceptive information within the BNST could have drastic behavioral impact. For instance, the reinstatement of bi-directional GABA plasticity (i.e., iLTP) in the ovBNST following hunger-satiety could signal to the cortical structures a need to shift responding strategies to stop food-seeking behaviors. When iLTD persists despite homeostatic correction (feeding), cortical structures receive a "hunger" signal and continue to engage circuits that promote preparatory and consummatory behaviors, including locomotion and drinking. Convergent findings from animal and human research postulate that together with the mPFC (Killcross and Coutureau, 2003), activity of the OFC manages the activity of subcortical pathways including the striatum (Gremel and Costa, 2013; Pauls et al., 2014; Gremel et al., 2016). Perhaps based on the firing frequency of glutamatergic inputs to the OFC, the OFC selects the appropriate circuit to activate (either goal-oriented or habit) downstream from the cortex (Gremel et al., 2016). In the presence of increased striatal DA reactivity, circuits may be weighted for preferential activation by cortical structures, resulting in a loss of striatal circuit plasticity. In this way, inadequate resolution of homeostatic emotion by

#### REFERENCES


subcortical structures through aberrant neural plasticity may promote maladaptive behavioral responding ultimately dictated by the OFC (Hardung et al., 2017).

### CONCLUSION

Using compulsive drinking in humans (primary polydipsia) and rodents (schedule-induced polydipsia) we suggest a putative neurobiological framework for the etiology of compulsive behaviors. Thirst, hunger, and satiety constitute essential homeostatic emotions, or subjective feelings that drive motivated, goal-directed behavioral selection needed for survival. In polydipsia, homeostatic emotions fail to illicit appropriate adaptive behaviors. How the brain might switch from a state responsive to shifts in homeostasis to one that is unresponsive was surmised by identifying key over-lapping brain changes between polydipsia associated with schizophrenia, schizophrenia without polydipia, and obsessive compulsive disorder. Composite data suggest that DA-regulated neural plasticity within the striatum and regions of the cortex underlie compulsive phenotypes. Ecological and validated animal models of compulsive drinking, or SIP, confirm a role for increased striatal DA activity in compulsive behaving. Additionally, literature points toward aberrant (non-exclusive) monoamine (e.g., DA) modulation of goal-directed and habitual behavior within cortico-limbic-striato-thalamic-cortical loops. Exciting new evidence suggests that non-representative encoding of homeostatic emotions (hunger/satiety) by a cluster of nuclei in the extended amygdala, promote the development of compulsions observed in SIP. The BNST's role in valence attribution or "valence surveillance" and its extensive web of connectivity within the cognitive, affective, and rewardrelated circuits highlights the importance of accurate encoding and bottom-up (to the cortex) transduction of homeostatic emotions. Aberrant neuroplasticity within bottom-up and topdown circuits that support homeostatic emotion signaling likely contribute to maladaptive behavioral selection and inappropriate responding. Future research can explore our postulated mechanism of compulsive behaving.

### AUTHOR CONTRIBUTIONS

TB and EH equally contributed to the design, writing, and editing of the manuscript.





and augments stress- and psychostimulant-induced hyperlocomotion. Neuropsychopharmacology 19, 105–113. doi: 10.1016/S0893-133X(98)00004-9





disorders (SIWIS). J. Nerv. Ment. Dis. 172, 552–555. doi: 10.1097/00005053- 198409000-00008


a multi-modal magnetic resonance study. Neuroimage 52, 62–68. doi: 10.1016/ j.neuroimage.2010.04.012


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Banasikowski and Hawken. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Predictive Value of Impulsivity and Risk-Taking Measures for Substance Use in Substance Dependent Offenders

Nathalie M. Rieser 1‡ , Lilach Shaul 1‡ , Matthijs Blankers 1,2,3 , Maarten W. J. Koeter 1† , Gerard M. Schippers <sup>1</sup> and Anna E. Goudriaan1,2 \*

<sup>1</sup>Department of Psychiatry, Amsterdam Institute for Addiction Research, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands, <sup>2</sup>Department of Research, Arkin Mental Health Care, Amsterdam, Netherlands, <sup>3</sup>Netherlands Institute of Mental Health and Addiction, Trimbos Institute, Amsterdam, Netherlands

#### Edited by:

Trevor W. Robbins, University of Cambridge, United Kingdom

#### Reviewed by:

Ashley Acheson, University of Arkansas for Medical Sciences, United States Matthias Brand, University of Duisburg-Essen, Germany

#### \*Correspondence:

Anna E. Goudriaan a.e.goudriaan@amsterdamumc.nl

†Maarten W.J. Koeter passed away during this research project

‡Both authors contributed equally to this manuscript and share first authorship

#### Specialty section:

This article was submitted to Motivation and Reward, a section of the journal Frontiers in Behavioral Neuroscience

> Received: 31 January 2019 Accepted: 06 August 2019 Published: 19 September 2019

#### Citation:

Rieser NM, Shaul L, Blankers M, Koeter MWJ, Schippers GM and Goudriaan AE (2019) The Predictive Value of Impulsivity and Risk-Taking Measures for Substance Use in Substance Dependent Offenders. Front. Behav. Neurosci. 13:192. doi: 10.3389/fnbeh.2019.00192 Impulsivity and risk-taking are known to have an important impact on problematic substance use and criminal behavior. This study examined the predictive value of baseline self-report and behavioral impulsivity and risk-taking measures [Delay Discounting Task (DDT), Balloon Analogue Risk Task (BART) and Behavioral Inhibition, Behavioral Activation Scale (BIS/BAS)] in 12-months follow-up substance use outcomes (e.g., use of alcohol, cannabis and other substances) and criminal recidivism (yes/no). Participants were 213 male offenders with a substance use disorder (SUD) under probation supervision. Bivariate regression analyses showed that BIS and BAS levels were associated (respectively) with the use of alcohol and cannabis. Multiple regression analysis showed that BIS was negatively associated with alcohol use at follow-up, whereas cannabis use at baseline and BAS predicted cannabis use at follow-up. At a trend level, interactions between delay discounting and risk-taking, and interactions between baseline cannabis use and BAS and BART predicted cannabis use at follow-up. Other substance use at follow-up was solely predicted by baseline other substance use. Overall, the findings provide marginal support for the predictive utility of impulsivity and risk-taking in accounting for variability in substance use among offenders with a SUD. This may be partly explained by the fact that only a limited number of psychological factors was assessed in this study. The studied population consists of a severe group, in which relapse into substance use or criminal behavior likely is related to complex, interacting biopsychosocial factors, of which impulsivity measures play a relatively small part.

Keywords: addiction, dependence, criminality, violence, probation, BART, BIS/BAS, delay discounting

## INTRODUCTION

This article examines the predictive utility of self-report and behavioral impulsivity and risk-taking measures on substance use in offenders with a substance use disorder (SUD). In SUDs, higher impulsivity has been linked to both the development of SUD, and to a more severe course, such as evidenced by earlier treatment dropout and more frequent relapses in SUDs (Stevens et al., 2014). Central to many dual-process theories about SUDs are the higher impulsivity and diminished control functions, compared with a focus on more immediate rewards, and specifically, responsivity towards drug-related cues, as for example in the I-RISA (Impaired-Response Inhibition Salience Attribution) model by Goldstein and Volkow (REF; Goldstein and Volkow, 2002; Verdejo-García and Bechara, 2009). This makes persons with SUDs who both experience a high reward responsivity to drug cues (e.g., by a higher cue reactivity, and a focus on more immediate rewards), in combination with less cognitive control—as for instance in higher impulsivity more vulnerable to relapse. A large number of studies corroborate that impulsivity and risk-taking are associated with a broad range of problematic behaviors such as SUDs or at-risk substance use (e.g., Moeller and Dougherty, 2001; Lejuez et al., 2003; Bornovalova et al., 2005; Perry and Carroll, 2008; Verdejo-García et al., 2008; de Wit, 2009; Dick et al., 2010; MacKillop et al., 2011; Smith et al., 2014) and criminal behavior (e.g., Gottfredson and Hirschi, 1990; White et al., 1994; Nofziger, 2009; Ribeaud and Eisner, 2016).

In addition, a strong and consistent association has been found between substance abuse and crime (e.g., Pihl and Peterson, 1995; Haggård-Grann et al., 2006; Bennett et al., 2008). Although impulsivity and risk-taking are associated with substance abuse and crime in general, these associations may differ across crime-types (e.g., violent, nonviolent; Cherek et al., 1997) and classes of substances (e.g., cocaine, heroin use; Bornovalova et al., 2005).

Impulsivity has been defined as ''a predisposition toward rapid, unplanned reactions to internal or external stimuli without regard to the negative consequences of these reactions to the impulsive individual or to others'' (Moeller et al., 2001, p.1784). It is a multifaceted construct whose facets can be assessed both by self-rated [e.g., Behavioral Inhibition, Behavioral Activation Scale (BIS/BAS)] and behavioral (neurocognitive) measures. Behavioral aspects of impulsivity are for instance delay discounting and risk-taking (Dougherty et al., 2015). The Delay Discounting Task (DDT) is a frequently used task that measures the preference for smaller immediate rewards over larger delayed rewards. The BIS/BAS scale is an instrument assessing reactivity to reward and punishment (Carver and White, 1994). Risk-taking behavior, the ''propensity to seek out novel, stimulating but potentially harmful experiences'' (Dougherty et al., 2015, p.1502) can be estimated with the Balloon Analogue Risk Task (BART), a computer task in which the propensity of an increase in gains (by pumping a balloon) over a risk of loss of the total accrued amount (when the balloon explodes after a pump) is measured.

Behavioral risk-taking as measured with the BART, and similar risky decision making tasks like the Cambridge Gamble Task—has been associated with substance use (Wills et al., 1994; Bornovalova et al., 2005; Schneider et al., 2012; Hanson et al., 2014). In this study, we, therefore, hypothesize that higher risk-taking would lead to a higher substance use, which was also shown by Fernie et al. (2010), who found that risk-taking predicted alcohol use in a group of 75 social drinkers. There are some indications that risk-taking in the BART is associated with alcohol use, but other studies indicate no differences. Ashenhurst et al. (2011) reported that higher risk-taking propensity was associated with lower alcohol use disorder symptoms in a sample of 158 non-treatment seeking heavy alcohol drinkers. Moreover, Ashenhurst et al. (2011) proposed that risk-taking may be an influential factor at initiation of alcohol use, but as use progresses, the relationship may turn in the opposite direction. Hanson et al. (2014) found a predictive effect of riskier choice in the BART and more frequent use of marijuana and other drug use in the past 18 months in a sample of 24 marijuana users and 34 non-users. Wichary et al. (2015) investigated risk-taking in male and female prisoners and non-prisoners. They reported an increased level of risk-taking in female prisoners compared to female non-prisoners, but no difference between male prisoners compared to male non-prisoners.

Delay discounting has also been associated with substance use, as indicated by two meta-analyses (Amlung et al., 2017). MacKillop et al. (2011) conducted a meta-analysis and reported a relation between substance use and delay discounting, but with a small magnitude of effect and high heterogeneity of effect size. They also presented a number of studies not showing a relationship for both alcohol and cannabis use. From the 64 studies analyzed, 27 studied substance use and 74% of the studies reported a higher delay discounting in the SUD samples compared to controls. In a more recent meta-analysis by Amlung et al. (2017), a small, but highly significant effect size was found for steeper delay discounting in SUD, and this relationship was stronger for studies focusing on severity of substance use problems, compared to studies including quantity by frequency measures of substance use. Results from single studies do indicate that several factors may impact the relation between substance use and delay discounting, as for instance gender: in a study on delay discounting and alcohol use in 65 college students or graduated students, higher levels of delay discounting were associated with higher levels of drinking in female college students, but not in their male counterparts (Yankelevitz et al., 2012). Although the meta-analyses did not indicate differential effects for specific substances, Moallem and Ray (2012) reported a steeper delay discounting rate in heavy drinkers who smoked (n = 213), compared to heavy drinkers (n = 107) or smokers (n = 67) solely. When drinking in combination with smoking is viewed as more severe substance use, this finding converges with the conclusion of the meta-analysis by Amlung, that severity measures have a stronger relation to steeper delay discounting. Impulsivity in substance using offenders was correlated with substance use in a sample of 80 drug court participants when measured using self-report, but not when impulsivity was measured with a DDT (Jones et al., 2015). In conclusion, both the meta-analyses indicate a link between substance use and delay discounting, whereas some other individual studies or in some subgroups—no relations were found. Thus, higher delay discounting in SUD samples is present, although the meta-analyses both indicate a small—but significant—magnitude of effect and high heterogeneity of effect size, indicating that the strength of the association of higher delay discounting in SUD samples differs across studies and may be stronger with more severe levels of substance use problems.

Regarding the relation between delay discounting and criminal activity, a smaller number of studies has been published. Åkerlund et al. (2016) analyzed the link between delay discounting at age 13 with their criminal behavior up to age 31 in 6,749 males. Adolescents showing a higher delay discounting rate had a higher risk to be involved in criminal behavior in the future. A study in 86 male offenders (prisoners and ex-prisoners) reported a significant group difference in delay discounting between non-offenders, prisoners and ex-prisoners; ex-prisoners showed a higher discounting rate compared to the other groups (Hanoch et al., 2013). Lastly, Piquero et al. (2018) recently conducted a long-term analysis in a longitudinal sample of over 400 boys of the predictive effect of delay discounting (instead of a task, they asked one question each at age 18, 32 and 48) on criminal behavior (number of convictions until age 56). Higher delay discounting was associated with more convictions.

Lee et al. (2017) found a bi-directional relation between delay discounting and property crime 1 and 2 years later in a study in 526 undergraduates. In another study among 63 male and female offenders, higher delay discounting rates were found in offenders compared to 70 non-offenders (Arantes et al., 2013). In the only study that examined offenders with a SUD, discounting rates and substance use among 80 offenders with a SUD were higher compared to noncriminal students (Jones et al., 2015). In a very small study by Cherek et al. (1997), parolees who had a history of violent crime (n = 9) displayed higher discounting rates than parolees without such a history (n = 21). Lastly, higher levels of delay discounting predicted property crimes, but not violent crimes later on Nagin and Pogarsky (2004). Mixed findings were present in a study by White et al. (1994), who reported a predictive value of cognitive and behavioral impulsivity at age 10 in a sample of 400 boys for delinquency at age 12–13, but no differences between stable nondelinquents, other delinquents and stable-serious delinquents in delay discounting at age 10 and 12–13. Wilson and Daly (2006) also found no difference in the discounting rates of young offenders (n = 91) compared to high school students (n = 284). Summarizing, previous research found evidence for an association between DDT and delinquency, but the relation has not uniformly been demonstrated, and studies in combined populations of offenders with problematic substance use are virtually non-existent (Jones et al., 2015).

BIS/BAS levels in young adults have been linked to alcohol, cannabis and methamphetamine use (e.g., O'Connor and Colder, 2005; Pardo et al., 2007; Simons et al., 2008). The BIS is associated with the avoidance of punishment, whereas the BAS is related to disinhibited behavior (Gray, 1975). The BAS was positively correlated with alcohol and cannabis use, whereas the BIS revealed a negative relation to these two substances (Pardo et al., 2007; Simons et al., 2008). To our best knowledge, there exists no previous research about the association of BIS/BAS levels with criminal recidivism and substance use in a criminal population. We hypothesize that higher BAS and lower BIS may promote substance use and criminal behavior.

In sum, relatively few studies assessed the association of impulsivity and risk-taking with substance use or future criminal behavior in offenders, using laboratory behavioral measures of impulsivity and risk-taking (e.g., Cherek et al., 1997; Mathias et al., 2002; Munro et al., 2007; Chen et al., 2008). These laboratory behavioral measures are especially important for the assessment of offender's impulsiveness and risk-taking, as they provide measures that are less susceptible to simulation than self-report measures. In addition, very few of these studies have focused on multiple aspects of impulsivity. As impulsivity is a multifaceted construct, it can be argued that being impulsive on several of these aspects—e.g., having a focus on steeper delay discounting and a higher risk-taking propensity may exacerbate the effects on potential future substance use and criminal behavior more than only having a present reward orientation or high risk-taking. The purpose of our study is to examine the predictive utility of baseline self-reported and behavioral impulsivity and risk-taking measures, and interactions between impulsivity factors and baseline substance use and impulsivity measures on follow-up use of: (1) alcohol; (2) cannabis; (3) other substance use; and (4) criminal behavior in offenders with a SUD, using a self-rated measure of impulsivity (BIS/BAS) and behavioral measures of impulsivity (DDT) and risk-taking propensity (BART). We hypothesized that higher baseline scores on DDT, BAS and BART, and lower BIS scores would be associated with higher substance use (i.e., alcohol, cannabis and other substances) and higher criminal behavior at follow-up.

### MATERIALS AND METHODS

#### Study Design

A cluster-site, controlled trial (CRT) was conducted to examine the effectiveness of a brief motivation enhancing intervention for offenders with SUDs. The reported results were part of a larger study (see Shaul et al., 2016 for additional information). Within the 220 offenders under probation supervision, participants followed either the motivation enhancing sessions or supervision as usual. The probation officer was set as the cluster variable and the participants were allocated to the two conditions by cluster randomization. This means, 73 probation officers of six probation offices were randomized to perform either supervision with the motivation increasing intervention (intervention condition) or supervision as usual (control condition). With the allocation to the probation officer, participants were also allocated to the supervision they will follow. To control for a potential bias of the motivation enhancing intervention, only data of offenders from the control condition were included for substance use outcome.

#### Recruitment and Assessment Procedures

The probation officers gave all the eligible offenders information about the study. The interested offenders were invited for the baseline assessment (T1). Baseline assessment took place in a private consulting room at the drug-probation office and consisted of a face-to-face interview, and three computerized neurobehavioral tests. A 17-inch laptop computer with a computer mouse was used to run the three neurobehavioral testprograms. Written informed consent for the offender's study participation was obtained prior to baseline assessment. The follow-up (T2) took place on average 14.4 months (SD = 3.76) after baseline (T1). Offenders were paid e15 at baseline and e20 at follow-up for participation. The CRT was approved by the Medical Research Ethics Committee of the Academic Medical Centre, University of Amsterdam. The trial is registered at the Dutch Trial Register, number NTR2420.

#### Participants

A total of 220 male parolees were included in the study, recruited from four addiction probation offices of five out of eleven District Courts in Netherlands (for more information regarding the inclusion process see Shaul et al., 2016). For 27 months, beginning in May 2010 until August 2012, all offenders meeting inclusion criteria were invited to participate. Inclusion criteria were: (i) a sufficient command of the Dutch language to understand interview questions and questionnaires; (ii) male gender; (iii) at least one prior sentence; (iv) regular use of alcohol and/or illicit drugs, i.e., using at least 3 days a week of which for alcohol: consuming at least five or more glasses per day; and (v) currently under a court-order supervision executed by an addiction probation service in a noncustodial setting. Exclusion criteria were: (i) a history of neurological problems or severe psychiatric disorders like schizophrenia, psychotic disorder, or bipolar disorder; (ii) only convicted for driving under influence; and (iii) illegal stay in the Netherlands. Of the 220 participants at baseline, 217 completed the DDT, 212 completed the BART and 209 filled in the BIS/BAS. We had to exclude two participants; one due to the diagnosis of schizophrenia (exclusion criterion i) and one participant due to not using any substances regularly (inclusion criterion iv). Five additional participants completed the BART or DDT at follow-up instead of baseline and were therefore excluded from our further analysis, leading to a final sample of n = 213. Out of the included 213 parolees, 160 participants finished the full procedure. Previous findings in our group showed that the motivation enhancing intervention had no significant effect on criminal recidivism at follow-up (Shaul et al., 2016), and effects of this intervention on treatment entry and substance use are being reported in a separate article (Shaul et al., 2016). Since the earlier publication found no difference between the two conditions regarding criminality, we used the whole sample (n = 213) to predict criminal behavior at follow-up. However, the effect of the motivation enhancing sessions on substance use at follow-up is still being analyzed, therefore, we used solely the participants in the no-intervention (n = 106) subgroup for the prediction of substance use at a 12-month follow-up.

#### Measures

A semi-structured interview based on the MATE-crimi (Schippers et al., 2011) was conducted both at baseline (T1) and 12 months follow-up (T2) assessment, including demographic questions and questions regarding lifetime, 12 months and 30 days information from offenders about their substance use, treatment history and criminal behavior.

Substance use was measured both at T<sup>1</sup> and T<sup>2</sup> using the Measurements in the Addiction for Triage and Evaluation (MATE 2.1; Schippers et al., 2010, 2011). We distinguished between three classes of substances: alcohol, cannabis and other substances, and used different entities per class. For alcohol, we used the number of units of the last 30 days, for cannabis the amount of grams, and for other substances the total number of days used in the last 30 days before T<sup>1</sup> and T2. Because use was lower for other substances (cocaine, crack, other stimulantia, ecstasy, heroine, other opiates and other substances), we added all these substances together and analyzed the effect of the aggregated use of other substances. As the measure of other substances variable included multiple substances it is possible that participants could show a sum of more than 30 days on T<sup>1</sup> and T<sup>2</sup> measures. This measure shows us how many days these substances were used.

#### Delay Discounting Task (DDT)

A computerized version of the DDT by Wittmann et al. (2007) was used to assess impulsive-choice behavior. This shorter functional magnetic resonance imaging (fMRI)-compatible version was included to limit the assessment time and in order to enable comparison to other SUD studies of the authors. The task consisted of six blocks, each containing eight trials on which participants made a choice between an immediate (lower) and a delayed (higher) hypothetical monetary reward. Delay in days (i.e., 5, 30, 180, 365, 1,095, 3,650) and delayed reward in euros (range 476–524 Euro) were equal for all trials of a given block, while the immediate reward value varied across trials within each block (range 0–476 Euro), in which the first two trials of one block were used to narrow down the delay equivalent depending on the responses made (see Wittmann et al., 2007 for exact adjustments). Block order varied and was randomized across participants. As proposed by Myerson et al. (2001), the area under the discounting curve (AUC) was used as a dependent measure; with lower AUC values denoting more discounting by delay (more impulsivity, or inversely, less self-control).

Data of 27 participants were considered non-systematic using the proposed algorithm developed by Johnson et al. (2010) to identify cases with indifference points that were not monotonically decreasing with delay. Specifically, a case was defined as non-systematic if: (i) two or more individual indifference points were greater than their preceding indifference point by a magnitude greater than 20% of the larger later reward; or (ii) the last indifference point was not less than the first indifference point by at least a magnitude equal to 10% of the larger later reward (Johnson et al., 2010). For participants with just one outlier point of indifference according to the former criteria (N = 11), the AUC was replaced by an adjusted AUC through linear interpolation of that point of indifference, leading to exclusion of 16 participants.

#### Balloon Analogue Risk Task (BART)

The BART (Lejuez et al., 2002) was used to assess risk-taking propensity. Due to time constraints, a version with 20 trials was chosen, as versions with 10–30 trials are a methodologically sound choice (Wallsten et al., 2005). As correlations for the total score are acceptable for the first 10 trials (∼0.6) and good for trials 11–20 (∼0.8) with little change for the 10 trials that follow (21–30: ∼0.8; Wallsten et al., 2005; Dahne et al., 2013), we opted for a 20-trial BART version. This was also done for feasibility reasons (time restrictions). During each of the 20 trials, participants inflated a picture of a balloon by pressing a pump button on the screen with a laptop mouse. Each pump increased the risk of the balloon exploding (average breaking point being 64 pumps) and the potential earning (rising by 5 cents). In each trial the balloon's potential earning that was accumulated in a temporary bank could be assured by clicking a collect button on the screen, thus transferring the earning from that particular balloon into a permanent bank. If a balloon exploded before that, the potential earning in the temporary bank for that balloon was lost and a new trial began. Participants received no precise information about the probability of explosion and the task contained no practice trials (for additional task details see Lejuez et al., 2002). The two outcome measures used were: (1) the total number of balloons that exploded during the task; and (2) the average number of pumps on trails where the balloon did not explode (i.e., adjusted average pumps).

#### Behavioral Inhibition, Behavioral Activation Scale (BIS/BAS Scale)

We used the BIS/BAS scale (Carver and White, 1994; Dutch version: Putman et al., 2004) to assess two general motivational systems underlying behavior. The BIS assesses the affective response to punishment, regulates avoidance of punishment and is associated with suppressing behavior and negative affect. The BAS assesses the affective response of upcoming rewards and is associated with the attainment of positively valued stimuli. The BAS scales are subdivided into the three categories; BAS drive, BAS fun-seeking and BAS reward sensitivity.

### Statistical Analysis

Statistical analyses were performed using SPSS 24.0 statistical software package (SPSS Inc., Chicago, IL, USA). Prior to the analyses, we converted all scores to z-scores entered into the bivariate, multiple, moderated and binary logistic regressions, in order for variables to have comparable impact. As suggested by Babyak (2004), we first conducted a bivariate linear regression with the potential predictors (DDT, BART, BIS/BAS) and the dependent variables (alcohol, cannabis and other substance use at follow-up) individually in order to minimize the amount of predictors a priori and prevent over-fitting of our models. The impulsivity and risk-taking measures with a p-value < 0.10 were included in the further analysis (Babyak, 2004). Second, we performed a multiple regression to investigate whether the measures examined a priori predicted the use of alcohol, cannabis or other substances in the last 30 days before 12 months follow-up assessment (T2). We used the exclusion criteria listwise as proposed by Field (2009). To correct for the baseline use of the specific substance (alcohol, cannabis, and other substances), we entered the T<sup>1</sup> substance use measures in the first block. The impulsivity measures were entered in the second block, also using the enter method to assess which measures would show the highest impact. This resulted in three different multiple regression models for the prediction of alcohol use, cannabis use, other substance use at follow-up (T2). The impulsivity and risk-taking measures (BIS/BAS, DDT and BART) were entered using the forward step (likelihood ratio) method as suggested by Field (2009). The cut-off p-value to enter was set at 0.05 and the one to remove was set at 0.10 (Hosmer and Lemeshow, 2000).

Furthermore, to assess interaction effects of impulsivity measures on the multiple regression, we conducted moderated regressions. To predict substance use (alcohol or cannabis) at follow-up, we first entered substance use (either alcohol or cannabis use) at baseline and the hypothesized impulsivity measures into the first block and the interaction effect of the substance use at baseline with impulsivity measures and interactions between the impulsivity measures into the second block.

We hypothesize that a higher alcohol use at follow-up would be associated with a lower BIS, a lower delay discounting (present orientation) and their interactions (lower BIS in combination with lower delay discounting; lower delay discounting and higher BAS; lower BIS and higher BAS). We entered the measures as moderator, whereas alcohol use at baseline was set as the independent variable and alcohol use at follow-up as the dependent variable. In a second model predicting alcohol use at follow-up, we hypothesize that a lower BIS, a higher BART and interactions of the impulsivity measures (moderators) and of baseline alcohol use and impulsivity measures are associated with increased alcohol use. Analyzing the moderating effect of the impulsivity measures on cannabis use at follow-up—cannabis use at baseline as an independent variable—we expected a higher BAS and a higher delay discounting to be associated with higher cannabis use. Furthermore, higher values of combinations of impulsivity measures were expected to be linked to an increased cannabis use at follow-up.

## RESULTS

Although not all variables were perfectly normally distributed, no serious violations of normality such as platy kurtosis requiring transformations were observed (Stevens, 1996). The demographic information is displayed in **Table 1**.

### Bivariate Analysis

When predicting alcohol use at follow-up in the bivariate analysis, only the BIS displayed a p < 0.10 and was therefore entered in the following multiple linear regression as only predictor. The same occurred for the BAS, which predicted cannabis use at follow-up and was included in the multiple regression for cannabis use. However, no impulsivity measure predicted the use of other substances with a p < 0.10 and no predictor was entered into the model. More detailed results of the bivariate analyses are shown in **Table 2**.

When predicting criminal recidivism for property crime, the BAS displayed a p < 0.10 and was therefore entered as the only impulsivity measure in the binary logistic regression analysis. No impulsivity measures showed a p < 0.10 when predicting criminal recidivism for violent crime or all types of crime.

#### TABLE 1 | Characteristics of offenders in the two groups.


Note: <sup>a</sup> including: heroine, other opiate, crack, cocaine, other stimulantia, ecstasy and other substances.

TABLE 2 | Results of bivariate linear regression analysis for predictors of substance use individually.


Note: <sup>a</sup>measured using Area under curve (AUC), <sup>b</sup>measured using average adjusted pumps, <sup>∗</sup>p < 0.10.

Detailed results of the bivariate analysis for criminal recidivism are reported in **Table 3**.

#### Multiple Linear Regression Predicting Substance Use in the Last 30 Days at Follow-Up

In the multiple regression analysis with alcohol use at follow-up as the dependent variable, the alcohol use at baseline was entered (β = 0.211, p = 0.073), the model was not significant; F(1,71) = 3.310, p = 0.073, R <sup>2</sup> = 0.211, adjusted R <sup>2</sup> = 0.045. After including BIS (second block), the model showed a significant, albeit limited amount of explained variance; F(2,70) = 3.770, p = 0.028, R <sup>2</sup> = 0.312, adjusted R <sup>2</sup> = 0.097, indicating a small goodness of fit according to Cohen, 1992). Alcohol use at baseline was marginally associated with alcohol use at follow-up (β = 0.217, p = 0.060), and the association with BIS was statistically significant (β = −0.230, p = 0.047).

In the first block of predicting cannabis use at follow-up, cannabis use at baseline entered the model with a R <sup>2</sup> of 0.424 (adjusted R <sup>2</sup> = 0.179), which indicated a small goodness of fit according to Cohen (1992). The model and the regression coefficient were significant; F(1,73) = 15.96, p < 0.001, respectively β = 0.424, p < 0.001). Additionally in the second block, we found a significant prediction of cannabis use at follow up by cannabis use at baseline and BAS; F(2,72) = 13.064, p < 0.001. The R 2 for the overall model was 0.516 (adjusted R <sup>2</sup> = 0.266), which also indicated a medium goodness of fit according to Cohen (1992). Therefore, the two variables together explained more variance than cannabis use at baseline solely. Cannabis use at baseline


Note: <sup>a</sup>measured using Area under curve (AUC), <sup>b</sup>measured using average adjusted pumps, <sup>∗</sup>p < 0.10.

(β = 0.369, p = 0.001) and BAS (β = 0.300, p = 0.005) were significant predictors in the model.

#### Moderated Regression Analyses

In the moderated regression analyses with alcohol use at follow up as the dependent variable—no predictors or interactions beyond the predictive value of the BIS (main effect of BIS (β = −0.28, p = 0.04), reached significance (p < 0.05) or a trend level (p < 0.10). The original model without the interactions, explained 18% of the variance (R <sup>2</sup> = 0.18, adjusted R <sup>2</sup> = 0.11); F(5,61) = 2.5, p = 0.04. The moderated regression model did not reach a significant level (p = 0.18). In **Figure 1**, we report the interaction graphs between alcohol use and the measures BART, BIS and DDT.

The moderated regression analysis to predict cannabis use at follow up, indicated in the first block, as expected from the multiple linear regression analyses, a significant main effect of cannabis use at baseline (β = 0.49, p ≤ 0.001) and BAS (β = 0.25, p = 0.02), but also of BIS (β = −0.22, p = 0.046). The moderated model also reached significance (F(10,63) = 5.14, p ≤ 0.001, R <sup>2</sup> = 0.49, adjusted R <sup>2</sup> = 0.40). With the interaction effects in the model, besides the main effects of cannabis use at baseline (β = 0.32, p = 0.02) and BAS (β = 0.35, p ≤ 0.002), and a trend for BIS (β = −0.20, p ≤ 0.07), significant interactions were present for baseline cannabis use\*BAS (β = 0.395, p = 0.013), a trend for baseline cannabis use\*BART (β = −0.25, p = 0.09), and a trend for BART\*delay discounting (β = −0.22, p = 0.046). To have a better understanding of the tendencies or direction, we included **Figure 2** reporting the interactions between cannabis use and BART, BAS and DDT and in **Figure 3** the interaction between cannabis use, BART and DDT.

To summarize, no significant interaction effects for prediction of alcohol use at follow-up were present, but in cannabis use at follow-up, interaction effects were present. In patients with a higher BAS, an increased use at baseline was strongly associated with higher cannabis use at follow up. However, in patients with a lower BAS, the association was minor. For the BART, lower levels of cannabis use at baseline in combination with higher risky decision making, predicted higher cannabis use at follow-up, whereas this relation was less strong in those with higher levels of cannabis use at baseline, although this only was a trend (p = 0.09). An interaction of both high BART and high delay discounting was predictive of more cannabis use at follow-up, although only at a trend level (p = 0.07).

#### DISCUSSION

We aimed to investigate the predictive value of impulsivity (DDT and BIS/BAS) and risk-taking (BART) measures for substance use and criminal recidivism at follow-up in a sample of substance using criminals. Results showed that self-rated impulsivity measures (BIS/BAS) were associated with substance use at follow-up. Specifically, a higher BIS predicted lower alcohol use at follow-up, whereas a higher cannabis use at baseline and BAS predicted an increased cannabis use at followup. For cannabis use, baseline use interacted with impulsivity measures to predict cannabis use at follow-up, and a (trendlevel) interaction between delay discounting and risky decision making (BART) predicted higher cannabis use at follow-up. Other substance use at follow-up was not predicted by BIS/BAS impulsivity measures or any of the behavioral impulsivity measures and was only associated with baseline other substance use. Our hypotheses were therefore partly confirmed.

When looking at other substances, only baseline use was associated with use at follow-up. This may be due to a less frequent use of the other substances and a higher amount of non-users which resulted in a reduced power. The relationship between impulsivity, risk-taking and substance use might differ across substances in the other substance use class. For example, in a study in crack cocaine users, higher levels of risk-taking and impulsivity were present compared to those of heroin users (Bornovalova et al., 2005). Therefore, the combination of varying levels of impulsivity in the other substance use category may have had a reducing effect on power, although the analysis of the

FIGURE 1 | Scatterplot indicating the interaction between alcohol use and the measures Balloon Analogue Risk Task (BART, measured using average adjusted pumps), Behavioral Inhibition Scale (BIS) and Delay Discounting (measured using Area under curve, AUC). A higher measure on BART indicated a higher risk-taking because of more average adjusted pumps to a balloon. Whereas a lower AUC points to more discounting by delay, higher impulsivity and less self-control.

individual other substances was not an option, given their lower prevalence of use.

Previous research demonstrated that a higher BIS is associated with a more inhibited personality style (Hoppenbrouwers et al., 2015). Furthermore, higher BIS scores have been related to the avoidance of potentially dangerous environments (Campbell-Sills et al., 2004). Therefore, it seems plausible that increased inhibition, as evidenced in higher BIS scores are protecting for alcohol use as found in our study, which is also in line with previous findings in a study in undergraduates (Pardo et al., 2007). However, baseline alcohol use had no influence on alcohol use at follow-up, indicating that BIS had a stronger effect on future alcohol use than the baseline use, which means that personality traits relating to impulsivity may be more important when predicting future alcohol use.

Scores on the BAS have been associated with a higher attention to positively valued stimuli (Hoppenbrouwers et al., 2015), impulsive reward-seeking behavior (Carver and White, 1994) and disinhibited behavior (Gray, 1975). Therefore, a higher BAS could lead to a higher anticipation of pleasure and reward-seeking behavior, thus explaining the relation to increased cannabis use in our study in criminal offenders. This finding is in line with prior research (Pardo et al., 2007). Baseline cannabis use predicted follow-up use as well, and at a trend level, interactions between both higher risktaking behavior as measured in the BART and higher delay discounting, and higher cannabis use at baseline and higher BAS were also predictive of higher cannabis use at followup, indicating that combinations of higher impulsivity have an additive effect on predicting cannabis use at follow-up, while also interacting with baseline cannabis use. In the last case, an opposite effect of the main effect of baseline cannabis use was present, with higher BAS and higher BART values impacting cannabis use at follow-up more when baseline cannabis use was lower.

No direct associations between BART or DDT and alcohol, cannabis, or other substance use were present in our study. This could be related to the fact that in this severe sample, probably with a higher level of impulsivity compared to the general population, the influence of these factors is limited through a restriction of range. Swogger et al. (2010) found no association between psychopathy and BART in male criminals and argued that caution is needed when generalizing results from non-criminal to criminal samples. They also suggested that the

diagnostic benefit of the BART among inmates may be limited. To be able to detect differences between inmates and other samples, a BART version with higher rewards for risky behavior may be superior (Bornovalova et al., 2009).

As already seen in the introduction, some inconsistent results were reported in the literature regarding the association between DDT and offending, respectively. Prior research with the DDT demonstrated positive findings, for example, an increased delay discounting in offenders compared to students (Arantes et al., 2013), predictive value for property and violent crime in undergraduates (Lee et al., 2017), increased delay discounting in a drug court sample compared to non-criminal university students (Jones et al., 2015), and delayed discounting measured in early lifetime (13 or 18, 32 and 48, respectively) as a predictive value for criminal convictions until the age of 31 or 56, respectively (Åkerlund et al., 2016; Piquero et al., 2018). However, negative findings are also present for instance in two studies where: no differences in delay discounting between delinquents and non-delinquents were found (White et al., 1994; Wilson and Daly, 2006). Property and violent criminality may be associated differently with delay discounting, as high levels of delay discounting have been associated with property crime, but not violent crime, which was predicted by poor impulse control (Nagin and Pogarsky, 2004). Our results indicate no predictive value of DDT for criminal behavior, which may be related to the male sample that we included, whereas other studies included both males and females. Since gender differences exist in the DDT (Yankelevitz et al., 2012), this may have had an influence. The DDT used in this study was a shorter version than usual DDT tasks, which also may have led to a less optimal measurement of delay discounting, as also indicated by the exclusion of 16 participants due to non-systematic data on the DDT. Further, the statistical analyses differed; we wanted to predict criminal behavior and substance use in parolees, whereas the prior studies calculated a comparison between offenders and students or predicted crime in students or in a healthy sample. Thirdly, most studies analyzed a longer time period than we used (e.g., 18 years, Åkerlund et al., 2016; 38 years, Piquero et al., 2018). Lastly, the DDT itself differed; for example, Piquero et al. (2018) asked the participants only one question in each survey year to assess delay discounting, whereas Arantes et al. (2013) used a DDT with amounts between \$500–\$4,000 with delays between 1–8 years. These are higher amounts and later hypothetic payout than in the version of the DDT that was used in this study (Wittmann et al., 2007).

Another factor which may be related to future substance use that has been discussed in this field is one's attitude towards the future. An optimistic attitude towards the future is linked to less risky behavior, whereas a negative orientation towards the future correlates with higher substance use and more risk-taking (e.g., Wills et al., 2001; Apostolidis et al., 2006a,b; Henson et al., 2006). Juveniles on probation who are more positive about their future were less involved in substance use and more likely to reject risky behaviors (Robbins and Bryan, 2004). The willingness to take risks can vary and be dependent on attitudes toward the future (Wilson and Daly, 2006). Participants in our sample recently left the prison and may show a more optimistic perspective for their future and may be keen to change their behavior. Therefore, they may display less impulsive behavior just after their stay in prison.

Wise and Koob (2014) discussed the development and maintenance of SUDs including positive (increase of behavior with positive stimuli) and negative (increase of behavior in order to remove or avoid a negative condition) reinforcement. When starting to use a substance, positive reinforcement, involving more impulsive behavior, is essential. They hypothesize that after developing an addiction, negative reinforcement predominates, involving elements of compulsivity [defined as ''actions inappropriate to the situation that persist, have no obvious relationship to the overall goal, and often result in undesirable consequences'' (Wise and Koob, 2014, p.257)]. Therefore, impulsivity may have a larger impact at the beginning of an addiction, but for the maintenance of an addiction, other factors may become more influential. Still, we found effects of the BIS/BAS, of a combination of high delay discounting and high BART and an interaction of BAS and BART with baseline cannabis use, on follow-up of cannabis use, indicating that combinations of higher levels of impulsivity and risk-taking can impact future substance (alcohol and cannabis use). These findings are consistent with addiction models that indicate a central role for impulsivity, other executive functions, and the underlying diminished functioning of the dorsolateral prefrontal cortex and anterior cingulate cortex (Goldstein and Volkow, 2002; Verdejo-García and Bechara, 2009). Thus, a combination of increased impulsivity, risk-taking and/or a preference for immediate rewards over delayed rewards, may exert its influence on future alcohol and cannabis, through changes in striatalfrontal brain circuitry, on top of the predictive effect that use at baseline has. Our sample consisted of parolees with a long history of criminal behavior and it may be possible that impulsivity at this period may have a smaller impact, than in younger or at-risk populations. This assumption may be supported by the findings in extremely violent prisoners, where Værøy et al. (2015) did not find an association between higher impulsivity (UPPS) and increased physical aggression (AQ-RSV). A further study found that premeditated aggression, which is defined as a planned action, predicted criminal recidivism, whereas impulsive aggression did not (Swogger et al., 2015), also indicating that the role of impulsivity may be limited, at least for some forms of (aggressive) criminality.

#### Limitations

The sample consisted of male parolees and the effects are not generalizable to female offenders or offenders with comorbid mental illnesses. Additionally, the sample differed widely in the range of substance use. A few parolees had been using solely one substance, leading to a high portion of non-users in the ''other substance'' group (43.5%), limiting the power to detect differences for this specific analysis. Furthermore, the intervention or probationary service which all of these parolees followed, could have had a diminishing effect on impulsivity, meaning that the sessions may have reduced the influence of impulsivity on substance use. In this study, we could explain between 10%–40% of variance, which means that there are other predictors that we did not measure, such as other personality traits, acute substance intoxication during offense, childhood experiences, genetic predisposition, their neighborhood and its criminogenic behavior and/or relationships (Zimmerman, 2010). In this study, no counterbalancing was employed and thus, fatigue may have impacted the neurocognitive assessment, potentially impacting the power of predicting substance use at a later point. For the delay discounting, a quality check was done ensuring data integrity (see DDT), indicating that only in a small minority of cases, this was the case. Also, given the fact that the DDT and BART were not speed tasks, we think fatigue only may have impacted the data minimally. Another limitation may lie in a ''restriction of range'' effect: as our sample consisted of criminals with problematic substance use, this likely reflects a population in which impulsivity is higher than in the general population, and thus, impulsivity measures may have had more limited effects in our study. Lastly, the analyzed time at risk may

#### REFERENCES


have been too short. Further studies should assess a longer time period after prison.

### CONCLUSION

Several impulsivity and risk-taking tasks had a predictive impact on alcohol and cannabis use at follow-up in male, substance using parolees. Assessing behavioral inhibition, behavioral activation, impulsivity and risk-taking propensity in parolees seem to be a valuable addition in order to prevent substance use. Using these scores, parolees could be assigned to an intervention that focuses on reducing impulsivity and/or risk-taking behavior. However, additional research is needed in order to improve the assessment of predicting criminal recidivism and substance use, taking into consideration other variables which may explain the complex roles of impulsivity and risk-taking in criminal behavior and substance use.

#### ETHICS STATEMENT

The trial was approved by the Medical Research Ethics Committee of the Academic Medical Centre, University of Amsterdam. The trial is registered at the Dutch Trial Register, number NTR2420. Participants provided written informed consent prior to study participation.

#### AUTHOR CONTRIBUTIONS

GS, MK, and LS designed the study. NR and LS drafted the first version of the manuscript. AG and MB amended the draft and critically reviewed the article. All authors approve submission of the manuscript.

#### FUNDING

This work was supported by Dutch Organisation for Health Research (ZonMw; grant number 31160207). AG is supported by grant number 91713354 from the Dutch Scientific Health organisation: ZonMw.


marijuana-dependent individuals. Exp. Clin. Psychopharmacol. 18, 99–107. doi: 10.1037/a0018333


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Rieser, Shaul, Blankers, Koeter, Schippers and Goudriaan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Effect of Schedule-Induced Behavior on Responses of Spontaneously Hypertensive and Wistar–Kyoto Rats in a Delay-Discounting Task: A Preliminary Report

Sergio Ramos <sup>1</sup> , Gabriela E. López-Tolsa<sup>1</sup> , Espen A. Sjoberg2,3 and Ricardo Pellón<sup>1</sup> \*

<sup>1</sup>Animal Learning and Behavior Laboratory, Departamento de Psicología Básica I, Universidad Nacional de Educación a Distancia (UNED), Madrid, Spain, <sup>2</sup>Animal Behavior Laboratories, Department of Behavioral Science, Oslo Metropolitan University, Oslo, Norway, <sup>3</sup>Schools of Health Sciences, Kristiania University College, Oslo, Norway

#### Edited by:

Margarita Moreno, University of Almería, Spain

#### Reviewed by:

Emily R. Hawken, Queen's University, Canada Caridad López-Granero, University of Zaragoza, Spain

> \*Correspondence: Ricardo Pellón rpellon@psi.uned.es

#### Specialty section:

This article was submitted to Motivation and Reward, a section of the journal Frontiers in Behavioral Neuroscience

Received: 31 January 2019 Accepted: 25 October 2019 Published: 13 November 2019

#### Citation:

Ramos S, López-Tolsa GE, Sjoberg EA and Pellón R (2019) Effect of Schedule-Induced Behavior on Responses of Spontaneously Hypertensive and Wistar–Kyoto Rats in a Delay-Discounting Task: A Preliminary Report. Front. Behav. Neurosci. 13:255. doi: 10.3389/fnbeh.2019.00255 Delay discounting is the loss of the subjective value of an outcome as the time to its delivery increases. It has been suggested that organisms can become more tolerant of this delay when engaging in schedule-induced behaviors. Schedule-induced behaviors are those that develop at a high rate during intermittent reinforcement schedules without the need of arranged contingency to the reinforcer, and they have been considered as a model of compulsivity. There is evidence that relates compulsivity to greater delay discounting. The rate of delay discounting represents how impulsive the subject is, as the rate of discounting increases the higher the impulsivity. Thus, the main purpose of this study was to undertake a preliminary evaluation of whether developing schedule-induced behaviors affects performance in a delay-discounting task, by comparing spontaneously hypertensive rats (SHRs) and Wistar–Kyoto (WKY) rats. The rats were exposed to a task that consisted of presenting the subjects with two levers: one produced a small, immediate food reinforcer while the other one produced a larger, delayed reinforcer. During Condition A, the levers were presented, and a water bottle and a running wheel were available in the conditioning chambers; during Condition B, only the levers were presented. SHR and WKY rats developed schedule-induced behaviors during Condition A and showed no difference in discounting rates, contradicting previous reports. Lick allocation during response-reinforcer delays and the inter-trial interval (ITI) showed, respectively, pre- and post-food distributions. Discounting rates during Condition B (when rats could not engage in schedule-induced behaviors) did not reach statistical significance difference among strains of animals, although it was observed a tendency for WKY to behave more self-controlled. Likewise it was not found any effect of scheduleinduced behavior on discounting rates, however, a tendency for WKY rats to behave more impulsive during access to drink and run seems to tentatively support the idea of schedule-induced behavior as a model of compulsivity in those rats, being impulsivity simply defined as an excess in behavior.

Keywords: delay discounting, schedule-induced behavior, impulsivity, compulsivity, SHR vs. WKY rats

### INTRODUCTION

Impulsive behavior is seen in everyday life, but the nature of this heterogeneous concept means that there is no consensus on its definition. In clinical psychology, impulsivity has been defined as a wide range of symptoms or factors that interact with each other, including impatience, interruption of activities, difficulty to plan ahead, difficulty to wait, excessive spending and/or substance abuse (American Psychiatric Association, 2013). On the other hand, impulsivity has been operationally defined in animal behavior research as a preference for smaller immediate reinforcers over large delayed ones (Fox et al., 2008; Hamilton et al., 2015), as an aversion to the delay (Sonuga-Barke et al., 1992; Richards et al., 2011), or as the inability to wait or to withhold a response (Richards et al., 2011). Although the operational definition of impulsivity seems to be heterogeneous as well, Sosa and dos Santos (2019) argue that different paradigms study the same behavioral tendency.

Impulsivity has been studied using different standardized intertemporal choice procedures, like the simple- and adjustingdelay discounting tasks (Sosa and dos Santos, 2019). One of the advantages of using standardized procedures is that they facilitate the comparison of analogous behaviors across species (Richards et al., 2011), although in some cases standardization is reduced due to methodological changes associated with accommodating the procedures to various species (Sjoberg, 2017).

Delay discounting is the loss of the subjective value of an outcome as the time to its delivery increases (Vanderveldt et al., 2016). Impulsivity has been evaluated using delay-discounting tasks since Mazur (1987) proposed the use of an adjusting-delay procedure (Sjoberg and Johansen, 2018; see also Bickel et al., 2015). In Mazur's experiment, pigeons were presented with a choice between two options, one that gave a small reward after a short delay, and the other that gave a large reward after a long delay. The value of the longer delay increased or decreased across the blocks of trials depending on the subject's responses, while the delay to the small reward was held constant. The purpose of this adjusting-delay procedure was to find out the subjective indifference point at which the value of both options was the same for each given subject (Vanderveldt et al., 2016).

One variation of the procedure suggested by Mazur (1987) is the simple delay-discounting procedure, which consists on presenting the subject with two options: one that gives a small immediate reinforcer (SS) and another that gives a larger delayed reinforcer (LL; Fox et al., 2008). When using animals, the delay to LL increases throughout the experiment and the animals usually start preferring the LL option until the delay reaches a value at which organisms change their preference to the SS option (Fox et al., 2008). Contrary to the adjusting-delay procedure, the value of the delay changes independently of the subject's responses, but the time at which the subject changes its preference (choosing LL in 50% of the trials) is equivalent to the indifference point in the adjusting-delay procedure (Hamilton et al., 2015). As pointed out by Sjoberg and Johansen (2018), this means that discounting behavior could be interpreted in one of two ways: either as a continuous measure, showing a degree of impulsive behavior, or as a switch in preference from large to small reinforcers. The more sensitive the subject is to the delay, the more impulsive it will be considered. Mazur (1987) also proposed a hyperbolic function as a mathematical model to describe the discounting rate, which predicts the preference reversal in humans and in different animal species, suggesting that different organisms discount in a similar way (Vanderveldt et al., 2016).

Two different strains of rats have been typically used to study impulsivity using delay-discounting tasks: the Spontaneously Hypertensive Rat (SHR) and the Wistar–Kyoto (WKY) rat (Adriani et al., 2003; Fox et al., 2008; Hand et al., 2009; Aparicio et al., 2019). The SHR is considered a valid rodent model of Attention-Deficit/Hyperactivity Disorder (ADHD) because rats from this strain exhibit behavioral characteristics similar to those seen in humans with ADHD, such as impaired sustained attention, learning insufficiencies, resistance to extinction, hyperactivity, hypersensitivity to delayed consequences, impulsivity, motor impulsiveness and behavioral variability (Sagvolden, 2000; Fox et al., 2008; Sontag et al., 2010; Orduña, 2015; Aparicio et al., 2019). WKY rats, on the other hand, do not exhibit so much excessive behavior and they are more tolerant of delayed consequences, so this strain is commonly used as a control group for the SHR. Its validity as a control has been questioned, as differences exist across vendor strains, with the WKY/NHsd strain being proposed as a model of inattention (Sagvolden et al., 2008, 2009; Sagvolden and Johansen, 2012). Other strains of rats, such as Lewis and Fisher 344 (Anderson and Woolverton, 2005) or Roman High- and Low-Avoidance (Moreno et al., 2010), have been also used to study impulsivity, although SHR and WKY rats have been the most used (Sontag et al., 2010).

Fox et al. (2008) carried out an experiment to determine if SHR behaved more impulsively than WKY rats. For that, they used a delay-discounting task in which two levers were presented: a press to one of the levers delivered one food pellet immediately, while a press to the other delivered three food pellets after a delay. This delay increased as the experiment progressed, and subsequently decreased. SHRs chose the LL option significantly less often than WKY rats, regardless of delay order. SHRs will also express higher degrees of impulsivity if the delays are presented in random order (Fox et al., 2008; Aparicio et al., 2019).

The delay-aversion theory proposes that the delay function should be understood as the overall trial length or the overall waiting time, meaning that impulsivity is the result of an inability to endure long trials in order to secure large rewards (Sonuga-Barke et al., 1992). However, as reviewed by Sjoberg and Johansen (2018), this theory holds little validity in animal research. Instead, the delay between response and reinforcer is the strongest predictor of impulsivity in animal research, although in humans both the delay and the trial length together explain the phenomenon, known as the dual-component model (Marco et al., 2009).

It was recently suggested that schedule-induced behavior might help organisms improve their performance in temporal tasks such as the temporal bisection or fixed-interval schedules (Ruiz et al., 2016), differential reinforcement of low rates (DRL; Bruner and Revusky, 1961; Segal and Holloway, 1963), or the peak procedure (Mattel and Portugal, 2007), by providing an alternative activity for the organisms. Seemingly, scheduleinduced behaviors can make waiting-time less aversive, so having the opportunity to engage in them should make organisms more self-controlled. Furthermore, DRL schedules and the peak procedure have been considered tasks that also measure impulsivity (see Pellón et al., 2018), although they were not designed to study choice (Mattel and Portugal, 2007; Sosa and dos Santos, 2019). Better performance on these tasks would imply that subjects are behaving more self-controlled.

When organisms are trained under intermittent reinforcement schedules they normally develop excessive patterns of behaviors during the inter-reinforcers intervals, even when there is no explicit contingency between their occurrence and the delivery of the reinforcer; those behaviors are called schedule-induced behaviors (Falk, 1971; Killeen and Pellón, 2013). Traditionally, schedule-induced behaviors have been included in a category different than operants (Falk, 1971) because they seem to be induced by a low probability of reinforcement (Staddon, 1977). Nevertheless, Pellón et al. (2018) have recently proposed that schedule-induced behaviors are induced by events in the environment and maintained by delayed reinforcement (Killeen and Pellón, 2013; Ruiz et al., 2016; Álvarez et al., 2016).

The most studied example of schedule-induced behavior is schedule-induced drinking, which consists of a small, regular and persistent drinking after each food pellet is delivered when food-deprived rats are exposed to an intermittent food-presentation schedule (Íbias and Pellón, 2011). After some training, the high pattern of drinking concentrates in the first 15–20 s of the interval (Álvarez et al., 2016), although if access to water is restricted to the last part of the interval, it will develop showing a similar distribution in latter portions of the interval (López-Crespo et al., 2004).

On the other hand, it has been suggested that scheduleinduced drinking is a model of compulsivity (Moreno and Flores, 2012). Compulsivity is defined as performing an act persistently and repetitively, inappropriate to the situation, and with no obvious relation to the overall goal, in order to prevent perceived negative consequences leading to functional impairment of the organism (Oldham et al., 1996; Dalley et al., 2011). Considering the definitions of compulsion and schedule-induced behavior, it can be noticed that both occur persistently, repetitively and with no obvious relationship to the overall goal (there is no arranged contingency between the behavior and the obtaining of reinforcement, though relations can be established based on the notion of proximity—see Killeen and Pellón, 2013).

It has been observed that patients with Parkinson's disease and Obsessive-Compulsive Disorder (OCD) with comorbid impulsive-compulsive behaviors showed elevated delaydiscounting rates compared to patients without comorbid compulsive behavior or healthy participants (Housden et al., 2010; Pinto et al., 2014; Sohn et al., 2014). If schedule-induced behavior functions as compulsive behavior, its development should make other organisms behave more impulsive, as it has been observed with humans. SHR rats could serve as a model of patients with comorbid impulsive-compulsive behaviors because of their elevated discounting rate and excessive behavior compared to WKY rats that could simulate the discounting rates of the patients that do not develop a comorbid compulsive behavior.

The aim of the present study was to observe the effect of having the opportunity to develop schedule-induced behavior during a delay-discounting task. To achieve that, rats were exposed to a delay-discounting task and the experiment was run in two successive conditions, one in which subjects could develop schedule-induced behavior, and the other in which they could not. If schedule-induced behavior acts as compulsive behavior, rats will show steeper discounting (higher degree of impulsivity) when the opportunity to develop schedule-induced behavior is available. By contrast, if schedule-induced behavior causes waiting time to be less aversive rats under schedule-induced behavior should discount less, i.e., be less impulsive.

### MATERIALS AND METHODS

#### Subjects

Six SHR rats and six WKY rats were used as subjects in this experiment. SHRs were obtained from Janvier Laboratories (France) and WKY rats from Envigo Laboratories (United Kingdom). The different origin of the animals is based on findings that support differences across vendors in the responsiveness of specific strains of rats (Sagvolden et al., 2009). The average group weight at the beginning of the experiment was 245.3 g (range: 225–278) for SHR and 237.5 g (range: 210–264) for WKY. At the beginning of the experiment, all subjects were 15 weeks old. They were housed individually in an environmentally controlled room where temperature was 22◦C, relative humidity was maintained at 55%, and there was a 12:12 h light-dark cycle (lights on at 8:00 AM). The home cages were made of transparent Plexiglas and measured 18 × 32.5 × 20.5 cm. Experimental sessions were conducted during the light part of the cycle, Monday through Sunday, at about the same time every day. Subjects were maintained at 85% of their free-feeding weights, following the standard growth curve for each strain, by restricting the amount of food they received every day. Water was freely available in the home cages. Each rat was weighed daily before the experimental session and supplemental feeding was delivered between 30 min and 1 h after the experimental session ended. All rats only had previous testing experience with the same delay-discounting task using both levers of the conditioning chambers, but they had no previous experience with schedule-induced behavior or any other experimental preparation.

#### Apparatus

Sessions were conducted using eight Letica LI-836 conditioning chambers. The conditioning chambers measured 29 × 24.5 × 35.5 cm and were enclosed in sound-attenuating boxes with a fan mounted on one of the walls that provided an ambient noise of approximately 60 dB. The front wall of each chamber was made of aluminum, the right and the rear walls were made of black Plexiglas, and the remaining wall was made of transparent Plexiglas. The floor consisted of a 16-bars stainless metal grid. The front panel of each chamber was equipped with two levers located at each side and a food tray located between them, 3.7 cm above the floor. The right wall had a 3.2 cm × 3.9 cm aperture situated 20 cm from the front panel and 7 cm above the floor. A bottle of water could be mounted behind the wall, and the rat could reach the spout from inside the aperture. Licks were recorded through the contact of the rat's tongue to the spout, which completed an electric circuit between the floor and the spout. At the rear wall, access to an activity wheel mounted outside the conditioning chambers was permitted. The activity wheel was made of stainless metal, measured 9 cm wide and had a diameter of 34 cm. Turns in the wheel were recorded using a magnet system which counted a turn each time it was closed. The houselight was mounted behind the front panel and provided general illumination during the sessions. Forty-five miligram food pellets (Bio-Serv, Frenchtown, NJ, USA) were delivered into the food tray. A MED-PC application under a Microsoft Windows XP environment provided environmental control and recorded lever presses, licks, and turns.

#### Procedure

The present experiment consisted of a delay-discounting task, in which two levers were presented to the subjects; after a response, one lever delivered one food pellet immediately and the other delivered three food pellets after an increasing delay. The experiment followed an A-B design, so each subject faced the task twice, once in each condition. Each condition began with some pre-training (see below).

During the delay-discounting phase in Condition A, rats had access to the bottle with water and the running wheel in the conditioning chambers. Access to the bottle and the wheel was not permitted in Condition B. Each bottle was filled with 150 ml of fresh tap water before experimental sessions.

Sessions in all phases were divided into 10 blocks. Each block consisted of six trials: two forced and four free trials. Trials started when the levers were inserted into the chamber after a response the levers were retracted, the food pellet(s) delivered and a 10-s inter-trial interval (ITI) began (except during the delay-discounting phase, more details provided in a section below). During forced trials only one lever was presented at a time, the order of which was randomized, and during free trials, both levers were presented. The houselight was turned on at the beginning of the experimental sessions and turned off at the end.

#### Pre-training

During phase 1 of pre-training, the two levers delivered one food pellet immediately after a single response (FR1). The aim of this phase was to control for lever-bias, such that if a rat showed a preference for one lever over the other, with all else being equal, then this could be counter-balanced when assigning SS and LL levers. Assigning the large reinforcement to the less-preferred lever ensured that rats would press it because of the magnitude of the reinforcer, not because of lever-bias. This phase lasted one session.

Phase 2 was similar to the prior one, except that one of the levers delivered three food pellets and the other delivered just one. The lever that delivered three food pellets was the opposite of the preferred one during the previous phase. The purpose of phase 2 was to conduct a preference test, ensuring that rats preferred a large reinforcer (three food pellets) rather than a small reinforcer (one food pellet) in the absence of any experimental manipulation. Rats stayed in this phase until they chose the larger option in at least 66% of the trials during three consecutive sessions. Rats that did not reach the criterion in a maximum of 10 sessions were removed from the experiment.

A schematic representation of the pre-training is outlined in **Figure 1**.

session. Phase 2 (lower panel) was a preference test for the large reinforcer. One of the levers delivered one food pellet and the other delivered three food pellets, both immediately. This phase lasted 10 sessions or until the subject reached the criterion of 66% of the choices to the large reinforcer in three consecutive sessions. Presentation of food pellets was followed by a 10-s inter-trial interval (ITI) in which levers were retracted.

#### Delay Discounting

This phase consisted of a delay-discounting task, during which two levers were presented to the rats: a press to one of the levers delivered one food pellet immediately (SS option) and one press to the other lever delivered three food pellets after a delay (LL option). Trials were similar to previous phases, except that after a press to the LL lever, the lever was retracted, and the delay started, the three food pellets were delivered after the delay, and then the 10-s ITI began. Therefore, the trial length grew as the value of the delay increased for the LL option. Because sessions finished after 60 choices regardless of the trials' duration, it was controlled the potential influence of reinforcement rate on choice. The delay between the response and the reinforcer increased 3 s per session until 36 s. So, the delays were: 0, 1, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33 and 36 s. A 1-s delay was included to illustrate the difference between the presence and absence of a delay, even if the delay was short. The delay-discounting phase lasted 14 sessions. See **Figure 2** for a schematic representation of the task.

#### Data Analysis

Lever presses, licks and wheel turns were recorded. The percentage (%) of responses to the LL option in the free choice trials was calculated considering data from the free choice trials only. The indifference point is 50%, so if the percentage of LL responses was above 50%, subjects were considered to be behaving self-controlled, while an LL percentage below 50% is indicative of impulsive behavior. The number of licks and turns were recorded every 1-s bin. The mean number of licks for each session and subject were calculated, as well as the mean number of licks per 1-s bin during the response-reinforcer delays and the ITI. The average of licks during each 1-s bin was calculated by dividing the number of licks in each bin by the times that bin occurred during the experiment. For example, the mean licks in bin 1 included data from all the delays, whereas the mean licks in bin 30 included data from delays of 30 s and longer.

Differences in the percentage of LL responses were analyzed using an analysis of variance (ANOVA) with a repeated measures factor, delay value (with 14 levels), and fixed factors: strain (with two levels), and condition (with two levels). Differences in the proportion of licks were analyzed using an ANOVA with a repeated measures factor, bins (with 36 levels for the proportion of licks during the delay, and 10 levels for the proportion of licks during the ITI), and fixed factors: strain (with two levels), and ITI (with two levels). Differences between strains in the total licks per session were also analyzed using an ANOVA with the repeated measures factor of delay value (with 14 levels) and strain as a fixed factor (with two levels). In all cases, statistical significance was set at a minimum p < 0.05.

In order to provide a quantitative measure of impulsivity, data of each subject was fitted to Mazur's Hyperbolic Model (Mazur, 1987), and the sensitivity to the delay was calculated with the following equation:

$$V = A/(1 + kD),$$

where V represents the subjective value of the large delayed reward, A corresponds to the mean proportion of LL responses under 0 s delay as the curve start point, k is a free parameter that represents the rate of discounting, and D is the value of the delay. The best-fitting parameters were obtained by the leastsquares method using Microsoft Excel solver, with the constraint that k should be greater or equal to zero (k > = 0). The goodness of fit was calculated using the coefficient of determination (R 2 ). The hyperbolic function describes the hyperbolic discounting rate of the reward as the value of the delay changes. Greater values of k mean more impulsivity. For a better understanding of Mazur's Hyperbolic Model, see Mazur (1987). To compare the values of k one-way ANOVAs were used, comparing groups (SHR and WKY) or conditions (Condition A and B) as factors. The significance level was established at a minimum of p < 0.05.

### RESULTS

#### Pre-training

During pre-training, a preference for the large option was acquired by all rats. At the end of pre-training of Condition A, the mean choice of the large option for SHR rats was 0.93 ± 0.02 (mean ± SEM) and for WKY rats was 0.93 ± 0.03, and at the end of pre-training of Condition B, the mean choice of the LL option for the SHR group was 0.89 ± 0.02 and 0.98 ± 0.08 for the WKY group. These results ensured that subjects chose the lever because of the magnitude of the reinforcement and not because of any side-bias for the lever.

#### Temporal Discounting

**Figure 3** shows the mean percentage of choices for the LL lever across the delays of each strain at each condition. Scheduleinduced behavior could be developed during Condition A, but

not in Condition B. In both conditions a main effect was found for delay (Condition A: F(3,32) = 148.112, p < 0.001, η = 0.937; Condition B: F(2,22) = 71.415, p < 0.001, η = 0.877), as the duration of the delay increased, the percentage of responses to the LL option decreased. Statistical analysis of the percentage of LL responses found that differences between strains were neither significant in Condition A (F(1,10) = 0.006, p = 0.940, η = 0.001), nor in Condition B (F(1,10) = 1.517, p = 0.246, η = 0.132). It was also found that differences between conditions were not statistically significant for SHR (F(1,10) = 0.491, p = 0.499, η = 0.047) or WKY (F(1,10) = 0.844, p = 0.380, η = 0.078). Although no statistical differences were obtained, it can be observed that SHR tended to behave slightly less impulsively during Condition A in comparison to Condition B, while WKY rats behaved more impulsively during Condition A.

**Table 1** shows individual values of parameter k, and the mean of each group in each condition. Regarding the group data in Condition A, the discounting rate represented by k was similar for both strains (0.15 for SHR rats, and 0.14 for WKY rats). In Condition B, however, the SHR group discounted faster (0.18) than the WKY group (0.11), but this difference was not statistically significant (F(1,11) = 3.132, p = 0.107). Comparing each strain with itself in both conditions, k value for SHR group in Condition A (0.15) was smaller than in Condition B (0.18). Conversely, WKY rats presented a larger k value in Condition A (0.14) than in Condition B (0.11). Statistical comparisons between these k values yielded non-significant results (p > 0.05). These results confirm what was observed in **Figure 1**. No statistical differences between groups were found in the adjustment of the individual data to the model, neither between conditions nor strains (A, SHR vs. WKY: F(1,11) = 0.161, p = 0.697; B, SHR vs. WKY: F(1,11) = 0.28, p = 0.871; SHR, condition A vs. B: F(1,11) = 0.403, p = 0.540; WKY, condition A vs. B: F(1,11) = 0.355, p = 0.564).

#### Schedule-Induced Drinking

Regarding schedule-induced behaviors, only schedule-induced drinking was developed, not schedule-induced running. Rats ran


Note. Mean ± SEM. R<sup>2</sup> is the coefficient of determination.

no more than 10 turns per session in at least half of the sessions during the experiment (except subjects WKY-1 and SHR-6), so data on schedule-induced running were not analyzed.

**Figure 4** depicts the mean number of licks at each delay value. A main effect was obtained for strain (F(1,10) = 6.075, p = 0.033, η = 0.378), with SHR giving more licks than WKY; and for delay (F(3,26) = 15.320, p > 0.000, η = 0.605), with more licks being given as the delay value increased. The interaction delay × strain was also statistically significant (F(3,26) = 3.875, p > 0.024, η = 0.279) and post hoc tests indicated that differences occurred in sessions with delay values of 9, 15, 18, 27 and 36 s. It can be discerned that SHR started developing schedule-induced drinking from the fourth session (6-s delay), showing its maximum number of licks when the delay was 27 s, with a mean of 737 licks. In contrast, WKY rats started to develop schedule-induced drinking when the delay value was 12 s, reaching the maximum number of licks when the delay was 33 s, with a mean of 342 licks. SHRs drank more than WKY rats throughout the procedure.

Circles depict data from SHR rats, and squares from WKY rats. Vertical bars denote standard error of the mean.

**Figure 5** shows the proportion of licks given every 1-s bin during the delay and the ITI for the average of all delaydiscounting sessions. **Figure 5A** compares the proportion of licks in every 1-s bin during the delay for both strains of rats. The analysis revealed a main effect for bin (F(2,18) = 9.262, p < 0.002, η = 0.481) and for the interaction bin × strain (F(2,18) = 4.465, p < 0.031, η = 0.309), but not a main effect of strain (F(1,10) = 2.229, p = 0.166, η = 0.182). Post hoc tests (Bonferroni) indicated that differences in favor of higher licking by SHRs occurred at bins 3 (p = 0.05), and 4, 5, 6 and 7 (p < 0.05), where licks peaked for both strains (although a bit earlier for SHR than WKY).

**Figure 5B** depicts the proportion of licks during the ITI after choosing LL or SS alternatives. In the ITI after choosing the LL alternative differences between strains were not significant (F(1,10) = 2.424, p = 0.151, η = 0.195), neither strain licked during those time periods. However, in the ITI after choosing the SS alternative, a main effect was found for strain (F(1,10) = 6.045, p < 0.034, η = 0.377), with SHR licking more than WKY rats. Post hoctests (Bonferroni) indicated that differences in favor of higher licking by SHRs occurred at bins 2, 3, 4, 5, 9 and 10 (p < 0.05). Statistical analyses also confirmed that rats of both strains drank significantly more during the ITI after choosing the SS alternative than after choosing the LL alternative (SHR: F(1,10) = 8.710, p < 0.015, η = 0.466; WKY: F(1,10) = 6.910, p < 0.025, η = 0.409).

#### DISCUSSION

The purpose of the present experiment was to evaluate if schedule-induced behaviors turn waiting-time less aversive, thus making rats behave more self-controlled, or if schedule-induced behaviors could serve as a model of compulsivity, by making rats behave more impulsively. We failed to find any effect of schedule-induced behavior on delay discounting in either strain. Differences between strains are typically observed when schedule-induced behavior could not be expressed (Fox et al., 2008; Íbias and Pellón, 2011; Aparicio et al., 2019). However, some studies found no strain differences (Adriani et al., 2003; Pardey et al., 2009; Garcia and Kirkpatrick, 2013; Botanas et al., 2016), which mirrors the results in the present study.

Preference for the LL option decreased as the value of the delay increased, which is consistent with several studies that evaluated the performance of SHR and WKY rats in delaydiscounting tasks (Fox et al., 2008; Aparicio et al., 2019). Nevertheless, the discounting rate in our experiment is more progressive and reached a lower percentage of responses to the LL option than previously reported, probably due to an effect of increasing the delay by only 3 s per session, instead of increasing it to the double of the previous delay (Adriani et al., 2003; Anderson and Woolverton, 2005; Fox et al., 2008; Hand et al., 2009; Moreno et al., 2010; Íbias and Pellón, 2011; Aparicio et al., 2019). These results support the relationship between the length of the delay to the reinforcer and its subjective value.

Differences in the discounting rate between SHR and WKY rats have been reported in previous studies (Bizot et al., 2007; Fox et al., 2008; Hand et al., 2009; Sutherland et al., 2009; Íbias and Pellón, 2011, 2014; Wooters and Bardo, 2011; Orduña, 2015; Orduña and Mercado, 2017; Aparicio et al., 2019). In our experiment, no strain differences were observed, although visual inspection shows a trend for SHRs to discount more steeply than WKYs between delays of 9 and 18 s in Condition B.

Perhaps the possibility to engage in schedule-induced behavior prevented the expected results of steeper delay discounting in SHR in comparison to WKY, which resumed to a certain extent (but not to statistical significance) when there was no possibility to engage in schedule-induced behavior. During Condition B, in comparison to Condition A, SHRs tended to be more impulsive while WKYs tended to be more self-controlled. These tendencies did not reach statistical significance perhaps due to the previous experience on delay discounting without the possibility to engage in schedule-induced behavior, and/or because each delay value lasted only one session, or the small sample size of the groups (though similar to previous reports: e.g., Fox et al., 2008; Orduña and Mercado, 2017).

Schedule-induced behaviors do not seem to have much of an effect on the performance of SHR animals, which could be seen as a reflection of their normal excessive behavioral base rate (Sagvolden, 2000; Adriani et al., 2003; Fox et al., 2008; Hand et al., 2009; Aparicio et al., 2019). However, the tendency of WKY

rats to behave more impulsively when they were able to engage in schedule-induced behavior is seemingly notorious. **Figure 6** shows that 50% of the WKY rats displayed clearly steeper delaydiscounting functions under Condition A in comparison to Condition B (WKY 1, WKY 4 and WKY 5), in contrast to the opposite result observed just in one rat (WKY 3). WKY rats do not usually show excessive behavior (Sagvolden, 2000; Adriani et al., 2003; Fox et al., 2008; Hand et al., 2009; Aparicio et al., 2019), but the excessiveness provided by the opportunity to engage in a schedule-induced behavior seemed to increase their activity during the task, making them behave like SHR subjects.

Differences between strains were observed in the development of schedule-induced behaviors. SHR developed schedule-induced drinking faster and at a higher rate than WKY rats, similar to what has been found in previous studies (Íbias and Pellón, 2011, 2014). Even though rats had simultaneous access to a bottle of water and a running wheel, only schedule-induced drinking was developed. This might be due to competition between licking and running, in which drinking was probably favored by the length of the interreinforcer interval used in this experiment (Roper, 1978; Pellón and Killeen, 2015), as it is one of the parameters that determine which schedule-induced behaviors are more likely to develop (Roper, 1978). Differences in schedule-induced drinking rates can be another reason to separate out SHR and WKY strains (see Moreno et al., 2010).

Distribution of licks during the response-reinforcer delay and the ITI exhibits the characteristic inverted U-shape function normally observed for schedule-induced drinking (e.g., López-Crespo et al., 2004; Íbias and Pellón, 2011; Álvarez et al., 2016). Since the delay initiated by the response becomes longer than the ITI, the inverted U-shape form is more defined in that time interval (Íbias and Pellón, 2011). Peak of the distribution of licks is usually located at the beginning of the interval, occurring during the first 15–20 s in the case of a regular presentation of food pellets (e.g., López-Crespo et al., 2004). In our study, licks during the response-reinforcer delay occurred during the first 3–9 s. This earlier location could be due to the short duration of the delays on the first delay-discounting sessions and the short experience with each delay value (only one session per delay) that could make it difficult for the rats to adjust to the interval length.

One of the arguments against schedule-induced behavior being considered operant is the early temporal location towards immediately after (rather than before) delivery of the reinforcer (Falk, 1971; López-Crespo et al., 2004; for an account of this, see Killeen and Pellón, 2013). In the present study, however, when rats had the opportunity to drink both before and after food reinforcement (LL trials), they allocated licks only during pre-food (the response-reinforcer delay) and not post-food (the ITI) periods, developing licking during the ITI just when there was no response-reinforcer delay (SS trials). These data can be interpreted as if pressing the lever and licking from the spout were part of the same behavioral pattern maintained by intermittent food reinforcement (Ruiz et al., 2016), being reinforcer effective given the systematic presence of behaviors in the context of spread reinforcement in time (Killeen and Pellón, 2013; Álvarez et al., 2016).

Impulsivity can be observed in two forms: cognitive impulsivity, determined by the choice, and motor impulsivity, understood as excessive behavior (Chudasama et al., 2003; Winstanley et al., 2004). Schedule-induced behaviors have been regarded as signs of motor impulsivity, but Íbias and Pellón (2011) proposed that schedule-induced behaviors could also reflect cognitive impulsivity because the amount of scheduleinduced drinking relates to parameters of the upcoming reinforcer (López-Crespo et al., 2004), making schedule-induced behaviors indistinguishable from operant behaviors (Killeen and Pellón, 2013).

Nevertheless, developing schedule-induced behaviors during Condition A made rats exhibit excessive behavior. Our results seem to indicate that there is no difference between cognitive and motor impulsivity. This is in line with the view that schedule-induced behaviors are operants (Killeen and Pellón, 2013), but goes against the categorization of impulsivity in two types (cognitive and motor) because it does not seem to be a difference between them in the present study. Scheduleinduced drinking could be part of the same behavioral pattern that determines the choice of subjects, as reported by other authors (Cleaveland et al., 2003; Machado and Keen, 2003; López-Tolsa and Pellón, in preparation). Similar findings have been observed in DRL schedules (Segal and Holloway, 1963) and the peak procedure (Mattel and Portugal, 2007), tasks that both involve self-control. Thus, impulsivity, in general, could be understood as excessive activity—hyperactivity (see also Íbias and Pellón, 2014).

Tendencies showed by each strain in this experiment point to be similar to previous findings reported with humans. Housden et al. (2010) compared Parkinson's disease patients that had or did not have comorbid impulsive-compulsive spectrum behaviors. They found that patients with comorbid impulsivecompulsive spectrum behaviors showed highly elevated delay discounting. Furthermore, Pinto et al. (2014) compared OCD patients with participants with an obsessive personality, although both are marked by compulsions, they seem to differ in impulsivity, as they found that OCD patients discounted faster in intertemporal choice procedures. Finally, Sohn et al. (2014) found that OCD patients showed more impulsivity than healthy people in different tasks.

Schedule-induced drinking has been proposed as a model of compulsivity (Moreno and Flores, 2012) because they share features like excessiveness, persistence and having no obvious relation to the overall goal (reinforcement). Considering the results described above with humans and the tendencies seen in our experiment, we can start suggesting, but not stating, that schedule-induced behavior could serve as a model of compulsivity, in the sense that performance of SHR could make them comparable to patients with Parkinson's disease with comorbid compulsivity and patients with OCD, while the tendency of WKY rats resembled participants in control groups of those studies (patients with Parkinson's disease without comorbid compulsivity, patients with obsessive personality, and healthy people), but only in Condition B. By allowing rats to engage in schedule-induced behaviors in Condition A (which models compulsive behavior), we favored the occurrence of compulsive behavior in both strains of rats. This conclusion, however, requires further experimental confirmation given the limitation of the current statistical results and the previous experience of the subjects.

In conclusion, our results failed to document any effect of schedule-induced behavior on delay discounting, suggesting that schedule-induced behaviors do not reduce impulsivity. Nevertheless, a tendency can be observed that seems to support the idea of schedule-induced behavior as a model of compulsivity. No strain statistical differences were found, which suggest that the SHR strain is unsuitable as a model for compulsivity. Furthermore, the distribution and location of licks within time intervals of the delay-discounting task support the notion of schedule-induced behavior as operant.

### ETHICS STATEMENT

All care and experimental procedures were in accordance with the Spanish Royal Decree 53/2013 regarding the protection of experimental animals and with the European Union Council Directive 2010/63. UNED bioethics committee approved the experimental protocol.

## AUTHOR CONTRIBUTIONS

SR designed research, performed the experiment, performed statistical analyses and wrote the initial version of the manuscript. GL-T designed research, assisted in programming the task, assisted in data analyses, contributed with writing. ES designed research, programmed the task, reviewed the writing. RP designed research, supervised work, reviewed the writing.

### FUNDING

All research aspects related to this article were supported by grant PSI2016-80082-P from Ministerio de Economía, Industria y Competitividad, Secretaría de Estado de Investigación, Desarrollo e Innovación, Spanish Government (RP, Principal Investigator). SR was supported by a Spanish Government grant from Ministerio de Educación y Cultura. GL-T was awarded a Scholarship to undertake PhD-level studies abroad (255325/383739) by the Consejo Nacional de Ciencia y Tecnología (CONACYT), Mexican Government. ES was awarded an OsloMet Research and Development Grant for Staying Abroad (17/00164).

#### REFERENCES


### ACKNOWLEDGMENTS

We thank Antonio Rey for excellent technical support and Javier Íbias for commentaries on prior versions of the manuscript.


**Conflict of Interest**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Ramos, López-Tolsa, Sjoberg and Pellón. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Impulsivity and Compulsivity After Subthalamic Deep Brain Stimulation for Parkinson's Disease

Sara Scherrer† , Andrew H. Smith† , Jaimie Gowatsky, Christina A. Palmese, Joohi Jimenez-Shahed, Brian H. Kopell, Helen S. Mayberg and Martijn Figee\*

Nash Family Center for Advanced Circuit Therapeutics, Icahn School of Medicine at Mount Sinai, New York, NY, United States

Impulsivity and compulsivity are prominent non-motor problems in Parkinson's disease (PD). Despite 20 years of research, there is still an ongoing debate as to whether subthalamic deep brain stimulation (STN DBS) for PD exacerbates or improves these symptoms. Here, we review how STN DBS affects clinical symptoms and neurocognitive aspects of impulsivity and compulsivity. When comparing patients post- to pre-surgery, in the majority of studies STN DBS for PD is associated with a decrease in clinically diagnosed impulse-control disorders and disorders of compulsivity. To avoid confounds, such as post-surgical decreases in dopaminergic medication doses, comparisons can also be made between DBS "On" versus "Off" conditions. These experimentally assayed effects of STN DBS with respect to neurocognitive aspects of impulsivity and compulsivity are more mixed. STN DBS improves behavioral flexibility without impairing negative feedback learning, delay discounting, or inhibitory control, as long as stimulation is restricted to the dorsal STN. However, STN DBS may drive impulsive actions when a subject is faced with competing choices. We discuss how motivated responses may be either enhanced or impaired by STN DBS depending on engagement of dorsal or ventral STN-mediated circuits. Future studies should combine structural and functional circuit measures with behavioral testing in PD patients on and off medication and stimulation. A more sophisticated understanding of how to modulate corticostriatal-thalamo-cortical loops will increase the likelihood that these circuit manipulation techniques can successfully be applied to a wider range of neuropsychiatric disorders.

Keywords: Parkinson, deep brain stimulation, subthalamic nucleus, impulsivity, compulsivity

#### INTRODUCTION

Parkinson's Disease (PD) is a neurodegenerative disorder affecting basal ganglia systems controlling motor and non-motor functions. Impulsivity and compulsivity are prevalent non-motor features of PD associated with lack of self-control. Whereas impulsivity involves diminished control over prematurely expressed actions, compulsivity refers to diminished control over repetitive, ritualistic thoughts and behaviors. In patients with PD, impulsive and compulsive symptoms can cause suffering and functional impairments, and are often unresponsive to, or even induced by, PD medications. The standard treatment for PD is dopamine replacement therapy (DRT), with either levodopa or dopamine agonists. Although these drugs are highly effective for motor symptoms such as rigidity, bradykinesia and resting tremor, they are also associated with a 2- to 3.5-fold increased odds of developing impulsive or compulsive behaviors (Weintraub et al., 2009; Santangelo et al., 2013; Kim et al., 2015).

#### Edited by:

David Belin, University of Cambridge, United Kingdom

#### Reviewed by:

Lindsay Halladay, Santa Clara University, United States Jean-Luc Houeto, CIC INSERM 1402 University of Poitiers, France Sebastien Carnicella, INSERM U1216 Grenoble Institut des Neurosciences (GIN), France

\*Correspondence:

Martijn Figee martijn.figee@mssm.edu †These authors have contributed equally to this work

#### Specialty section:

This article was submitted to Motivation and Reward, a section of the journal Frontiers in Behavioral Neuroscience

> Received: 11 November 2019 Accepted: 17 March 2020 Published: 23 April 2020

#### Citation:

Scherrer S, Smith AH, Gowatsky J, Palmese CA, Jimenez-Shahed J, Kopell BH, Mayberg HS and Figee M (2020) Impulsivity and Compulsivity After Subthalamic Deep Brain Stimulation for Parkinson's Disease. Front. Behav. Neurosci. 14:47. doi: 10.3389/fnbeh.2020.00047

For PD patients developing incapacitating motor fluctuations or side-effects on DRT, subthalamic deep brain stimulation (STN DBS) has proven to be an effective augmentation strategy. STN DBS, relative to DRT, may allow for less dopaminergic striatal stimulation as a result of a reduction in the total levodopa equivalent daily dose (LEDD) by an average of 73% (Lhommée et al., 2012). In theory, these medication changes may decrease the risk for impulsive and compulsive behaviors, implying that STN DBS is a beneficial option for PD patients suffering from these symptoms. However, it is possible that the stimulation is not limited to the motor subdivision of the STN, and also affects cognitive-associative and/or limbic subdivisions, which could either improve or exacerbate impulsivity and compulsivity. Indeed, the literature is conflicted regarding the effects of STN DBS on impulsivity and compulsivity in PD, with evidence for both improvement and worsening of these symptoms.

This article aims to elucidate the effects of STN DBS on impulsivity and compulsivity in PD, taking a new approach to reviewing the literature. Considering the connection of dopamine with impulsivity and compulsivity, we explore the behavioral influence of post-surgical changes in dopaminergic medication doses. In addition, we did not limit our search of the literature to inventories of clinical symptoms and diagnoses before and after surgery. Rather, we focused on changes in neurocognitive paradigms that reflect different aspects of impulsivity and compulsivity, and that can be quantified during experimental changes in stimulation delivery.

#### Impulsivity

Impulse control disorders (ICDs) as classified in the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5, American Psychiatric Association, 2013) are conditions involving problems of emotional and behavioral self-control, including intermittent explosive disorder (IED), kleptomania, pyromania, and other specified or unspecified ICDs. More commonly observed in PD are diminished control over gambling, sexual behaviors, eating, shopping, hobbyism, non-goal directed actions (punding) and medication use (Dopamine Dysregulation Syndrome, DDS) (Weintraub et al., 2009). The Questionnaire for Impulsive-Compulsive Disorders in Parkinson's Disease Rating Scale (QUIP-RS) is the most widely used and well-validated scale for measuring likelihood and severity of ICDs in PD (Weintraub et al., 2012). As the name suggests, this scale intends to measure severity of both impulsive and compulsive disorders. Its questions therefore relate to impulsive as well as compulsive aspects of problematic behaviors, i.e., excessive urges, distressing desires, and obsessive thinking. The total score, however, is defined as severity of ICD without sub-scores for impulsive or compulsive aspects.

Complicating efforts to disambiguate impulsivity and compulsivity in commonly used scales, there has not always been consensus about the definition of impulsivity itself. Many authors agree upon a definition of impulsivity as a tendency to act prematurely with little foresight, a failure to suppress inappropriate motor, cognitive or emotional responses. This definition implies that impulsivity is a multifaceted construct requiring assessment with multiple distinct paradigms. The most widely used paradigms assess impulsivity as: (1) impaired negative feedback learning, e.g., the Iowa Gambling Task (Bechara et al., 1997) or the Probabilistic Selection Task (Frank et al., 2007); (2) a preference for small immediate reward over larger delayed reward, i.e., delayed discounting tasks (Loewenstein, 1988); (3) making responses that are premature or should be withheld, e.g., the Go/No Go Task (Ballanger et al., 2009) or Stroop test (Stroop, 1935).

### Compulsivity

The most common disorders of compulsivity are defined in the DSM-5 under Obsessive-Compulsive and Related Disorders (OCRD). These include obsessive-compulsive disorder (OCD), body dysmorphic disorder, hoarding disorder, trichotillomania (hair pulling disorder) and excoriation (skin-picking) disorder. The DSM-5 also allows for the identification of compulsions in other mental disorders, defining compulsions as repetitive behaviors or mental acts, such as hand washing, ordering, checking, praying, counting, or repeating words, which the individual feels driven to perform in response to an obsession or according to rules that must be applied rigidly. Compulsions are usually aimed at preventing or reducing anxiety, distress, or dreaded events, despite insight that the behaviors are not realistically connected to these outcomes. Severity of the prototypical disorder of compulsivity, OCD, can be measured with clinician-rated scales such the Maudsley Obsessive-Compulsive Inventory (MOCI, Hodgson and Rachman, 1977), or the Yale-Brown Obsessive-Compulsive Scale (Y-BOCS, Goodman et al., 1989). Examples of transdiagnostic compulsivity questionnaires are the DSM-5 obsessive– compulsive spectrum scale (LeBeau et al., 2013) and the Padua Inventory (Sternberger and Burns, 1990).

As a cognitive construct, compulsivity, like impulsivity, may be decomposed into various factors of a mainly cognitive, affective or motivational nature (Figee et al., 2016). First, similar to impulsivity, compulsivity implies engagement in selfdefeating repetitive behaviors, which may be related to altered reward or punishment sensitivity. Second, the diminished ability to ignore or stop unwanted ideas or actions suggests the presence of cognitive or behavioral inflexibility, as measured with reversal-learning tasks (i.e., Wisconsin Card Sorting Task, Grant and Berg, 1948) or attentional set-shifting tasks (i.e., Trail Making Task, Ardouin et al., 1999). Third, habitual responding and diminished goal-directed activity suggests excessive habitlearning, as measured with instrumental or model-based learning paradigms (Daw et al., 2005). However, firm consensus about the definition of compulsivity as a cognitive construct remains to be established. Further, paradigms which explicitly test compulsivity are difficult to develop because compulsive behaviors are often triggered by person-specific circumstances that are challenging to recreate in generalizable tasks. In addition, there may be overlap between impulsivity and compulsivity particularly in reinforcement learning paradigms. Nevertheless, paradigms testing cognitive and behavioral flexibility and habit-learning tap into important aspects of compulsivity which we use in the present review.

## METHODS

### Search Strategy

fnbeh-14-00047 April 22, 2020 Time: 15:16 # 3

We conducted a literature search using the Pubmed database for articles published between January 1st 2002 and June 27th 2019. Keywords for impulsivity associated with subthalamic DBS in PD were based on ICDs, associated scales, and neurocognitive paradigms:

Subthalamic DBS, Parkinson, Impulsivity, Impulse control disorder, Intermittent Explosive Disorder, Kleptomania, pyromania, gambling, sexual behaviors, eating, shopping, hobbyism, punding, Dopamine Dysregulation syndrome, QUIP-RS, extinction learning, reward choice, response inhibition, Barratt Impulsiveness Scale, The Iowa Gambling Task, Instrumental Learning Task, The Game of Dice Task, The Temporal Discounting Task, Deal or No Deal, Cambridge Gambling Task, The Probabilistic selection task, Status Quo Task, Auditory two-alternative forced choice task, The Go/No-Go task, The Stroop Test.

Similarly, for compulsivity we used the following keywords:

Subthalamic DBS, Parkinson, compulsivity, Obsessive-Compulsive Disorder (OCD), OCRD, body dysmorphic disorder, hoarding, trichotillomania, excoriation, hand washing, ordering, checking, praying, counting or repeating words, Maudsley Obsessive-Compulsive Inventory, Yale-Brown Obsessive-Compulsive Scale, Obsessive–Compulsive Spectrum Scale, Padua Inventory, reward punishment sensitivity, cognitive inflexibility, habit learning, probabilistic reversal-learning, attentional setshifting tasks, habit-learning, devaluation, Trail Making Task, Wisconsin Card Sorting Task, Probabilistic classification task.

After the initial search, a reference analysis was conducted to find additional reports. We excluded studies published before 1999, reviews, animal studies, and computational modeling studies. Only studies investigating the effects of bilateral STN DBS in PD patients were included. Clearly defined assessment tools and dopaminergic medication status at time of testing had to be described. Reports in languages other than English, or that did not include statistical tests, were excluded.

### Impulsivity Measures

We searched for studies reporting ICDs related to STN-DBS in PD, and for studies measuring DBS-related impulsivity with the following scales or paradigms:

Barratt Impulsiveness Scale (BIS-11, Patton and Stanford, 1995): a 30-item questionnaire designed to assess impulsive personality traits, with sub-scores for attentional impulsivity, motor impulsivity and non-planning impulsivity.

Negative Feedback Learning (**Table 1**): Iowa Gambling Task (IGT, Bechara et al., 1997), Game of Dice Task (GDT, Brand et al., 2005), Instrumental Learning Task (ILT, Seymour et al., 2016), and Probabilistic Selection Task (Frank et al., 2004, 2007). These paradigms may be used for measuring impulsivity defined as impaired learning from negative feedback. In the IGT and GDT, participants have to balance their choices between safe and advantageous options (card decks or dice rolls with low reward and small losses) and risky and disadvantageous options


TABLE

1 | Negative feedback learning.

(high rewards and high losses). In the ILT, participants choose from four abstract pictures based on feedback from previous trials where their choice was either rewarded (receiving tokens) or punished (losing tokens). In all of these tasks, impulsive participants may insufficiently learn from previous loss-trials or series. A probabilistic selection task may also be used to measure impulsivity defined as impaired learning from negative feedback, or (see below) as premature responding under highconflict conditions (when presented with conflicting positive reward probabilities). Impulsive participants may insufficiently learn from their previous losses and/or respond faster in highconflict trials.

Delay Discounting (**Table 2**): Delay discounting tasks may be used to measure impulsivity defined as discounting of larger delayed reward over smaller immediate reward. Examples include the Kirby Delay-Discounting Task (Kirby and MarakoviC, 1996 ´ ). Impulsive participants may impulsively discount future reward, i.e., show increased delay discounting. The Cambridge Gambling Task (Rogers et al., 1999), used by Torta et al. (2012), measures delay aversion, identifying subjects who repeatedly pick the initial bet offered, and results are here presented alongside studies of delay discounting, the measure of impulsive tendencies it most closely resembled.

Inhibitory control (**Table 3**): Go/No-Go task (Donders, 1969), Stop Signal Task [SST, (Logan and Cowan, 1984)], Status Quo Task (SQT, Fleming et al., 2010) and Stroop test (Stroop, 1935), Simon (Simon and Rudell, 1967). These tasks may be used to measure impulsivity defined as impaired inhibitory control over prepotent motor or cognitive responses. In Go/No-Go, SST and SQT, participants are presented with a stimulus that requires them either to respond (Go) or withhold a response (No-Go/Stop). Impulsive participants may make more commission errors (Go responses on No-go or Stop trials) and/or anticipation errors (responding too fast). SQT is a modified Go/No-Go measuring response inhibition under high conflict. A ball appears either between two lines (Go) or outside the lines (No-Go). High conflict occurs when the ball is almost touching the line. In the Stroop test, words representing colors are written in colors incongruent to the written word, and participants are required to state aloud the ink color as opposed to reading the written word. Impulsive participants may make more commission errors (reporting the word instead of the color). In the Simon task participants respond to visual cues prompting either contra- or ipsilateral movements.

Responding Under High Conflict (**Table 4**): ILT, Probabilistic selection tasks, delay discounting, and SQT, discussed above, can all be utilized to gauge impulsive behavior under so called "high conflict" scenarios. These are situation where the range of options available for selection are less readily distinguished from one another. Other paradigms have also been used for this purpose, such as an auditory forced choice task (London et al., 2019).

#### Compulsivity Measures

In addition to studies reporting the prevalence of OCD and OCRD before and after STN DBS in PD, we searched for studies measuring OCD-severity using the Maudsley Obsessive-Compulsive Inventory (MOCI, Hodgson and Rachman, 1977)



 of PD cases across different conditions are given. Also provided is number of months between surgery and evaluation,

 and ↓

denotes impairment,

 nucleus of the thalamus (VIM), globus pallidus interna. (Gpi), not applicable (N/A). All DBS experiments

 and – denotes no change. For ease of interpretation,

 effects of SNT DBS activation in experimental

 impulsivity construct utilized, and experimental

 sessions are highlighted

 involved STN DBS unless specified otherwise.

 in a separate column.

 \*No P-value from

 results. ↑

and healthy controls (HC). Distribution

denotes improvement

Sub-thalamic

an experimental

 nucleus (STN), ventral intermediate

 session comparing

 Stimulation

 On vs. Stimulation

 Off.

 in performance


or the Yale-Brown Obsessive- Compulsive Scale (Y-BOCS, Goodman et al., 1989). The MOCI is a self-rated 30-item scale with sub-scores for checking, cleaning, slowness, and doubting. The Y-BOCS is a clinician-rated 10-item scale with sub-scores for obsessions and compulsions. In addition, we searched for studies examining aspects of compulsivity in PD related STN DBS utilizing one or more of the following paradigms: Habit learning tasks (Daw et al., 2005; Gillan et al., 2011) measure compulsivity defined as the dominance of habits over goal-directed learning. Participants learn to respond to stimuli with rewarding or negative reinforcing outcomes which are eventually devalued. Compulsive participants keep habitually responding to devalued stimuli, or show more model-free learning (predicted by reinforcement history) than model-based

learning (adapted to devaluation). Perseveration (**Table 5**): Wisconsin Card Sorting Task (WCST, Grant and Berg, 1948) is a reversal learning task which may be used to measure compulsivity defined as cognitive inflexibility. Cards must be sorted on the basis of either number, form or color with the sorting rule alternating after 10 correct responses, a fact learned by the participant through trial and error. Compulsive participants make more perseverative errors, indicated by two successive sorts on an incorrect dimension. The original Wisconsin Card Sorting Task (WCST) uses 128 response cards, and a Modified Wisconsin Card Sorting Task (MWCST) simplifies the task, using only 48 response cards (Nelson, 1976).

Attentional set shift (**Table 6**): Trail Making Task (TMT) Part B (Corrigan and Hinkeldey, 1987) may be used to measure compulsivity defined as impaired attentional setshifting. Participants must draw a line alternating between letters and numbers (A-1-B-2-C-3, etc.). Compulsive individuals tend to have longer completion times due to less attentional flexibility.

## RESULTS

### Impulse Control Disorders

We found 12 studies investigating the prevalence of ICDs (gambling, compulsive shopping, binge eating, hypersexuality, punding, DDS) in a total of 582 PD patients pre- and (6– 12 months) post STN DBS (Ardouin et al., 1999; Lim et al., 2009; Lhommée et al., 2012; Shotbolt et al., 2012; Eusebio et al., 2013; Kim et al., 2013; Amami et al., 2015; Castrioto et al., 2015; Gee et al., 2015; Merola et al., 2017). The combined prevalence of ICDs in these patients was 28% (N = 162) before STN DBS and 6% (N = 32) after STN DBS (with variable post-surgical LEDD decreases). In the 162 patients that already showed ICDs before DBS, post-surgical improvement of ICDs was observed in 86% (139 patients), including full remission in 68%. Worsening of ICDs after STN-DBS was found in only 3% of all patients (Pallanti et al., 2010; Lhommée et al., 2012). Although new onset of ICDs after STN-DBS was reported in 38 patients, most of these cases were transient (Shotbolt et al., 2012; Kim et al., 2013; Amami et al., 2015). Factors that were associated with ICDs after DBS were personality disorders, dyskinesias and higher post-surgical LEDD.

fnbeh-14-00047 April 22, 2020 Time: 15:16 # 6

TABLE

4 |

Responding

 under high conflict.


TABLE 5 |

Perseveration.

sub-thalamic

 nucleus (STN), globus pallidus interna (GPi).

∗No P-value from an experimental

 session comparing

 Stimulation

 On vs. Stimulation

 Off.


 nucleus (STN), globus pallidus interna (GPi).

6 |

no change. For ease of interpretation, ∗STN and GPi patients were jointly analyzed. \*\*No P-value from an experimental

 effects of SNT DBS activation in experimental

 sessions are highlighted

 session comparing

 Stimulation

 On vs. Stimulation

 Off.

 in a separate column. Trail making task (TMT), sub-thalamic

### Impulsivity Clinical Scales

fnbeh-14-00047 April 22, 2020 Time: 15:16 # 9

Impulsivity scores on the Barratt Impulsiveness Scale (BIS-11, Patton and Stanford, 1995) were measured in six studies (total of 165 patients) comparing scores either between DBS On and Off conditions (Torta et al., 2012; Seinstra et al., 2016), or between DBS On and PD-patients On medication or healthy controls (Hälbig et al., 2009; Evens et al., 2015; Hagelweide et al., 2018; Irmen et al., 2019). One study reported significantly worse impulsivity scores in a DBS-group compared to healthy controls (Hälbig et al., 2009). For all other studies, no differences in BIS-11 impulsivity scores were found between DBS On and control groups.

#### Negative Feedback Learning

Impulsivity defined as impaired negative feedback learning was measured in five studies in a total of 95 PD STN DBS patients (**Table 1**; Frank et al., 2007; Boller et al., 2014; Castrioto et al., 2015; Evens et al., 2015; Seymour et al., 2016). An early study using a probabilistic selection task reported that negative feedback learning was impaired by dopaminergic medication in PD patients without DBS, but not by DBS activation in PD patients who had been implanted (Frank et al., 2007). While a subsequent study using the GDT to evaluate negative feedback learning in implanted patients did not replicate the deleterious effect of dopaminergic medication (Boller et al., 2014), a different study called attention to the much lower doses of dopaminergic medication used post-implantation, and showed a relationship between post-implantation medication dose decreases and recovery of negative feedback learning as measured by the IGT (Castrioto et al., 2015). Both of these two studies (Boller et al., 2014; Castrioto et al., 2015), as well as a separate study using the IGT (Evens et al., 2015), replicated that initial finding (Frank et al., 2007) that DBS does not impair negative feedback learning. Only one study (Seymour et al., 2016), employing an ILT, suggested decreased negative feedback learning secondary to DBS activation. In conclusion, the reviewed data suggest that dopaminergic medication increases impulsivity defined as impaired negative feedback learning, but STN DBS does not. Although negative feedback learning may even improve after STN DBS, this appears to be primarily related to the postsurgical decrease of dopaminergic medication.

#### Delay Discounting

Four studies measured DBS-related delay discounting in a total of 108 PD patients (**Table 2**; Torta et al., 2012; Evens et al., 2015; Seinstra et al., 2016; Seymour et al., 2016). None of these studies found evidence that DBS activation interferes with subjects' ability to delay claiming a reward in order to maximize total reward, suggesting that DBS does not affect this aspect of impulsivity.

#### Inhibitory Control

Impulsivity defined as premature responding to stimuli was measured in a total of 130 STN-DBS PD patients (**Table 3**; Jahanshahi et al., 2000; Dujardin et al., 2001; Schroeder et al., 2002; van den Wildenberg et al., 2006; Page and Jahanshahi, 2007; Ballanger et al., 2009; Hershey et al., 2010; Georgiev et al., 2016; Aiello et al., 2017). An early study (van den Wildenberg et al., 2006) was not able to show a significant difference (P = 0.08) between DBS On and Off conditions when examining rate of NoGo commission errors. However, two subsequent studies (Ballanger et al., 2009; Hershey et al., 2010) did show that stimulation impaired inhibitory control, making subjects respond to NoGo signals more frequently. Notably, Hershey et al. (2010) used an innovative strategy of recruiting only subjects who had STN electrodes placed at different depths on contralateral sides, enabling them to fine map the anatomy most relevant to impulse control. Specifically, they identified the ventral STN as most relevant to impaired inhibition, consistent with the ventral STN's closer connection to association cortex compared to the dorsal STN, which is thought to be more closely connected to sensorimotor cortex.

Authors employing the Simon task also found evidence for the functional relevance of a ventral-dorsal STN axis, namely that dorsal stimulation leads to more inhibitory capacity compared to ventral territory stimulation (van Wouwe et al., 2017). Georgiev et al. (2016) were unable to confirm higher rates of NoGo commission errors during STN stimulation, and also did not observe increases in premature responding (anticipation errors). One study showed significant impairment in NoGo paradigm response inhibition prior to DBS being switched on (Aiello et al., 2017), but these subjects were evaluated only 5 days post-surgery, making it unclear whether acute postoperative factors contributed to cognitive deficits. Jahanshahi et al. (2000) showed a heightened error rate on the Stroop test, which was specific to STN stimulation, being unaffected by globus pallidus interna (GPi) stimulation. Two subsequent studies (Schroeder et al., 2002; Page and Jahanshahi, 2007) were unable to replicate this detrimental impact on Stroop error rate. Dujardin et al. (2001) looked at Stroop performance preand post-surgery, but did not experimentally compare DBS On and Off conditions. In conclusion, although initial small studies report stimulation-related impaired inhibitory control, subsequent studies in larger samples showed no deleterious effect of stimulation, or even improved inhibitory control, especially with stimulation of the dorsal STN.

#### Responding Under High Conflict

Impulsive responding under high conflict scenarios was measured in four studies in a total of 65 STN DBS patients (**Table 4**; Frank et al., 2007; Zaehle et al., 2017; London et al., 2019). Frank et al. (2007) and Seymour et al. (2016), already discussed above, were able to utilize their respective paradigms to assess whether exacerbation of impulsivity by STN DBS could be more easily appreciated under conditions of high conflict. Only Frank et al. (2007) observed a significant effect, with the DBS On condition characterized by an impairment in the ability of subjects to slow down during situations requiring more careful evaluation. Along similar lines, Zaehle et al. (2017) looked for a stimulation-difficulty interaction on "false alarm" rates during the SQT, but reported a non-significant interaction term. In contrast, London et al. (2019) who assigned each subject "impulsivity indexes" based on sequential same-sided clicks in an auditory two-alternative forced choice task, did report a positive finding. Specifically, they showed that stimulation increased the impulsivity index only during trials with higher levels of conflict. Together, these studies suggest that STN DBS negatively affects impulse control under high conflict conditions.

### Disorders of Compulsivity

fnbeh-14-00047 April 22, 2020 Time: 15:16 # 10

Subthalamic deep brain stimulation for OCD gained attention after two patients were treated with STN DBS for PD with comorbid OCD (Mallet et al., 2002). Both cases experienced improvement in motor as well as obsessive-compulsive symptoms. Two weeks after the procedure, the patients were free from compulsions, and obsessions improved by 58% in patient 1 and by 64% in patient 2. Since then, the literature on co-morbid OCD and OCDR in patients with PD undergoing STN DBS is surprisingly sparse.

## Compulsivity

#### Clinical Scales

Only two studies (Alegret et al., 2001; Hälbig et al., 2009) used obsessive-compulsive symptom scales during STN DBS for PD, both using the MOCI. One of these studies administered the MOCI pre and post STN DBS (n = 15), finding that STN DBS improved obsessive-compulsive traits. The pre-surgical score average of 8.40 decreased significantly at the 3 months follow-up, down to 5.47 (Alegret et al., 2001). A second study reported no difference in MOCI scores when making an intersubject comparison between 16 STN PD patients (M = 6.8, SD = 2.59) and 37 PD patients treated only with medication (M = 6.79, SD = 3.85) (Hälbig et al., 2009). No studies were found that administered the Y-BOCS in Parkinson's patients treated with STN DBS.

#### Habit Formation

We were unable to find studies that applied habit-learning paradigms in STN DBS PD patients. However, we did find studies measuring compulsivity defined as impaired perseveration or attentional set-shifting (see below).

#### Perseveration

A total of six publications were reviewed which evaluated the effects of STN DBS on compulsivity in a total of 271 subjects using a perseveration paradigm (**Table 5**), which aims to measure persistence of unrewarded responses and inability to update decision rules. All protocols utilized the WCST or the MWCST. Rates of perseverative errors, reflecting inappropriate continued employment of ineffective selection strategies, are considered to reflect cognitive inflexibility. Isolating the behavioral rigidity common in individuals with compulsive behaviors and disorders, this metric has been widely employed to evaluate one facet of compulsivity in the STN DBS population.

Upon review of perseverative error rates reported in these studies, it appears that, overall, STN DBS has no deleterious effect on, if not improves, this index of compulsivity. Three studies (Ardouin et al., 1999; Contarino et al., 2007; Tröster et al., 2016) found no significant effect of STN DBS on perseverative errors. All three of these studies used a withinsubjects design, comparing patients before and after device implantation. The study by Tröster et al. (2016) differed in that a subset of patients were randomized to delayed device activation, and the authors used the original WCST instead of the less ambiguous MWCST (Nelson, 1976). Another study comparing pre- and post-operative subjects (Aono et al., 2014) also found no difference when using the WCST, but did find significant performance improvement on the MWCST. Castelli et al. (2006) also reported significant improvement on the MWCST postsurgery. Only one study (Jahanshahi et al., 2000) experimentally assessed the effects of stimulation by conducting testing both with the stimulator on and with the stimulator switched off. These authors found that STN stimulation, but not GPi stimulation, decreased the number of MWCST perseverative errors.

#### Attentional Set-Shifting

Another means of quantifying inflexibility is via measuring ability to shift attention. Seven studies with a total of 155 STN DBS patient (Ardouin et al., 1999; Jahanshahi et al., 2000; Alegret et al., 2001; Brusa et al., 2001; Dujardin et al., 2001; Perozzo et al., 2001; Castelli et al., 2006) deployed the TMT, measuring time to completion (**Table 6**). Shorter time to completion was taken to reflect less rigid and more flexible cognition. Two early studies (N = 49 and 15, respectively) compared STN DBS patient pre- and post-implantation and found improvement, with patients able to complete the task more quickly once DBS had begun (Ardouin et al., 1999; Alegret et al., 2001). Two other early studies with a smaller combined sample size (N = 9 and 20, respectively) using a similar pre-/post-implantation design did not find a significant effect (Dujardin et al., 2001; Perozzo et al., 2001), nor did a later study assessing 55 patient pre- and post-implantation (N = 55). Two studies used controlled experimental designs, but had smaller samples sizes, reflecting the greater difficulty of conducting such studies. The first (Jahanshahi et al., 2000), found that stimulation significantly shortened TMT completion time in four STN DBS patients, while the second (Brusa et al., 2001) detected only a non-significant decrease in mean completion time when stimulation was switched on in a mixed sample of STN and GPi DBS patients off medication (Brusa et al., 2001).

### DISCUSSION

To date, studies in DBS PD patients have employed a wide variety of strategies to dissect out the specific behavioral relevance of the STN, a key relay station in cortico-stiatal-thalamocortico (CSTC) loops. The sheer diversity of approaches, both conceptually and experimentally, is evident from the above review, and the range of paradigms can make it difficult to discern larger patterns in the literature. Twenty years of work on DBS in PD has, though, produced a number of findings consistent with a facilitation of cognitive-behavioral flexibility alongside restoration of purposeful movements, although possibly at the cost of some diminished capacity for behavioral inhibition especially under high-conflict conditions.

Subthalamic deep brain stimulation for PD is thought to exert its primary effect by restoring the balance of activity between direct and indirect pathways (Jakobs et al., 2019). Early studies reinforced concerns that damage to the integrity of these pathways in PD could lead to an unwanted increase in impulsive and compulsive behaviors when DBS was applied. However, when evaluating the extant literature on this topic, we found that the majority of studies identified a decrease in ICDs after STN DBS. Nevertheless, there are important caveats. ICDs are binary clinical diagnoses based on the detrimental impact of behavioral patterns on social functioning over time, which means they are not experimentally tractable constructs. Evidence about ICDs is therefore limited to pre- and postsurgery comparisons, as opposed to laboratory manipulations. However, pre- and post-comparisons are subject to confounds, such as post-operative LEDD decreases (Castrioto et al., 2015). Dopaminergic medications are associated with ICDs and DDS (Castrioto et al., 2015), and dose reductions (a major goal of DBS surgery) may be the predominant factor influencing changes in ICD prevalence, leading to a dramatic decrease post-surgery (5%) compared to pre-surgery (28%). Avoiding such confounds was an important motivation behind our prioritizing identification of non-binary measures such as scales (e.g., the BIS) and cognitivebehavioral constructs (e.g., delay discounting) more amenable to controlled evaluation via electrode current manipulation. This approach revealed the value of targeting different subdomains of impulsivity, with different tasks yielding different findings: (1) The evidence suggests that negative feedback learning is impaired by dopaminergic medication but not by STN DBS. (2) We found no evidence that delay discounting tendencies are exacerbated by STN DBS. (3) With response inhibition, the picture is more mixed. Earlier studies suggest that STN DBS may impair inhibitory control. However, later studies suggest that the effects on inhibitory control may be

dependent on the location of stimulation, with impairments after ventral stimulation but improvements after dorsal stimulation. (4) Finally, in so called "high conflict" scenarios, where a greater degree of ambiguity is built into the patient's task using more involved study designs, we identified evidence of heightened impulsivity after STN DBS.

Studies of compulsivity-related diagnoses and measures during STN DBS for PD were much sparser than probes of impulsivity. For unclear reasons, studies of compulsivity were also less likely to report experimental current manipulations than pre-/post-surgery comparisons, which as discussed above are subject to confounds. One study (Alegret et al., 2001) found a decrease in OCD severity (MOCI) scores post-STN electrode activation. In terms of specific facets of compulsivity, only one study experimentally quantified perseverative error rates in DBS-On vs. –Off conditions (Jahanshahi et al., 2000), finding that stimulation decreased perseverative errors. Similarly, only two studies compared set shifting under On vs. Off conditions, with one finding a significant increase in flexibility (Jahanshahi et al., 2000), and one finding a non-significant improvement (Brusa et al., 2001). Thus, although all three of the identified experimental studies were directionally consistent with diminished compulsivity during DBS, the extremely limited total sample size warrants substantial caution in drawing generalized conclusions. Further support for the relevance of the STN to compulsivity comes from the literature on STN DBS for treatment of OCD in non-PD patients. Although this body of evidence is beyond the scope of the current review, in line with our summary of ventral territory-specific stimulation effects on impulsivity, amelioration of OCD symptoms has generally been attempted via use of more ventral electrode contacts within the STN (Mallet et al., 2008) (**Figure 1**).

The results discussed in this review raise the possibility that there may be benefits as well as trade-offs inherent in

modulating the balance between the action suppressing and promoting functions of CSTC loops. Dampening of suppressive STN (indirect) pathways may enhance behavioral flexibility, but at the cost of impaired self-control when faced with competing choices. STN DBS is able to normalize motor control by reducing excessive beta band activity in STN and motor cortex (Whitmer et al., 2012). However, appropriate motor responses when faced with competing choices may also require coherence of theta and delta bands between the STN and medial prefrontal cortex, to facilitate the appropriate delay of motivated responses (Zavala et al., 2014). The reviewed data suggest that STN DBS may facilitate these effects in a circuit-specific manner, with dorsal stimulation being associated with less impulsivity relative to ventral stimulation. The dorsal STN shows connectivity with (pre-) motor and prefrontal motor control areas, whereas the more ventral STN is linked to limbic circuits regulating motivational control. This suggests that for impulsive PD patients, stimulation should probably be restricted to dorsal STNpathways. However, symptoms suggestive of motivational deficits such as apathy or depression, were recently found to improve with stimulation of ventral limbic pathways (Petry-Schmelzer et al., 2019). Optimizing non-motor outcomes during STN DBS may therefore depend on reversing patient-specific imbalances in distinct motor and motivational circuits.

Efforts to learn about impulsivity and compulsivity via STN DBS in PD patients rely on convenience samples, and while they present unique opportunities to causally test hypotheses in humans, there are important limitations that must be borne in mind. First, it is not possible to test specificity of regional effects, since it would be ethically problematic to place electrodes into brain regions without a reasonable amount of prior evidence to suggest clinical benefit. This limitation is not present in the animal literature, where circuit manipulations may be performed in line with principles of experimental design. Indeed, early rodent studies of experimental STN lesions demonstrated an increase in premature responding during the 5-choice serial reaction time test, although there was also evidence of increased perseveration (Baunez et al., 2011). More recent work in rodents examining repeated sessions of STN high frequency stimulation has also shown premature responding during measurement of impulse control (Aleksandrova et al., 2013). Experiments performed in model systems have also pointed to mechanistic hypotheses about DBS beyond straightforward inactivation of STN neurons, such as possible facilitation of GABAergic STN efferents (Windels et al., 2000). Second, all implanted subjects were diagnosed with PD, and have CSTC loops that have already been forced to adapt to the characteristic neuropathological changes that underlie PD. In contrast to noninvasive neuromodulation technologies, it is not possible to enroll healthy controls. Third, we did not review the effects

#### REFERENCES

Aiello, M., Eleopra, R., Foroni, F., Rinaldo, S., and Rumiati, R. I. (2017). Weight gain after STN-DBS: the role of reward sensitivity and impulsivity. Cortex 92, 150–161. doi: 10.1016/j.cortex.2017.04.005

of DBS on other domains that are relevant for disorders of impulsivity and compulsivity, such as reward sensitivity, or risk tolerance. Fourth, in an effort to characterize particular aspects of impulsivity and compulsivity, a long list of investigator-specific tasks has been developed and refined. These task differences limit the applicability of wide-spread meta-analysis statistical techniques that rely on carefully harmonized phenotypes, and illustrate the value of ongoing NIMH initiatives to promote common data elements (Barch et al., 2016). The ideal set-up for disentangling the causal effects of disease, dopaminergic medication, surgery, and stimulation would be to prospectively evaluate pre-surgical impulsivity and compulsivity on and off medication in a large cohort of patients using a common set of measures, followed by post-surgical evaluations on and off medication and DBS. However, no study to date has applied this design. In addition, further investigation is needed into how deliberate positioning of electrode contacts at different points along the STN axes may differentially affect impulsivity and compulsivity (Mallet et al., 2007).

### CONCLUSION

In conclusion, despite small samples sizes, logistical challenges, and methodological heterogeneity, the evidence reviewed here tentatively suggests that dampening of suppressive STNmediated pathways in PD decreases the risk of ICDs and compulsivity-related diagnoses (via dopaminergic medication dose decreases) and enhances cognitive-behavioral flexibility, but at the cost of impaired self-control when faced with competing choices. We need a much fuller understanding of structural and functional circuits encompassing the STN, and of how DBS and dopaminergic medication can optimally interact with these circuits to alleviate motor and non-motor symptoms in PD. This needed advance could be achieved by combining electroencephalography and tractography with behavioral testing on and off medication and stimulation in prospective PD DBS cohorts. Progress on this front is a prerequisite for application of human circuit manipulation technologies to a wider range of debilitating neuropsychiatric disorders involving disordered behavior.

#### AUTHOR CONTRIBUTIONS

SS, AS, and MF contributed to the conception and design of the review and wrote the first draft of the manuscript. SS, JG, and MF performed the search and reviewed all articles. All authors contributed to manuscript revision, and read and approved the submitted version.

Alegret, M., Junque, C., Valldeoriola, F., Vendrell, P., Pilleri, M., Rumia, J., et al. (2001). Effects of bilateral subthalamic stimulation on sleep in Parkinson's disease. Arch. Neurol. 58, 1223–1227. doi: 10.1001/archneur.58.8.1223

Aleksandrova, L. R., Creed, M. C., Fletcher, P. J., Lobo, D. S., Hamani, C., and Nobrega, J. N. (2013). Deep brain stimulation of the subthalamic nucleus

increases premature responding in a rat gambling task. Behav. Brain Res. 15, 76–82. doi: 10.1016/j.bbr.2013.02.011


stimulation on reward processing. Neuropsychologia 75, 11–19. doi: 10.1016/j. neuropsychologia.2015.05.005



Parkinson's disease, with a literature review. Parkinson. Relat. Disord. 18, 10–16. doi: 10.1016/j.parkreldis.2011.08.016


**Conflict of Interest:** JJ-S has received consulting fees from Boston Scientific, Abbott and Medtronic, and research support from Abbott and Medtronic. BK has received consulting fees from Abbott, Medtronic and Blackrock Therapeutics. HM has received consulting and intellectual property licensing fees from Abbott Labs Neuromodulation (previously St. Jude Medical Inc.).

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Scherrer, Smith, Gowatsky, Palmese, Jimenez-Shahed, Kopell, Mayberg and Figee. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

digital media

of impactful research

article's readership