The VTA is an area of the midbrain dopamine reward system located in the midbrain area between the substantia nigra and the rubrum (Corbett and Wise, 1980). It is a heterogeneous and highly complex structure with multiple types of neurons, including neurons that release GABA and glutamate, though the behavioral impact of VTA activity has long been considered to be driven exclusively by dopaminergic output (Yamaguchi et al., 2007). VTA glutamate neurons selectively express vesicular glutamate transporter type 2 (VGLUT2) mRNA and are found mainly in the medial VTA, representing approximately 2–3% of the total VTA cell population (Nair-Roberts et al., 2008). These neurons project to several areas in the brain, including the NAc, LH, VP, amygdala, and medial prefrontal cortex (mPFC) (Yamaguchi et al., 2011; Hnasko et al., 2012). Approximately 20–40% of VTA cells are GABAergic (Carr and Sesack, 2000; Nair-Roberts et al., 2008; Margolis et al., 2012) and project to several forebrain regions (Fields et al., 2007). It has been reported that the in vivo optogenetic activation of VTA GABA neurons in adult mice disrupts reward consumption while activation of VTA GABAergic projections to the NAc results in the detectable release of GABA but does not alter reward consumption (van Zessen et al., 2012).

Ventral tegmental area is the main source of dopamine in the midbrain reward system. VTA DA neurons serve a central role in regulating motivation, reward, and other behaviors (Wise, 1989, 2004; Berridge and Kringelbach, 2015; Pignatelli and Bonci, 2015; Schultz, 2016; Watabe-Uchida et al., 2017). The VTA sends projections to the NAc and also integrates glutamate input from areas such as the PFC, amygdala, and hippocampus (HIP) (Liu et al., 2008). Studies of related damage, receptor blockade, and drug addiction have shown that DA neurons play an important role in reward circuits. A blockade of DA receptors or damage to DA neurons reduces the positive effects of intracranial self-stimulation (ICSS) and the addictive effect of drugs such as cocaine and amphetamines (Fouriezos and Wise, 1976; Wise, 2006). Input-output analyses using TRIO and cTRIO techniques (Schwarz et al., 2015) developed in recent years enable dissection of the VTA DA circuits according to their output targets (Beier et al., 2015). Lammel et al. (2011, 2012) demonstrated that VTA DA neurons signal various reward values on the basis of their input and output circuits. Using optogenetics, researchers can selectively excite or inhibit the VTA neurons. The pathway from the LDT to the VTA DA to the NAc (NAcLat) signals a reward and that from the lateral habenula (LHb) to the VTA DA to the mPFC signals aversion (Nieh et al., 2013).

The reward prediction error (RPE) hypothesis provides a simple account of the role of DA in the learning of rewards and the behaviors associated with them. Cai et al. (2020) recorded and manipulated VTA DA neurons in mice during fear extinction, in which process, their fiber photometry recordings showed, the DA neurons in the medial and lateral VTA displayed distinct activity profiles. Further, the medial VTA activity more closely reflected the RPE while the lateral VTA activity more closely reflected a salience-like signal. This research showed that distinct signals in the medial and lateral VTA dopamine neurons modulate fear extinction at various times. Moreover, in a study applying machine learning to analyze the relationship between the VTA DA and RPE, (Lowet et al., 2020) suggested that the combined activity of VTA DA neurons encodes not just the mean but rather the complete distribution of reward through an expectable code. Meanwhile, (Dabney et al., 2020) reported on distributional reinforcement learning in the brain and also found a distributional code for value in DA-based reinforcement learning, thereby providing strong evidence for a neural realization of distributional reinforcement learning.

Despite the robustness of the RPE account of DA signals, some researchers have challenged this “canonical” view. Thus, several studies have reported that dopamine neurons are activated by aversive stimuli in addition to reward stimuli (Matsumoto and Hikosaka, 2009). To explain the functional diversity of DA, (Matsumoto and Hikosaka, 2009; Bromberg-Martin et al., 2010) argued that DA neurons of one type encode the motivational value, being excited by reward stimuli and inhibited by aversive events, and support the brain’s systems for seeking goals, evaluating outcomes, and learning value. DA neurons of another type encode motivational salience, being excited by both rewards and aversive stimuli, and support the brain’s systems for orienting, cognitive processing, and motivation. In addition to their value- and salience-coding activities, both types of DA neurons transmit alerting signals. Moreover, these two populations of DA neurons are spatially separated, with the former being distributed mainly in the dorsolateral substantia nigra compact area (SNc) and the latter in the ventromedial SNc and lateral VTA. An unresolved question concerns the proportion of DA neurons excited by aversive stimuli and whether they are the specific subpopulation that projects to specific targets. Notably, a recent study in mice found that roughly equivalent numbers of putative VTA DA neurons are activated and inhibited by tail pinch (Zweifel et al., 2011). Studies of combing optogenetics have found that the single-unit activity of DA neurons in mice increased in response to an aversive stimulus (Cohen et al., 2012).

The midbrain includes two main neural dopamine pathways, its cortex and limbic system. The former leads from the VTA to the prefrontal cortex and the latter from the VTA to the NAc. The NAc serves as an important downstream target of the VTA in the midbrain limbic system pathway. It receives input from VTA DA neurons and is also an important downstream target of dorsal raphe nucleus (DRN) serotonin neurons (Groenewegen et al., 1999; Ikemoto, 2007; Watabe-Uchida et al., 2012). The NAc comprises the ventral part of the striatum. Most of the cells in the NAc, as in the dorsal striatum, are GABA-containing MSNs, with the remainder being cholinergic and GABAergic inter-neurons (Meredith, 1999). Striatal regions contain subpopulations of these MSNs, the so-called “direct” and “indirect” pathways (Gerfen et al., 1990; Surmeier et al., 2007).

The MSNs of the direct pathway predominantly co-express D1 receptors and the endogenous opioid peptide dynorphin while the MSNs of the indirect pathway predominantly co-express D2 receptors and the endogenous opioid peptide enkephalin (Humphries and Prescott, 2010; Sesack and Grace, 2010). According to traditional explanations, the action of DA on the D1 receptors is coupled with the G-protein (G_s, stimulatory) and associated with the activation of adenylate cyclase, which tends to excite the MSNs of the direct pathway (Albin et al., 1989b). Elevation in the activity of these cells increases GABAergic and dynorphin input to the mesolimbic system and negative feedback on the midbrain dopamine cells. By contrast, the action of DA on the D2 receptors is coupled with glucose-inhibited (Gi, inhibitory) and associated with the inhibition of the adenylate cyclase, which often inhibits the MSNs of the indirect pathway (Albin et al., 1989a; Surmeier et al., 2007) and reduces the GABAergic input of enkephalin from the VP area. Through multiple synaptic connections, then, the inhibition of the indirect pathway at the level of the NAc, ultimately, activates the thalamus (Kelley, 2004).

The various projection pathways of the MSNs in the striatum indicate that the behavioral results that they guide may be inconsistent when they perform related adjustments of the nervous system. The canonical understanding is that the D1 receptors that express striatal neurons convey information directly to the output nuclei of the basal ganglia whereas D2-expressing neurons transmit information indirectly through pallidal neurons (Yawata et al., 2012; Bock et al., 2013; MacAskill et al., 2014). The NAc D1-MSNs encode rewards whereas D2-MSNs code aversion (Hikida et al., 2010; Lobo et al., 2010; Kravitz et al., 2012; Tai et al., 2012). Studies using mouse models have suggested that the optogenetic stimulation of the D1 receptors in the dorsal striatum induces persistent reinforcement while activation of the D2 receptors induces transient punishment (Kravitz et al., 2010, 2012).

Recent studies, however, have challenged the canonical view. Combining optogenetics and electrophysiology, (Kupchik et al., 2015) found that 50% of the VP cells in mice received input from D1-MSNs. Furthermore, researchers found that the optical stimulation of dyn-expressing cells in discrete subregions of the NAc neural pathway drove opposite motivated behavioral states, meaning that light stimulation of dyn-expressing cells in the vNAcSh drove aversive behavior. Conversely, the stimulation of dyn-expressing cells in the dNAcSh elicited reward behavior. The activation of dynorphin neurons in the ventral part of the NAc in mice overlaps for the most part with D1 receptors, leading to aversive production (Al-Hasani et al., 2015). D1 MSNs projecting to the LH are capable of causing anhedonia and behavioral despair (O’Connor et al., 2015; Massaly et al., 2019; Pignatelli et al., 2021). A mouse model for stress-induced anhedonia and passive coping showed that these phenomena were associated with increases in the synaptic strength of the excitatory synapses in the ventral hippocampus (VH) on D1-MSNs in the NAc medial shell (NAcmSh) and with LH projecting D1-MSN (Pignatelli et al., 2021). Using the mouse model, (O’Connor et al., 2015) found that D1-MSNs provided the dominant source of accumbal inhibition to the LH and rapid control over feeding through the LH GABA neurons. The molecular evidence suggests that mice lacking D2 receptors are less sensitive to the reward effect of cocaine. Combining this finding with evidence that human addicts experience reduced binding of the D2 receptors expressed in the NAc, it appears that the D2 receptors play a crucial role in encoding rewards (Volkow et al., 2007). Furthermore, researchers suggest that D1- and D2-MSNs may exert a concurrent action in reward-related behaviors (Steinberg et al., 2014; Soares-Cunha et al., 2016, 2018; Vicente et al., 2016; Natsubori et al., 2017) selectively activated DA neurons in transgenic rats using genetically targeted tools and demonstrated that NAc D1 and D2 receptors alike contribute to VTA ICSS.

The BF contains many types of neurons, the most common being cholinergic, GABAergic, and glutamatergic neurons. There is some variability in the type, morphology, and projection pattern of neurotransmitters across these types of neurons (Henny and Jones, 2008; Záborszky et al., 2018). Neurons in the BF express various types of calcium-binding proteins, including calmodulin, calpain, and parvalbumin (PV), as well as other molecular tags (e.g., neuropeptide-Y) and growth hormone inhibitors [e.g., somatostatin (SOM)] (Zaborszky et al., 2012). Owing to the highly complex structure and composition of this region, the relationship of the numerous types of neurons in this region with the regulatory mechanisms of reward and aversion has not yet been fully established.

Cholinergic neurons account for 20% of BF neurons and mainly project to the neocortical carotid and other cortices (Woolf, 1991). They also co-express with other neurotransmitters such as epinephrine, norepinephrine, DA, GABA, glycine, VGLUT1, VGLUT2, and enkephalin. Cholinergic neurons also promote local cortical state regulation and coordinate the activities of cortical circuits involved in various functions (Záborszky et al., 2018). Given the role of these neurons in cognitive regulation, the central nervous system of elderly populations and patients with dementia can be expected to display obvious cholinergic dysfunction related to memory loss (Bartus et al., 1982). Optogenetic studies have found that nearly all cholinergic neurons show an activation response to the main enhancer, such as a water reward or an air sac punishment (Gu and Yakel, 2011; Gu et al., 2012). In addition, cholinergic neurons transmit strengthening signals to the cerebral cortex that can induce plastic changes in V1 to generate reward signals (Eggermann et al., 2014; Liu, 2015).

In recent years, researchers have identified GABAergic and glutamatergic neurons in the BF, which also play important roles in the neural regulation of the brain (Hassani et al., 2009). Non-cholinergic neurons in the BF are involved in the regulation of behavioral states, such as attention and sleep (Hangya et al., 2015; Xu et al., 2015). Optogenetic stimulation of BF PV neurons in mice can regulate the function of the cortex by inducing GABAergic resonance (Kim et al., 2015). Studies using mouse models have also reported that light-activated cholinergic, glutamatergic, and PV-expressing GABAergic neurons in the BF promote wakefulness, while activated SOM-expressing GABAergic neurons enhance sleep (Xu et al., 2015).

The cholinergic, glutamatergic, SOM, and PV neurons of the BF interact: Glutamatergic neurons have outputs to cholinergic and PV neurons, while SOM neurons project to the other three types of neurons (Xu et al., 2015). Both cholinergic and non-cholinergic neurons in the BF can enhance cortical activity, but cholinergic neurons respond mainly to enhancers (Hangya et al., 2015). However, non-cholinergic neurons provide clear clues for predicting the motivation of the enhancement-phase response (Lin and Nicolelis, 2008; Avila and Lin, 2014) and are related to the operational measures of attention (Hangya et al., 2015). Therefore, cholinergic and non-cholinergic neurons may have complementary and synergistic functions in arousal and cognition. Recent studies using mouse models have found non-cholinergic neurons in the BF to be involved in the regulation of reward (Zhu et al., 2017; Faget et al., 2018), but the specific mechanisms of neural regulation remain unclear.

The BF receives many neuronal inputs from and also projects to other brain regions. Thus, cholinergic neurons from numerous sub-regions project to various brain regions. Those from the HDB and MCPO project mainly to the olfactory bulb and the piriform and internal olfactory cortices. Cholinergic neurons from the MS and VDB project mainly to the hippocampus and other brain regions, while those from the VP and SI project primarily to the basolateral amygdala (BLA) and neocortical brain regions (Zaborszky et al., 2012). Cholinergic neurons in the subregions of the HDB, MCPO, MS, and VDB project to orexin neurons in the LH (Sakurai et al., 2005). In addition, cholinergic neurons in the BF receive inputs from several cortical areas, such as the orbitofrontal, motor, and insular cortices, as well as from brain regions, including the lateral septum, central amygdaloid nucleus (CEA), paraventricular nucleus of the hypothalamus, DRN, and parabrachial nucleus (Gielow and Zaborszky, 2017). In studies of the projections of BF non-cholinergic neurons, researchers have reported that PV, GABAergic, and glutamatergic neurons project to each other in the brain regions that encode reward modulation, including the NAc, VTA, and LH (Zaborszky et al., 2012; Do et al., 2016). SOM neurons in the BF mainly receive input from the LS and NAc and project to the LH, VTA, and LHb (Zhu et al., 2017). However, further investigation is needed to determine whether the mutual projection relationships among these neurons are involved in the neuromodulation of reward and their underlying mechanisms.

Located on the dorsal side of the BF, the VP plays a critical role in processing and executing motivated behaviors in the basal ganglia (Mogenson et al., 1980). It has a wide range of upstreams and downstreams in the brain. Researchers have offered a comprehensive interpretation of the central position of the VP as the “common pathway” of reward processing in the brain (Kalivas and Volkow, 2005; Smith et al., 2009). However, the complexity of the VP in terms of the several types of neurons and their interactions has impeded the study of the mechanism of neural regulation underlying reward and aversion.