Abstract
We review recent progress in the study of roles of octopaminergic (OA-ergic) and dopaminergic (DA-ergic) signaling in insect classical conditioning, focusing on our studies on crickets. Studies on olfactory learning in honey bees and fruit-flies have suggested that OA-ergic and DA-ergic neurons convey reinforcing signals of appetitive unconditioned stimulus (US) and aversive US, respectively. Our work suggested that this is applicable to olfactory, visual pattern, and color learning in crickets, indicating that this feature is ubiquitous in learning of various sensory stimuli. We also showed that aversive memory decayed much faster than did appetitive memory, and we proposed that this feature is common in insects and humans. Our study also suggested that activation of OA- or DA-ergic neurons is needed for appetitive or aversive memory recall, respectively. To account for this finding, we proposed a model in which it is assumed that two types of synaptic connections are strengthened by conditioning and are activated during memory recall, one type being connections from neurons representing conditioned stimulus (CS) to neurons inducing conditioned response and the other being connections from neurons representing CS to OA- or DA-ergic neurons representing appetitive or aversive US, respectively. The former is called stimulus–response (S–R) connection and the latter is called stimulus–stimulus (S–S) connection by theorists studying classical conditioning in vertebrates. Results of our studies using a second-order conditioning procedure supported our model. We propose that insect classical conditioning involves the formation of S–S connection and its activation for memory recall, which are often called cognitive processes.
Introduction
Biogenic amines regulate various functions of central nervous systems in vertebrates and invertebrates (Blenau and Baumann, ). In vertebrates, dopamine (DA) pathways are involved in the coordination of motor behavior, motivation, addiction, and reward-based learning of a wide range of sensory stimuli (Schultz, 1998, 2006). In insects, DA appears to play roles in regulating motor behavior (Blenau and Baumann, ) and arousal (Andretic et al., ), and octopamine (OA), the invertebrate counterpart of noradrenaline, plays roles in desensitizing sensory inputs and regulating various forms of behavior (Roeder, 1999), including aggression (Stevenson et al., 2005; Hoyer et al., ; Zhou et al., 2008) and sleep (Crocker et al., ). In this article, we review recent advances in studies on the roles of OA-ergic and DA-ergic neurons in classical conditioning in insects, focusing on findings from our behavioral and pharmacological studies on crickets Gryllus bimaculatus.
Roles of Aminergic Neurons in Formation of Olfactory Memory in Honey Bees and Fruit-Flies
Insects are useful animal models for the study of cellular and molecular mechanisms underlying learning and memory (Giurfa, ; Heisenberg, ; Davis, ; Menzel and Giurfa, ; Menzel et al., ; Keene and Waddell, ). This is mainly because insect brains consist of a relatively small number (<106) of neurons and the organization of “microbrains” (Mizunami et al., , ) is relatively simple, and thus their functions are highly susceptive to genetic, pharmacological, and other means of experimental manipulation. One of the pioneering studies on the roles of aminergic neurons in insect learning was performed in honey bees by Hammer (). A hungry honey bee extends its proboscis in response to sucrose stimulation applied to its antennae, proboscis, or tarsi. The proboscis extension response can be conditioned by pairing an odor applied to the antennae (conditioned stimulus, CS) with sucrose stimulation (unconditioned stimulus, US) (Kuwabara, ; Erber et al., ). Hammer () observed that pairing of an odor with intracellular stimulation of the VUMmx1 neuron, a putative octopamine immunoreactive neuron (Kreissl et al., ) that exhibited responses to sucrose stimulation, induced a conditioning effect. Hence, he concluded that this neuron mediates the reinforcing property of sucrose reward in olfactory conditioning. Later, Hammer and Menzel () showed that local injection of octopamine into the antennal lobes and the calyces of the mushroom bodies, termination areas of the VUMmx1 neuron, substituted the sucrose US in olfactory conditioning; the antennal lobes are primary olfactory centers and the mushroom bodies are higher-order olfactory and multi-sensory association centers (Erber et al., ; Heisenberg et al., ; Mizunami et al., ,; Okada et al., ; Heisenberg, ; Davis, ; Menzel and Giurfa, ). In addition, Farooqui et al. () showed that RNA interference of OA receptors or pharmacological blockade of OA receptors by mianserin in the antennal lobe impaired olfactory conditioning.
Another pioneering study was performed in the fruit-fly Drosophila by Schwaerzel et al. (2003). Fruit-flies can be conditioned to choose an odor associated with sucrose, or avoid an odor associated with electric shock (Dudai et al., ; Tully and Quinn, 1985). Schwaerzel et al. (2003) demonstrated that transgenic flies defective in OA or DA synthesis exhibited defects in appetitive olfactory learning with sucrose reward or aversive olfactory learning with electric shock, respectively. Subsequent studies in fruit-flies confirmed that OA- or DA-ergic neurons convey signals for sucrose reward or electric shock punishment, respectively, in olfactory conditioning in larval (Schroll et al., 2006; Honjo and Furukubo-Tokunaga, ; Selcho et al., 2009) and adult flies (Riemensperger et al., 2005; Tomchik and Davis, 2009; Gervasi et al., ), although a few exceptions have also been found (Kim et al., , Sitaraman et al., 2008). Notable findings in these studies are that photoactivation of OA-ergic or DA-ergic neurons paired with an odor stimulation successfully induced an appetitive or aversive conditioning effect, respectively, in larval flies (Schroll et al., 2006) and that photoactivation of a class of DA-ergic brain neurons also induced an aversive conditioning effect in adult flies (Claridge-Chang et al., ). Roles of DA in conveying aversive US have also been demonstrated in honey bees in olfactory conditioning of the sting extension reflex, in which an odor was paired with electric shock punishment (Vergoz et al., 2007).
Procedures for Olfactory and Visual Pattern Conditioning in Crickets
We studied the roles of OA-ergic and DA-ergic signaling in classical conditioning in crickets Gryllus bimaculatus. We have demonstrated that crickets have excellent olfactory learning capabilities: for example, they can learn (1) to associate an odor with reward by a single operant or classical conditioning trial (Matsumoto and Mizunami, , ), (2) to form a lifetime olfactory memory (Matsumoto and Mizunami, ), (3) to memorize seven pairs of odors at the same time (Matsumoto and Mizunami, ), and (4) to associate one odor with reward and another odor with punishment in one visual context and to associate the opposing in another visual context (Matsumoto and Mizunami, ). Their high learning capabilities may reflect their omnivorous foraging habit, i.e., they test many potential food items to assess whether they are edible or not. Capacity for forming visual place memory has also been demonstrated in crickets (Wessnitzer et al., 2008). Moreover, we have shown that crickets are suitable materials for the study of molecular mechanisms underlying learning and memory by using pharmacological manipulation (Matsumoto et al., , ) and RNA interference (Takahashi et al., 2009).
We used a “classical conditioning and operant testing” procedure, which is based on a high capability of crickets to transfer memory formed in a classical conditioning situation to an operant testing situation (Matsumoto and Mizunami, ; Unoki et al., 2005, 2006). For appetitive olfactory conditioning, crickets were individually placed in a beaker, and one of two odors (e.g., banana and apple odors) was presented to their antennae and then water reward was presented to their mouth (Figure 1A). For aversive olfactory conditioning, one of these two odors was presented to the antennae before presenting 2 M sodium chloride solution to the mouth. In the odor preference test, animals were allowed to freely choose between two odor sources. Each odor source consisted of a container containing a filter paper soaked with a solution of odor essence. The time that the animals touched the gauze net covering the top of each container with their mouths or palpi was measured for evaluating relative odor preference of the animals (Figure 1B). For visual pattern conditioning, either of a white-center and black-surround pattern or a black-center and white-surround pattern was paired with water reward or sodium chloride punishment (Figure 1C). In the preference test, the time that the animals touched each of the two patterns was measured for evaluating relative preference (Figure 1D).
Figure 1
These procedures were highly effective for achieving conditioning. In the case of appetitive olfactory conditioning, for example, one conditioning trial was sufficient to establish conditioning, with its memory lasting for several hours (mid-term memory; see Figure 4; Matsumoto and Mizunami, ; Unoki et al., 2005). Two appetitive conditioning trials with a 5-min interval induce memory that lasts for at least 1 day (see Figure 4), which matches protein-synthesis-dependent long-term memory because it is blocked by injection of a protein-synthesis inhibitor, cycloheximide, into the hemolymph before conditioning (Matsumoto et al., ).
Roles of Aminergic Neurons in Formation of Olfactory Memory in Crickets
We studied the effect of OA and DA receptor antagonists on appetitive and aversive olfactory conditioning in crickets (Unoki et al., 2005). Crickets injected with epinastine or mianserin, antagonists of insect OA receptors (Roeder et al., 1998; Degen et al., ), into the hemolymph before conditioning exhibited a complete impairment of appetitive conditioning to associate an odor with water reward (Figure 2A); the preference for the rewarded odor after conditioning did not significantly differ from that before conditioning. On the other hand, these animals exhibited no impairment of aversive learning with saline punishment (Figure 2B). The latter observation shows that OA receptor antagonists do not impair sensory function, motor function, or motivation necessary for learning, and we thus conclude that OA is specifically involved in conveying water reward. We also found that fluphenazine, chlorpromazine, or spiperone, antagonists of insect DA receptors (Degen et al., ; Mustard et al., ), completely impaired aversive learning with sodium chloride punishment (Figure 2C) but did not affect appetitive learning with water reward (Figure 2D). The latter finding indicates that DA receptor antagonists do not impair the sensory function, motor function, or motivation necessary for learning, and thus we conclude that DA is specifically involved in conveying sodium chloride punishment. It should be cautioned that the specificity of antagonists used in our studies is not necessarily perfect (see Discussion in Unoki et al., 2005). However, two different kinds of OA receptor antagonists impaired appetitive learning but not aversive learning and three different kinds of DA receptor antagonists impaired aversive learning but not appetitive learning, suggesting that the impairments were due to the blockade of OA or DA receptors. Thus, we concluded that OA- or DA-ergic neurons convey information about appetitive or aversive US, respectively, for olfactory conditioning in crickets (Unoki et al., 2005).
Figure 2
Roles of Aminergic Neurons in Formation of Visual Pattern Memory and Color Memory
We next studied the effect of OA and DA receptor antagonists on appetitive and aversive conditioning of visual pattern (Unoki et al., 2006) and color (Nakatani et al., ). Crickets injected with epinastine or mianserin, OA receptor antagonists, into the hemolymph exhibited a complete impairment of appetitive learning to associate a visual pattern with water reward, but aversive learning to associate a visual pattern with sodium chloride punishment was unaffected (Figure 3A). In contrast, fluphenazine, chlorpromazine, or spiperone, DA receptor antagonists, completely impaired aversive learning but not appetitive learning (Figure 3B). We also found the same for color learning: OA receptor antagonists impaired appetitive color learning with water reward without affecting aversive color learning with sodium chloride punishment. In contrast, DA receptor antagonists impaired aversive color learning without affecting appetitive color learning (Nakatani et al., ). These findings indicate that the roles of OA-ergic and DA-ergic neurons in conveying information about appetitive and aversive US, respectively, are ubiquitous in learning of odor, visual pattern and color stimuli, suggesting that these neurons serve as the general reward or punishment system for insect learning.
Figure 3
In mammals, midbrain DA-ergic neurons play major roles in a wide range of visual, auditory, and somatosensory stimuli and thus are considered to serve as a general reward system (Schultz, 1998, 2006). It appears that the roles of aminergic neurons in conveying reinforcement signals are conserved across different phyla, but the kind of reinforcement signal that each biogenic amine mediates is different: DA mediates appetitive reinforcement in mammals but mediates aversive reinforcement in insects. Future studies on the roles of DA in learning in phylogenetically ancient species may clarify how different roles of DA in positive or negative reinforcement have emerged during the course of evolution. Neurotransmitters meditating negative reinforcement in mammals are less known, but roles of noradrenaline, serotonin, or DA in some forms of aversive learning have been suggested (Daw et al., ; Harley, ; Wise, 2004; Schultz, 2006).
Different Dynamics of Appetitive Memory and Aversive Memory
In the work described above, we noticed that the time courses of appetitive memory and aversive memory fundamentally differ (Unoki et al., 2005, 2006; Nakatani et al., ). In Figure 4, the time course of memory after 1-trial, 2-trial, and 6-trial appetitive olfactory conditioning and that after 2-trial and 6-trial aversive olfactory conditioning are shown. A group subjected to 1-trial appetitive conditioning exhibited a significant level of retention at 30-min after conditioning: the preference for rewarded odor was significantly greater than that before conditioning. The memory did not significantly decay from 30 min to 1 h after conditioning, but it was completely diminished at 1 day after conditioning. Two-trial and 6-trial groups exhibited a significant level of 30-min retention, with no significant decay of retention from 30 min to 1 day after conditioning. Two-trial aversive conditioning group exhibited a significant level of 30-min retention. However, the memory decayed to a non-significant level at 1 h after conditioning. Six-trial aversive conditioning group exhibited a significant decay of retention from 30 min to 1 h after training. Thereafter, a significant level of retention was maintained with no significant decay from 1 h to 1 day after conditioning. In short, aversive olfactory memory exhibited a prominent decay from 30 min to 1 h after conditioning, whereas appetitive olfactory memory exhibited little decay from 30 min to 1 day after conditioning.
Figure 4
We found similar distinction of dynamics between appetitive memory and aversive memory for visual pattern conditioning (Unoki et al., 2006) and color conditioning (Nakatani et al., ) in crickets. It was obvious that the number of conditioning trials, and hence the levels of initial acquisition, also influenced memory dynamics, but effects of these factors did not account for the difference in memory dynamics observed after appetitive and aversive learning: reward memory was sustained even when the level of 30-min retention was low, and punishment memory exhibited a characteristic decay even when the level of 30-min retention was high.
Comparisons with studies in other species of insects showed that our finding that aversive memory is less durable than appetitive memory is not specific to the type of US we used (water as appetitive US and sodium chloride as aversive US) or the species used (crickets). In fruit-flies, it has been reported that punishment memory after conditioning of an odor with electric shock punishment decays much faster than reward memory after conditioning of an odor with sucrose reward, regardless of intensity of electric shock and the concentration of the sucrose solution (Tempel et al., 1983). A recent study also suggested that aversive olfactory memory with quinine or saline punishment is less durable than appetitive olfactory memory with sucrose reward in fruit-fly larvae (Honjo and Furukubo-Tokunaga, ). Thus, dynamics of punishment memory and reward memory differ for different intensities or kinds of unconditioned stimulus (US; water or sugar as reward and saline, quinine, or electric shock as punishment), for different species of insects (crickets and fruit-flies), and for different conditioning paradigms (individual conditioning in crickets and group conditioning in fruit-flies).
By examining the literature in human psychology, we have proposed that these findings in insects are comparable to findings in humans (Nakatani et al., ). Many studies in educational psychology have proposed that punishment is highly effective for immediately suppressing behavior of children at school or home, but the effect tends to be short-lived compared to the effect of reward (Peine and Howarth, ; Gershoff, ; Driscoll, ). However, to our knowledge, no convincing evidence supporting the argument that the dynamics of reward and punishment memory fundamentally differ has been obtained in rigorously controlled animal experiments (Walters and Crusec, 1977), except for several studies on insects discussed above. Obviously, our proposal that different dynamics after punishment and reward learning is conserved across phyla is highly speculative, and more studies on vertebrates are clearly needed to evaluate the validity of this idea.
If different dynamics of punishment and reward memory is conserved across different phyla, what is the possible adaptive significance for it? We have proposed that it is related to a different significance of reward and punishment leaning for survival in a changing environment (Nakatani et al., ). Since the environment is constantly changing, stimuli that once served as predictors of punishment may change to predict reward or vice versa. Consider that an inedible food item in one season may become profitable in the next season. In this case, long-term retention of avoidance of stimuli that once predicted aversive stimuli is not necessarily beneficial, because it reduces the opportunity to obtain useful resources in the future. Stimuli that once predicted reward may also be changed to predict punishment, but long-term retention of preference for once-rewarded stimuli has no such cost, because animals can re-learn to avoid such stimuli when they encounter the stimuli again. Thus, we have proposed that different adaptive significance of durability of memory between reward and punishment learning is the basis of different dynamics of reward and punishment memories (Nakatani et al., ).
Participation of Octopaminergic and Dopaminergic Neurons in Appetitive and Aversive Memory Recall
We then studied the effect of an OA or DA receptor antagonist on appetitive or aversive memory recall (retrieval). The results suggested that intact OA-ergic or DA-ergic signaling is necessary for recall of appetitive or aversive memory, respectively, after olfactory learning and visual pattern learning (Mizunami et al., ). Crickets were subjected to appetitive or aversive olfactory conditioning and were injected with an OA or DA receptor antagonist before retention test. Injection of epinastine or mianserin, OA receptor antagonists, completely impaired appetitive olfactory memory recall but had no effect on aversive olfactory memory recall (Figure 5A). On the other hand, injection of fluphenazine, chlorpromazine, or spiperone, DA receptor antagonists, completely impaired aversive memory recall but had no effect on appetitive memory recall (Figure 5B). This is in accordance with observations in honey bees that disruption of OA-ergic transmission in the antennal lobe, the primary olfactory center, by an OA receptor antagonist (mianserin) or by RNA interference of the OA receptor gene disrupted appetitive olfactory memory recall (Farooqui et al., ), although the results of the study by Farooqui et al. were not conclusive as we have discussed before (Mizunami et al., ). We also found that OA and DA receptor antagonists impaired appetitive and aversive memory recall, respectively, in visual pattern conditioning (Mizunami et al., ). Therefore, we concluded that intact synaptic transmission from OA- and DA-ergic neurons is needed for the recall of appetitive memory and aversive memory, respectively, in both olfactory and visual pattern learning.
Figure 5
We noticed that our findings are not consistent with conventional neural models of insect classical conditioning. Figure 6A depicts perhaps the best model proposed to account for the roles of extrinsic and intrinsic neurons of mushroom bodies in olfactory conditioning in the fruit-fly Drosophila (Schwaerzel et al., 2003). This model assumes that (1) “CS” neurons (intrinsic neurons of the mushroom body, called Kenyon cells) that convey signals about a CS make synaptic connections with dendrites of “CR” neurons (efferent (output) neurons of the mushroom body lobe), activation of which leads to a CR (conditioned response) that mimics UR (unconditioned response), but these synaptic connections are silent or very weak before conditioning, (2) OA- or DA-ergic efferent neurons projecting to the lobes (“OA/DA” neurons), which convey signals for appetitive or aversive US, respectively, make synaptic connections with axon terminals of “CS” neurons, and (3) the efficacy of the synaptic transmission from “CS” neurons to “CR” neurons that induces a conditioned response (CS–CR or S–R connection) is strengthened by coincident activation of “CS” neurons and “OA/DA” neurons during conditioning (assuming Kandelian synaptic plasticity; see Abrams and Kandel, ). In short, this model assumes that presentation of a CS after conditioning activates the CS–CR or S–R connection to induce a CR. Thus, this model is characterized as an S–R model (Figure 7A), following terminology in studies on classical conditioning in higher vertebrates (Rescorla, 1988; Pickens and Holland, 2004; Holland, ). It can be pointed out that the S–R model accounts for most forms of classical conditioning in invertebrates, including classical conditioning of gill withdrawal reflex in the mollusk Aplysia, where pairing of a gentle tactile stimulus to the siphon (CS) and a strong tactile stimulus to the gill (US) results in an enhancement of the efficacy of synaptic transmission from siphon sensory neuron to gill motor neuron (Abrams and Kandel, ; Kandel, ; Roberts and Glanzman, 2003), which is characterized as an S–R connection. The model by Schwaerzel et al. (2003), however, is inconsistent with our findings because it predicts that activation of OA- or DA-ergic neurons is not required for appetitive or aversive memory recall, respectively.
Figure 6
Figure 7
We have proposed a new model (Figure 6B), with minimal modifications of the model proposed by Schwaerzel et al. (2003). We have assumed that (1) activation of “OA/DA” neurons and resulting release of OA or DA are needed to “gate” the sensori-motor pathway from the “CS” neurons to “CR” neurons after conditioning and (2) synaptic connection from “CS” neurons to “OA/DA” neurons representing US is strengthened by coincident activation of “CS” neurons and “OA/DA” neurons by pairing of a CS with a US (assuming Hebbian synaptic plasticity). The latter connection is termed a CS–US or S–S connection (Figure 7B; Rescorla, 1988; Pickens and Holland, 2004), In short, our model assumes that two kinds of memory traces are formed by conditioning and that activation of both memory traces is needed for memory recall. This model corresponds to a hybrid of the S–R and S–S models (for explanation of the S–S model, see legends of Figure 7).
An alternative possibility to explain our findings is that different sets of “OA/DA” neurons govern reinforcement and memory retrieval processes, respectively. This is achieved by modifying the model shown in Figure 6A by assuming other “OA/DA” neurons in neural pathways downstream of the “CR” neurons. This model, however, failed to account for our results with second-order conditioning described below.
Evaluation of Our Model by Using a Second-Order Conditioning Procedure
The critical assumption of our model is that the pathway from neurons representing CS to OA/DA neurons representing appetitive or aversive US (S–S connection) is strengthened by conditioning. We evaluated this assumption by using a second-order conditioning procedure (Mizunami et al., ). Second-order conditioning (Figure 6C) is a procedure for testing whether a CS can acquire the reinforcing property of a US, by pairing a CS (CS1) with a US and then pairing another CS (CS2) with CS1 (Rescorla, 1988). Our model predicts that blockade of OA or DA receptors (on axon terminals of “CS” neurons) during the initial CS1–US pairing stage does not impair the enhancement of synapses from “CS” neurons to “OA/DA” neurons. This is because blockade of OA or DA receptors should not affect normal activities of “CS” neurons and “OA/DA” neurons (Figure 6C). Hence, CS1 should act as a reinforcer in the second CS1–CS2 pairing stage, although CS1 per se does not induce a CR (notice that impairments of first-order conditioning by DA/DA receptor antagonists are shown in Figures 2 and 3). Therefore, our model predicts that blockade of OA or DA receptors during the first CS1-US pairing stage does not impair second-order conditioning, but the same treatment during the second CS2–CS1 pairing stage or during the final retention test impairs second-order conditioning.
We first studied whether second-order conditioning can be achieved in crickets (Figure 8; Mizunami et al., ). We used an olfactory stimulus as CS1 and a visual pattern as CS2. For appetitive or aversive conditioning, an odor (CS1) was paired with water or sodium chloride solution, respectively, and then a visual pattern (CS2) was paired with an odor (CS1). A group of animals that was subjected to appetitive second-order conditioning trials exhibited significantly increased preference for the CS2 (Figure 8A). In contrast, control groups that were each subjected to unpaired presentations of stimuli at the first or second conditioning stage exhibited no significantly increased preference for the CS2 (Figure 8A), thus indicating that the increased preference for the CS2 in the experimental group is truly the result of second-order conditioning. Similarly, we showed that second-order aversive conditioning could be achieved (Figure 8B).
Figure 8
We then studied the effect of OA or DA receptor antagonist on appetitive or aversive second-order conditioning (Figure 9). A group of animals injected with epinastine before the first conditioning stage of appetitive second-order conditioning exhibited a significantly increased preference for the CS2, thus indicating that blockade of OA receptor during the first stage does not impair appetitive second-order conditioning (Figure 9A). In contrast, the group injected with epinastine before the second stage (Figure 9B) or before final test (Figure 9C) for appetitive second-order conditioning exhibited no significantly increased preference for the CS2, indicating a complete impairment of second-order conditioning. Similarly, blockade of DA receptor antagonist at the first stage did not impair aversive second-order conditioning, but that at the second stage or final test impaired aversive second-order conditioning (Figures 9D–F).
Figure 9
The results were in full accordance with our model. In addition, the results were inconsistent with an alternative model in which different sets of OA/DA neurons participate in reinforcement and memory recall, because the model predicts that blockade of OA/DA receptors at any stage of conditioning impairs second-order conditioning.
Our finding that OA-ergic signaling and DA-ergic signaling are needed for appetitive and aversive memory recall, respectively, is in accordance with some previous findings in honey bees and fruit-flies but not with other findings, as we have discussed elsewhere (Mizunami et al., ). Thus, we proposed that activations of OA- or DA-ergic neurons are needed for memory recall in some forms of classical conditioning in insects but not in other forms (Mizunami et al., ). The critical factors for determining the requirement of OA- or DA-ergic signaling in appetitive/aversive memory recall remain to be clarified.
Future Perspectives
Many questions concerning the roles of OA- or DA-ergic neurons in insect learning remain to be addressed. Firstly, although there has been a substantial progress in identifying DA- and OA-ergic neurons involved in appetitive and aversive learning in honey bees (Hammer, ) and fruit-flies (Schroll et al., 2006; Claridge-Chang et al., ), more studies need to be performed to clarify the morphology and physiology of DA or OA-ergic neurons participating in insect learning. In crickets, it has been suggested that the mushroom bodies are involved in olfactory learning (in Acheta domesticus: Scotto-Lomassese et al., 2003), and thus OA- and DA-ergic neurons projecting to the mushroom bodies need to be anatomically and physiologically characterized. Detailed immunohistochemical studies for OA and DA will provide the first step for this. Secondly, we have proposed that different dynamics of appetitive and aversive memory are the result of different dynamics of biochemical processes after activation of OA and DA receptors, respectively (Nakatani et al., ). To examine this hypothesis, physiology, and biochemistry of neurons participating in association of CS and US, most probably Kenyon cells of the mushroom bodies, need to be characterized. Thirdly, our successful demonstration of cross-modal second-order conditioning suggests that OA- or DA-ergic neurons participating in olfactory learning also participate in visual pattern learning (Mizunami et al., ), and this suggestion needs to be examined by electrophysiologically characterizing responses of OA- and DA-ergic neurons during olfactory learning and visual learning. Fourthly, mammalian midbrain DA-ergic neurons have been suggested to convey prediction error for rewarding outcomes, namely, they encode the error between expected and actual rewarding outcomes (Schultz, 1998, 2006), and it is important to clarify whether or not OA-ergic (resp. DA-ergic) neurons in insects work in the same way or not in appetitive (resp. aversive) learning, respectively. Further examination of these questions should contribute to clarification of phylogenetically conserved principles of brain systems underlying learning and memory among vertebrates and invertebrates, as well as each specific functional specialization in each group.
Statements
Acknowledgments
Our study was supported by grants from the Japan Society for Promotion of Science. We thank Ai Hatano for providing data on one-trial aversive olfactory conditioning shown in Figure 4B.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
AbramsT. W.KandelE. R. (1988). Is contiguity detection in classical conditioning a system or a cellular property? Learning in Aplysia suggests a possible molecular site. Trends Neurosci.11, 128–135.10.1016/0166-2236(88)90137-3
2
AndreticR.van SwinderenB.GreenspanR. J. (2005). Dopaminergic modulation of arousal in Drosophila. Curr. Biol.15, 1165–1175.
3
BlenauW.BaumannA. (2001). Molecular and pharmacological properties of insect biogenic amine receptors: lessons from Drosophila melanogaster and Apis mellifera. Arch. Insect Biochem. Physiol.48, 13–38.
4
Claridge-ChangA.RoordaR. D.VrontouE.SjulsonL.LiH.HirshJ.MiesenböckG. (2009). Writing memories with light-addressable reinforcement circuitry. Cell139, 405–415.10.1016/j.cell.2009.08.034
5
CrockerA.ShahidullahM.LevitanI. B.SehgalA. (2010). Identification of a neural circuit that underlies the effects of octopamine on sleep:wake behavior. Neuron65, 670–681.10.1016/j.neuron.2010.01.032
6
DavisR. L. (2005). Olfactory memory formation in Drosophila: from molecular to systems neuroscience. Annu. Rev. Neurosci.28, 275–302.
7
DawN. D.KakadeS.DayanP. (2002). Opponent interactions between serotonin and dopamine. Neural Netw.15, 603–616.10.1016/S0893-6080(02)00052-7
8
DegenJ.GeweckeM.RoederT. (2000a). Octopamine receptors in the honey bee and locust nervous system: pharmacological similarities between homologous receptors of distantly related species. Br. J. Pharmacol.130, 587–594.10.1038/sj.bjp.0703338
9
DegenJ.GeweckeM.RoederT. (2000b). The pharmacology of a dopamine receptor in the locust nervous tissue. Eur. J. Pharmacol.396, 59–65.10.1016/S0014-2999(00)00182-5
10
DriscollM. P. (2005). Psychology of Learning for Instruction. Boston: Pearson.
11
DudaiY.JanY. N.ByersD.QuinnW. G.BenzerS. (1976). Dunce, a mutant of Drosophila deficient in learning. Proc. Natl. Acad. Sci. U.S.A.73, 1684–1688.
12
ErberJ.MasuhrT. H.MenzelR. (1980). Localization of short-term memory in the brain of the bee, Apis mellifera. Physiol. Entomol.5, 343–358.
13
FarooquiT.RobinsonK.VaessinH.SmithB. H. (2003). Modulation of early olfactory processing by an octopaminergic reinforcement pathway in the honeybee. J. Neurosci.23, 5370–5380.
14
GershoffE. T. (2002). Corporal punishment by parents and associated child behaviors and experiences: a meta-analytic and theoretical review. Psychol. Bull.128, 539–579.
15
GervasiN.TchénioP.PreatT. (2010). PKA dynamics in a Drosophila learning center: coincidence detection by rutabaga adenylyl cyclase and spatial regulation by dunce phosphodiesterase. Neuron65, 516–529.10.1016/j.neuron.2010.01.014
16
GiurfaM. (2003). Cognitive neuroethology: dissecting non-elemental learning in a honeybee brain. Curr. Opin. Neurobiol.13, 726–735.
17
HammerM. (1993). An identified neuron mediates the unconditioned stimulus in associative olfactory learning in honeybees. Nature366, 59–63.10.1038/366059a0
18
HammerM. R.MenzelR. (1998). Multiple sites of associative odor learning as revealed by local brain microinjections of octopamine in honeybees. Learn. Mem.5, 146–156.
19
HarleyC. W. (2004). Norepinephrine and dopamine as learning signals. Neural Plast.11, 1–14.10.1155/NP.2004.191
20
HeisenbergM. (2003). Mushroom body memoir: from maps to models. Nat. Rev. Neurosci.4, 266–275.
21
HeisenbergM.BorstA.WagnerS.ByersD. (1985). Drosophila mushroom body mutants are deficient in olfactory learning. J. Neurogenet.2, 1–30.10.3109/01677068509100140
22
HollandP. (2008). Cognitive versus stimulus-response theories of learning. Learn. Behav.36, 227–241.
23
HonjoK.Furukubo-TokunagaK. (2009). Distinctive neuronal networks and biochemical pathways for appetitive and aversive memory in Drosophila larvae. J. Neurosci.29, 852–862.10.1523/JNEUROSCI.1315-08.2009
24
HoyerS. C.EckartA.HerrelA.ZarsT.FischerS. A.HardieS. L.HeisenbergM. (2008). Octopamine in male aggression of Drosophila. Curr. Biol.18, 159–167.
25
KandelE. R. (2001). The molecular biology of memory storage: a dialogue between genes and synapses. Science294, 1030–1038.10.1126/science.1067020
26
KeeneA. C.WaddellS. (2007). Drosophila olfactory memory: single genes to complex neural circuits. Nat. Rev. Neurosci.8, 341–354.
27
KimY. C.LeeH. G.HanK. A. (2007). D1 dopamine receptor dDA1 is required in the mushroom body neurons for aversive and appetitive learning in Drosophila. J. Neurosci.27, 7640–7647.10.1523/JNEUROSCI.1167-07.2007
28
KreisslS.EichmüllerS.BickerG.RapusJ.EckertM. (1994). Octopamine-like immunoreactivity in the brain and subesophageal ganglion of the honeybee. J. Comp. Neurol.348, 583–595.
29
KuwabaraM. (1957). Bildung des bedingten Reflexes von Pavlovs Typus bei der Honigbiene, Apis mellifera. J. Fat. Sci. Hokkaido Univ. Ser. VI.13, 458–464.
30
MatsumotoY.HatanoA.UnokiS.MizunamiM. (2009). Stimulation of the cAMP system by the nitric oxide-cGMP system underlying the formation of long-term memory in an insect. Neurosci. Lett.467, 81–85.
31
MatsumotoY.MizunamiM. (2000). Olfactory learning in the cricket Gryllus bimaculatus. J. Exp. Biol.203, 2581–2588.
32
MatsumotoY.MizunamiM. (2002a). Temporal determinants of olfactory long-term retention in the cricket Gryllus bimaculatus. J. Exp. Biol.205, 1429–1437.
33
MatsumotoY.MizunamiM. (2002b). Lifetime olfactory memory in the cricket Gryllus bimaculatus. J. Comp. Physiol. A.188, 295–299.
34
MatsumotoY.MizunamiM. (2004). Context-dependent olfactory learning in an insect. Learn. Mem.11, 288–293.
35
MatsumotoY.MizunamiM. (2006). Olfactory memory capacity of the cricket Gryllus bimaculatus. Biol. Lett.2, 608–610.
36
MatsumotoY.NojiS.MizunamiM. (2003). Time course of protein synthesis-dependent phase of olfactory memory in the cricket Gryllus bimaculatus. Zool. Sci.20, 409–416.
37
MatsumotoY.UnokiS.AonumaH.MizunamiM. (2006). Critical role of nitric oxide-cGMP cascade in the formation of cAMP-dependent long-term memory. Learn. Mem.13, 35–44.
38
MenzelR.GiurfaM. (2006). Dimensions of cognition in an insect, the honeybee. Behav. Cogn. Neurosci. Rev.5, 24–40.
39
MenzelR.LeboulleG.EisenhardtD. (2006). Small brains, bright minds. Cell124, 237–239.10.1016/j.cell.2006.01.011
40
MizunamiM.OkadaR.LiY.StrausfeldN. J. (1998b). Mushroom bodies of the cockroach: The activity and identities of neurons recorded in freely moving animals. J. Comp. Neurol.402, 501–519.10.1002/(SICI)1096-9861(19981228)402:4<501::AID-CNE5>3.3.CO;2-D
41
MizunamiM.UnokiS.MoriY.HirashimaD.HatanoA.MatsumotoY. (2009). Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in an insect. BMC Biol.7, 46.10.1186/1741-7007-7-46
42
MizunamiM.WeibrechtJ. M.StrausfeldN. J. (1998a). Mushroom bodies of the cockroach: Their participation in place memory. J. Comp. Neurol.402, 520–537.10.1002/(SICI)1096-9861(19981228)402:4<520::AID-CNE6>3.3.CO;2-B
43
MizunamiM.YokohariF.TakahataM. (1999). Exploration into the adaptive design of the arthropod “microbrain.”Zool. Sci.16, 703–709.
44
MizunamiM.YokohariF.TakahataM. (2004). Further exploration into the adaptive design of the arthropod “microbrain”: I. Sensory and memory-processing systems. Zool. Sci.21, 1141–1151.
45
MustardJ. A.BlenauW.HamiltonI. S.WardV. K.EbertP. R.MercerA. R. (2003). Analysis of two D1-like dopamine receptors from the honeybee Apis mellifera reveals agonist-independent activity. Mol. Brain Res.113, 67–77.10.1016/S0169-328X(03)00091-3
46
NakataniY.MatsumotoY.MoriY.HirashimaD.NishinoH.ArikawaK.MizunamiM. (2009). Why the carrot is more effective than the stick: Different dynamics of punishment memory and reward memory and its possible biological basis. Neurobiol. Learn. Mem.92, 370–380.
47
OkadaR.IkedaJ.MizunamiM. (1999). Sensory responses and movement-related activities in extrinsic neurons of the cockroach mushroom bodies. J. Comp. Physiol. A.185, 115–129.
48
PeineH. A.HowarthR. (1975). Children and Parents. Baltimore: Penguin Books.
49
PickensC. L.HollandP. C. (2004). Conditioning and cognition. Neurosci. Behav. Rev.28, 651–661.
50
RescorlaR. A. (1988). Pavlovian conditioning. Am. Psychol.40, 151–160.10.1037/0003-066X.43.3.151
51
RiemenspergerT.VöllerT.StockP.BuchnerE.FialaA. (2005). Punishment prediction by dopaminergic neurons in Drosophila. Curr. Biol.15, 1953–1960.
52
RobertsA. C.GlanzmanD. L. (2003). Learning in Aplysia: looking at synaptic plasticity from both sides. Trends. Neurosci.26, 662–670.
53
RoederT. (1999). Octopamine in invertebrates. Prog. Neurobiol.59, 533–561.
54
RoederT.DegenJ.GeweckeM. (1998). Epinastine, a highly specific antagonist of insect neuronal octopamine receptors. Eur. J. Pharmacol.349, 171–177.
55
SchrollC.RiemenspergerT.BucherD.EhmerJ.VöllerT.ErbguthK.GerberB.HendelT.NagelG.BuchnerE.FialaA. (2006). Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae. Curr. Biol.16, 1741–1747.
56
SchultzW. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol.80, 1–27.
57
SchultzW. (2006). Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol.57, 87–115.10.1146/annurev.psych.56.091103.070229
58
SchwaerzelM.MonastiriotiM.ScholzH.Friggi-GrelinF.BirmanS.HeisenbergM. (2003). Dopamine and octopamine differentiate between aversive and appetitive olfactory memories in Drosophila. J. Neurosci.23, 10495–10502.
59
Scotto-LomasseseS.StrambiC.StrambiA.AouaneA.AugierR.RougonG.CayreM. (2003). Suppression of adult neurogenesis impairs olfactory learning and memory in an adult insect. J. Neurosci.23, 9289–9296.
60
SelchoM.PaulsD.HanK. A.StockerR. F.ThumA. S. (2009). The role of dopamine in Drosophila larval classical olfactory conditioning. PLoS ONE4, e5897.10.1371/journal.pone.0005897
61
SitaramanD.ZarsM.LaferriereH.ChenY. C.Sable-SmithA.KitamotoT.RottinghausG. E.ZarsT. (2008). Serotonin is necessary for place memory in Drosophila. Proc. Natl. Acad. Sci. U.S.A.105, 5579–5584.
62
StevensonP. A.DyakonovaV.RillichJ.SchildbergeK. (2005). Octopamine and experience-dependent modulation of aggression in crickets. J. Neurosci.25, 1431–1341.10.1523/JNEUROSCI.4258-04.2005
63
TakahashiT.HamadaA.MiyawakiK.MatsumotoY.MitoT.NojiS.MizunamiM. (2009). Systemic RNA interference for the study of learning and memory in an insect. J. Neurosci. Methods179, 9–15.
64
TempelB. L.BoniniN.DawsonD. R.QuinnW. G. (1983). Reward learning in normal and mutant Drosophila. Proc. Natl. Acad. Sci. U.S.A.80, 1482–1486.
65
TomchikS. M.DavisR. L. (2009). Dynamics of learning-related cAMP signaling and stimulus integration in the Drosophila olfactory pathway. Neuron64, 510–521.10.1016/j.neuron.2009.09.029
66
TullyT.QuinnW. G. (1985). Classical conditioning and retention in normal and mutant Drosophila melanogaster. J. Comp. Physiol. A.157, 263–277.
67
UnokiS.MatsumotoY.MizunamiM. (2005). Participation of octopaminergic reward system and dopaminergic punishment system in insect olfactory learning revealed by pharmacological study. Eur. J. Neurosci.22, 1409–1416.
68
UnokiS.MatsumotoY.MizunamiM. (2006). Roles of octopaminergic and dopaminergic neurons in mediating reward and punishment signals in insect visual learning. Eur. J. Neurosci.24, 2031–2038.
69
VergozV.RousselE.SandozJ-C.GiurfaM. (2007). Aversive learning in honeybees revealed by the olfactory conditioning of the sting extension reflex. PLoS ONE2, e288.10.1371/journal.pone.0000288
70
WaltersG. C.CrusecJ. E. (1977). Punishment. San Francisco: Freeman.
71
WessnitzerJ.ManganM.WebbB. (2008). Place memory in crickets. Proc. R. Sci. B.275, 915–921.
72
WiseR. A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci.5, 483–494.
73
ZhouC.RaoY.RaoY. (2008). A subset of octopaminergic neurons are important for Drosophila aggression. Nat. Neurosci.11, 1059–1167.
Summary
Keywords
octopamine, dopamine, classical conditioning, memory recall, olfactory learning, visual learning, crickets
Citation
Mizunami M and Matsumoto Y (2010) Roles of Aminergic Neurons in Formation and Recall of Associative Memory in Crickets. Front. Behav. Neurosci. 4:172. doi: 10.3389/fnbeh.2010.00172
Received
29 June 2010
Accepted
19 October 2010
Published
17 November 2010
Volume
4 - 2010
Edited by
Jean-Christophe Sandoz, National Center for Scientific Research, France
Reviewed by
Jean-Christophe Sandoz, National Center for Scientific Research, France; Randolf Menzel, Freie Universität Berlin, Germany; Monique Gauthier, University Paul Sabatier Toulouse 3, France
Copyright
© 2010 Mizunami and Matsumoto.
This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: Makoto Mizunami, Graduate School of Life Sciences, Hokkaido University, Sapporo 060-0810, Japan. e-mail: mizunami@sci.hokudai.ac.jp
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.