Roles of octopamine and dopamine in appetitive and aversive memory acquisition studied in olfactory conditioning of maxillary palpi extension response in crickets
Yukihisa Matsumoto
Chihiro-Sato Matsumoto
Ryo Wakuda3 
Makoto Mizunami- 1Faculty of Science, Hokkaido University, Sapporo, Japan
- 2Faculty of Liberal Arts, Tokyo Medical and Dental University, Ichikawa, Japan
- 3Graduate School of Life Science, Hokkaido University, Sapporo, Japan
Elucidation of reinforcing mechanisms for associative learning is an important subject in neuroscience. Based on results of our previous pharmacological studies in crickets, we suggested that octopamine and dopamine mediate reward and punishment signals, respectively, in associative learning. In fruit-flies, however, it was concluded that dopamine mediates both appetitive and aversive reinforcement, which differs from our suggestion in crickets. In our previous studies, the effect of conditioning was tested at 30 min after training or later, due to limitations of our experimental procedures, and thus the possibility that octopamine and dopamine were not needed for initial acquisition of learning was not ruled out. In this study we first established a conditioning procedure to enable us to evaluate acquisition performance in crickets. Crickets extended their maxillary palpi and vigorously swung them when they perceived some odors, and we found that crickets that received pairing of an odor with water reward or sodium chloride punishment exhibited an increase or decrease in percentages of maxillary palpi extension responses to the odor. Using this procedure, we found that octopamine and dopamine receptor antagonists impair acquisition of appetitive and aversive learning, respectively. This finding suggests that neurotransmitters mediating appetitive reinforcement differ in crickets and fruit-flies.
Introduction
Associative learning provides animals with the ability to adapt their behavior to a change in the environment. Elucidation of reinforcing mechanisms for associative learning is an important subject in neuroscience. In mammals, there is strong evidence that midbrain dopamine neurons mediate appetitive reinforcement signals (Schultz, 2006, 2013), and results of some studies have suggested that they also mediate aversive reinforcement signals (Matsumoto and Hikosaka, 2009). In crickets, we obtained pharmacological evidence suggesting that octopamine receptor antagonists (epinastine and mianserin) impair appetitive learning but not aversive learning, whereas dopamine receptor antagonists (flupenthixol, fluphenazine, chlorpromazine, and spiperone) impair aversive learning but not appetitive learning, and we thus proposed that octopamine and dopamine neurons mediate reward and punishment, respectively, in associative learning (Unoki et al., 2005, 2006; Mizunami et al., 2009; Nakatani et al., 2009). This is in accordance with suggestions from pharmacological studies on honey bees that octopamine participates in appetitive learning with sucrose reward (Hammer and Menzel, 1998) and dopamine participates in aversive learning with electric shock punishment (Vergoz et al., 2007). Based on results of recent studies using transgenic fruit-flies, however, it was concluded that different sets of dopamine neurons mediate appetitive and aversive reinforcements, while octopamine neurons transmit sweet taste signals to dopamine neurons in appetitive learning with sugar reward (Kim et al., 2007; Selcho et al., 2009; Burke et al., 2012; Liu et al., 2012; Lin et al., 2014). These findings in fruit-flies differ from those in crickets and urge us to re-examine the validity of our suggestions.
It should be pointed out that tests on the effects of conditioning in our studies in crickets were preformed 30 min after training or later, due to limitations of our previous “classical conditioning and operant testing” procedure (Matsumoto and Mizunami, 2002; Unoki et al., 2005), and thus the possibility that octopamine and dopamine are needed for 30 min memory retention but not for initial acquisition of learning was not ruled out. In our previous studies, crickets that received conditioning trials were allowed to rest for at least 15 min in the conditioning beaker before being transferred to the test apparatus because we speculated that disturbance immediately after conditioning may disrupt consolidation of memory. Moreover, after transferring the crickets to the test arena, crickets were first placed in a waiting chamber for at least 4 min for acclimation. Thus, we were not able to test acquisition performance immediately after training. This is in contrast to studies in flies in which the effect of conditioning was tested immediately after training. Therefore, it is critically important to establish procedures to allow investigation of possible roles of octopamine and dopamine in initial acquisition of learning in crickets.
In this study, we first established a conditioning procedure to allow evaluation of acquisition performance in crickets. We observed that crickets extended their maxillary palpi and vigorously swung them when water was applied to the antennae, which we refer to as maxillary palpi extension responses (MERs). Some odors such as vanilla and maple odors easily evoked MERs, while other odors such as peppermint and apple odors rarely induced MERs. In this study, we found that the MER to an odor is increased by pairing of the odor with water reward. This is analogous to olfactory conditioning of proboscis extension responses (PER) in honey bees, in which pairing of an odor and sucrose reward leads to an increase of the PER (Menzel and Giurfa, 2006; Giurfa, 2007; Menzel, 2012; Matsumoto et al., 2014). Moreover, we found that crickets exhibit a decrease in the MER to some odors by pairing of the odors with sodium chloride punishment, thus allowing the study of appetitive conditioning and aversive conditioning in a very similar experimental situation. We used these procedures to determine whether octopamine and dopamine are indeed required for acquisition of appetitive and aversive learning in crickets.
Materials and Methods
Insects
Adult male crickets, Gryllus bimaculatus, at 1–2 weeks after the imaginal molt were used. They were reared in a 12 h: 12 h light: dark cycle at 27 ± 2°C and were fed a diet of insect pellets and water ad libitum. Three days before the start of the experiment, crickets were individually placed in 100 ml glass beakers and fed a diet of insect pellets ad libitum but were deprived of drinking water to enhance their motivation to search for water.
Conditioning
We used differential appetitive or aversive conditioning procedures with water reward or sodium chloride punishment. In differential appetitive conditioning, individual animals received five trials, in which one of the two odors (peppermint and apple odors) was paired with water (unconditioned stimulus, US) and another odor was presented alone without pairing with US with an inter-trial interval (ITI) of 5 min (Figure 1). We refer to the former odor as paired odor (or conditioned stimulus, CS) and the latter odor as unpaired odor. Hypodermic syringes of 1 ml each were used for conditioning. A small filter paper was attached to the needle of the syringe. The syringe was filled with water, and the filter paper was soaked in peppermint or apple essence. For odor presentation, the filter paper was placed within 1 cm of the cricket's head. At 3 s after the onset of odor presentation, a drop of water was given to the mouth of the cricket for 2 s. The presence or absence of an MER was recorded during the first 3 s of odor presentation. In differential aversive conditioning experiment, individual animals received six trials, in which one of the two odors (vanilla and maple odors) was paired with a high concentration (20%) of sodium chloride solution (aversive US) and another odor was presented alone. Crickets exhibited MERs to these odors with high percentages (60–80%) prior to conditioning. The sequence of odor presentations was pseudo-randomized to avoid a possible sequential effect (Figure 1).
Figure 1. Procedures for differential olfactory conditioning with water reward or sodium chloride punishment. Crickets were subjected to presentation of one odor (paired odor: CS) paired with water (appetitive US) or a high concentration (20%) of sodium chloride solution (aversive US) and another odor (unpaired odor) without pairing with US five or six times each with pseudo-random sequences and with 5-min intervals. Retention was tested at 1 day after training. For pharmacology, 3 μL of saline or saline containing 2 μM epinastine or 200 μM flupentixol was injected into the hemolymph at 20 min before training.
We also performed experiments using absolute appetitive conditioning procedures, in which an odor is paired with water reward, to facilitate comparison with results of our previous studies using absolute conditioning procedures (Unoki et al., 2005; Mizunami et al., 2009). The methods and the results of absolute appetitive conditioning are described in Supplementary Figures S3–S5.
Retention Test
For evaluation of retention performance, the rates of MER to the paired odor and the unpaired odor were compared (Figure 1) at 1 day after conditioning. After the last odor presentation in the test, a drop of water was presented to the mouth or an antenna and the resulting MER was tested. Crickets that did not exhibit an MER to water US were not used for data evaluation. Crickets discarded by this criterion were less than 3% in total.
Pharmacology
For pharmacology, saline containing flupenthixol (Mustard et al., 2003) or epinastine (Roeder et al., 1998) was injected into the head haemolymph at 20 min before the start of training (Unoki et al., 2005). These drugs were purchased from Sigma-Aldrich (Tokyo, Japan).
Data Analysis
Occurrence of MER to odor presentation was measured during acquisition and in retention tests. In all experiments, percentage of MER (%MER) was calculated as the number of crickets that exhibited MER to the CS in the total number of crickets studied. Cochrans Q-test was used for within-group comparison of %MER during acquisition. McNemar's test was used for pairwise comparison of %MER between the odor paired with the US and the odor presented alone in acquisition. In the retention test, McNemar's test was used to compare %MER to the paired odor and the unpaired odor.
Results
Maxillary Palpi Extension Response of Crickets
Maxillary palpi of crickets are equipped with a number of olfactory receptors, contact chemoreceptors and mechanoreceptors, and crickets use these receptors to locate nearby food or water sources (Klein, 1981). When crickets are stationary, their maxillary palpi are usually held loosely beneath the mouthpart (Figure 2A). Upon application of water to the antennae, crickets extended and vigorously swung their maxillary palpi (Figure 2B). This response, which we term maxillary palpi extension response (MER), accompanied raising of the head and extension and swinging of the labial palpi and was immediately followed by vigorous swinging of the antennae, protraction of the mouth forward and upward, and frequent initiation of locomotor actions (Supplementary Movie 1). In short, MER is an initial phase of exploratory behavior in search for water or odor source.
Figure 2. Maxillary palpi extension response (MER) of the cricket. (A) When a cricket is stationary, its maxillary palpi are typically held loosely beneath the mouthparts (red circles). (B) Upon presenting a drop of water to an antenna of the cricket, the cricket extended (red circles) and vigorously swung its maxillary palpi while raising its head.
We observed that crickets often extended and vigorously swung their maxillary palpi when a small piece of filter paper soaked with an essence of some odors, such as vanilla and maple odors, was presented near the antennae. In contrast, presentation of other odors, such as peppermint odor and apple odor, rarely induced MER (10% or less). In subsequent experiments, peppermint and apple odors were used for appetitive conditioning with water reward, while vanilla and maple odors were used for aversive conditioning with sodium chloride punishment.
Acquisition and Retention in Differential Aversive MER Conditioning
We first attempted to establish a procedure for aversive olfactory conditioning of MER. We used a high concentration (20%) of sodium chloride solution as aversive US and vanilla and maple odors as CSs. We observed that repeated presentation of these odors alone without pairing with US led to a slight decrease of %MER to the odor (see Figure 3A). Therefore, in order to discriminate pairing-specific decrement of %MER (associative conditioning effect) from this non-associative effect (habituation), we used a differential conditioning procedure in which one of the two odors (vanilla and maple odors) was paired with US (paired odor; CS) and another odor was presented alone (unpaired odor) to allow comparison of MERs to the CS and the unpaired odor. The odors were presented six times each in a pseudo-random sequence and with 5-min ITIs. Percentages of MER to vanilla odor and those to maple odor were similarly high (>70%) in the first trials, and acquisition and retention performances of the vanilla CS group did not differ from those of the maple CS group (Supplementary Figure S1). Therefore, data from the two subgroups, vanilla CS group and maple CS group, were pooled. Percentage of MER to the CS significantly decreased with increase in the number of trials (Figure 3A, Cochran's Q-test: χ2 = 38, df = 5, p = 0.00000032). For the unpaired odor, we observed a slight decrease of %MER by repeated presentation of the odor, although the difference was not statistically significant (Cochran's Q-test: χ2 = 5.1, df = 5, p = 0.41). Percentages of MER to the CS did not significantly differ from that to the unpaired odor in the 1st trial (McNemar's test: χ2 = 0.0, df = 1, p = 1.0) and in the 2nd trial (χ2 = 2.3, p = 0.13), but it was significantly greater than that to the unpaired odor in the 3rd (χ2 = 6.8, p = 0.0094), 4th (χ2 = 4.9, p = 0.027), 5th (χ2 = 4.2, p = 0.041), and 6th trials (χ2 = 7.1, p = 0.0077). Thus, we conclude that the decrease of %MER to the paired odor is pairing-specific.
Figure 3. Differential aversive olfactory conditioning of MER with sodium chloride US. (A) Acquisition performance. Percentages of MER to an odor paired with 20% sodium chloride solution (paired odor, open circle) and those to an odor presented alone (unpaired odor, filled circle) in each conditioning trial are shown. Crickets were subjected to six pairing trials to associate an odor with sodium chloride solution (aversive US) and to presentation of another odor without pairing with the US with 5-min intervals, and percentages of MERs to the paired odor (gray graph) and the unpaired odor (black graph) were compared. The number of animals tested is shown in parentheses. (B) Retention performance at 1-day after conditioning. Percentages of MERs to the paired odor (white bar) and the unpaired odor (black bar) are shown. Percentage of MER to the paired odor was significantly lower than that to the unpaired odor. *p < 0.05; **p < 0.01.
Next, retention performance was tested at 1 day (24 h) after six-trial aversive conditioning. Aversively conditioned crickets exhibited a significantly lower %MER to the CS (paired odor) than to the unpaired odor (Figure 3B, McNemar's test: χ2 = 7.1, df = 1, p = 0.0077). Thus, 1-day aversive memory is in large part pairing-specific.
Acquisition and Retention in Differential Appetitive MER Conditioning
We next attempted to establish a procedure for appetitive conditioning with water reward. In one group of crickets, one of two odors (peppermint and apple odors) was paired with water and the other odor was presented alone without pairing with US for five times each in a pseudo-random sequence and with 5-min ITIs (Figure 1). Because acquisition and 1-day retention performances in the peppermint CS group did not differ from those in the apple CS group (Supplementary Figure S2), data from the two sub-groups were pooled. In the first trial, %MER to either peppermint or apple odor was low (< 15%), and it significantly increased with increase in the number of trials (Figure 4A, Cochran's Q-test: χ2 = 53, df = 4, p = 0.000000000086). For the unpaired odor, we observed a slight decrease of %MER by repeated presentation of the odor, although the difference was not statistically significant (Cochran's Q-test: χ2 = 2.4, df = 4, p = 0.67). The %MER to the paired odor did not significantly differ from that to the unpaired odor in the 1st trial (McNemar's test: χ2 = 0, df = 1, p = 1), but it was significantly greater than that to the unpaired odor in the 2nd (χ2 = 8.6, p = 0.0033), 3rd (χ2 = 14, p = 0.00014), 4th (χ2 = 18, p = 0.000023), and 5th trials (χ2 = 25, p = 0.00000056). Thus, we conclude that the increase of %MER to the paired odor is pairing-specific.
Figure 4. Differential appetitive olfactory conditioning of MER with water US. (A) Acquisition performance. Percentages of MER to the odor paired with water (paired odor: black graph) and the odor presented alone (unpaired odor; gray graph) are shown. Animals were subjected to five pairing trials to associate an odor with water and to presentation of another odor without pairing with the US with 5-min intervals. The number of animals tested is shown in parentheses. (B) Retention performance at 1 day after conditioning. Percentages of MERs to the paired odor (black bar) and those to the unpaired odor (white bar) are shown. The %MER to the paired odor was significantly higher than that to the unpaired odor. **p < 0.01; ***p < 0.001.
Retention performance was tested at 1 day (24 h) after conditioning. The group that received differential appetitive conditioning exhibited a significantly larger %MER to the paired odor than to the unpaired odor (Figure 4B, McNemar's test: χ2 = 20, df = 1, p = 0.0000076). Thus, 1-day appetitive memory is in large part pairing-specific.
We also attempted to establish a procedure of absolute appetitive olfactory conditioning of the MER, in which an odor was paired with water reward (Supplementary Figures S3, S4). This was to facilitate comparison with results of our previous studies using a dual-choice preference test, in which we used absolute conditioning procedure (Matsumoto and Mizunami, 2002; Unoki et al., 2005; Mizunami et al., 2009). We found that a few pairing trials of absolute appetitive conditioning are sufficient to achieve pairing-specific associative effect and that 30-min or 1 day memory is in large part CS-specific (Supplementary Figure S4), in agreement with our findings using a dual-choice preference test.
Epinastine, but not Flupenthixol, Impairs Acquisition of Differential Appetitive MER Conditioning
We then proceeded to a study on the effects of dopamine and octopamine receptor antagonists on appetitive and aversive learning using differential procedures. We used epinastine (Roeder et al., 1998) and flupenthixol (Mustard et al., 2003), potent antagonists of insect octopamine and dopamine receptors, respectively, since they were the most effective drugs to impair appetitive and aversive learning among drugs we used in our studies in crickets (Unoki et al., 2005, 2006; Mizunami et al., 2009; Nakatani et al., 2009; Matsumoto et al., 2013).
We first tested the effects of epinastine and flupentixol on differential MER conditioning with water reward. Three groups of crickets were each injected with 3 μl of saline (saline group) or saline containing 2 μM epinastine (epinastine group) or 200 μM flupenthixol (flupenthixol group) at 20 min prior to 5 trials of appetitive conditioning. The doses and the timing of injection were based on previous studies (Unoki et al., 2005, 2006; Mizunami et al., 2009; Nakatani et al., 2009). The saline and flupenthixol groups exhibited significant increases in %MER to the CS with progress of training (Figures 5A,E, Cochran's Q-test: saline: χ2 = 36, df = 4, p = 0.00000025; flupenthixol: χ2 = 46, p = 0.0000000022), but the epinastine group did not (Figure 5C, Cochran's Q-test: χ2 = 8.5, df = 4, p = 0.074). In these three groups, %MER to the unpaired odor, on the other hand, did not change with progress of training (Figures 5A,C,E, Cochran's Q-test: saline: χ2 = 4.82, df = 4, p = 0.31; epinastine: χ2 = 0.76, p = 0.94; flupenthixol: χ2 = 3.0, p = 0.55). The saline and flupenthixol groups exhibited significantly higher %MER to the CS than to the unpaired odor in the 3rd (Figures 5A,E, McNemar's test: saline: χ2 = 8.6, df = 1, p = 0.0033; flupenthixol: χ2 = 12, p = 0.00069), 4th (saline: χ2 = 12, p = 0.00051; flupenthixol: χ2 = 14, p = 0.00018), and 5th trials (saline: χ2 = 13, p = 0.00030; flupenthixol: χ2 = 20, p = 0.0000076). However, %MER to appetitive CS in the epinastine group did not significantly differ from that to the unpaired odor in all trials (Figure 5C, McNemar's test: p > 0.05). The results indicate that epinastine, but not flupenthixol, impairs acquisition of appetitive learning with water reward. We thus suggest that octopamine, but not dopamine, participates in acquisition of appetitive learning.
Figure 5. Epinastine, but not flupenthixol, impairs acquisition of appetitive MER conditioning. At 20 min prior to differential appetitive conditioning, crickets in three groups were each injected with 3 μl of saline (saline group, A,B) or saline containing 2 μM epinastine (epinastine group, C,D) or 200 μM flupenthixol (flupenthixol group, E,F). (A,C,E) Acquisition performance of the saline group (A), epinastine group (C) and flupenthixol group (E). Percentages of MER to the paired odor (black graphs) and those to the unpaired odor (gray graphs) in each conditioning trial are shown. The saline group and flupenthixol group exhibited effective acquisition, but the epinastine group did not. (B,D,F) One-day retention performance of the saline group (B), epinastine group (D), and flupenthixol group (F). Percentages of MER to the paired odor and those to the unpaired odors are shown. The saline group and flupenthixol group exhibited significantly higher %MER to the CS (black bars) than to the unpaired odor (white bars), indicating that the memory is CS-specific. In contrast, in the epinastine group, percentage of MER to the CS was as low as that to the unpaired odor, indicating no CS-specific memory. ***p < 0.001; **p < 0.01; NS: non-significant (p > 0.05).
Retention performance was tested at 1 day (24 h) after conditioning. The saline and flupenthixol groups exhibited a high %MER to the CS (>60%), and it was significantly greater than that to the unpaired odor (Figures 5B,F, McNemar's test: saline: χ2 = 12, df = 1, p = 0.00051; flupenthixol: χ2 = 21, p = 0.0000044). In the epinastine group, on the other hand, %MER to the CS was low (< 40%) and it did not significantly differ from that to the unpaired odor (Figure 5D, χ2 = 0.25, p = 0.62). Thus, the epinastine group exhibited no CS-specific memory, whereas 1-day retention of CS-specific memory was intact in the flupenthixol group as in the saline group.
We also confirmed that epinastine, but not flupenthixol, impaired initial acquisition and 1-day retention using a five-trial absolute appetitive conditioning procedure (Supplementary Figure S5). Thus, we suggest that octopamine, but not dopamine, is required for acquisition of appetitive learning, regardless of the procedure being differential or absolute.
Flupenthixol, but not Epinastine, Impairs Acquisition of Aversive MER Conditioning
In the final experiment, we tested the effects of epinastine and flupenthixol on differential MER conditioning with sodium chloride punishment. Three groups of crickets were each subjected to injection of 3 μl saline (saline group) or saline containing 2 μM epinastine (epinastine group) or 200 μM flupenthixol (flupenthixol group) at 20 min before 6 trials of differential aversive conditioning. In the saline and epinastine groups, %MER to aversively conditioned odor (CS, paired) significantly decreased with progress of training (Figures 6A,C, Cochran's Q-test: saline: χ2 = 42, df = 5, p = 0.000000048; epinastine: χ2 = 39, p = 0.00000021). Percentages of MER to the unpaired odor slightly decreased with progress of training, but the difference was not statistically significant (Cochran's Q-test: saline: χ2 = 5.0, df = 5, p = 0.42; epinastine: χ2 = 10, p = 0.076). The saline and epinastine groups exhibited significantly lower %MER to the CS than to the unpaired odor in the 3rd (Figures 6A,C, McNemar's test: saline: χ2 = 7.1, df = 1, p = 0.0077; epinastine: χ2 = 10, p = 0.0015), 4th (saline: χ2 = 13, p = 0.00030; epinastine: χ2 = 6.8, p = 0.0094), 5th (saline: χ2 = 10, p = 0.0015; epinastine: χ2 = 10, p = 0.0015), and 6th trials (saline: χ2 = 9.1, p = 0.0026; epinastine: χ2 = 10, p = 0.0015). In the flupenthixol group, on the other hand, there were no significant changes of %MER not only to the control odor (unpaired odor) but also to aversive CS with training (Figure 6E, Cochran's Q-test: to CS: χ2 = 5.0, df = 5, p = 0.42; to the unpaired odor: χ2 = 7.9, p = 0.16). The %MER to aversive CS in this group did not significantly differ from that to the unpaired odor in all trials (McNemar's test: p > 0.05). The results indicate that flupenthixol, but not epinastine, impairs acquisition of aversive learning with sodium chloride US. We thus suggest that dopamine, but not octopamine, is required for acquisition of aversive learning.
Figure 6. Flupenthixol, but not epinastine, impairs acquisition of aversive MER conditioning. At 20 min prior to differential aversive conditioning, crickets in three groups were each injected with 3 μl of saline (saline group, A,B) or saline containing 2 μM epinastine (epinastine group, C,D) or 200 μM flupenthixol (flupenthixol group, E,F). (A,C,E) Acquisition performance of the saline group (A), epinastine group (C) and flupenthixol group (E). Percentages of MER to the paired odor (gray graph) and those to the unpaired odor (black graphs) in each conditioning trial are shown. The saline and epinastine groups exhibited significantly lower %MER to the aversive CS (gray graph) than to the unpaired odor (black graph), but the flupenthixol group exhibited no significantly different %MER to the CS than that to the unpaired odor. (B,D,F) Retention performance at 1 day after conditioning of the saline group (B), epinastine group (D) and flupenthixol group (F). Percentages of MER to the paired odor and the unpaired odor are shown. The saline and epinastine groups exhibited significantly lower %MER to the aversive CS (white bars) than to the unpaired odor (black bars), indicating CS-specific aversive memory. In the flupenthixol group, on the other hand, %MER to the CS did not significantly differ from that to the unpaired odor, indicating no CS-specific memory. The number of animals tested is shown in parentheses.**p < 0.01; ***p < 0.001; NS: non-significant (p > 0.05).
In a 1-day retention test, the saline and epinastine groups exhibited significantly lower %MER to the aversive CS than to the unpaired odor (Figures 6B,D, McNemar's test: saline: χ2 = 8.1, df = 1, p = 0.0044; epinastine: χ2 = 12, p = 0.0005). In the flupenthixol group, on the other hand, %MER to the aversive CS did not significantly differ from that to the unpaired odor (Figure 6F, χ2 = 0.36, p = 0.55). Thus, the epinastine group exhibited CS-specific memory as did the saline group, but the flupenthixol group did not.
Discussion
We showed that olfactory conditioning of MER provides an excellent paradigm to investigate acquisition performance of appetitive and aversive conditioning in crickets. MER is an initial phase of exploratory movements in search for water or odor sources, and the occurrence of MER to an odor can be easily judged if crickets and their palpi are stationary prior to water or odor presentation. Crickets exhibited high percentages of MER to some odors but not to other odors, and we used the former odors in aversive conditioning and the latter odors in appetitive conditioning. We observed that repeated presentation of one of the former odors alone led to a slight decrease of the MER, although the difference was not statistically significant. We thus used a differential aversive conditioning procedure for evaluation of the pairing-specific effect. In contrast, we observed that repeated presentation of one of the latter odors alone did not lead to a change of the MER. In appetitive or aversive conditioning, we observed a significant difference in the MER between the paired and unpaired odors in the 2nd or 3rd trial. Thus, only a few trials are sufficient to induce a conditioning effect. Retention tests performed 1 day after appetitive or aversive conditioning demonstrated that the memory is in large part CS-specific. Olfactory MER conditioning thus provides a robust measure of acquisition and retention in olfactory learning in crickets.
In addition to differential conditioning procedures used in this study, we established absolute appetite conditioning procedure to associate an odor with water reward (Supplementary Figure S3), and we observed that only a few trials are sufficient to induce a conditioning effect (Supplementary Figure S4A) and memory at 30 min or 1 day after 5-trial absolute appetitive conditioning is in large part CS-specific (Supplementary Figure S4B). Therefore, both of the differential and absolute olfactory MER conditioning procedures provide robust measures of acquisition and retention in olfactory learning in crickets.
We now have two procedures to test the effect of olfactory conditioning training, the MER test and the operant dual-choice preference test. The former is useful for studying mechanistic aspects of learning and memory, while the latter provides insights into the behavioral significance of them. Crickets should open up further possibilities for studying mechanistic and adaptive aspects of learning and memory.
Comparison of MER Conditioning with Conditioning of Feeding Responses in Other Insects
Conditioning of MER by pairing an odor with water reward described here is analogous to olfactory conditioning of mouthpart movements reported in several insects, especially that of proboscis extension response (PER) in honey bees (Giurfa and Sandoz, 2012). We observed 60–80% MERs to an odor paired with water US in crickets after four pairing trials, the percentages being slightly less than those of PERs to sucrose-associated odor in honey bees (more than 80% after three or more trials; Menzel and Giurfa, 2006; Matsumoto et al., 2014) and being comparable or slightly higher than those of PERs in the moth Manduca sexta (50–70%; Daly and Smith, 2000), those of maxilla-labium extension responses in Camponotus ants (40–60%; Guerrieri et al., 2011) and those of maxillary palpi opening reactions in the desert locust Schistocerca gregaria (40–50%; Simões et al., 2011). In all of those studies, the conditioned response (mouthpart movement) was measured in harnessed insects, but we measured it from freely moving animals placed in a small beaker in this study. We found that harnessed crickets also exhibit MER, but the percentage of MER was not as high as that in crickets placed in a beaker (data not shown).
We were able to evaluate aversive conditioning by a decrease of the MER to odors paired with sodium chloride punishment. This allows evaluation of appetitive conditioning and aversive conditioning in very similar experimental conditions and thus facilitates comparisons of the neural basis and molecular basis of acquisition in appetitive and aversive conditioning. For future studies, it will be more ideal to find odors that produce MER at a rate of about 50% and use them in both appetitive and aversive conditioning. To our knowledge, little effort has been directed toward establishing a procedure to use the decrease in the rate of feeding response for evaluation of aversive conditioning in other species of insects.
Roles of Octopamine and Dopamine in Appetitive and Aversive Conditioning
We found that octopamine and dopamine receptor antagonists specifically impair acquisition of appetitive learning with water reward and aversive learning with sodium chloride punishment, respectively (Figures 5, 6, Supplementary Figure S5). It should be pointed out that the experimental procedures for appetitive and aversive learning were not exactly the same, namely, the odors used as CS and the number of conditioning trials differed. Therefore, whether such differences might account for different effects of antagonists should be discussed. This possibility, however, is unlikely since we observed that the effects of antagonists were conserved in experiments with different kinds of CS and different number of trials (Unoki et al., 2005, 2006; Mizunami et al., 2009; Nakatani et al., 2009; Mizunami and Matsumoto, 2010). Therefore, we suggest that octopamine and dopamine mediate reinforcement signals for acquisition of appetitive and aversive learning, respectively, in crickets. In accordance with our suggestion, it has been suggested in honey bees that octopamine participates in appetitive learning with sucrose reward (Hammer and Menzel, 1998) and that dopamine participates in aversive learning with electric shock punishment (Vergoz et al., 2007).
Based on results of recent studies on fruit-flies, however, it was concluded that different sets of dopamine neurons mediate appetitive and aversive reinforcements in olfactory conditioning with sucrose, water, or electric shock, while octopamine neurons convey sweet taste signals of sucrose reward to dopamine neurons (Burke et al., 2012; Liu et al., 2012; Lin et al., 2014; Yamagata et al., 2015: For an alternative view, see Kim et al., 2013.). Thus, we suggest that neurotransmitters mediating appetitive reinforcement differ in crickets and fruit-flies, at least in learning with water reward, whereas those mediating aversive reinforcement are the same. In order to confirm that octopamine and dopamine mediate appetitive and aversive reinforcement, respectively, we are currently performing studies using RNAi (Shinmyo et al., 2006; Takahashi et al., 2009) and CRISPR/Cas9 systems (Cong et al., 2013; Mali et al., 2013) to knockdown or knockout expression of octopamine and dopamine receptor genes in crickets.
Concerning signals that animals use for reinforcement, we recently obtained evidence that the prediction error theory, which states that the discrepancy, or error, between actual reward and predicted reward determines whether learning occurs (Rescorla and Wagner, 1972), is applicable to crickets (Terao et al., 2015). This theory is known as the best theory to account for associative learning in mammals (Schultz, 2006, 2013) and there is strong evidence that dopamine neurons in the midbrain mediate appetitive prediction error signals (Schultz, 2013), although whether they also mediate aversive prediction error signals is controversial (Matsumoto and Hikosaka, 2009; Fiorillo, 2013). For crickets, we reported results of pharmacological analysis suggesting that octopamine neurons mediate appetitive prediction error signals (Terao et al., 2015), although it remains to be determined whether dopamine neurons mediate aversive prediction error signals in crickets. Future electrophysiological studies on dopamine and octopamine neurons projecting to the mushroom body, a higher-order associative center participating in olfactory learning (Menzel and Giurfa, 2006; Davis, 2011; Menzel, 2012), are needed to clarify neural mechanisms of prediction error computation in associative learning in crickets.
In conclusion, we suggest that there is an unexpected diversity in neurotransmitters mediating appetitive reinforcement, while those mediating aversive reinforcement are conserved, among different species of insects. There arises a question of to what extent the neurotransmitter mechanisms of learning differ between different insect species and how such diversity has evolved (see also Mizunami et al., 2015). Further studies on a number of insects, such as honey bees, moths, and cockroaches, are needed for answering this interesting question.
Author Contributions
YM designed and carried out experiments, analyzed and interpreted data, and wrote the manuscript. CM, RW, and SI carried out experiments and analyzed and interpreted data. MM designed experiments, analyzed and interpreted data, and wrote the manuscript.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This study was supported by Grants-in-Aid for Scientific Research from the Ministry of Education, Science, Culture, Sports and Technology of Japan (No.24370030 and No. 25440163).
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/article/10.3389/fnbeh.2015.00230
References
Burke, C. J., Huetteroth, W., Owald, D., Perisse, E., Krashes, M. J., Das, G., et al. (2012). Layered reward signalling through octopamine and dopamine in Drosophila. Nature 492, 433–437. doi: 10.1038/nature11614
Cong, L., Ran, F. A., Cox, D., Lin, S., Barretto, R., Habib, N., et al. (2013). Multiplex genome engineering using CRISPR/Cas systems. Science 339, 819–823. doi: 10.1126/science.1231143
Daly, K. C., and Smith, B. H. (2000). Associative olfactory learning in the moth Manduca sexta. J. Exp. Biol. 203, 2025–2038.
Davis, R. L. (2011). Traces of Drosophila memory. Neuron 70, 8–19. doi: 10.1016/j.neuron.2011.03.012
Fiorillo, C. D. (2013). Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341, 546–549. doi: 10.1126/science.1238699
Giurfa, M. (2007). Behavioral and neural analysis of associative learning in the honeybee: a taste from the magic well. J. Comp. Physiol. A 193, 801–824. doi: 10.1007/s00359-007-0235-9
Giurfa, M., and Sandoz, J. C. (2012). Invertebrate learning and memory: fifty years of olfactory conditioning of the proboscis extension response in honeybees. Learn. Mem. 19, 54–66. doi: 10.1101/lm.024711.111
Guerrieri, F. J., D'Ettorre, P., Devaud, J. M., and Giurfa, M. (2011). Long-term olfactory memories are stabilised via protein synthesis in Camponotus fellah ants. J. Exp. Biol. 21, 3300–3304. doi: 10.1242/jeb.059170
Hammer, M. R., and Menzel, R. (1998). Multiple sites of associative odor learning as revealed by local brain microinjections of octopamine in honeybees. Learn. Mem. 5, 146–156.
Kim, Y. C., Lee, H. G., and Han, K. A. (2007). D1 dopamine receptor dDA1 is required in the mushroom body neurons for aversive and appetitive learning in Drosophila. J. Neurosci. 27, 7640–7647. doi: 10.1523/JNEUROSCI.1167-07.2007
Kim, Y. C., Lee, H. G., Lim, J., and Han, K. A. (2013). Appetitive learning requires the alpha1-like octopamine receptor OAMB in the Drosophila mushroom body neurons. J. Neurosci. 33, 1672–1677. doi: 10.1523/JNEUROSCI.3042-12.2013
Klein, U. (1981). Sensilla of the cricket palp. Fine structure and spatial organization. Cell Tissue Res. 219, 229–252. doi: 10.1007/BF00210145
Lin, S., Owald, D., Chandra, V., Talbot, C., Huetteroth, W., and Waddell, S. (2014). Neural correlates of water reward in thirsty Drosophila. Nat. Neurosci. 17, 1536–1542. doi: 10.1038/nn.3827
Liu, C., Plaçais, P. Y., Yamagata, N., Pfeiffer, B. D., Aso, Y., Friedrich, A. B., et al. (2012). A subset of dopamine neurons signals reward for odour memory in Drosophila. Nature 488, 512–516. doi: 10.1038/nature11304
Mali, P., Yang, L., Esvelt, K. M., Aach, J., Guell, M., DiCarlo, J. E., et al. (2013). RNA-guided human genome engineering via Cas9. Science 339, 823–826. doi: 10.1126/science.1232033
Matsumoto, M., and Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841. doi: 10.1038/nature08028
Matsumoto, Y., Hirashima, D., and Mizunami, M. (2013). Analysis and modeling of neural processes underlying sensory preconditioning. Neurobiol. Learn. Mem. 101, 103–113. doi: 10.1016/j.nlm.2013.01.008
Matsumoto, Y., and Mizunami, M. (2002). Temporal determinants of olfactory long-term retention in the cricket Gryllus bimaculatus. J. Exp. Biol. 205, 1429–1437.
Matsumoto, Y., Sandoz, J. C., Devaud, J. M., Lormant, F., Mizunami, M., and Giurfa, M. (2014). Cyclic nucleotide–gated channels, calmodulin, adenylyl cyclase and calcium/calmodulin-dependent kinase II are required for late but not early long-term memory formation in the honey bee. Learn. Mem. 21, 272–286. doi: 10.1101/lm.032037.113
Menzel, R. (2012). The honeybee as a model for understanding the basis of cognition. Nat. Rev. Neurosci. 13, 758–768. doi: 10.1038/nrn3357
Menzel, R., and Giurfa, M. (2006). Dimensions of cognition in an insect, the honeybee. Behav. Cogn. Neurosci. Rev. 5, 24–40. doi: 10.1177/1534582306289522
Mizunami, M., Hamanaka, Y., and Nishino, H. (2015). Toward elucidating diversity of neural mechanisms underlying insect learning. Zool. Lett. 1, 8. doi: 10.1186/s40851-014-0008-6
Mizunami, M., and Matsumoto, Y. (2010). Roles of aminergic neurons in formation and recall of associative memory in crickets. Frontiers Behav. Neurosci. 4:172. doi: 10.3389/fnbeh.2010.00172
Mizunami, M., Unoki, S., Mori, Y., Hirashima, D., Hatano, A., and Matsumoto, Y. (2009). Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in an insect. BMC Biol. 7:46. doi: 10.1186/1741-7007-7-46
Mustard, J. A., Blenau, W., Hamilton, I. S., Ward, V. K., Ebert, P. R., and Mercer, A. R. (2003). Analysis of two D1-like dopamine receptors from the honey bee Apis mellifera reveals agonist-independent activity. Mol. Brain Res. 113, 67–77. doi: 10.1016/S0169-328X(03)00091-3
Nakatani, Y., Matsumoto, Y., Mori, Y., Hirashima, D., Nishino, H., Arikawa, K., et al. (2009). Why the carrot is more effective than the stick: different dynamics of punishment memory and reward memory and its possible biological basis. Neurobiol. Learn. Mem. 92, 370–380. doi: 10.1016/j.nlm.2009.05.003
Rescorla, R. A., and Wagner, A. R. (1972). “A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement,” in Classical Conditioning II, eds A. Black and W. R. Prokasy (New York, NY: Academic Press), 64–99.
Roeder, T., Degen, J., and Gewecke, M. (1998). Epinastine, a highly specific antagonist of insect neuronal octopamine receptors. Eur. J. Pharmacol. 349, 171–177. doi: 10.1016/S0014-2999(98)00192-7
Schultz, W. (2006). Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115. doi: 10.1146/annurev.psych.56.091103.070229
Schultz, W. (2013). Updating dopamine reward signals. Curr. Opin. Neurobiol. 23, 229–238. doi: 10.1016/j.conb.2012.11.012
Selcho, M., Pauls, D., Han, K. A., Stocker, R. F., and Thum, A. S. (2009). The role of dopamine in Drosophila larval classical olfactory conditioning. PLoS ONE 4:e5897. doi: 10.1371/journal.pone.0005897
Shinmyo, Y., Mito, T., Uda, T., Nakamura, T., Miyawaki, K., Ohuchi, H., et al. (2006). Brachyenteronis necessary for morphogenesis for the posterior gut but not for anteroposterior axial elongation from the posterior growth zone in the intermediate-germband cricket Gryllus bimaculatus. Development 133, 4539–4547. doi: 10.1242/dev.02646
Simões, P., Ott, S. R., and Niven, J. E. (2011). Associative olfactory learning in the desert locust, Schistocerca gregaria. J. Exp. Biol. 214, 2495–2503. doi: 10.1242/jeb.055806
Takahashi, T., Hamada, A., Miyawaki, K., Matsumoto, Y., Mito, T., Noji, S., et al. (2009). Systemic RNA interference for the study of learning and memory in an insect. J. Neurosci. Methods 179, 9–15. doi: 10.1016/j.jneumeth.2009.01.002
Terao, K., Matsumoto, Y., and Mizunami, M. (2015). Critical evidence for the prediction error theory in associative learning. Sci. Rep. 5:8929. doi: 10.1038/srep08929
Unoki, S., Matsumoto, Y., and Mizunami, M. (2005). Participation of octopaminergic reward system and dopaminergic punishment system in insect olfactory learning revealed by pharmacological study. Eur. J. Neurosci. 22, 1409–1416. doi: 10.1111/j.1460-9568.2005.04318.x
Unoki, S., Matsumoto, Y., and Mizunami, M. (2006). Roles of octopaminergic and dopaminergic neurons in mediating reward and punishment signals in insect visual learning. Eur. J. Neurosci. 24, 2031–2038. doi: 10.1111/j.1460-9568.2006.05099.x
Vergoz, V., Roussel, E., Sandoz, J. C., and Giurfa, M. (2007). Aversive learning in honeybees revealed by the olfactory conditioning of the sting extension reflex. PLoS ONE 2:e288. doi: 10.1371/journal.pone.0000288
Keywords: classical conditioning, memory acquisition, dopamine, octopamine, appetite learning, aversive learning, Gryllus bimaculatus, insects
Citation: Matsumoto Y, Matsumoto CS, Wakuda R, Ichihara S and Mizunami M (2015) Roles of octopamine and dopamine in appetitive and aversive memory acquisition studied in olfactory conditioning of maxillary palpi extension response in crickets. Front. Behav. Neurosci. 9:230. doi: 10.3389/fnbeh.2015.00230
Received: 11 April 2015; Accepted: 13 August 2015;
Published: 01 September 2015.
Edited by:
Bertram Gerber, Leibniz Institut für Neurobiologie, GermanyReviewed by:
Troy Zars, University of Missouri, USAPaul Anthony Stevenson, University of Leipzig, Germany
Andreas Stephan Thum, University of Konstanz, Germany
Copyright © 2015 Matsumoto, Matsumoto, Wakuda, Ichihara and Mizunami. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Makoto Mizunami, Faculty of Science, Hokkaido University, Sapporo 060-0810, Japan, mizunami@sci.hokudai.ac.jp