Memory Formation in Reversal Learning of the Honeybee

In reversal learning animals are first trained with a differential learning protocol, where they learn to respond to a reinforced odor (CS+) and not to respond to a non-reinforced odor (CS−). Once they respond correctly to this rule, the contingencies of the conditioned stimuli are reversed, and animals learn to adjust their response to the new rule. This study investigated the effect of a protein synthesis inhibitor (emetine) on the memory formed after reversal learning in the honeybee Apis mellifera. Two groups of bees were studied: summer bees and winter bees, each yielded different results. Blocking protein synthesis in summer bees inhibits consolidation of the excitatory learning following reversal learning whereas it blocked the consolidation of the inhibitory learning in winter bees. These findings suggest that excitatory and inhibitory learning may involve different molecular processes in bees, which are seasonally dependent.

. Odors are used as CSs, and sucrose solution as US for hungry bees. Several forms of memory developing in series and in parallel have been described leading to lifelong memory under appropriate conditions (Menzel, 1990).
Memory formation has been shown to consist of distinctive phases; each depends on different molecular pathways: short-term and mid-term memories depend on existing proteins, and two forms of long-term memory (LTM) are controlled by different signaling cascades (Menzel and Müller, 1996;Menzel, 1999;Müssig et al., 2010). Notably, early LTM (eLTM) depends on translation and late LTM (lLTM) depends on transcription processes (Wüstenberg et al., 1998;Friedrich et al., 2004). When applied shortly prior to acquisition, emetine, a protein synthesis blocker which inhibits translation processes, is known to inhibit eLTM consolidation in the honeybee. The effects of transcription and translation inhibitors have been studied so far only in simple forward pairing paradigms (Wüstenberg et al., 1998;Menzel et al., 2001;Friedrich et al., 2004) and in extinction paradigms (Stollhoff et al., 2005;Stollhoff and Eisenhardt, 2009).
In the honeybee, reversal learning was found to have a heritable component which is manifested in the rapidity to reverse from the former CS− to the new CS+ association (Ferguson et al., 2001). However, Ben-Shahar et al. (2000) found differences in the extinction rate of the former CS+ during the reversal phase, which were derived from the bees' behavioral state: nurses showed faster rates of extinction than foragers. Taken together, these finding suggest that two dissociable processes constitute the reversal learning, i.e., excitatory learning and inhibitory learning.
Using local anesthetics to block the main output region of the mushroom body (MB), Devaud et al. (2007) were able to demonstrate that the acquisition of reversal learning requires an intact MB activity, whereas simple differential learning (the first phase in reversal paradigm) was spared. It was also shown that experiencing olfactory reversal learning improves the bee's future performance

IntroductIon
In classical conditioning, animals learn to associate an originally neutral stimulus (CS) with a biologically significant stimulus (US) if the CS is followed by the US (forward pairing). Animals are also capable of acquiring an opposite contingency for a given CS, i.e., the absence of the US. Following Pavlov's (1927) terminology, differential conditioning consists of two such contingencies, where the stimulus which precedes the appearance of the US (CS+) retains an excitatory valence, and the one which predicts the absence of the US (CS−) retains an inhibitory one. In reversal learning the animal is first introduced to differential conditioning and once such discrimination has been learned, the stimuli's contingencies are reversed and the animal learns to adapt its response to the new rule. Following Pavlov (1927), forward pairing of CS with reinforcement generates excitatory learning whereas extinction leads to inhibitory learning. Thus reversal learning is a paradigm entailing rather more complex learning than a simple acquisition and extinction, as the animal has to form such new associations on the background of inverted contingencies. The molecular underpinnings of acquisition and extinction learning are believed to differ, particularly in regard to the requirement of protein synthesis. In a wide range of experimental preparations, protein synthesis inhibition was found to block memory formation of acquisition learning (e.g., Davis and Squire, 1984;Abel et al., 1997;Lattal and Abel, 2001). In extinction on the other hand, the administration of protein synthesis inhibitors yielded conflicting results which probably depend on the experimental protocol used (e.g., Flood et al., 1977;Berman and Dudai, 2001;Stollhoff et al., 2005;Duvarci et al., 2006). Altogether, reversal learning provides an adequate paradigm to study both acquisition and extinction learning and memory.
The honeybee (Apis mellifera) serves as a valid model for the study of the underlying mechanisms of learning and memory (Menzel et al., 2006) for which many paradigms of conditioning were tested. It was found that the results follow the rules of classical conditioning as known from laboratory mammals (Bitterman Memory formation in reversal learning of the honeybee Neurobiology, Institut für Biologie, Freie Universität Berlin, Berlin, Germany In reversal learning animals are first trained with a differential learning protocol, where they learn to respond to a reinforced odor (CS+) and not to respond to a non-reinforced odor (CS−). Once they respond correctly to this rule, the contingencies of the conditioned stimuli are reversed, and animals learn to adjust their response to the new rule. This study investigated the effect of a protein synthesis inhibitor (emetine) on the memory formed after reversal learning in the honeybee Apis mellifera. Two groups of bees were studied: summer bees and winter bees, each yielded different results. Blocking protein synthesis in summer bees inhibits consolidation of the excitatory learning following reversal learning whereas it blocked the consolidation of the inhibitory learning in winter bees. These findings suggest that excitatory and inhibitory learning may involve different molecular processes in bees, which are seasonally dependent.

Ravit Hadar* and Randolf Menzel
in solving further discrimination reversals (Komischke et al., 2002), a feature that might serve to optimize bee's foraging efficiency when food-source profitability changes. However, those studies were designed so, that the temporal spacing of each phase from the next allowed only the formation of short-term and mid-term memories in this paradigm.
Here the effect of emetine on the eLTM formed after reversal learning was investigated in order to elucidate the consolidation of excitatory and inhibitory associations formed after reversal learning, into eLTM. To this end, each learning phase took place on a different day, when translation-dependent memories are formed. Two groups of honeybees were used: summer bees and winter bees, because it was observed in earlier experiments (Menzel et al., 2001) that inhibiting transcription factors yields different results in summer and winter bees, specifically, winter bees did not develop long-lasting memory following spaced conditioning. We found that blocking protein synthesis during consolidation of reversal learning inhibits the consolidation of the excitatory learning in summer bees whereas consolidation of inhibitory learning was blocked in winter bees.

General procedures related to behavIor
The experiments were conducted in Berlin, Germany using honeybees (A. mellifera carnica) from the colonies of the laboratory. Experiments were carried out in summer time (July/August 2009), using bees raised in outdoor hives, and in winter time (November/ December 2009), using bees kept in small flight cages (1 m 3 ) in a glasshouse. One day prior to the experimental procedure, foraging bees were caught at the hive entrance when leaving the hive; they were then immobilized by cooling and harnessed in small metal tubes. In the evening bees were fed to satiation with a 1-M sucrose solution. On each experimental day, bees were fed in the afternoon to satiation and then kept in a dark and humid box at room temperature (∼22°C, ∼70% humidity).

condItIonInG of the per
All acquisition and retrieval trials shared a standardized protocol; each acquisition trial began by positioning a test bee in front of an exhaust fan. Odor stimuli (CS) were applied after 10 s (duration 4 s) and were delivered through 5 ml syringe, each containing a filter paper soaked with 4 μL of pure odorant, 2-octanone, and 1-hexanol (Sigma-Aldrich Chemie GmbH). Computer-controlled magnetic valves were used for the delivery of the odorants, allowing constant air flow. The presentation of the US started 3 s after odor onset by touching the antennae with a toothpick soaked in sucrose solution to induce proboscis extension. US delivery lasted for 4 s during which animals were allowed to lick sucrose solution with the proboscis (hence 1 s overlap between CS and US).
On unrewarded trials (CS−) all conditions remained the same, except there was no presentation of the US (sucrose). A positive response was scored if the proboscis was extended during the CS and before the US.

reversal learnInG protocol
On the first day animals were subjected to a differential conditioning protocol with two odorants A and B (2-octanone and 1-hexanol), one forward paired with the US (sucrose solution), the other unrewarded (day 1: A+ vs. B−). Each odorant was presented six times in a pseudo-randomized order and the sequence of odor presentation was identical for all subjects (ABBABAABABBA). Odor identities were counter balanced across subjects.
The intertrial interval was 10 min. On the following day the reinforcement pattern was reversed (day 2: A− vs. B+) whereas all other conditions remained constant. Retention tests were carried out on the third day, where both odorants were presented in the absence of reward.
Acquisition curves are presented as percentages of bees showing conditioned PER for each pair of CS+ and CS− presentations, which constitute one block trial.

data analysIs
Only animals that survived until the retention test and then showed an unconditioned response to sucrose were included. The ordinates give the probability of PER responses. The McNemar χ 2 test (Zar, 1997) was used (SigmaStat) for within-group comparison of the CR to the different odors. The G-test for contingency tables (log likelihood ratio) was used when testing the differences in CR for each odor for between group comparisons.

control experIMents
Control experiments were designed in order to rule out a general effect of emetine on performance. On the first day bees were subjected to a differential conditioning protocol as described above. On the following day, bees were assigned randomly to two groups and were injected with either emetine or PBS, and after 30 min a retention test for both odors (in the absence of a reward) was carried out. On the third day bees underwent another retention test for both odors.

Emetine inhibits the new excitatory learning when applied in summer
On the first experimental day, summer bees were trained to differentiate between two odorants, one being rewarded (A+) whereas the other was presented alone (B−). Each odor was presented six times; by the last differential learning trials the proportions of CRs to the A+ and B− were 76 and 2%, respectively (McNemar's Test: χ 2 = 112.00, p < 0.001, df = 1; Figure 1A).
On the following day, 30 min prior to reversal learning training, bees were randomly assigned to two groups; one being injected with emetine and the other with PBS (phosphate buffer used as saline for emetine). All bees were then trained to the reversed rule (A− vs. B+). By the last differential learning trials the proportions of CRs to A− and B+ were 4 and 49% in the PBS group (McNemar's Test: χ 2 = 29.03, p = <0.001, df = 1) and 8 and 39% with the PBS injected group (G test: G = 4.254, p < 0.05, df = 1), which indicates an emetine-treatment induced interference with consolidation of excitatory learning.

Control experiments: when applied 24 h after differential learning, emetine has no effect on memory retrieval
On the first experimental day, summer bees underwent a differential conditioning protocol, as described above. By the last differential learning trials the proportions of bees exhibiting the CR to the A+ and B− were 74 and 5%, respectively (McNemar's Test: χ 2 = 53.01, p < 0.001, df = 1; Figure 2A). On the following day, 30 min prior to a retention test, bees were randomly assigned to two groups; one in the emetine group, respectively (McNemar's Test: χ 2 = 21.04, p = <0.001, df = 1; Figure 1B). Emetine injections had no effect on acquisition during this phase.
On the third day, 24 h after the reversal learning, all bees were subjected to a retention test for both odorants ( Figure 1C). The group injected with PBS scored significantly higher for odor B than for odor A (McNemar's Test: χ 2 = 4.267, p < 0.05, df = 1) indicating that the reversal rule had been learned and was remembered. In contrast, the emetine injected group scored the same for both odors (McNemar's Test: χ 2 = 0.050, NS, df = 1), indicating that this group did not remember the reversed rule. Moreover, the emetine injected group scored significantly lower for odor B in comparison  As in the summer experiments, emetine injections had no effect on the acquisition curves during this phase. On the third day a retention test for both odorants was carried out ( Figure 3C). As in summer bees, the group injected with PBS scored significantly higher for odor B than for odor A (McNemar's Test: χ 2 = 9.091, p < 0.05, df = 1) indicating that bees learned to associate odor B with reward. As in summer, no significant difference between the two odorants was observed in the emetine injected group (McNemar's Test: χ 2 = 0.571, NS, df = 1). However, as opposed to the results achieved in the summer where emetine inhibited the excitatory association, here the emetine injected group scored significantly higher for odor A, when compared with the PBS group (G test: G = 4.0422, p < 0.05, df = 1).

Control experiments: when applied 24 h after differential conditioning, emetine has no effect on memory retrieval
On the first experimental day, winter bees underwent a differential conditioning protocol, as described above. By the last differential learning trials the proportions of bees exhibiting CRs to the A+ and B− were 65 and 15%, respectively (McNemar's Test: χ 2 = 27.57, p < 0.001, df = 1; Figure 4A). On the following day, 30 min prior to a retention test, bees were randomly assigned to two groups; one being injected with emetine and the other with PBS. At the retention test both groups scored significantly higher to odor A than to odor B; PBS group 64 and 38%, respectively (McNemar's Test: χ 2 = 4.26, p < 0.05, df = 1), emetine group 74 and 42%, respectively (McNemar's Test: χ 2 = 4.76, p < 0.05, df = 1; Figure 4B). There was no significant difference in the proportions of bee exhibiting the CR between the two experimental groups (G test odor A: PBS vs. Eme = 0.74, NS, df = 1; G test odor B: PBS vs. Eme = 0.15, NS, df = 1). Another retention test was carried out on the third day, again, both groups scored significantly higher for odor A than for odor B, PBS group 70 and 41%, respectively (McNemar's Test: χ 2 = 5.06; p < 0.05; df = 1) emetine group 73 and 38%, respectively being injected with emetine and the other with PBS. At the retention test both groups scored significantly higher to odor A than to odor B, as shown by their CRs; PBS group 73 and 39%, respectively (McNemar's Test: χ 2 = 8.47, p < 0.005, df = 1), emetine group 75 and 27%, respectively (McNemar's Test: χ 2 = 13.13, p < 0.001, df = 1; Figure 2B). There was no significant difference in the proportions of bees exhibiting a CR between the two experimental groups (G test odor A: G PBS vs. G Eme = 0.039, NS, df = 1; G test odor B: G PBS vs. G Eme = 1.31, NS, df = 1). Another retention test was carried out on the third day. Again both groups scored significantly higher for odor A than for odor B, PBS group 64 and 37%, respectively (McNemar's Test: χ 2 = 5.78, p < 0.05, df = 1) emetine group 71 and 31%, respectively (McNemar's Test: χ 2 = 7.68, p < 0.01, df = 1; Figure 2C). Again both groups did not differ in their proportion of bees exhibiting a CR to both odorants (G test odor A: G PBS vs. G Eme = 0.035, NS, df = 1; G test odor B: G PBS vs. G Eme = 0.032, NS, df = 1). A general non-specific effect of emetine on learning and memory can thus be ruled out.

Emetine inhibits the new inhibitory learning when applied in winter
The same protocol was applied to winter bees (Figure 3). By the last differential learning trials on day 1 the proportions of bees exhibiting the CR to the A+ and B− were 63 and 7%, respectively (McNemar's Test: χ 2 = 68.01, p = <0.001, df = 1; Figure 3A). On the following day, 30 min prior to reversal learning training, bees were randomly assigned to two groups; one being injected with emetine and the other with PBS (phosphate buffer used as saline for emetine). All bees were then trained the reversed rule (A− vs. B+). By the last differential learning trials the proportions of bees exhibiting the CR to the A− and B+ were 22 and 57% in the PBS group (McNemar's Test: χ 2 = 14.06, p = <0.001, df = 1), respectively, and 25 and 61% in the emetine group (McNemar's Test: χ 2 = 17.05, p = <0.001, df = 1), respectively ( Figure 3B). injections, a reversal protocol was applied (A− vs. B+), each stimulus was presented six times, shown here in six blocks, (solid lines for A+ vs. dashed lines for B−), at the last trials both PBS and emetine groups exhibited a reversed preference (McNemar's Test, p < 0.05 for both groups). (C) On day 3, all groups were subjected to retention tests for both odorants in the absence of a reward. The PBS group showed a significant preference for odor B (McNemar's Test, N = 40, p < 0.05), the emetine group showed no preference (McNemar's Test, NS, N = 45) and scored significantly higher for odor A than PBS group (G test, p < 0.05).
The present study tested the different effects of protein synthesis inhibitor on reversal learning in two groups of honeybees, summer and winter bees.
The main findings from these experiments are that the requirements for protein synthesis in winter bees and summer bees appear to differ with respect to the kind of memory consolidation. In general, emetine did not fully block reversal learning in either summer bees, or in winter bees. In summer bees emetine injected shortly before reversal learning impaired the manifestation of the new CS-US relation but did not affect the extinction of the original preference, when tested 24 h later. In the winter bees however, emetine yielded an inverse effect: the manifestation of the new CS-US relation remained intact, whereas the extinction of the original preference was blocked, when tested 24 h later. These results suggest a double dissociation with respect to the protein synthesis requirements in reversal learning: emetine targets different memories (excitatory memory vs. inhibitory memory), and this effect is different with respect to the line of bees used ( summer vs. winter).
It has already been suggested that seasonal variations in honeybees might result in a range of changes from behavior over neurotransmitter and pheromones levels to protein metabolism (Crailsheim, 1986;Currie and Jay, 1988;Harris and Woodring, 1992;Balderrama et al., 1996). Winter bees used in this study were kept under rather artificial conditions. They were housed in small flight cages under circadian illumination, humidity, and temperature conditions that mimicked summer. The bees were foraging for sucrose and pollen, and the colony did not form a winter cluster. The queen continued or started to lay eggs at a low rate. It was observed that in contrast to summer bees, these bees did not form transcription-dependent lLTM after multiple spaced conditioning trials (Menzel et al., 2001). Thus the hormonal status of winter bees that are exposed to simulated summer conditions must be different from real summer bees. So far it has been believed that these differences affect consolidation of lLTM but our study shows that they also affect consolidation of translation-dependent eLTM (McNemar's Test: χ 2 = 6.05, p < 0.05, df = 1; Figure 4C). Again both groups did not differ in the proportion of bees exhibiting the CR to both odorants (G test odor A: G PBS vs. G Eme = 0.05, NS, df = 1; G test odor B: G PBS vs. G Eme = 0.09, NS, df = 1). As in summer, a general non-specific effect of emetine on learning and memory can thus be ruled out.

dIscussIon
Two learning processes take place while an animal experiences a reversed CS-US contingency: a new excitatory learning and a new extinction learning of the original memory. Unlike a regular extinction, reversal learning involves the continued delivery of a reinforcer and a manifestation of a new preference is hence formed. It has been long known that new memories must be stabilized if they are to persist; this process is called consolidation and requires a cascade of intracellular events (McGaugh, 2000;Dudai, 2004).
The administration of amnestic agents during a discrete time window following learning can disturb the formation of long-term memories. In the honeybee, the systemic administration of emetine, a translation inhibitor, shortly before an absolute appetitive conditioning yields no effect on the learning process but blocks consolidation of long-term memory when tested at 24 h after acquisition (Stollhoff et al., 2005).
The effect of protein synthesis inhibitors on memory formation has been investigated so far in honeybees in either simple forward conditioning, or in regular extinction paradigms (Wüstenberg et al., 1998;Friedrich et al., 2004;Stollhoff et al., 2005). In addition, eLTM is affected by actinomycin D, a transcription-inhibitor, under spaced conditioning but not under massed conditioning (Menzel et al., 2001). Under a regular extinction paradigm, the emetine effect depends on the number of retrieval trials presented. When applied systematically 30 min before the presentation of two retrieval trials (non-rewarded CS presentations), it blocks the extinction learning at a 24-h retention test, whereas for five retrieval trials the spontaneous recovery at 24 h retention test is blocked (Stollhoff et al., 2005).  and molecular separations between excitatory and inhibitory memory traces. If the transition from short to long-term memory would lead to a separation at the network level, a specific control by hormonal factors expressing the differences between summer and winter bees may be more easily understood. In such a scenario the transfer of excitatory and inhibitory memory traces to the specific networks for long-term memory store would be differently controlled by these hormonal factors.
In this study, differences between summer and winter bees are also evident in the acquisition curves and the retention tests of the reversal learning, irrespective of the experimental groups. Summer bees display general lower levels of proboscis extension response during the acquisition of the reversal learning, compared to winter bees. This also holds true for the retention test on the third day. Such a disparity might result from different brain levels of the biogenic amine octopamine, which is known to influence response threshold to sucrose (Page and Erber, 2002), and its brain levels are correlated both with age and behavioral specialization of bees Wagener-Hulme et al., 1999). In addition, injections of octopamine to specific brain regions served as a substitute for sucrose in an associative learning (Hammer and Menzel, 1998), again pointing to its involvement in the processing of sucrose reward.

acknoWledGMents
The authors would like to thank Dorothea Eisenhardt for her fruitful discussions and helpful comments. Johannes Felsenberg is gratefully acknowledged for his valuable support during the experiments. We thank the Academic Editor and the reviewers for their fruitful comments. The first author also acknowledges support from the Deutscher Akademischer Austauschdienst (DAAD) for providing a doctoral scholarship grant. in a learning-dependent way. In the future it would be interesting to investigate the effects of transcription inhibitors on the lLTM of reversal learning.
Translation-dependent memory consolidation requires existing mRNA and a mechanism that targets the synthesized proteins to the respective synaptic sites. Our findings suggest that the excitatory and inhibitory memory traces after olfactory reversal conditioning are differently dependent on cellular mechanisms that express the seasonal hormonal changes. In Drosophila the short-term memory trace of excitatory aversive conditioning and that of extinction learning of such excitatory learning (thus a form of inhibitory learning) appear to depend on different molecular mechanisms of the same neurons, the gamma lobe Kenyon cells of the MB (Schwärzel et al., 2002). It is also known in Drosophila that the transition from short-term to long-term olfactory aversive memory is accompanied by a shift from gamma lobe related Kenyon cells of the MB to vertical lobe related cells (Pascual and Preat, 2001). It is not known, however, whether the consolidation of such excitatory and inhibitory memory traces in Drosophila involves only translation or both transcription and translation. We also do not know yet for the bee whether the effects we see may also require transcription. Recently, using a series of single-gene Drosophila mutants, Qin and Dubnau (2010) found that extinction of olfactory aversive 1 day memory depends on different molecular mechanisms than those involved in associative learning. Other supporting evidence for the dissociation between classical learning and extinction learning arise from vertebrate studies in which pharmacological and genetics disruptions were shown to affect extinction but not classical conditioning (e.g., Cain et al., 2002;Marsicano et al., 2002). Taken together, we interpret our data as supporting the concept developed for Drosophila with respect to the mechanistic