Signaling Pathways for Long-Term Memory Formation in the Cricket

Unraveling the molecular mechanisms underlying memory formation in insects and a comparison with those of mammals will contribute to a further understanding of the evolution of higher-brain functions. As it is for mammals, insect memory can be divided into at least two distinct phases: protein-independent short-term memory and protein-dependent long-term memory (LTM). We have been investigating the signaling pathway of LTM formation by behavioral-pharmacological experiments using the cricket Gryllus bimaculatus, whose olfactory learning and memory abilities are among the highest in insect species. Our studies revealed that the NO-cGMP signaling pathway, CaMKII and PKA play crucial roles in LTM formation in crickets. These LTM formation signaling pathways in crickets share a number of attributes with those of mammals, and thus we conclude that insects, with relatively simple brain structures and neural circuitry, will also be beneficial in exploratory experiments to predict the molecular mechanisms underlying memory formation in mammals.


INTRODUCTION
Brain structures and neural circuitry of insects are relatively simple, and they are therefore useful for exploratory experiments to predict the molecular mechanisms underlying memory formation in mammals. Memory in insects as well as that in vertebrates is a dynamic process organized in two main types: short-term memory (STM) and long-term memory (LTM). The former is defined as protein synthesis-independent memory, and the latter is defined as protein synthesis-dependent memory. They can be distinguished by their temporal courses and molecular mechanisms (Kandel, 2001). It is a common understanding that while STM is based on temporal changes in the synaptic strength due to covalent modifications of pre-existing proteins, LTM is supported by long-lasting alteration in the strength of synaptic function demanding for transcription and translation of genes, among a wide variety of animals including mice, sea hares Aplysia and fruit flies Drosophila (Montarolo et al., 1986;DeZazzo and Tully, 1995). The cAMP pathway is demonstrated to be critical for LTM formation in all of these animals (Bartsch et al., 1995;Yin et al., 1995;Abel et al., 1997). The cAMP pathway is a signaling cascade beginning with an increase in intracellular cAMP that activates cAMP dependent protein kinase (PKA). PKA phosphorylates the transcription factor cAMP-responsive element-binding protein (CREB) that leads to LTM formation. The nitric oxide (NO)-cGMP pathway is another system playing critical roles in the formation of LTM in sheep (Kendrick et al., 1997), great pond snails Lymnaea (Kemenes et al., 2002), and honey bees (Müller, 1996(Müller, , 2000.
In this review, we will summarize the results of our pharmacological behavioral studies on the molecular mechanisms of the formation of LTM in the cricket Gryllus bimaculatus and propose an updated model of LTM formation. The main results introduced in this review are shown in Table 1.
Crickets provide several advantages to investigate memoryrelated molecules. First, they demonstrate remarkable ability of olfactory learning and memory, including that requires cognitive functions. For example, they exhibit robust olfactory memory maintained throughout their lifetime (Matsumoto and Mizunami, 2002a), contextual learning (Matsumoto and Mizunami, 2004), high capacity of memory storage , second-order conditioning , and sensory preconditioning (Matsumoto et al., 2013a). In addition, they have remarkable visual learning ability Nakatani et al., 2009;Matsumoto et al., 2013b). Second, effective approaches that greatly facilitate analysis of the molecular basis of learning and memory are feasible. Recent progress in genetics allowed establishment of gene knockdown by RNA interference (RNAi) (Takahashi et al., 2009;Awata et al., 2016) and genome editing by the CRISPR/cas9 system (Awata et al., 2015) in crickets, adding to the well-established pharmacological methods (Unoki et al., 2005Matsumoto et al., , 2009Matsumoto et al., , 2016Mizunami et al., 2014;Sugimachi et al., 2016). Third, there has been a good accumulation of knowledge that bridges between the nervous system and behavior of crickets gained by extensive neuroethological studies in crickets (Stevenson and Schildberger, 2013;Hedwig, 2016).

Experimental Procedures
In our previous works in crickets, we have developed and extensively studied the olfactory associative conditioning, in which an odor is paired with reinforcement stimulus Mizunami, 2000, 2002b;Matsumoto et al., 2015). Similar conditioning protocols applied to two different types of visual stimuli, visual-pattern  or color-vision (Nakatani et al., 2009), paired with reinforcement stimuli have also been established. All of these procedures use classical conditioning for training and operant testing for memory tests (Matsumoto and Mizunami, 2002b;Matsumoto et al., 2003) and is performed on individual, isolated cricket. This protocol is built on the fact that crickets are able to transfer memory formed by classical conditioning in a beaker, half-compelled to receive the training, to the environment that allows freedom of choice in a larger testing chamber.
We will slightly go through the details of conditioning taking olfactory appetitive conditioning of an odor with water reward as an example. Before the experiment, crickets are each isolated in a beaker without water for 3 days, which enhances water consumption. A syringe containing water with a piece of filter paper set near the needle tip is used in conditioning training. Odor essence is applied to the filter paper to present the odor. The cricket receives the odor around its antennae for 3 s, and then receives a drop of water reward to the mouth. On water application, crickets attempt to drink it indicating that water serves as an appetitive stimulus. Retention scores of memory formed by single pairing of an odor with water reward (single-trial conditioning) is as high as that formed by repeated pairings of odor-reward association (multiple-trial conditioning) at 30 min after training, but it declines over a period of several hours and is no longer observed at 1 day after training .
Multiple-trial conditioning consist of two or more repetition of odor-reinforcement trials with inter-trial intervals (ITIs) that induces long-lasting memory beyond 1 day under adequate  Matsumoto et al. (2003. The concentrations of the administrated drugs were as follows: L-NAME (400 µM), ODQ ( conditions (e.g., number of trials = 4, ITI = 5 min). Multipletrial conditioning in our previous studies includes absolute conditioning (A+) and differential conditioning (A+, B−). Absolute conditioning can be described as repetition of appetitive conditioning trials. Differential conditioning combines appetitive and aversive conditioning trials in an alternating order. For olfactory aversive conditioning of an odor with sodium chloride punishment, similar syringe containing 20% sodium chloride solution is used. The crickets show immediate retraction from sodium chloride solution, indicating that it functions as an aversive stimulus. In previous works, we used differential conditioning that leads to robust memory , but we eventually switched to absolute conditioning for the simplicity of analysis Mizunami et al., 2014). Before and after olfactory associative conditioning, crickets were tested for their odor preferences between two odors during a 4 min testing period. Tests were performed operantly, allowing a cricket to search and choose from two odor sources, a control odor and a conditioned odor, provided in the testing chamber. Relative odor preference index for each cricket was calculated from the visiting time for each of the odor sources, as a ratio of rewarded-odor visiting time to the total visiting time. Visiting time was recorded when odor source was explored by the mouth parts of the cricket.
In our pharmacological behavioral experiments, basically, we injected 3 µl of saline containing a drug into the hemolymph of the cricket's head using a microsyringe 20 min before the onset of training (see Table 1 legend for drug doses). All of the drugs used in our experiments had been confirmed for their efficacy in physiological or biochemical researches in insects.

Memory Phases
As is the case with other animals (DeZazzo and Tully, 1995), memory induced by multiple-trial conditioning in crickets can be further distinguished into several memory phases with different retention curves. In our previous work applying differential conditioning in crickets, we have demonstrated that olfactory memory can be subdivided into at least two memory phases, STM and LTM. The peak memory score induced by sufficient multipletrial conditioning with sufficient ITIs is retained without decline for a few days (Matsumoto and Mizunami, 2002b), but when injected with a protein synthesis inhibitor (e.g., cycloheximide), memory retention score started to diminish from 5 h after training, and completely disappeared at 8 h after training (Matsumoto et al., 2003). The results indicate that there are two types of memory phases discriminated by the sensitivity to a protein synthesis inhibitor. One type is named LTM that requires protein synthesis and at least maintained for several days (Matsumoto and Mizunami, 2002b). The other type is STM which does not require novel protein synthesis (Matsumoto et al., 2003). The STM peaks immediately after the training until 4 h after training and disappears at 8 h after training. Differential conditioning may be a rather complicated learning task involving both appetitive and aversive learning. Thus, we are switching the conditioning paradigm to the simpler absolute conditioning in recent works. The memory phases in absolute conditioning should be clarified by further investigation.

cAMP Signaling Pathway
The cAMP signaling system has been demonstrated to be essential in LTM formation in mice (Abel et al., 1997), Drosophila (Yin et al., 1995;Isabel et al., 2004) and Aplysia (Bartsch et al., 1995). LTM formation in all of these species requires phosphorylation of transcription factor CREB (cAMP-responsive element-binding protein) by PKA (cAMP-dependent protein kinase) which is activated by an increase of intracellular cAMP (Bartsch et al., 1995;Yin et al., 1995;Abel et al., 1997).
We investigated whether cAMP signaling is necessary for LTM formation in the cricket . Crickets were each injected with inhibitors of key enzymes of cAMP signaling into the hemolymph prior to multiple-trial conditioning. We used either 2 ,5 -dideoxyadenosine (DDA) or SQ22536 as an adenylyl cyclase (AC) inhibitor, and either KT5720 or Rp-8-br-cAMPS as a PKA inhibitor. In a retention test 1 day after training, all of the groups of crickets failed to exhibit increased preference to the conditioned odor in comparison to that before conditioning (Figure 1). On the other hand, they showed normal scores of 30-min memory retention similar to the control group that had received injection of cricket saline. These observations indicate that these drugs fully impair LTM formation but have no effect on STM formation, motivation, sensory or motor functions. On the other hand, when these drugs were administered after conditioning, they did not impair LTM, indicating that it is during conditioning that activation of cAMP signaling is necessary for LTM formation.
The results of our experiments using 'LTM-inhibiting' drugs showed that cAMP signaling is necessary for LTM formation FIGURE 1 | Effects of drug injection prior to multiple-trial conditioning on LTM. Injected drugs are the cAMP signaling inhibitors (DDA, KT5720), the cGMP signaling inhibitor (ODQ) and the protein synthesis inhibitor (CHX). Odor preferences of crickets were tested before conditioning (white bars) and at 1-day after conditioning (shaded bars). The results of statistical comparisons are shown as means + SE. Significant differences of the PIs are indicated by asterisks (WCX test). NS p > 0.05, * * * p < 0.001. The number of animals is shown at each data point. Modified from Matsumoto et al. (2009). in the cricket, but is it also sufficient for LTM formation? To address this issue, we investigated whether forced LTM formation occurs by upregulating the cAMP signaling pathway during single-trial conditioning, which does not form LTM. Crickets were each injected with an AC activator (forskolin) or a cAMP analog (DB-cAMP, 8-br-cAMP) into the hemolymph prior to single-trial conditioning. In a retention test 1 day after the conditioning, higher preference scores for the conditioned odor in comparison to that before conditioning were observed in all of the groups, and their scores were as high as that of crickets that had been trained by multiple-trial conditioning Mizunami et al., 2014). Moreover, crickets co-injected with a protein synthesis inhibitor (cyclohexymide) and one of the activators of cAMP signaling paired with single-trial conditioning did not exhibit 1-day memory retention. These results suggest that activators of cAMP signaling induce protein-dependent LTM.
To investigate whether NO-cGMP signaling is necessary for LTM formation in the cricket, crickets were each injected with an NOS inhibitor (L-NAME) or an sGC inhibitor (ODQ) prior to multiple-trial conditioning . These groups of crickets did not show 1-day memory retention, whereas 30-min memory retention remained intact . These observations indicate that inhibition of NO-cGMP signaling fully impairs LTM formation but has no effect on STM formation. We also obtained comparable results using RNAi: injection of NOS dsRNA fully impaired 1-day retention but not 30-min retention in 7th-instar nymphal crickets (Takahashi et al., 2009).
The results of our experiments using 'LTM-inhibiting' drugs showed that NO-cGMP signaling is required to establish LTM in the cricket. Next, we investigated whether externally applied activators of NO-cGMP signaling paired with singletrial conditioning can facilitate LTM formation. Crickets each injected with an NO donor (SNAP, NOR3) or a cGMP analog (8-br-cGMP) before the single-trial conditioning showed significantly high retention level at 1 day after conditioning, which was almost identical to that in saline-injected group at 1 day after multiple-trial conditioning . Moreover, crickets co-injected with a protein synthesis inhibitor (cyclohexymide) and an activator of NO-cGMP signaling paired with single-trial conditioning did not exhibit 1-day memory retention, indicating that activators of NO-cGMP signaling pathway induce formation of protein-dependent memory, that is, LTM.

NO-cGMP Signaling Stimulates cAMP Signaling to Induce LTM
Our pharmacological behavioral experiments using 'LTMinhibiting' drugs or 'LTM-inducing' drugs suggested that NO-cGMP signaling and cAMP signaling are both necessary and sufficient for cricket LTM formation, particularly in the conditioning process.
Next, to determine which of the two pathways, NO-cGMP signaling or cAMP signaling, precedes the other in the LTM formation cascade, we varied the combinations of 'LTMinhibiting' drugs or 'LTM-inducing' drugs paired with singletrial conditioning and evaluated their effects. For example, we investigated whether cAMP mediates the forced LTM formation by combining a cGMP analog injection with singletrial conditioning . While LTM induction by combination of a cGMP analog (8-br-cGMP) and single-trial conditioning was unaffected by co-injection of an NOS inhibitor (L-NAME), it was completely impaired by co-injection of an AC inhibitor (DDA).
Induction of LTM by single-trial conditioning paired with 'LTM-inducing' drugs related to cAMP signaling (AC activator forskolin, cAMP analog DB-cAMP) was unaffected by 'LTMinhibiting' drugs related to NO-cGMP signaling (L-NAME, ODQ) . In contrast, induction of LTM by single-trial conditioning paired with 'LTM-inducing' drugs related to NO-cGMP signaling (SNAP, 8-br-cGMP) was fully blocked by 'LTM-inhibiting' drugs related to cAMP signaling (DDA, KT5720). The results suggest that in the LTM induction process, the AC-cAMP pathway works downstream of the NO-cGMP pathway, and not vice versa.

Biological Pathways Intervening Between NO-cGMP Signaling and cAMP Signaling
Next, we investigated biological pathways intervening between cGMP and AC activation. PKG, a cGMP-dependent protein kinase, is one of the possible targets of cGMP. Working in parallel with PKA, PKG enhances the phosphorylation of CREB in mice (Lu and Hawkins, 2002). Working in parallel with the cAMP pathway, NO-cGMP-PKG signaling pathway governs the induction of long-term hyper-excitability on receiving a noxious stimulation in nociceptive sensory neurons of Aplysia (Lewin and Walters, 1999). We investigated the roles of PKG in olfactory memory in the cricket. LTM formation was not affected by external application of PKG inhibitor KT5823, whether it was induced by multiple-trial conditioning or by single-trial conditioning combined with 8-br-cGMP.
Thus, we switched our target to cyclic nucleotide-gated cation channel (CNG channel). CNG channels are Ca 2+ -permeable channels activated by cAMP and/or cGMP. A CNG channel inhibitor [L-cis diltiazem (L-DIL), 3,4,-dechlorobenzamil (DCB)] fully impaired LTM, but not STM, formed by multiple-trial conditioning. Moreover, the CNG channel inhibitor L-DIL fully impaired LTM induced by combination of a cGMP analog (8-br-cGMP) and single-trial conditioning, while L-DIL did not affect LTM induced by 'LTM-inducing' drugs related to cAMP signaling (forskolin, DB-cAMP) paired with single-trial conditioning. From the results, it can be suggested that CNG channel plays its role downstream of cGMP and upstream of AC activation in the LTM formation process.
In Drosophila, it has been shown that AC is activated by either G-protein or calcium-calmodulin (Ca 2+ /CaM) (Livingstone et al., 1984). CaM is a principal Ca 2+ -binding messenger protein in the central nervous system. We examined whether CaM mediates the signaling pathway from CNG channel to AC activation. A CaM inhibitor (W-7) fully impaired LTM formed by multiple-trial conditioning. Moreover, the CaM inhibitor W-7 fully impaired LTM induced by a cGMP analog (8-br-cGMP) paired with single-trial conditioning, while it had no effect on LTM induced by 'LTM-inducing' drugs related to cAMP signaling (forskolin, DB-cAMP) paired with single-trial conditioning. Next, we investigated whether rise in calcium concentration mediates signaling from CNG channel to CaM in LTM formation process. Crickets injected with a calcium (Ca 2+ ) ionophore (A23187) paired with single-trial conditioning exhibited LTM. The LTM induced by A23187 was unaffected by co-injection of an sGC inhibitor (ODQ) or a CNG channel inhibitor (L-DIL) but was completely impaired by co-injection of a CaM inhibitor (W-7) or an AC inhibitor (DDA). The results indicate that Ca 2+ /CaM mediates signaling from CNG channel to AC, filling the gap of LTM formation cascade.
Ca 2+ /CaM-dependent serine/threonine kinase II (CaMKII), which is one of the Ca 2+ /CaM effector enzymes, supports various learning and memory systems as a key signaling molecule in vertebrates (Coultrap and Bayer, 2012). This is especially because CaMKII have the ability to modulate its own kinase activity by autophosphorylation. In the fruit fly Drosophila, synthesis of CaMKII in mushroom bodies has been reported to be necessary for olfactory LTM formation (Ashraf et al., 2006;Akalal et al., 2010;Malik et al., 2013). The mushroom body is known as a multisensory association center as well as a secondary olfactory center essential for olfactory learning and memory (Heisenberg, 2003;Davis, 2011). In cockroaches, an increase of phosphorylated CaMKII is observed in pre-and post-synaptic structures in the mushroom body calyx after learning to associate an olfactory stimulus with a visual stimulus (Lent et al., 2007). In our recent report, we demonstrated that CaMKII inhibitors impair the olfactory LTM formation in honey bees . Are these roles of CaMKII in olfactory memory processing introduced above also true for crickets? In crickets, a CaMKII inhibitor (KN-62 or KN-93) fully impaired induction of LTM, but not STM, paired with multiple-trial conditioning. Moreover, KN-62 fully impaired induction of LTM by a Ca 2+ ionophore (A23187) paired with single-trial conditioning, but not that by a cAMP analog, indicating that CaMKII works upstream of AC for LTM formation cascade. Because KN-62 did not impair LTM induced by a cAMP analog, it was rather surprising to find out that KN-62 or KN-93 inhibits LTM induction with folskolin, an AC activator. The best working theory to explain these observations is that there is an interaction between CaMKII and FIGURE 2 | A model of biochemical pathways for LTM formation in associative olfactory conditioning. The model is proposed on the basis of the present findings in crickets and some documented findings in insects (see text). Single-trial conditioning induces only short-term synaptic plasticity that underlies protein synthesis-independent short-term memory (STM). Multiple-trial conditioning activates NO-cGMP signaling, and this activates cyclic nucleotide-gated (CNG) channel, Ca 2+ /CaM, CaMKII and then adenylyl cyclase (AC)-cAMP-PKA signaling. This in turn activates cAMP-responsive element-binding protein (CREB), which results in transcription and translation of genes that are necessary for achieving long-term plasticity of synaptic connection upon other neurons that underlies LTM. NOS, NO synthase; sGC, soluble guanylyl cyclase; Arg, arginine; Gs, Gq, receptor (R)-coupled G-protein; OA, octopamine; ACh, acetylcholine; nAChR, nicotinic acetylcholine receptor; mAChR, muscarinic acetylcholine receptor; PLC, phospholipase C; IP3, inositol 1,4,5-triphosphate; RyR, ryanodine receptor; ER, endoplasmic reticulum. AC, conceivably through formation of macromolecular complex in a similar manner demonstrated in mammalian CaMKII (Coultrap and Bayer, 2012;Lisman et al., 2012), and when KN-62 or KN-93 binds to CaMKII, AC activation by forskolin may be impaired.

A Model of the Signaling Pathways for LTM Formation
A putative model of the signaling pathways for olfactory LTM formation in crickets is shown in Figure 1, updated from our previous model . The new model illustrates the simplest of all the signaling pathways that account for the results summarized in Table 1, which describes the outcomes of co-injection experiments. The following documented findings in several insects are incorporated in this model: (1) in vitro alpha-bungarotoxin (BGT)-sensitive nicotinic acetylcholine receptors (nAChRs) are able to trigger NO synthesis in Kenyon cells of insects (Bicker et al., 1996;Zayas et al., 2002), (2) NO production by NO synthase is stimulated by Ca 2+ /CaM in Drosophila (Regulski and Tully, 1995), (3) in vitro muscarinic acetylcholine receptors (mAChR) activate CaM by calcium release from the endoplasmic reticulum (ER) via PLC/IP 3 signaling (Hasebe and Yoshino, 2016), (4) calcium release via ryanodine receptors (RyRs) on the ER induces LTM in crickets (Sugimachi et al., 2016), (5) AC is activated by either the G-protein coupled receptor or Ca 2+ /CaM in Drosophila (Livingstone et al., 1984) and (6) PKA activates CREB which leads to LTM formation in Drosophila (Yin et al., 1995).
Anatomical studies of NO-generating neurons and NOreceptive neurons have been performed in some insects. Putative NO synthase have been revealed histochemically in some neurons of the mushroom body and the antennal lobe, a primary olfactory center, in honey bees (Bicker, 2001), locusts (Müller and Bicker, 1994) and cockroaches (Ott and Elphick, 2002), while immunoreactivity to NO-induced cGMP has been observed in other neurons of the same centers (Bicker et al., 1996;Bicker, 2001). To determine the brain region of NO-generating neurons and NO-receptive neurons in crickets, we investigated the expression patterns of the NOS gene and SGCβ gene by whole-mount in situ hybridization (Takahashi et al., 2009). The SGCβ gene is coding the β subunit of sGC. We observed a high expression level of NOS mRNA in outer Keyon cells of the mushroom body, but not in inner Kenyon cells, in addition to several somata around the antennal lobe and at the base of the visual center optic lobe. On the other hand, we observed a significant level of expression of sGC mRNA in inner Keyon cells. Therefore, NO production is presumed to take place in outer Kenyon cells, and NO permeates into nearby inner Kenyon cells.
One of our next steps is to clarify whether several biological molecules depicted in Figure 2 indeed contribute to LTM formation in crickets using both pharmacological study and RNAi. The target molecules include nAChR, mAChR, PLC, IP 3 and CREB, which have not been shown to be involved in cricket LTM formation. There are several LTM-related signaling pathways other than those mentioned in this review in other animals, such as N-methyl-D-aspartic acid (NMDA) receptor signaling (Giese et al., 2015;Wang and Peng, 2016), insulin receptor signaling (Zhao and Alkon, 2001;Zhao et al., 2004;Dou et al., 2005;Chambers et al., 2015;Kojoma et al., 2015), mitogenactivated protein kinase (MAPK) signaling (Alfieri et al., 2011;Philips et al., 2013;Shobe et al., 2016), and mechanistic target of rapamycin (mTOR) signaling (Bekinschtein et al., 2007;Blundell et al., 2008;Huang et al., 2013;Buffington et al., 2014;Hylin et al., 2018). Whether these signaling pathways are related to LTM formation in crickets is another issue.
We have established conditioning procedures for different sensory modalities for crickets: olfactory conditioning, visualpattern conditioning and color-vision conditioning. Each conditioning can be classified into two categories: appetitive conditioning and aversive conditioning. Thus, we can examine whether the finding of biochemical cascades in olfactory appetitive learning is applicable to other learning paradigms. For example, in appetitive visual LTM formation, we have shown that NO-cGMP signaling works upstream of cAMP signaling (Matsumoto et al., 2013b). We have also shown that at least NO-cGMP signaling participates in aversive visual LTM formation (Matsumoto et al., 2013b). Thus, we conclude that signaling cascades for LTM formation is shared between olfactory and visual learning.

CONCLUSION
In this review, we overviewed the biochemical cascades for LTM formation based on the results of co-injection experiments with different combinations of LTM-inducing drugs for 'gain of function' and LTM-inhibiting drugs for 'loss of function.' From our pharmacological behavioral studies, we proposed an updated model in which multiple-trial conditioning triggers the NO-cGMP signaling that activates the downstream cAMP signaling through the CNG channel, Ca 2+ /CaM and CaMKII, leading to the formation of protein synthesis-dependent LTM. A number of molecular actors involved in LTM formation in crickets, such as NOS, NO, cGMP, cAMP, PKA and CaMKII, are known to be involved in mammalian LTM formation. Thus, we conclude that insects, with relatively simple brain structures and neural circuitry, will also be beneficial in exploratory experiments to predict the molecular mechanisms underlying cognitive functions and memory formation in mammals.

AUTHOR CONTRIBUTIONS
YM, CM, and MM wrote the manuscript and approved the final version.

FUNDING
This study was supported by Grants-in-Aid for Scientific Research from the Ministry of Education, Science, Culture, Sports and Technology of Japan to YM (Grant No. 16K07434) and to MM (Grant Nos. 16H04814 and 16K18586).