Modeling signal transduction in classical conditioning with network motifs

Introduction

Motor learning is fundamental to neural circuits controlling voluntary movements in order to appropriately plan and adapt them to environmental situations. In recent years a great deal of effort has gone into elucidating the cellular and molecular mechanisms that underlie learning processes in the brain. To understand the signal transduction events involved in these processes a relatively simple form of associative learning called classical conditioning has been extensively studied. In this form of learning, a predictive relationship is acquired between a neutral stimulus that ordinarily produces no behavior, like the ringing of a bell, and a noxious stimulus such as an airpuff to the eye that produces a defensive motor behavior, in this case an eyeblink. After only a few pairings of the bell and airpuff animals and humans learn to blink in response to the bell alone. The generation of motor responses to novel stimuli is necessary for survival and therefore this form of learning is rapid and robust. Voluntary movements are controlled by motor signals generated by widespread brain regions acting in parallel, including the motor cortex, basal ganglia, and cerebellum, and the recognition of such large-scale network activity in brain function has lead to the development of the distributed processing module (DPM) model of behavior and cognition (Houk, 2005). How the DPM model applies to classical conditioning through the activity of distributed brain networks that coordinate training input signals with adaptive motor output is outlined by Houk (2010). Using this as a foundation, we are constructing a multilevel model of classical conditioning starting here with the molecular events that underlie an in vitro form of eyeblink classical conditioning initially described several years ago (Keifer et al., 1995) and recently reviewed in Keifer and Zheng (2010).

Biological networks are constructed of repeated simplified patterns called network motifs (Alon, 2007). These building block networks have their own limited information processing capacity and combine to build larger more complex systems. Network motifs can be found in a variety of organisms including bacteria, yeast, plants, and animals, as well as intracellular transcription networks for gene expression and signal transduction processes in neuronal circuits of interest here. That biological systems follow general rules is not new. However, as applied to studies of learning, models for signal transduction processes are typically designed as sequential and parallel pathways that can be daunting in their complexity. The application of network motifs to such models reveals simplifying recurring patterns, or modules, in the design structure of these signal transduction pathways. Using these simplified principals, the structure of potential unknown signaling pathways may also be predicted. As a first step, in this report we apply common biological network motifs mainly using feed-forward loops (FFLs) to model signal transduction during in vitro eyeblink classical conditioning.

Signal Transduction Events in in vitro Classical Conditioning

A neural analog of eyeblink classical conditioning can be studied using an isolated brainstem preparation from turtles (Keifer and Zheng, 2010). This preparation, illustrated in Figure 1A, is unique because turtle brain tissue is highly resistant to hypoxia such that the entire brain and brainstem can be maintained for hours or days in a dish allowing for in vitro studies of learning processes. Considerable experimental advantage is achieved since neuronal circuits under study are accessible to recording electrodes and application of pharmacological compounds or small interfering RNAs (siRNAs) using incubation procedures. Instead of using a tone or airpuff as in behaving animals, paired stimulation of the auditory nerve (the “tone” conditioned stimulus, CS) with the trigeminal nerve (the “airpuff” unconditioned stimulus, US) generates neural discharge in the abducens nerve, which controls blinking in this species, that is characteristic of conditioned eyeblink responses (Figure 1B, arrow). Once CS–US pairing is initiated, conditioned responses (CRs) are acquired rapidly in about 1 h, or during the second pairing session, compared to controls receiving unpaired stimuli that show no CRs (Figure 1C). Each pairing session during the training consists of 50 paired CS–US stimuli (lasting 25 min in duration) followed by a 30-min rest period in which there is no stimulation. Our studies indicate that trafficking of AMPA type glutamate receptors (AMPARs) containing GluR1 and GluR4 subunits support conditioning in this preparation. Based on data describing the action of multiple signal transduction elements, a two-stage model of synaptic AMPAR delivery in the abducens motor neurons during acquisition of eyeblink conditioning was developed and is illustrated in Figure 1D (Zheng and Keifer, 2009; Keifer and Zheng, 2010). First, GluR1-containing AMPAR synaptic incorporation occurs early in conditioning to activate silent synapses containing only NMDA receptors (NMDARs) that convey the auditory CS (Figure 1D, Early). This step is initiated within 15 min of training with the phosphorylation of protein kinase A (PKA) and the calcium/calmodulin-dependent protein kinases (CaMK) II and IV (Zheng and Keifer, 2009). This leads to activation of the transcription factor cAMP response element-binding (CREB) protein which ultimately results in production of brain-derived neurotrophic factor (BDNF) that is required for AMPAR delivery and CR acquisition (Li and Keifer, 2008, 2009). BDNF is hypothesized to activate extracellular signal-regulated kinase (ERK) by signaling through the BDNF receptor tropomyosin-related kinase B (TrkB) that induces synaptic delivery of GluR1-containing AMPARs by translocation of existing receptors into the synapse. The incorporation of GluR1 into auditory nerve synapses early in conditioning is an essential step that serves to activate NMDARs, thereby allowing post-synaptic intracellular calcium (Ca²⁺) entry that triggers the second stage of AMPAR trafficking for conditioning involving GluR4 subunits (Mokin et al., 2007). Therefore, the first stage of AMPAR delivery is NMDAR-independent while the second stage requires NMDAR activation. In the second step of AMPAR trafficking, the delivery of GluR1 is followed by replacement of those subunits with synaptic incorporation of newly synthesized GluR4-containing AMPARs that underlies the acquisition of CRs (Figure 1D, Late; Mokin et al., 2007; Zheng and Keifer, 2009; Keifer and Zheng, 2010). Pharmacological data indicate that this step requires the coordinated actions of Ca²⁺-dependent and independent isoforms of PKC, as well as CaMKII and ERK (Zheng and Keifer, 2008, 2009).

FIGURE 1

Figure 1. Summary of the in vitro classical conditioning model system. (A) Drawing of the turtle brainstem viewed from above (upper) and the side (lower). The preparation consists of the pons alone produced by transections indicated by the dotted lines. The trigeminal (Trigem) and auditory (Aud) cranial nerves used for delivery of the US and CS, respectively, are indicated as is the abducens (Abd) nerve from which neural discharge corresponding to blink responses is recorded. Cb, cerebellum. (B) Extracellular recording of a burst discharge recorded from the abducens nerve that is representative of a CR (arrow) occurring during the CS and an unconditioned response (UR) initiated by the US. The CS–US paired stimuli are indicated. (C) Typical CR acquisition curves generated from paired (conditioned) or unpaired stimuli. CRs are usually recorded in the second pairing session or after about an hour; unpaired stimuli produce no CRs. (D) Summary of the signal transduction pathways for two stages of AMPAR trafficking during in vitro conditioning. Synaptic delivery of GluR1-containing AMPARs occurs shortly after CS–US pairing (Early) followed by delivery of GluR4-containing AMPARs (Late) that results in the acquisition of CRs. Details are provided in the text.

The multistage physiologically based model shown in Figure 1D necessarily illustrates signal transduction events during conditioning as a series of sequential steps. However, it fails to convey the multiple interactions among the elements of these molecular pathways and the cooperativity in their actions. Here, as a way of improving our understanding of these events, we present a new motif-based model that is founded on generalized design principles derived from a broad selection of biological systems (Alon, 2007). Our initial goals are: (1) to construct a biologically realistic model of signal transduction for classical conditioning based on network motifs, and (2) to predict the potential structure of unknown signaling pathways to guide hypothesis construction for experimental examination. The motif-based model describes the two stages of synaptic AMPAR trafficking during conditioning in Parts A and B below. Potential interactions of these networks form a more generalized network motif pattern that is presented in Part C. The model helps to clarify potential interactions between signal transduction elements during acquisition of conditioning, timing of these synaptic processes, and guides hypotheses for as yet undiscovered signaling events that may occur during coincidence detection of the CS and US. The application of network motif design principals to signal transduction mechanisms shown here should have broad applications to studies of learning.

Analysis

Repeated simplified patterns, or network motifs, form the building blocks of information processing systems. Some common ones are shown in Figure 2 and are discussed extensively by Alon (2007). The FFL is a network pattern having three nodes, X, Y, and Z, in which X has a direct path to Z and an indirect path to Z through Y (Figure 2). Coherent FFLs are those in which the direct path has the same overall sign as the indirect path, while the indirect path in the incoherent FFL has the opposite sign as the direct one. The complexity of the FFL in particular can be enhanced by duplication of one of its nodes as illustrated in Figure 3. The basic FFL is shown in Figure 3 (left) as is the network structure resulting from a single duplication of X which forms a multi-input FFL (right), in this case a two-input FFL. Therefore, complexity of the information processing capacity of the network can be increased by simple replication. The FFL is a prominent network motif in transcriptional (Shen-Orr et al., 2002) and neuronal networks (White, 1985; Milo et al., 2002). It’s structure and function are reviewed by Mangan and Alon (2003). FFLs have also been proposed to support the function of large neuronal arrays in the DPM model of brain function (Houk et al., 1993; Houk, 2005, 2010), for example, as anticipatory commands for sensorimotor control.

FIGURE 2

Figure 2. Some common network motifs found in biological networks, including signal transduction and neuronal circuits. Adapted from Alon (2007).

FIGURE 3

Figure 3. The basic feed-forward loop (FFL) has a direct pathway from X to Z and an indirect pathway to Z through Y (left). Complexity of the FFL is increased by a simple replication of one of its nodes, in this case X, which forms a multi-input FFL.

Part A. Multi-Input FFL with Diamond Motif for GLUR1 Delivery

The FFL network is prominent in signal transduction cascades engaged by classical conditioning. Figure 4 illustrates the network motif design for the early signaling events leading to synaptic GluR1 AMPAR incorporation during in vitro classical conditioning. In the initial stages of conditioning, PKA and CaMKs II and IV are phosphorylated within 15 min of CS–US pairing (Zheng and Keifer, 2009). The phosphorylation of PKA and CaMKII are not sensitive to application of the NMDAR antagonist AP-5 and therefore these early events in conditioning are NMDAR-independent. PKA and CaMKIV activate CREB at Ser133 and there is feed-forward phosphorylation of GluR1 at Ser845, the PKA site, and Ser831, the CaMKII/PKC site, which are required for synaptic delivery of AMPARs (Derkach et al., 2007; Zheng and Keifer, 2009) as illustrated in Figure 4. Activation of CREB results in transcription of proBDNF (the precursor of BDNF) and possibly the recently characterized secreted form of the turtle tolloid-like metalloprotease gene tTLLs (dashed line indicates hypothetical pathway in Figure 4). The reptilian ortholog of the tolloid metalloprotease gene family, tTLL, is most similar to mammalian tolloid-like 2 (mTLL-2) in sequence structure and bone morphogenetic protein-1 (BMP-1) in its domain architecture (Sabirzhanov et al., 2007). During in vitro classical conditioning, the precursor proBDNF is converted to the mature and active form BDNF, indicated in Figure 4, through extracellular proteolytic cleavage by tTLLs whose transcription is induced early in conditioning (see also Figure 1; Keifer et al., 2009). This process is similar to the one described during long-term potentiation (LTP) in mammals in which the conversion of proBDNF occurs through the action of extracellular proteases (Pang et al., 2004; Yang et al., 2009). BDNF is known to have a critical role in synaptic plasticity mechanisms (Bramham and Messaoudi, 2005). In conditioning, it is necessary for synaptic GluR1 delivery (Li and Keifer, 2008, 2009) and results in activation of ERK. BDNF/TrkB activation of intracellular ERK has been proposed for other forms of synaptic plasticity described in the mouse hippocampus and in Aplysia (Patterson et al., 2001; Sharma et al., 2006). Phosphorylation of ERK is also required for synaptic incorporation of GluR1-containing AMPARs during conditioning (Zheng and Keifer, 2008; Li and Keifer, 2009). While GluR1 has direct sites for phosphorylation by PKA and CaMKII (as well as other kinases), MEK–ERK signaling pathways control synaptic AMPAR delivery through the Ras GTPase (Zhu et al., 2002).

FIGURE 4

Figure 4. A multi-input FFL with diamond motif describes the initial stage of signal transduction for synaptic GluR1-containing AMPAR delivery during in vitro classical conditioning. This step results in synaptic incorporation of GluR1 subunits to unsilence auditory nerve synapses that convey the CS by activation of NMDARs. See text for details.

The design architecture of signal transduction events leading to synaptic GluR1 delivery during conditioning can be described as a multi-input, in this case two-input, FFL as shown in Figure 4 (see also Figure 3, right). Two inputs from PKA and the CaMKs (X) directly, and indirectly through a CREB–BDNF–ERK pathway (Y), regulate GluR1 trafficking (Z). Embedded in the Y pathway is a diamond network motif for CREB–BDNF regulation. The multi-input FFL and diamond are strong network motifs found in neuronal networks and signal transduction processes of some relatively simple systems (Alon, 2007). The diamond motif is interesting because it can be combined to form multiple layers such as occurs in signaling cascades for protein phosphorylation. However, the diamond is not thought to be highly represented in transcription networks as we propose here (Milo et al., 2002; Alon, 2007). The bifan is more common (Figure 2). Our future studies may reveal that a bifan motif is instead embedded in this pathway or that this is a new motif for transcription networks found in brain. What is captured easily by this model as opposed to the one in Figure 1D is the coincident transcription of proBDNF and tTLLs leading to conversion of BDNF, and the cooperativity required among PKA, CaMKII, and ERK in the phosphorylation of GluR1 subunits for synaptic delivery. The multi-input FFL in particular has direct applications for classical conditioning because it is able to perform coincidence detection and is discussed below. The signal transduction process shown in Figure 4 begins with activation of PKA and the CaMKs. However, as indicted by the arrows at the beginning of the diagram, these are not the first events leading to in vitro classical conditioning. There is an initial signaling process for coincidence detection within minutes of the occurrence of the CS and US that precedes Part A and has yet to be described in detail experimentally. Data suggest this initial process involves phosphoinositide 3 (PI3)-kinase mediated activation of 3-phosphoinositide-dependent protein-kinase-1 (PDK-1) and is described further in Part B below.

Part B. Multi-Input FFL for GluR4 Delivery

The motif-based model for the known signal transduction events leading to synaptic delivery of GluR4-containing AMPARs in the second stage of conditioning is illustrated in Figure 5. Delivery of GluR4 AMPAR subunits is an NMDAR-dependent step. Synaptic incorporation of GluR1 AMPAR subunits into NMDAR-containing silent auditory nerve synapses in Part A (GluR1/NMDAR in Figure 5) unsilences those synapses allowing for post-synaptic Ca²⁺ entry into abducens motor neurons to activate Ca²⁺-sensitive conventional PKC (PKCc, namely α and β). Activation of PKC is required for GluR4-containing AMPAR trafficking and acquisition of CRs but pharmacological inhibition of PKC does not affect synaptic GluR1 delivery (Zheng and Keifer, 2008). In addition to the conventional PKCs, data suggest that PI3-kinase and its downstream target PDK-1 are also essential for conditioning. The PI3-kinase inhibitor Wortmannin blocks conditioning and inhibits phosphorylation of the atypical PKC isoform PKCζ as well as ERK (Zheng and Keifer, 2008). Additionally, recent Western blot data show significantly increased levels of phospho-PDK-1 within minutes after the onset of CS–US pairing, but not after unpaired stimuli, that is maintained during conditioning (Keifer, unpublished). PDK-1 is an interesting kinase that is required to bind to the activation loop of the ABC kinases, such as PKC, to prime further phosphorylation of substrates (Newton, 2003). PDK-1 may also directly activate ERK through phosphorylation of MEK (Sato et al., 2004). These pathways are illustrated in Figure 5. Finally, the GluR4 AMPAR subunit has three known phosphorylation sites. Ser842 is targeted by PKC (conventional isoforms), PKA, and CaMKII, Thr830 by PKC (conventional isoforms; Carvalho et al., 1999), and Thr855 by Jun N-terminal kinases (JNK; Thomas et al., 2008). In conditioning, synaptic delivery of GluR4-containing AMPARs is known to require PKCc and ERK. It is uncertain whether PKCζ has any direct interactions with AMPARs (illustrated as a dashed line in Figure 5) although there are likely indirect actions through regulatory proteins controlling trafficking (Migues et al., 2010).

FIGURE 5

Figure 5. A multi-input FFL describes synaptic delivery of GluR4-containing AMPARs in the second stage of conditioning. AMPARs containing GluR4 replace GluR1 subunits and underlie the acquisition of CRs. See text for details.

The signal transduction events leading to synaptic incorporation of GluR4-containing AMPARs are not yet as well described as for GluR1 during conditioning in this system. Nevertheless, it is instructive to model the known interactions in terms of the multi-input FFL-like architecture shown in Figure 5. This model helps to clarify two previous experimental findings. First, application of either chelerythrine (an inhibitor of PKCc, i.e., PKCαβ but not ζ) or the PKCζ pseudosubstrate peptide inhibitor ZIP (an inhibitor of PKCζ and downstream ERK but not PKCαβ) suppresses GluR4 delivery and conditioning (Zheng and Keifer, 2008). Therefore, the cooperative action of both conventional and atypical classes of PKC isoforms directly, or indirectly through ERK, is required for GluR4 trafficking during conditioning as depicted in Figure 5. Second, the model illustrates why PKCζ may remain phosphorylated in conditions in which GluR4 synaptic delivery is inhibited if the cooperative action of PKCc is also required. These ideas can be tested experimentally.

Part C. Network Interactions between the FFLs

Viewing these signal transduction pathways from a more global perspective, the FFL of Part A is hypothesized to directly interact with the FFL of Part B in a feed-forward manner in two distinct ways (Figure 6; blue arrows). First, the synaptic insertion of AMPARs containing GluR1 subunits into NMDAR-containing silent auditory nerve synapses activates them and results in post-synaptic intracellular Ca²⁺ entry required to initiate Part B (Figure 6; blue arrow in the middle). Second, PKA and CaMKII activated in Part A are likely to regulate GluR4 trafficking perhaps by direct phosphorylation of that subunit (Figure 6; outer two blue arrows). While this has not yet been directly demonstrated experimentally, this assertion is supported by the observation that once activated after onset of the conditioning stimuli both PKA and CaMKII are maintained in a phosphorylated state for hours allowing them to affect later AMPAR trafficking events (Zheng and Keifer, 2009). Moreover, blockade of PKA and CaMKII suppresses synaptic delivery of GluR4 and inhibits in vitro classical conditioning (Zheng and Keifer, 2009). An interesting possibility that can be appreciated in the model is that the PKA/CaMKII phosphorylation site of GluR4 (Ser842) might cooperatively interact with the PKCc site (Thr830) that, together with ERK, induces GluR4 synaptic delivery. The cooperative action of multiple AMPAR subunit phosphorylation sites suggested here for regulation of both GluR1- and GluR4-containing AMPAR trafficking during conditioning is similar to mechanisms proposed for synaptic GluR1 delivery during LTP (Derkach et al., 2007). Specifically, GluR1 subunits are thought to be mobilized by PKA-mediated phosphorylation at Ser845 followed by NMDAR/Ca²⁺-mediated phosphorylations involving Ser818/Ser831/Ser845 to induce GluR1 trafficking and potentiation of synaptic function. Complex molecular interactions such as these are difficult to capture using conventional physiological models such as the one in Figure 1 but are readily conveyed by motif-based architectures. Finally, in addition to these feed-forward actions of Part A on Part B, evidence suggests that PDK-1 is activated in the very earliest stages of conditioning (Keifer, unpublished) and it therefore is placed at the beginning of the signal transduction pathway shown in Figure 6. PDK-1 regulates signaling of the ABC kinases (Newton, 2003) and is required to prime not only PKA early in conditioning but also the PKCs in the later stages of conditioning. Therefore, there is an additional feed-forward pathway in which PDK-1 phosphorylation is initiated early and maintained during conditioning allowing it to play a role in regulating both steps of GluR1 and GluR4 trafficking (Figure 6; green arrows).

FIGURE 6

Figure 6. Network interactions between the two stages of AMPAR trafficking in conditioning. The later stage of synaptic incorporation of GluR4 AMPAR subunits (Part B) depends on the initial signal transduction events leading to synaptic delivery of GluR1 (Part A) and activation of PDK-1. Approximate activation time after the onset of CS–US pairing is indicated. See text for details.

The overall structure of the signal transduction pathways for conditioning shown in Figure 6 describes a coherent type-1 FFL (C1-FFL) with an AND gate as discussed by Alon (2007) and is depicted schematically in Figure 7A (see also, Figure 2). Importantly, the production of Z requires the activation of both X AND Y. Imagine the initial conditioning-induced signal transduction step involving PDK-1 (yet to be described experimentally in detail) as “X,” the GluR1 synaptic delivery step shown in Part A as “Y,” and the GluR4 delivery step in Part B as “Z” (Figure 7B). PDK-1 and associated signal transduction elements activate Part A (GluR1 synaptic delivery) and both PDK-1 and Part A jointly activate Part B (GluR4 delivery; Figure 7B). In molecular terms, the signal transduction pathways involving PDK-1 result in activation of PKA to initiate the pathway shown in Part A resulting in insertion of GluR1-containing AMPARs into auditory nerve synapses conveying the CS. Activation of these silent synapses allows for NMDAR-dependent Ca²⁺ entry to induce activation of PKC required for synaptic GluR4 delivery in Part B. At about the same time, PDK-1 also participates in the downstream priming of PKC resulting in synaptic insertion of GluR4 subunits and CR acquisition. Initiation of the signal transduction cascade in Part B could not occur without the combined inputs of GluR1-mediated NMDAR activation and the priming of PKC by PDK-1. Therefore, together they serve an AND function for synaptic incorporation of GluR4 during conditioning.

FIGURE 7

Figure 7. (A) General network structure of a C1-FFL with AND logic. (B) This network motif as applied to signal transduction during in vitro classical conditioning detailed in Figure 3 and described in the text.

General Discussion and Hypotheses

The C1-FFL Motif in Classical Conditioning

Feed-forward loops have been applied to many biological systems to simplify and explain their function. The most common FFL in biological networks has been proposed to be the C1-FFL (Alon, 2007). This structure broadly fits our classical conditioning signal transduction data and, interestingly, a simple duplication of this pattern forms a two-input FFL motif (Figure 3) that can perform coincidence detection (discussed below). Alternative models are not as attractive, at least for signal transduction events in classical conditioning. For example, the incoherent FFL (I1-FFL; Figure 2), in which the indirect path is antagonistic to the direct one, acts like a pulse generator that speeds up response time of the system (a sign-sensitive accelerator; Alon, 2007). This motif is apparently represented in transcription networks but does not have clear applications to data from classical conditioning (except perhaps to short latency CRs that are generated after cerebellar lesions). As discussed by Alon (2007), the C1-FFL with AND logic (Figure 7) acts generally as a sign-sensitive delay element in response to ON or OFF inputs. After ON activation by a signal from X, for example, Y needs time to accumulate to a threshold level and this results in a delay in the production of Z. If the input pulse is too short to result in a threshold accumulation of Y then there is no expression of Z. In this way, the network filters out ON inputs that are too brief. Therefore, the C1-FFL with AND logic is a persistence detector that is sensitive to the duration of input pulses and responds with a delay (Alon, 2007). Our model predicts that there are threshold effects in signal transduction that may explain the non-linearity of the conditioning process. That is, inputs from X AND Y are required for production of Z (for example, GluR4 delivery for CR acquisition) but Y takes time to reach threshold and the generation of CRs will be delayed. How are delayed responses produced at the molecular level? For classical conditioning, PDK-1 is activated early and is additionally required for the later step of GluR4 synaptic delivery. For PDK-1 to function later on it must “wait” for the accumulation of Y (GluR1 delivery) because production of Z (GluR4) is controlled by the combined actions of GluR1 synaptic insertion and sustained activity of PDK-1. The “waiting” period for PDK-1 during GluR1 accumulation could be achieved by its known autophosphorylation activity that sustains its function (Casamayor et al., 1999). Such a mechanism has been invoked for prolonged activation of synaptic CaMKII that also has a delayed function in learning processes (Mullasseril et al., 2007). The resultant delay in expression of Z (GluR4 synaptic delivery) is observed as gradual acquisition of CRs during training that is characteristic of classical conditioning.

Experimental evidence for the function of network motifs in biological systems is derived mainly from organisms such as bacteria. Mangan et al. (2003) presented evidence for the presence of a C1-FFL with AND logic in the arabinose system of E. coli by showing that transcription events to ON steps behaved like a sign-sensitive delay element. Such an FFL structure can be detected by measuring responses to ON and OFF inputs that are asymmetric in time. In that study, the onset of transcription of the araBAD promoter was delayed by about 20 min in response to a 10-mM cAMP ON step compared to the linear activation of lacZYA which does not have FFL connectivity. Neither promoter showed delayed kinetics to a cAMP OFF step. The FFL motif has also been found in the neuronal network of the nematode worm C. elegans whose synaptic connections have been mapped in detail (White, 1985; Milo et al., 2002). How would one look for evidence of the network motif structure proposed here for signal transduction pathways controlling learning? For in vitro classical conditioning, responsiveness of this system to ON and OFF inputs could be assessed by examination of the synaptic incorporation of GluR4-containing AMPARs or the phosphorylation levels of specific signal transduction elements that lead to GluR4 trafficking which might have greater sensitivity for measurement. For example, two candidates might be PKCζ and its downstream target ERK. Both of these signaling elements show delayed activation of at least 40 min after conditioning onset (Zheng and Keifer, 2008). One would need to work out the appropriate ON input and OFF input to this system to test experimentally for asymmetry in its responsiveness (that is, delayed responses to ON but not to OFF inputs) as evidence for a C1-FFL network motif. One possibility is that application of the PKA activator Sp-cAMPs to the in vitro preparation might be an effective ON input while its analog Rp-cAMPs, a competitive inhibitor of PKA, could serve as an OFF input. Responses of PKCζ and ERK protein phosphorylation would then be measured at selected time points using Western blot. Such physiological experiments could be used to test the model proposed here.

Coincidence Detection

The two-input FFL motif embedded in the initial stages of conditioning shown in Figure 4 and discussed in Part A is particularly relevant to models of classical conditioning because it can perform coincidence detection. Two different inputs may have a cooperative effect if they occur close enough together in time. The dynamics of this integrative effect depend on the strength of the two inputs, the threshold of the output to be achieved (activation threshold), and the rate function of activation or decay. In classical conditioning, molecular events leading to coincidence detection are usually applied to the initial association of a CS with a coincident or briefly delayed US. In order to form a learned association of the two stimuli they must occur close enough together in time to have a cooperative effect. Many ideas for coincidence detection have been founded on Donald Hebb’s original hypothesis that synaptic modifications underlying learning occur in response to simultaneous pre- and post-synaptic activity (Hebb, 1949). Today, theories of coincidence detection are based heavily on the function of the NMDAR (Tsien, 2000). The NMDAR uniquely fulfills Hebb’s postulate because it requires both presynaptic release of the neurotransmitter glutamate and post-synaptic depolarization in order to function by allowing intracellular Ca²⁺ entry and therefore is widely considered to be a molecular coincidence detector. However, data from in vitro eyeblink classical conditioning (Zheng and Keifer, 2009), operant conditioning in Aplysia (Lorenzetti et al., 2008), and learning in Drosophila (Gervasi et al., 2010) indicate that coincidence detection occurs well before or without any NMDAR-mediated steps in signal transduction. These mechanisms, instead, appear to use adenylate cyclase and PKA. For reinforcement learning in Aplysia, the Baxter/Byrne group (Lorenzetti et al., 2008) proposed that the adenylyl cyclase complex is a point of molecular convergence that raises cAMP/PKA levels past a certain threshold to produce behavioral responses (in the form of firing of cell B51) underlying operant conditioning of feeding. This model involves convergence of behaviorally relevant increases in Ca²⁺-dependent PKC and dopamine reward signals. Interestingly, a similar situation appears to be present in the mushroom body of Drosophila which is required for olfactory learning. Acetylcholine application, which increases intracellular Ca²⁺, paired with dopamine results in greatly enhanced increases in PKA compared to application of either stimulus alone (Gervasi et al., 2010). Although the initial coincidence detection event for the occurrence of the CS and US has yet to be described in molecular detail for in vitro classical conditioning, it is hypothesized to have a two-input C1-FFL network motif structure, like that in Figure 3 (right), and immediately precedes Part A described here. The two inputs represent the CS (X₁) and US (X₂) and utilize activation of IP3/PDK-1 and (non-NMDAR-mediated) intracellular Ca²⁺ influx/adenylate cyclase. We further postulate a convergent node Y that generally represents signal transduction events controlling the spatial proximity of molecular interactions initiated by the CS and US. Spatially localized kinase signaling is regulated by scaffolding proteins involved in subcellular compartmentalization such as A-kinase anchoring protein (AKAPs; Dessauer, 2009). Together, these inputs activate PKA and CaMKII (Z) leading to the molecular cascade detailed in Part A for GluR1 trafficking. Therefore, the CS and US must occur together both temporally and spatially to activate PKA/CaMKII. Further physiological and molecular studies will reveal the details of the initial coincidence detection events that lead to classical conditioning and whether they can be described adequately by a network motif architecture.

Motif-Based Models as a New Tool for Understanding Signal Transduction during Synaptic Plasticity and Learning

Current efforts in depicting the complex signaling events that underlie different forms of synaptic plasticity and learning are undoubtedly helpful in conveying the basic physiological events underlying these processes (e.g., Derkach et al., 2007; Lorenzetti et al., 2008; Keifer and Zheng, 2010; and Figure 1D). However, such models can be unsatisfying when trying to capture a more complete picture of the mechanisms involved. If it is accepted that biological processes follow some basic design principles, then the application of the network motif approach can be beneficial to such studies as we have tried to illustrate here for in vitro classical conditioning. First, motif-based models provide some underlying structure where there is otherwise none. Certain motifs appear to be more common than others in transcription, signal transduction, and neuronal networks, and this information can guide modeling at least as an initial starting point. Second, the network motif approach easily captures complex molecular interactions. For example, the multiple phosphorylation events that AMPAR subunits undergo during synapse modification can be clarified which may prove instructive for understanding a broad range of experimental results. Third, motif-based architectures applied to signal transduction can aid in predicting molecular pathways of as yet undiscovered processes, such as coincidence detection during conditioning, and contribute to the design of such experiments. The application of network motif design principals to models of signal transduction during learning should contribute to the elucidation of these mechanisms and provide a general framework to compare signal transduction across different forms of learning and model systems.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Supported by NIH grants NS051187 and P20 RR015567 which is designated as a Center of Biomedical Research Excellence (COBRE) to Joyce Keifer, and P01 NS44383 to James C. Houk. We thank Dr. Clive Bramham for helpful discussions and review of the manuscript.

References

Alon, U. (2007). An Introduction to Systems Biology. Boca Raton, FL: Chapman & Hall/CRC.

Bramham, C. R., and Messaoudi, E. (2005). BDNF function in adult synaptic plasticity: the synaptic consolidation hypothesis. Prog. Neurobiol. 76, 99–125.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Carvalho, A. L., Kameyama, K., and Huganir, R. L. (1999). Characterization of phosphorylation sites on the glutamate receptor 4 subunit of the AMPA receptors. J. Neurosci. 19, 4748–4754.

Pubmed Abstract | Pubmed Full Text

Casamayor, A., Morrice, N. A., and Alessi, D. R. (1999). Phosphorylation of Ser-241 is essential for the activity of 3-phosphoinositide-dependent protein kinase-1: identification of five sites of phosphorylation in vivo. Biochem. J. 342, 287–292.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Derkach, V. A., Oh, M. C., Guire, E. S., and Soderling, T. R. (2007). Regulatory mechanisms of AMPA receptors in synaptic plasticity. Nat. Rev. Neurosci. 8, 101–113.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dessauer, C. W. (2009). Adenylyl cyclase-A-kinase anchoring protein complexes: the next dimension in cAMP signaling. Mol. Pharmacol. 76, 935–941.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gervasi, N., Tchenio, P., and Preat, T. (2010). PKA dynamics in a Drosophila learning center: coincidence detection by rutabaga adenylyl cyclase and spatial regulation by dunce phosphodiesterase. Neuron 65, 516–529.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hebb, D. O. (1949). The Organization of Behavior. New York, NY: John Wiley.

Houk, J. C. (2005). Agents of the mind. Biol. Cybern. 92, 427–437.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Houk, J. C. (2010). “Voluntary movement: control, learning and memory,” in Encyclopedia of Behavioral Neuroscience, eds G. F. Koob, M. Le Moal, and R. F. Thompson (Oxford: Academic Press), 3, 455–458.

Houk, J. C., Keifer, J., and Barto, A. G. (1993). Distributed motor commands in the limb premotor network. Trends Neurosci. 16, 27–33.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Keifer, J., Armstrong, K. E., and Houk, J. C. (1995). In vitro classical conditioning of abducens nerve discharge. J. Neurosci. 15, 5036–5048.

Pubmed Abstract | Pubmed Full Text

Keifer, J., Sabirzhanov, B. E., Zheng, Z., Li, W., and Clark, T. G. (2009). Cleavage of proBDNF to BDNF by a tolloid-like metalloproteinase is required for acquisition of in vitro eyeblink classical conditioning. J. Neurosci. 29, 14956–14964.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Keifer, J., and Zheng, Z. (2010). AMPA receptor trafficking and learning. Eur. J. Neurosci. 32, 269–277.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Li, W., and Keifer, J. (2008). Coordinate action of pre- and postsynaptic brain-derived neurotrophic factor is required for AMPAR trafficking and acquisition of in vitro classical conditioning. Neuroscience 155, 686–697.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Li, W., and Keifer, J. (2009). BDNF-induced synaptic delivery of AMPAR subunits is differentially dependent on NMDA receptors and requires ERK. Neurobiol. Learn. Mem. 91, 243–249.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lorenzetti, F. D., Baxter, D. A., and Byrne, J. H. (2008). Molecular mechanisms underlying a cellular analog of operant reward learning. Neuron 59, 815–828.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mangan, S., and Alon, U. (2003). Structure and function of the feed-forward loop network motif. Proc. Natl. Acad. Sci. U.S.A. 100, 11980–11985.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mangan, S., Zaslaver, A., and Alon, U. (2003). The coherent feedforward loop serves as a sign-sensitive delay element in transcription networks. J. Mol. Biol. 334, 197–204.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Migues, P. V., Hardt, O., Wu, D. C., Gamache, K., Sacktor, T. C., Wang, Y. T., and Nader, K. (2010). PKMζ maintains memories by regulating GluR2-dependent AMPA receptor trafficking. Nat. Neurosci. 13, 630–634.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Milo, R., Shen-Orr, S., Itzkovitz, S., Kashtan, N., Chklovskii, D., and Alon, U. (2002). Network motifs: simple building blocks of complex networks. Science 298, 824–827.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mokin, M., Zheng, Z., and Keifer, J. (2007). Conversion of silent synapses into the active pool by selective GluR1-3 and GluR4 AMPAR trafficking during in vitro classical conditioning. J. Neurophysiol. 98, 1278–1286.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Mullasseril, P., Dosemeci, A., Lisman, J. E., and Griffith, L. C. (2007). A structural mechanism for maintaining the “on-state” of the CaMKII memory switch in the post-synaptic density. J. Neurochem. 103, 357–364.

Pubmed Abstract | Pubmed Full Text

Newton, A. C. (2003). Regulation of the ABC kinases by phosphorylation: protein kinase C as a paradigm. Biochem. J. 370, 361–371.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Pang, P. T., Teng, H. K., Zaitsev, E., Woo, N. T., Sakata, K., Zhen, S., Teng, K. K., Yung, W.-H., Hempstead, B. L., and Lu, B. (2004). Cleavage of proBDNF by tPA/plasmin is essential for long-term hippocampal plasticity. Science 306, 487–491.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Patterson, S. L., Pittenger, C., Morozov, A., Martin, K. C., Scanlin, H., Drake, C., and Kandel, E. R. (2001). Some forms of cAMP-mediated long-lasting potentiation are associated with release of BDNF and nuclear translocation of phospho-MAP kinase. Neuron 32, 123–140.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sabirzhanov, B. E., Keifer, J., and Clark, T. G. (2007). Characterization of a novel reptilian tolloid-like gene in the pond turtle, Pseudemys scripta elegans. Brain Res. 1154, 22–30.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sato, S., Fujita, N., and Tsuruo, T. (2004). Involvement of 3-phosphoinositide-dependent protein kinase-1 in the MEK/MAPK signal transduction pathway. J. Biol. Chem. 279, 33759–33767.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sharma, S. K., Sherff, C. M., Stough, S., Hsuan, V., and Carew, T. J. (2006). A tropomyosin-related kinase B ligand is required for ERK activation, long-term synaptic facilitation, and long-term memory in Aplysia. Proc. Natl. Acad. Sci. U.S.A. 103, 14206–14210.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shen-Orr, S., Milo, R., Mangan, S., and Alon, U. (2002). Network motifs in the transcriptional regulation network of Escherichia coli. Nat. Genet. 31, 64–68.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Thomas, G. M., Lin, D.-T., Nuriya, M., and Huganir, R. L. (2008). Rapid and bi-directional regulation of AMPA receptor phosphorylation and trafficking by JNK. EMBO J. 27, 361–372.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tsien, J. Z. (2000). Linking Hebb’s coincidence-detection to memory formation. Curr. Opin. Neurobiol. 10, 266–273.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

White, J. G. (1985). Neuronal connectivity in Caenorhabditis elegans. Trends Neurosci. 8, 277–283.

CrossRef Full Text

Yang, J., Siao, C.-J., Nagappan, G., Marinic, T., Jing, D., McGrath, K., Chen, Z.-Y., Mark, W., Tessarollo, L., Lee, F. S., Lu, B., and Hempstead, B. L. (2009). Neuronal release of proBDNF. Nat. Neurosci. 12, 113–115.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zheng, Z., and Keifer, J. (2008). Protein kinase C-dependent and independent signaling pathways regulate synaptic GluR1 and GluR4 AMPAR subunits during in vitro classical conditioning. Neuroscience 156, 872–884.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zheng, Z., and Keifer, J. (2009). PKA has a critical role in synaptic delivery of GluR1- and GluR4-containing AMPARs during initial stages of acquisition of in vitro classical conditioning. J. Neurophysiol. 101, 2539–2549.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zhu, J. J., Qin, Y., Zhao, M., van Aelst, L., and Malinow, R. (2002). Ras and Rap control AMPA receptor trafficking during synaptic plasticity. Cell 110, 443–455.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text