Context prediction analysis and episodic memory
- Laboratory of Neural Systems, Decision Science, Learning and Memory, Department of Psychology, University of Washington, Seattle, WA, USA
Events that happen at a particular place and time come to define our episodic memories. Extensive experimental and clinical research illustrate that the hippocampus is central to the processing of episodic memories, and this is in large part due to its analysis of context information according to spatial and temporal references. In this way, hippocampus defines ones expectations for a given context as well as detects errors in predicted contextual features. The detection of context prediction errors is hypothesized to distinguished events into meaningful epochs that come to be recalled as separate episodic memories. The nature of the spatial and temporal context information processed by hippocampus is described, as is a hypothesis that the apparently self-regulatory nature of hippocampal context processing may ultimately be mediated by natural homeostatic operations and plasticity. Context prediction errors by hippocampus are suggested to be valued by the midbrain dopamine system, the output of which is ultimately fed back to hippocampus to update memory-driven context expectations for future events. Thus, multiple network functions (both within and outside hippocampus) combine to result in adaptive episodic memories.
Introduction
Events are typically defined by the situations that are associated with significant outcomes. Each situation, or context, is multifaceted in that it includes not only the external sensory environment, but also our intrinsic motivational and emotional state, as well as social considerations. Thus it should not be surprising that the cells of the hippocampus, long thought to mediate episodic or event-based memories (e.g., O’Keefe and Nadel, 1978; Tulving, 2002), have been found to represent a variety of context-defining information. The challenge has been, however, to understand how these hippocampal representations of context-specific information are used to generate episodic memories. The following discusses the view that hippocampus represents context information in order to determine whether the expected contextual features match those currently being experienced. The identification of mismatches (termed context prediction errors) may lead to a cascading series of assessments through connected brain regions that could ultimately alter future decisions and update memories.
Spatial Context-Dependency of Hippocampal Neural Codes
It is generally accepted that hippocampus processes contextual information (e.g., Hirsh, 1974; Myers and Gluck, 1994; Anagnostaras et al., 2001; Maren, 2001; Fanselow and Poulos, 2005; Bouton et al., 2006). There are numerous demonstrations in which conditioned responses to contextual stimuli are eliminated with hippocampal damage while responses to discrete conditioned stimuli are unaffected (e.g., Kim and Fanselow, 1992; Phillips and LeDoux, 1992, 1994). Also, animals with hippocampal or entorhinal cortical damage do not show the normal decrement in conditioned responding after a shift in context (Penick and Solomon, 1991; Freeman et al., 1996a,b). The hippocampus of freely behaving animals is predisposed to represent context information within a spatial framework: during unrestrained navigation, hippocampal neurons fire selectively as animals traverse restricted areas of their environment, referred to as place fields (O’Keefe and Dostrovsky, 1971). As decades of research have shown (summarized in Mizumori, 2008), place fields are dynamic and integrated representations of multiple types of context-defining information. For example, changing any modality of cues, the motivational state, or the behaviors needed to perform the task result in alterations of place field properties, a process commonly referred to as remapping.
The readiness of place fields to remap is evident when changes are made to the visual environment (e.g., Ranck, 1973; O’Keefe, 1976; Olton et al., 1978; Muller and Kubie, 1987), such as its geometric features (e.g., Gothard et al., 1996; O’Keefe and Burgess, 1996; Wiener, 1996). Other sensory inputs also bias place field activity, including olfactory (Save et al., 2000), auditory (O’Keefe and Conway, 1978; McEchron and Disterhoft, 1999), and somatosensory information (Young et al., 1994). Thus, an extensive literature (Mizumori, 2008) verifies that hippocampal pyramidal neurons process multimodal sensory input. Hippocampal place fields are also sensitive to changes in a task’s reward structure (Smith and Mizumori, 2006b; Wikenheiser and Redish, 2011). Smith and Mizumori (2006b) explicitly tested this idea by training rats to distinguish Context A from Context B according to where reward was expected to be found. The motivational, sensory, and behavioral requirements of task performance were purposely held constant across the two contexts so that changes in place fields could be attributed to the recall of a different memory. Place fields remapped at the beginning of trials in Context B, a time when there is heightened uncertainty about the context conditions. In a similar experiment, Wikenheiser and Redish (2011) demonstrated that changes in reward contingency can modulate the trial-to-trial variability of hippocampal place cell activity, again suggesting that uncertainty can drive place field remapping. Finally, the relative contributions of sensory, motivation (including reward), or response information to a given place field vary with task demands, and this is evidenced by findings across many laboratories that place fields change when rats use identical information to solve tasks according to different mnemonic strategies (e.g., Ferbinteanu and Shapiro, 2003; Mizumori et al., 2004; Eschenko and Mizumori, 2007).
In sum, it is clear that place fields can come to represent different types of sensory, behavioral, and intrinsic information that have strong spatial and contextual (i.e., experience-dependent) features, thereby validating the term spatial context when referring to the informational content of place fields (Nadel et al., 1985; Mizumori et al., 1999, 2000; Jeffery et al., 2004). If this information indeed underlies the effective use and generation of episodic memories, Nadel (2008) has argued that the neural representations that define a familiar context must be relatively stable and predictable, maintaining their relationship to each other, until prediction errors are detected. This pattern of place field responses has indeed been documented in the literature. Stable place fields, then, can be said to reflect the integration of stable background sensory information, internal state or motivational information, as well as response or behavioral outcome expectancies within a spatial framework as a function of time.
To the extent that different cognitive strategies are mediated by different underlying memories, one network pattern of activated place cells is thought to reflect one memory, and a different pattern of activated place cells corresponds to a different memory (e.g., Samsonovich and McNaughton, 1997). When one refers to place field remapping, then, implicit is the notion that each map (or collection of activated place cells) is driven by a different memory.
Spatial Context Discrimination and Prediction
As striking as location-selective firing is to even the naive observer, equally impressive is the fact that the otherwise stable place fields readily remap when most any feature of the spatial context changes. This sensitivity to changes in context has led many to suggest that a fundamental operation of the hippocampus is to detect changes in contextual information so that animals can discriminate contexts (e.g., Smith and Mizumori, 2006a,b). The ability to distinguish contexts likely relies on computations that are often attributed to hippocampus, including those that underlie the flexible use of conjunctive, sequential, relational, and spatial algorithms (e.g., O’Keefe and Nadel, 1978; Foster et al., 1987; Eichenbaum et al., 1999; Wood et al., 2000; Eichenbaum and Cohen, 2001; O’Reilly and Rudy, 2001; Fortin et al., 2002). The latter, in turn, are thought to be supported by various forms of pattern separation and pattern completion neurocomputations (Mizumori et al., 2004, 2007b; Penner and Mizumori, 2012a,b).
A Context Discrimination Hypothesis (CDH) postulates that single hippocampal neuronal representations of context provide multidimensional data to population-based network computations that ultimately determine whether expected contextual features of a situation have changed (e.g., Mizumori et al., 1999, 2000, 2007a; Smith and Mizumori, 2006a,b; Mizumori, 2008). Initial suggestive evidence of this interpretation of hippocampal network function was the repeated observation that upon less than complete changes in a familiar context, many but not all place fields remap (e.g., Tanila et al., 1997; Mizumori et al., 1999; Brown and Skaggs, 2002; Knierim, 2002; Lee et al., 2004). The place fields that remained in the face of changes in a familiar context were considered to represent the stable or expected contextual features. The place fields that changed, then, could be thought of as representing current context information. The existence of these two types of place field responses gave rise to the notion that hippocampus compares expected and experienced context features (Mizumori et al., 1999). This idea begs the question, then, why does hippocampus represent both expected (learned) and current context information? These hippocampal spatial context representations (O’Keefe and Nadel, 1978; Nadel and Wilner, 1980; Nadel and Payne, 2002) may contribute to a match-mismatch type of analysis that evaluates the present context according to how similar it is to the context that an animal expects to encounter based on past experiences (e.g., Gray, 1982; Vinogradova, 1995; Mizumori et al., 1999, 2000; Gray, 2000; Lisman and Otmakhova, 2001; Hasselmo et al., 2002; Anderson and Jeffery, 2003; Jeffery et al., 2004; Hasselmo, 2005b; Smith and Mizumori, 2006a,b; Manns et al., 2007a; Nadel, 2008). Detected mismatches may be signaled by a change in the pattern of input from hippocampus or possibly by a specific input pattern. This question remains to be answered. Nevertheless, mismatch signals can be used to identify novel situations and to distinguish different contexts, functions that are necessary to define significant events or episodes. Mismatch signals also may engage neural mechanisms that determine the value of the mismatch so that existing memories can be updated and/or new memories can be formed. When context match signals are generated, the effect could be to strengthen currently active memory networks located elsewhere in the brain (e.g., neocortex). In this way, hippocampus may play different mnemonic roles depending on whether or not contexts actually change.
In support of the CDH, disconnecting hippocampus by fornix lesions impairs context discrimination (Smith et al., 2004), and hippocampal lesions reduce animals’ ability to respond to changes in a familiar environment (Good and Honey, 1991; Save et al., 1992a,b). Spatial novelty detection corresponds to selective elevation of the immediate early gene c-fos in hippocampus, and not in surrounding parahippocampal cortical regions (Jenkins et al., 2004). Also, as described above, hippocampal neurons show significantly altered firing patterns when rats experience spatial or non-spatial changes in a familiar environment (O’Keefe, 1976; Muller and Kubie, 1987; Wood et al., 1999; Fyhn et al., 2002; Ferbinteanu and Shapiro, 2003; Moita et al., 2004; Yeshenko et al., 2004; Leutgeb et al., 2005a,b; Puryear et al., 2006; Smith and Mizumori, 2006b; Eschenko and Mizumori, 2007). As an example, Smith and Mizumori (2006b) showed that hippocampal neurons develop context-specific responses only when rats were required to discriminate contexts. Discriminating neural responses were not observed when rats were allowed to randomly forage for the same amount of time. Further, Manns et al. (2007b) demonstrated that relative to match trials in an odor cue or object recognition task, CA1 neurons preferentially discharged when animals experienced a non-match situation in these same tasks. Also consistent with the CDH, neuroimaging studies of human performance shows that hippocampus becomes differentially active during match and mismatch trials (Kumaran and Maguire, 2007; Kuhl et al., 2010; Chen et al., 2011; Dickerson et al., 2011; Foerde and Shohamy, 2011; Duncan et al., 2012a,b).
The detection of changes in context is fundamentally important for the continual selection of appropriate behaviors that optimize performance and learning in a variety of tasks (e.g., navigation-based learning, instrumental conditioning, or classical conditioning). Context discrimination engages and prepares cellular mechanisms for rapid and new learning at potentially important times (Paulsen and Moser, 1998), as it is generally known that novelty detection increases attention and exploratory behaviors in a variety of tasks. Interestingly, hippocampal cell firing tends to occur during the “encoding phase” of the ongoing theta rhythm (Hasselmo, 2005a), which is increased during exploratory and investigatory behaviors (Vanderwolf, 1969). Thus, detection of a non-match situation can change the relationship between cell discharge and the local theta rhythm such that encoding functions are enhanced. Detection of matches, on the other hand, does not cause changes in the hippocampal neural activity profile, resulting in efferent messages that continue to retrieve/utilize the currently active memory network that recently drove the execution of successful responses. Context discrimination, then, can be viewed as being critical for the formation of new episodic memories because it leads to the separation in time and space one meaningful event from the next. Such division of memories could facilitate long-term information storage according to memory schemas (Tse et al., 2007; Bethus et al., 2010).
Since hippocampus seems particularly sensitive to changes in the expected (i.e., experience-dependent) context-defining features of a situation, its mismatch signals can be considered to reflect errors in predicted encounters with contextual features, or context prediction errors. Indeed, it has been shown that the greater the change in familiar context information, the more place fields remap (e.g., Leutgeb et al., 2005a). It should be noted, however, that when hippocampal cells respond differently to two contexts, it may be because they receive different afferent signals when animals experience different contexts. A second possibility is that the hippocampus receives input from memories stores that define context expectations, and this is actively compared within hippocampus to different input that defines the current context. The simultaneous presence of both familiar and new context information in hippocampus supports the latter possibility. In either case, however, transmission of a context prediction error signal from hippocampus may inform distal brain areas that a change in the context has occurred. Upon receipt of the context prediction error message, efferent midbrain-striatal structures may respond with changes in excitation or inhibition that reflect preparations for, or actual evaluation of, the subjective value of the context prediction error signal (e.g., Mizumori et al., 2004; Lisman and Grace, 2005; Humphries and Prescott, 2010; Penner and Mizumori, 2012a). On the other hand, a hippocampal signal indicating that there was no prediction error may enable plasticity mechanisms that ultimately allow new information to be incorporated into existing memory schemas (e.g., Mizumori et al., 2007a,b; Tse et al., 2007; Bethus et al., 2010). Thus, hippocampal context analyses become critical for the formation of new episodic memories not only because prediction signals provide a mechanism that separates in time and space one meaningful event from the next, but also because the outcome of the prediction error computation engages appropriate neuroplasticity mechanisms in efferent structures that promote subsequent adaptive decisions and memory.
The midbrain dopaminergic system is part of a neural network that assesses the value of behavioral outcomes. Reward-induced excitation of dopamine neurons scales to the magnitude of expected and encountered rewards regardless of the task demands (Schultz et al., 1997; Puryear et al., 2010; Jo et al., 2013): encounters with large rewards are accompanied by larger amplitude phasic dopamine responses than encounters with small amounts of reward. In addition, dopamine cells respond to unexpected reward absences by decreasing their firing rates (Schultz et al., 1997; Puryear et al., 2010). The reduction in firing when rewards are unexpectedly absent is greater if the expectation was for a large, and not small, reward. Further, these reward responses are context-dependent in a manner similar to what is observed for hippocampal place fields (Puryear et al., 2010), a result consistent with the view that hippocampal information guides reward value assessment systems of the brain (e.g., Mizumori et al., 2004; Lisman and Grace, 2005; Puryear et al., 2010) such that the significance of context prediction error messages can be determined (e.g., Penner and Mizumori, 2012a,b). The outcome of the value determination may ultimately come to bias future behavioral responding so that desired goals are more likely to be achieved (see review in Penner and Mizumori, 2012b).
Context-Dependent Temporal Information Processing and Episodic Memory
Different episodic memories contain information within unique spatial and temporal domains (Tulving, 2002). While there is growing clarity regarding how and why hippocampal neurons represent spatial context information (e.g., Knierim et al., 2006; McNaughton et al., 2006; Penner and Mizumori, 2012a,b; Pilly and Grossberg, 2012; Buzsaki and Moser, 2013), the mechanisms by which spatial context-defined events are distinguished temporally in the service of episodic memory functions remains unclear. However, current evidence support the hypothesis that hippocampus contains an organization structure that permits grouping of information into a number of different time scales (e.g., as reviewed in Buzsaki, 2006; Lisman and Redish, 2009; Grossberg and Pilly, 2013), that are relevant for comparing expected and experienced context information, and thus for episodic memory.
The activity of single neurons and neural networks naturally oscillate within hippocampus and across the brain according to a range of frequencies from low (e.g., 2–4 Hz) to high (e.g., 80 Hz) (Buzsaki, 2006). Oscillatory activity reflects alternating periods of synchronous neural firing: synchronous activity is associated with greater synaptic plasticity and stronger coupling amongst cells of an ensemble, while desynchronous periods are associated with less plasticity and weak signal strength (Hasselmo et al., 2002; Hasselmo, 2005a; Buzsaki, 2006). Thus, inherent in oscillatory neural activity is the ability to segregate on a (very short) time scale the processing of event-related information. In fact, the precise temporal alignment of place cell firing relative to the phase of the ongoing (theta) rhythm processes as animals move through the cell’s place field (referred to as phase precession; O’Keefe and Recce, 1993) suggests the possibility that context-dependent changes in place cell activity alters the state of synaptic plasticity in hippocampus according to experience. By extension, then, an important impact of altered contextual features is the corresponding change in the nature and/or efficiency of information read into and passed on from hippocampus. Importantly, hippocampal oscillations may enable more temporally precise context representations, and therefore episodic memory. This in turn may enable more precise detection of unexpected context information.
Hippocampal circuits are also observed to synchronize their activity at frequencies greater than the theta frequency. Of particular interest is the gamma band (30–80 Hz) which has been observed in many sensory and motor areas of cortex, hippocampus, parietal cortex, and striatum (e.g., Leung and Yim, 1993; Brosch et al., 2002; Csicsvari et al., 2003; Berke et al., 2004; Bauer et al., 2006; Hoogenboom et al., 2006; Womelsdorf et al., 2006). Orchestration of both excitatory and inhibitory networks within each structure underlies the generation of synchronized gamma oscillations (e.g., Whittington et al., 1995; Vida et al., 2006). Although the functional importance of gamma oscillations remains debated, information carried by the cells that participate in a gamma-burst is effectively accentuated against a background of disorganized neural activity. Therefore, it has been suggested that gamma-bursts represent a fundamental mechanism by which information becomes segmented, filtered, or highlighted within a structure, as well as a mechanisms by which to coordinate information across structures (Buzsaki, 2006). Theta and gamma have many common physiological and behavioral relationships, suggesting that they are components of a coordinated and larger scale oscillatory network. For example, similar to theta rhythms, single unit responses that are recorded simultaneously with gamma oscillations have been found to have specific phase relationships to the gamma rhythm, and both theta and gamma are at least in part regulated by dopamine (e.g., Berke, 2009; van der Meer and Redish, 2009; Kalenscher et al., 2010). Therefore, changes in context that induce remapping (i.e., that altered the constellation of activated neurons), likely change patterns of gamma activity that are characteristic of a familiar context. Since gamma oscillations may effectively select salient information that ultimately impacts decisions, learning, and behavioral responses (e.g., van der Meer and Redish, 2009; Kalenscher et al., 2010), it is predicted that changes in gamma oscillatory patterns are likely to alter future decisions and learning. Further, gamma-bursts should become more predictable as learning takes place within a given context. The relationship between place fields and the overriding theta and gamma rhythms is then an important mechanism by which spatially organized context information becomes temporally organized as animals experience an environment (Buzsaki, 2006; Buzsaki and Moser, 2013). Specifically, this type of temporal relationship may confer a high level of temporal alignment (and hence accuracy) between expected and experienced context information, and this in turn should increase the accuracy of their comparison.
Since multiple brain areas demonstrate rhythmic neural activity, neural oscillations are likely a fundamental mechanism for coordinating neural activity across the brain in the service of adaptive decisions, learning, and memory (e.g., Buzsaki, 2006; Fries, 2009; Monaco et al., 2011; Penner and Mizumori, 2012b). Numerous laboratories have now reported that synchronous neural activity (in particular coherence of the theta rhythm) can be detected within and between memory-related brain structures such as the hippocampus, striatum, or prefrontal cortex (Tabuchi et al., 2000; Engel et al., 2001; Fell et al., 2001; Varela et al., 2001; Siapas et al., 2005; DeCoteau et al., 2007a; Womelsdorf et al., 2007). For example, hippocampal theta activity can become synchronized with place cell firing, resulting in coordinated timing of spatial coding (O’Keefe and Recce, 1993; Gengler et al., 2005). Also, theta oscillations within the striatum can become entrained to the hippocampal theta rhythm (Berke et al., 2004; DeCoteau et al., 2007a). Stimulating the striatum can induce hippocampal theta activity (Sabatino et al., 1985) and increases high frequency theta power, which is thought to be important for sensorimotor integration (Hallworth and Bland, 2004). Also, hippocampal and striatal activity theta activity become increasingly coherent during goal-directed navigation (Allers et al., 2002; DeCoteau et al., 2007a). When neural activity is disrupted in the striatum via D2 receptor antagonism, striatal modulation of high frequency hippocampal theta activity is also disrupted. The result is that motor and spatial/contextual information is not integrated, and task performance is impaired (Gengler et al., 2005).
Particularly intriguing is a finding common to both hippocampus and striatum, and that is that synchronous activity occurs in specific task-relevant ways (e.g., Hyman et al., 2005; Jones and Wilson, 2005) particularly during times when rats are said to be engaged in decision making (e.g., Benchenane et al., 2010). For example, striatal theta is modified over the course of learning on an egocentric T-maze task, increasing in coherence as the rat chooses and initiates turn behavior (DeCoteau et al., 2007a,b). Rats that learn the task develop an antiphase relationship between hippocampal and striatal theta oscillations, while rats that do not learn the task also do not show this type of theta relationship. This coherence has also been observed during striatal-dependent classical conditioning (Kropf and Kuschinsky, 1993). Coherent theta oscillations across distant brain structures can be enhanced with application of dopamine (Benchenane et al., 2010). Dopamine, then, may play a crucial role in coordinating ensemble activity across brain areas during times of decision making during navigation. Functionally, this type of control by dopamine suggests that information about the saliency of reward may determine which brain systems become synchronized (and desynchronized). This in turn guides the nature and type of information that is used to update memories and to determine future responses.
Task demands, then, seem to dictate the nature of neural synchrony across distal brain structures, and this synchrony may take the form of comodulation of existing theta and gamma rhythms as well as the generation of an additional rhythm. The latter was illustrated in a recent study by Fujisawa and Buzsaki (2011). They demonstrated that the existence of very low frequency (4 Hz) entrainment of local field potentials, e.g., the 7–12 Hz theta oscillation, that emerges only during phases of a maze task when rats made decisions (i.e., in the stem of a T-Maze). During decision periods, the 4 Hz rhythm was phase locked to theta oscillations in both the prefrontal cortex and VTA. Some of the individual prefrontal and VTA neurons were also phase locked to hippocampal theta oscillation at this time. Importantly the 4 Hz rhythm was present only during a decision making period when theta oscillations were also present. The findings of this study suggest that a 4 Hz rhythm may coordinate activity in distal brains structures specifically as animals make decisions during goal-directed navigation.
Assuming that “orchestrating” rhythms such as the 4 Hz rhythm also incorporate hippocampal neural activity, place field remapping is expected to have effects that extend well beyond hippocampal computations. When an expected context changes, evidence shows that place fields change by either increasing or decreasing firing rates and/or changes in their spatial specificity and reliability. This period of place field change corresponds to the period of uncertainty that is generated with by the context change. Therefore, as uncertainty decreases, place fields become more stable, and this in turn should result in the generation of more stable comodulation of specific frequencies of EEG across brain structures.
To fully understand the role of the hippocampus in episodic memory requires an understanding of how hippocampus relates event-specific details to particular temporal features of a task. Recently many (Manns et al., 2007b; Pastalkova et al., 2008; Gill et al., 2011; McDonald et al., 2011; Kraus et al., 2013) have suggest the existence of “time cells” in addition to place cells in hippocampus. These studies demonstrate that the timing of cell firing by a subpopulation of hippocampal neurons is not directly related to spatial or behavioral metrics such as distance, location, or running speed but rather they appear tuned to the timing of task-relevant events. Another challenge for future research is to better define how hippocampal context prediction error signals are interpreted and valued by neural systems responsible for decisions based on the most recent context analysis, and ultimately, by neural systems that update memories (Lisman and Grace, 2005; Penner and Mizumori, 2012a,b). The midbrain dopaminergic system is likely involved in this assessment process since dopamine neurons are not only known to respond to changes in the learned value of rewards, and to cues that predict future rewards (Schultz et al., 1997), but also they respond to changes in reward values in a context-dependent manner when rats perform a hippocampal-dependent task (Puryear et al., 2010).
Dopamine neural responses to the expectation of rewards seem to be regulated at least in part by prefrontal cortex (Jo et al., 2013), suggesting that the prefrontal cortex may relay to dopamine neurons information based on stored memories about past consequences of behavior. Penner and Mizumori (2012b) recently reviewed an extensive literature that describes how dopamine cell responses to hippocampal-based context information can come to regulate subsequent choices and response selection as information is processed through iterative striatal-to-cortex information loops (Haber et al., 2000). The result of this sequence of striatal-cortical processing is the determination of the degree to which expected behavioral outcomes occurred, and the updating of long-term memory. The latter, in turn, updates the definition of expected context features that cortex subsequently provides hippocampus (Penner and Mizumori, 2012a,b; Mizumori and Jo, 2013). Indeed Martig and Mizumori (2011) have shown that inactivation of the VTA results in unstable hippocampal place fields and behavioral errors when rats perform a hippocampal-dependent spatial task. Since the outcome assessment by VTA impacts the subsequent stability of hippocampal neural codes and behavioral accuracy on a spatial working memory task, it is likely that hippocampus and the VTA-striatal circuitry comprise key components of an adaptive loop of neural processing that allows organisms to continuously update memories and memory representations according to the outcomes of choices made within circumscribed epochs of time.
Homeostatic Regulation of Neural Signals of Prediction Errors
An emerging view is that the brain has evolved in large part to allow organisms to accurately predict the outcomes of events and behaviors (e.g., Llinas and Roy, 2009; Buzsaki, 2013; Buzsaki and Moser, 2013; Mizumori and Jo, 2013). More specifically, it has been suggested that organisms have been able to adapt to environments and societies of increasing complexity because brains evolved more complex neural circuitry that support the ability to make dynamic and conditional decisions and predictions. These neural ensembles evolved to retain information over times of varying scales depending on the desired goal. Different brain areas are known to generate and retain sequences of information, and this ability can be accounted for by state-dependent changes in network dynamics (Mauk and Buonomano, 2004), internally generated oscillatory activity (Pastalkova et al., 2008), and/or dedicated “time cells” (Kraus et al., 2013). Thus, many elemental features of prediction analyses seem to be intrinsic, or self-generated. This property is likely very important for it may provide a mechanism by which prediction analyses can occur automatically so that organisms naturally seek control of the outcomes to their behaviors. What, then, might be the mechanism of a self-generated, and thus auto-regulated, brain prediction system? It is suggested here that such a mechanism may to some extent mirror principles of self-regulation at synaptic and neural circuit levels (e.g., Turrigiano, 1999, 2008, 2011; Marder and Prinz, 2003; Turrigiano and Nelson, 2004; Marder and Goaillard, 2006; Shetty et al., 2012; Mizumori and Jo, 2013).
Marder and Goaillard (2006) suggested that homeostatic neuroplasticity may be nested: calcium sensors may monitor neural firing rates, then up or down regulate the availability of glutamate receptors to ramp up or down firing rates toward an optimal firing rate set point. Groups of neurons or neural networks may sense changes in firing collectively to regulate experience-dependently population activity levels and patterns of activation. In this way homeostatic plasticity enables groups of neural circuits to find a balance between flexible and stable processing as needed to learn from experiences, and to be responsive to future changed inputs. The details of how networks of cells or their connections engage in homeostatic regulation remain to be discovered. Nevertheless, it is worth noting that homeostatic regulation at the neural systems level is clearly evident from studies of brain development, as well as from studies of reactive or compensatory neuroplasticity mechanisms that occur in response to experience (e.g., sensorimotor learning; Froemke et al., 2007) or brain injury (e.g., brain trauma or addiction; Robinson and Kolb, 2004; Nudo, 2011). While specific homeostatic neural plasticity mechanisms have not been used to account for complex learning, current theories of reinforcement- and context-based learning and memory commonly rely on the autoregulation of feedback loops.
A homeostatic framework could apply to the autoregulation of prediction analyses, which in turn will impact regulation of future decisions and memories. Such a framework includes variables that are monitored by sensors and then regulated by controllers, and thus it likely involves multiple, interactive, and hierarchically organized (auto-regulated) information loops analogous to what was described by Buzsaki (2013). At the cellular or synaptic levels, homeostatic plasticity mechanisms (Turrigiano et al., 1998; Marder and Goaillard, 2006; Turrigiano, 2011) may regulate cell excitability around a neural activity set point such that neurons retain maximal responsivity to future inputs. This process enables neurons to achieve a balance between synaptic stability and flexibility. Changes in calcium flux appear to be an important part of the sensing system that determines the current level of firing. It is hypothesized here that a prediction error, or mismatch signal, may result in higher or lower firing rates, at which time controller mechanisms should be engaged to bring the firing rates back to set point levels. Indeed reward prediction errors are illustrated by transient and significant reduced or elevated neural firing depending on the valence of the error (Schultz et al., 1997). Future research should focus on understanding the enabling and restorative mechanisms of prediction error signals. Of particular interest are mechanisms by which cortical memory may impact the threshold for signaling prediction errors. One cortical area of interest is the prefrontal cortex given (a) its intrinsic recurrent circuitry and detailed excitatory and inhibitory extrinsic connections (as reviewed in Arnsten et al., 2012) with both hippocampal/temporal lobe and reward valuation systems, and (b) given its role in attention and working memory (e.g., Fuster, 2006, 2008, 2009). That is, prefrontal cortex may orchestrate and coordinate the level of neural excitability in different prediction error brain areas according to homeostatic principles and in this way, bias the nature of the outputs of connected brain areas according to experience and recent outcomes of decisions. Prefrontal cortex also has strong functional connections with other cortical memory areas (e.g., parietal and temporal cortex), and is thus strategically positioned to influence long-term memory updating based on prediction error analyses. In this way the most recent memories can be fed forward to hippocampus for future context evaluation.
Although it is reasonable to assume that the prefrontal cortex controls or biases neural signaling in distal prediction brain regions, it should be noted that other sources of control of cell excitability may arise via direct interconnections amongst the multiple prediction detection areas of the brain. For example, a prediction error signal from the hippocampus could be transmitted to midbrain-striatal neurons along pathways that do not necessarily include the prefrontal cortex. Indirect support for this idea come from observations that conditions that produce error messages in the hippocampus change reward responses of dopamine neurons (Puryear et al., 2010; Jo et al., 2013), phasic theta comodulation is observed between hippocampus and striatum (DeCoteau et al., 2007a) during decision tasks, and comodulation of neural activity has been reported between prefrontal cortex and parietal cortex (Diwadkar et al., 2000). However, the identification of such correlations does not necessarily mean that this is the source of the comodulation or neural synchrony. Finding the source of neural regulation remains a challenge for the general field of systems neuroscience, one that may soon have answers with continued development of new methodologies such as optogenetic analyses.
In sum, homeostatic regulatory processes may contribute to the automatic and continuous self-regulatory nature of prediction error analysis, and ultimately decision making and episodic memory. Such a naturally adaptive mechanism optimizes the contribution of different types of prediction error signals to future decisions and actions according to the pattern of recent successes and failures in prediction.
Conflict of Interest Statement
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This work was supported by NIH grant MH58755. The author is most grateful for the significant research and theoretical contributions of numerous outstanding students over many years, most recently Wambura Fobbs, Yong Sang Jo, Sujean Oh, Marsha Penner, and Valerie Tryon.
References
Allers, K. A., Ruskin, D. N., Bergstrom, D. A., Freeman, L. E., Ghazi, L. J., Tierney, P. L., et al. (2002). Multisecond periodicities in basal ganglia firing rates correlate with theta bursts in transcortical and hippocampal EEG. J. Neurophysiol. 87, 1118–1122.
Anagnostaras, S. G., Gale, G. D., and Fanselow, M. S. (2001). Hippocampus and contextual fear conditioning: recent controversies and advances. Hippocampus 11, 8–17. doi: 10.1002/1098-1063(2001)11:1<8::AID-HIPO1015>3.3.CO;2-Z
Anderson, M. I., and Jeffery, K. J. (2003). Heterogeneous modulation of place cell firing by changes in context. J. Neurosci. 23, 8827–8835.
Arnsten, A. F. T., Wang, M. J., and Paspalas, C. D. (2012). Neuromodulation of thought: flexibilities and vulnerabilities in prefrontal cortical network synapses. Neuron 76, 223–239. doi:10.1016/j.neuron.2012.08.038
Bauer, M., Oostenveld, R., Peeters, M., and Fries, P. (2006). Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduces low-frequency activity in parieto-occipital areas. J. Neurosci. 26, 490–501. doi:10.1523/JNEUROSCI.5228-04.2006
Benchenane, K., Peyrache, A., Khamassi, M., Tierney, P. L., Gioanni, Y., Battaglia, F. P., et al. (2010). Coherent theta oscillations and reorganization of spike timing in the hippocampal-prefrontal network upon learning. Neuron 66, 921–936. doi:10.1016/j.neuron.2010.05.013
Berke, J. D. (2009). Fast oscillations in cortical-striatal networks switch frequency following rewarding events and stimulant drugs. Eur. J. Neurosci. 30, 848–859. doi:10.1111/j.1460-9568.2009.06843.x
Berke, J. D., Okatan, M., Skurski, J., and Eichenbaum, H. B. (2004). Oscillatory entrainment of striatal neurons in freely moving rats. Neuron 43, 883–896. doi:10.1016/j.neuron.2004.08.035
Bethus, I., Tse, D., and Morris, R. G. (2010). Dopamine and memory: modulation of the persistence of memory for novel hippocampal NMDA receptor-dependent paired associates. J. Neurosci. 30, 1610–1618. doi:10.1523/JNEUROSCI.2721-09.2010
Bouton, M. E., Westbrook, R. F., Corcoran, K. A., and Maren, S. (2006). Contextual and temporal modulation of extinction: behavioral and biological mechanisms. Biol. Psychiat. 60, 352–360. doi:10.1016/j.biopsych.2005.12.015
Brosch, M., Budinger, E., and Scheich, H. (2002). Stimulus-related gamma oscillations in primate auditory cortex. J. Neurophysiol. 87, 2715–2725.
Brown, J. E., and Skaggs, W. E. (2002). Concordant and discordant coding of spatial location in populations of hippocampal CA1 pyramidal cells. J. Neurophysiol. 88, 1605–1613.
Buzsaki, G., and Moser, E. I. (2013). Memory, navigation and theta rhythm in the hippocampal-entorhinal system. Nat. Neurosci. 16, 130–138. doi:10.1038/nn.3304
Chen, J., Olsen, R. K., Preston, A. R., Glover, G. H., and Wagner, A. D. (2011). Associative retrieval processes in the human medial temporal lobe: hippocampal retrieval success and CA1 mismatch detection. Learn. Mem. 18, 523–528. doi:10.1101/lm.2135211
Csicsvari, J., Jamieson, B., Wise, K. D., and Buzsaki, G. (2003). Mechanisms of gamma oscillations in the hippocampus of the behaving rat. Neuron 37, 311–322. doi:10.1016/S0896-6273(02)01169-8
DeCoteau, W. E., Thorn, C., Gibson, D. J., Courtemanche, R., Mitra, P., Kubota, Y., et al. (2007a). Learning-related coordination of striatal and hippocampal theta rhythms during acquisition of a procedural maze task. Proc. Natl. Acad. Sci. U.S.A. 104, 5644–5649. doi:10.1073/pnas.0700818104
DeCoteau, W. E., Thorn, C., Gibson, D. J., Courtemanche, R., Mitra, P., Kubota, Y., et al. (2007b). Oscillations of local field potentials in the rat dorsal striatum during spontaneous and instructed behaviors. J. Neurophysiol. 97, 3800–3805. doi:10.1152/jn.00108.2007
Dickerson, K. C., Li, J., and Delgado, M. R. (2011). Parallel contributions of distinct human memory systems during probabilistic learning. Neuroimage 55, 266–276. doi:10.1016/j.neuroimage.2010.10.080
Diwadkar, V. A., Carpenter, P. A., and Just, M. A. (2000). Collaborative activity between parietal and dorso-lateral prefrontal cortex in dynamic spatial working memory revealed by fMRI. Neuroimage 12, 85–99. doi:10.1006/nimg.2000.0586
Duncan, K., Ketz, N., Inati, S. J., and Davachi, L. (2012a). Evidence for area CA1 as a match/mismatch detector: a high-resolution fMRI study of the human hippocampus. Hippocampus 22, 389–398. doi:10.1002/hipo.20933
Duncan, K., Sadanand, A., and Davachi, L. (2012b). Memory’s penumbra: episodic memory decisions induce lingering mnemonic biases. Science 337, 485–487. doi:10.1126/science.1221936
Eichenbaum, H., and Cohen, N. J. (2001). From Conditioning to Conscious Recollection: Memory Systems of the Brain. New York: Oxford University Press.
Eichenbaum, H., Dudchenko, P., Wood, E., Shapiro, M., and Tanila, H. (1999). The hippocampus, memory, and place cells: is it spatial memory or a memory space? Neuron 425, 184–188.
Engel, A. K., Fries, P., and Singer, W. (2001). Dynamic predictions: oscillations and synchrony in top-down processing. Nat. Rev. Neurosci. 2, 704–716. doi:10.1038/35094565
Eschenko, O., and Mizumori, S. J. (2007). Memory influences on hippocampal and striatal neural codes: effects of a shift between task rules. Neurobiol. Learn Mem. 87, 495–509. doi:10.1016/j.nlm.2006.09.008
Fanselow, M. S., and Poulos, A. M. (2005). The neuroscience of mammalian associative learning. Annu. Rev. Psychol. 56, 207–234. doi:10.1146/annurev.psych.56.091103.070213
Fell, J., Klaver, P., Lehnertz, K., Grunwald, T., Schaller, C., Elger, C. E., et al. (2001). Human memory formation is accompanied by rhinal-hippocampal coupling and decoupling. Nat. Neurosci. 4, 1259–1264. doi:10.1038/nn759
Ferbinteanu, J., and Shapiro, M. L. (2003). Prospective and retrospective memory coding in the hippocampus. Neuron 40, 1227–1239. doi:10.1016/S0896-6273(03)00752-9
Foerde, K., and Shohamy, D. (2011). Feedback timing modulates brain systems for learning in humans. J. Neurosci. 31, 13157–13167. doi:10.1523/JNEUROSCI.2701-11.2011
Fortin, N. J., Agster, K. L., and Eichenbaum, H. B. (2002). Critical role of the hippocampus in memory for sequences of events. Nat. Neurosci. 5, 458–462.
Foster, T. C., Christian, E. P., Hampson, R. E., Campbell, K. A., and Deadwyler, S. A. (1987). Sequential dependencies regulate sensory evoked responses of single units in the rat hippocampus. Brain Res. 408, 86–96. doi:10.1016/0006-8993(87)90361-1
Freeman, J. H. Jr., Cuppernell, C., Flannery, K., and Gabriel, M. (1996a). Context-specific multi-site cingulate cortical, limbic thalamic, and hippocampal neuronal activity during concurrent discriminative approach and avoidance training in rabbits. J. Neurosci. 16, 1538–1549.
Freeman, J. H. Jr., Cuppernell, C., Flannery, K., and Gabriel, M. (1996b). Limbic thalamic, cingulate cortical and hippocampal neuronal correlates of discriminative approach learning in rabbits. Behav. Brain Res. 80, 123–136. doi:10.1016/0166-4328(96)00027-7
Fries, P. (2009). Neuronal gamma-band synchronization as a fundamental process in cortical computation. Annu. Rev. Neurosci. 32, 209–224. doi:10.1146/annurev.neuro.051508.135603
Froemke, R. C., Merzenich, M. M., and Schreiner, C. E. (2007). A synaptic memory trace for cortical receptive field plasticity. Nature 450, 425–429. doi:10.1038/nature06289
Fujisawa, S., and Buzsaki, G. (2011). A 4 Hz oscillation adaptively synchronizes prefrontal, VTA, and hippocampal activities. Neuron 72, 153–165. doi:10.1016/j.neuron.2011.08.018
Fuster, J. M. (2006). The cognit: A network model of cortical representation. Int. J. Psychophysiol. 60, 125–132. doi:10.1016/j.ijpsycho.2005.12.015
Fuster, J. M. (2009). Cortex and memory: emergence of a new paradigm. J. Cog. Neurosci. 21, 2047–2072. doi:10.1162/jocn.2009.21280
Fyhn, M., Molden, S., Hollup, S., Moser, M. B., and Moser, E. (2002). Hippocampal neurons responding to first-time dislocation of a target object. Neuron 35, 555–566. doi:10.1016/S0896-6273(02)00784-5
Gengler, S., Mallot, H. A., and Holscher, C. (2005). Inactivation of the rat dorsal striatum impairs performance in spatial tasks and alters hippocampal theta in the freely moving rat. Behav. Brain Res. 164, 73–82. doi:10.1016/j.bbr.2005.06.009
Gill, P. R., Mizumori, S. J. Y., and Smith, D. M. (2011). Hippocampal episode fields develop with learning. Hippocampus 21, 1240–1249. doi:10.1002/hipo.20832
Good, M., and Honey, R. C. (1991). Conditioning and contextual retrieval in hippocampal rats. Behav. Neurosci. 105, 499–509. doi:10.1037/0735-7044.105.4.499
Gothard, K. M., Skaggs, W. E., Moore, K. M., and Mcnaughton, B. L. (1996). Binding of hippocampal CA1 neural activity to multiple reference frames in a landmark-based navigation task. J. Neurosci. 16, 823–835.
Gray, J. A. (1982). The Neuropsychology of Anxiety: An Enquiry into the Functions of the Septo-Hippocamal System. Oxford: Oxford University Press.
Gray, J. A. (2000). The Neuropsychology of Anxiety: An Enquiry into the Functions of the Septo-Hippocamal System, 2nd Edn. Oxford: Oxford University Press.
Grossberg, S., and Pilly, P. K. (2013). How entorhinal grid cells may learn multiple spatial scales from a dorsoventral gradient of cell response rates in a self-organizing map. PLoS Comput. Biol. 8:e1002648. doi:10.1371/journal.pcbi.1002648
Haber, S. N., Fudge, J. L., and McFarland, N. R. (2000). Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382.
Hallworth, N. E., and Bland, B. H. (2004). Basal ganglia – hippocampal interactions support the role of the hippocampal formation in sensorimotor integration. Exp. Neurol. 188, 430–443. doi:10.1016/j.expneurol.2004.04.014
Hasselmo, M. E. (2005a). What is the function of hippocampal theta rhythm? – Linking behavioral data to phasic properties of field potential and unit recording data. Hippocampus 15, 936–949. doi:10.1002/hipo.20116
Hasselmo, M. E. (2005b). The role of hippocampal regions CA3 and CA1 in matching entorhinal input with retrieval of associations between objects and context: theoretical comment on Lee etal. (2005). Behav. Neurosci. 119, 342–345. doi:10.1037/0735-7044.119.1.342
Hasselmo, M. E., Hay, J., Ilyn, M., and Gorchetchnikov, A. (2002). Neuromodulation, theta rhythm and rat spatial navigation. Neural Netw. 15, 689–707. doi:10.1016/S0893-6080(02)00057-6
Hirsh, R. (1974). The hippocampus and contextual retrieval of information from memory: a theory. Behav. Biol. 12, 421–444. doi:10.1016/S0091-6773(74)92231-7
Hoogenboom, N., Schoffelen, J. M., Oostenveld, R., Parkes, L. M., and Fries, P. (2006). Localizing human visual gamma-band activity in frequency, time and space. Neuroimage 29, 764–773. doi:10.1016/j.neuroimage.2005.08.043
Humphries, M. D., and Prescott, T. J. (2010). The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Prog. Neurobiol. 90, 385–417. doi:10.1016/j.pneurobio.2009.11.003
Hyman, J. M., Zilli, E. A., Paley, A. M., and Hasselmo, M. E. (2005). Medial prefrontal cortex cells show dynamic modulation with the hippocampal theta rhythm dependent on behavior. Hippocampus 15, 739–749. doi:10.1002/hipo.20106
Jeffery, K. J., Anderson, M. I., Hayman, R., and Chakraborty, S. (2004). A proposed architecture for the neural representation of spatial context. Neurosci. Biobehav. Rev 28, 201–218. doi:10.1016/j.neubiorev.2003.12.002
Jenkins, T. A., Amin, E., Pearce, J. M., Brown, M. W., and Aggleton, J. P. (2004). Novel spatial arrangements of familiar visual stimuli promote activity in the rat hippocampal formation but not the parahippocampal cortices: a c-fos expression study. Neuroscience 124, 43–52. doi:10.1016/j.neuroscience.2003.11.024
Jo, Y. S., Lee, J., and Mizumori, S. J. Y. (2013). Effects of prefrontal cortical inactivation on neural activity in the ventral tegmental area. J. Neurosci. 33, 8159–8171. doi:10.1523/JNEUROSCI.0118-13.2013
Jones, M. W., and Wilson, M. A. (2005). Theta rhythms coordinate hippocampal-prefrontal interactions in a spatial memory task. PLoS Biol. 3:e402. doi:10.1371/journal.pbio.0030402
Kalenscher, T., Lansink, C. S., Lankelma, J. V., and Pennartz, C. M. (2010). Reward-associated gamma oscillations in ventral striatum are regionally differentiated and modulate local firing activity. J. Neurophysiol. 103, 1658–1672. doi:10.1152/jn.00432.2009
Kim, J. J., and Fanselow, M. S. (1992). Modality-specific retrograde amnesia of fear. Science 256, 675–677. doi:10.1126/science.1585183
Knierim, J. J. (2002). Dynamic interactions between local surface cues, distal landmarks, and intrinsic circuitry in hippocampal place cells. J. Neurosci. 22, 6254–6264.
Knierim, J. J., Lee, I., and Hargreaves, E. L. (2006). Hippocampal place cells: parallel input streams, subregional processing and implications for episodic memory. Hippocampus 16, 755–764. doi:10.1002/hipo.20203
Kraus, B. J., Robinson, R. J. II, White, J. A., Eichenbaum, H., and Hasselmo, M. E. (2013). Hippocampal “time cells”: time versus path integration. Neuron 78, 1–12. doi:10.1016/j.neuron.2013.04.015
Kropf, W., and Kuschinsky, K. (1993). Conditioned effects of apomorphine are manifest in regional EEG of rats both in hippocampus and in striatum. Naunyn Schmiedebergs Arch. Pharmacol. 347, 487–493. doi:10.1007/BF00166740
Kuhl, B. A., Shah, A. T., DuBrow, S., and Wagner, A. D. (2010). Resistance to forgetting associated with hippocampus-mediated reactivation during new learning. Nat. Neurosci. 13, 501–506. doi:10.1038/nn.2498
Kumaran, D., and Maguire, E. A. (2007). Match-mismatch processes underlie human hippocampal responses to associative novelty. J. Neurosci. 27, 8517–8524. doi:10.1523/JNEUROSCI.1677-07.2007
Lee, I., Yognarasimha, D., Rao, G., and Knierim, J. J. (2004). Comparison of population coherence of place cells in hippocampal subfields CA1 and CA3. Nature 430, 456–459. doi:10.1038/nature02739
Leung, L. S., and Yim, C. Y. (1993). Rhythmic delta-frequency activities in the nucleus accumbens of anesthetized and freely moving rats. Can. J. Physiol. Pharmacol. 71, 311–320. doi:10.1139/y93-049
Leutgeb, J. K., Leutgeb, S., Treves, A., Meyer, R., Barnes, C. A., McNaughton, B. L., et al. (2005a). Progressive transformation of hippocampal neuronal representations in “morphed” environments. Neuron 48, 345–358. doi:10.1016/j.neuron.2005.09.007
Leutgeb, S., Leutgeb, J. K., Barnes, C. A., Moser, E. I., McNaughton, B. L., and Moser, M. B. (2005b). Independent codes for spatial and episodic memory in hippocampal neuronal ensembles. Science 309, 619–623. doi:10.1126/science.1114037
Lisman, J. E., and Grace, A. A. (2005). The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron 46, 703–713. doi:10.1016/j.neuron.2005.05.002
Lisman, J. E., and Otmakhova, N. A. (2001). Storage, recall, and novelty detection of sequences by the hippocampus: elaborating on the SOCRATIC model to account for normal and aberrant effects of dopamine. Hippocampus 11, 551–568. doi:10.1002/hipo.1071
Lisman, J. E., and Redish, A. D. (2009). Prediction, sequences and the hippocampus. Philos. Trans R. Soc. Lond. B Biol. Sci. 364, 1193–1201. doi:10.1098/rstb.2008.0316
Llinas, R. R., and Roy, S. (2009). The ‘predictive imperative’ as the basis for self-awareness. Phil. Trans. R. Soc. Lond. B Biol. Sci. 364, 1301–1307. doi:10.1098/rstb.2008.0309
McDonald, C. J., Lepage, K. Q., Eden, U. T., and Eichenbaum, H. (2011). Hippocampal “time cells” bridge the gap in memory for discontiguous events. Neuron 71, 737–749. doi:10.1016/j.neuron.2011.07.012
Manns, J. R., Howard, M. W., and Eichenbaum, H. (2007a). Gradual changes in hippocampal activity support remembering the order of events. Neuron 56, 530–540. doi:10.1016/j.neuron.2007.08.017
Manns, J. R., Zilli, E. A., Ong, K. C., Hasselmo, M. E., and Eichenbaum, H. (2007b). Hippocampal CA1 spiking during encoding and retrieval: Relation to theta phase. Neurobiol. Learn. Mem. 87, 9–20. doi:10.1016/j.nlm.2006.05.007
Marder, E., and Goaillard, J. M. (2006). Variability, compensation and homeostasis in neuron and network function. Nat. Rev. Neurosci. 7, 563–574. doi:10.1038/nrn1949
Marder, E., and Prinz, A. A. (2003). Current compensation in neuronal homeostasis. Neuron 37, 2–4. doi:10.1016/S0896-6273(02)01173-X
Maren, S. (2001). Neurobiology of Pavlovian fear conditioning. Ann. Rev. Neurosci. 24, 897–931. doi:10.1146/annurev.neuro.24.1.897
Martig, A. K., and Mizumori, S. J. (2011). Ventral tegmental area disruption selectively affects CA1/CA2 but not CA3 place fields during a differential reward working memory task. Hippocampus 21, 172–184. doi:10.1002/hipo.20734
Mauk, M. D., and Buonomano, D. V. (2004). The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340. doi:10.1146/annurev.neuro.27.070203.144247
McEchron, M. C., and Disterhoft, J. F. (1999). Hippocampal encoding of non-spatial trace conditioning. Hippocampus 9, 385–396. doi:10.1002/(SICI)1098-1063(1999)9:4<385::AID-HIPO5>3.0.CO;2-K
McNaughton, B. L., Battaglia, F. P., Jensen, O., Moser, E. I., and Moser, M.-B. (2006). Path integration and the neural basis of the ‘cognitive map’. Nat. Rev. Neurosci. 7, 663–678. doi:10.1038/nrn1932
Mizumori, S. J. Y., Cooper, B. G., Leutgeb, S., and Pratt, W. E. (2000). A neural systems analysis of adaptive navigation. Mol. Neurobiol. 21, 57–82. doi:10.1385/MN:21:1-2:057
Mizumori, S. J. Y. (ed.) (2008). A Context for Hippocampal Place Cells During Learning. New York: Oxford University Press.
Mizumori, S. J. Y., and Jo, Y. S. (2013). Homeostatic regulation of memory systems and adaptive decisions. Hippocampus. doi:10.1002/hipo.22176
Mizumori, S. J. Y., Ragozzino, K. E., Cooper, B. G., and Leutgeb, S. (1999). Hippocampal representational organization and spatial context. Hippocampus 9, 444–451. doi:10.1002/(SICI)1098-1063(1999)9:4<444::AID-HIPO10>3.3.CO;2-Q
Mizumori, S. J. Y., Smith, D. M., and Puryear, C. B. (2007a). Hippocampal and neocortical interactions during context discrimination: electrophysiological evidence from the rat. Hippocampus 17, 851–862. doi:10.1002/hipo.20317
Mizumori, S. J. Y., Smith, D. M., and Puryear, C. B. (2007b). “Mnemonic contributions of hippocampal place cells,” in Neurobiology of Learning and Memory, eds J. L. Martinez Jr. and R. P. Kesner (New York: Academic Press), 155–190.
Mizumori, S. J. Y., Yeshenko, O., Gill, K. M., and Davis, D. M. (2004). Parallel processing across neural systems: implications for a multiple memory system hypothesis. Neurobiol. Learn. Mem. 82, 278–298. doi:10.1016/j.nlm.2004.07.007
Moita, M. A., Rosis, S., Zhou, Y., LeDoux, J. E., and Blair, H. T. (2004). Putting fear in its place: Remapping of hippocampal place cells during fear conditioning. J. Neurosci. 24, 7015–7023. doi:10.1523/JNEUROSCI.5492-03.2004
Monaco, J. D., Knierim, J. J., and Zhang, K. (2011). Sensory feedback, error correction, and remapping in a multiple oscillator model of place-cell activity. Front. Comput. Neurosci. 5:1–17. doi:10.3389/fncom.2011.00039
Muller, R. U., and Kubie, J. L. (1987). The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells. J. Neurosci. 7, 1951–1968.
Myers, C. E., and Gluck, M. (1994). Context, conditioning, and hippocampal representation in animal learning. Behav. Neurosci. 108, 835–847. doi:10.1037/0735-7044.108.5.835
Nadel, L. (2008). “The hippocampus and context revisited,” in Hippocampal Place Fields: Relevance to Learning and Memory, ed. S. J. Y. Mizumori (New York: Oxford University Press), 3–15.
Nadel, L., and Payne, J. D. (2002). “The hippocampus, way finding and episodic memory,” in The Neural Basis of Navigation: Evidence from Single Cell Recording, ed. P. E. Sharp (Norwell, MA: Kluwer Academic Publisher), 235–248.
Nadel, L., Willner, J., and Kurz, E. M. (1985). “Cognitive maps and environmental context,” in Context and Learning, eds P. Balsam, and A. Tomie (Hillside, NJ: Erlbaum), 385–406.
Nadel, L., and Wilner, J. (1980). Context and conditioning: a place for space. Physiol. Psychol. 8, 218–228.
Nudo, R. J. (2011). Neural bases of recovery after brain injury. J. Commun. Disord. 44, 515–520. doi:10.1016/j.jcomdis.2011.04.004
O’Keefe, J. (1976). Place units in the hippocampus of the freely moving rat. Exp. Neurol. 51, 78–109. doi:10.1016/0014-4886(76)90055-8
O’Keefe, J., and Burgess, N. (1996). Geometric determinants of the place fields of hippocampal neurons. Nature 381, 425–428. doi:10.1038/381425a0
O’Keefe, J., and Conway, D. H. (1978). Hippocampal place units in the freely moving rat: why they fire where they fire. Exp. Brain Res. 31, 573–590. doi:10.1007/BF00239813
O’Keefe, J., and Dostrovsky, J. (1971). The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Res. 34, 171–175. doi:10.1016/0006-8993(71)90358-1
O’Keefe, J., and Nadel, L. (1978). The Hippocampus as a Cognitive Map. Oxford: Oxford University Press.
O’Keefe, J., and Recce, M. L. (1993). Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus 3, 317–330. doi:10.1002/hipo.450030307
Olton, D. S., Branch, M., and Best, P. J. (1978). Spatial correlates of hippocampal unit activity. Exp. Neurol. 58, 387–409. doi:10.1016/0014-4886(78)90096-1
O’Reilly, R. C., and Rudy, J. W. (2001). Conjunctive representations in learning and memory: Principles of cortical and hippocampal function. Psychol. Rev. 108, 311–345. doi:10.1037/0033-295X.108.2.311
Pastalkova, E., Itskov, V., Amarasingham, A., and Buzsaki, G. (2008). Internally generated cell assembly sequences in the rat hippocampus. Science 321, 1322–1327. doi:10.1126/science.1159775
Paulsen, O., and Moser, E. I. (1998). A model of hippocampal memory encoding and retrieval: GABAergic control of synaptic plasticity. Trends Neurosci. 21, 273–278. doi:10.1016/S0166-2236(97)01205-8
Penick, S., and Solomon, P. R. (1991). Hippocampus, context, and conditioning. Behav. Neurosci. 105, 611–617. doi:10.1037/0735-7044.105.5.611
Penner, M. R., and Mizumori, S. J. Y. (2012a). Age-associated changes in the hippocampal-ventral striatum-ventral tegmental loop that impact learning, prediction and context discrimination. Front. Aging Neurosci. 4:1–12. doi:10.3389/fnagi.2012.00022
Penner, M. R., and Mizumori, S. J. Y. (2012b). Neural systems analysis of decision making during goal-directed navigation. Prog. Neurobiol. 96, 96–135. doi:10.1016/j.pneurobio.2011.08.010
Phillips, R. G., and LeDoux, J. E. (1992). Differential contribution of amygdala and hippocampus to cued and contextual fear conditioning. Behav. Neurosci. 106, 274–285. doi:10.1037/0735-7044.106.2.274
Phillips, R. G., and LeDoux, J. E. (1994). Lesions of the dorsal hippocampal formation interfere with background but not foreground contextual fear conditioning. Learn Mem. 1, 34–44. doi:10.1101/lm.1.1.34
Pilly, P. K., and Grossberg, S. (2012). How do spatial learning and memory occur in the brain? Coordinated learning of entorhinal grid cells and hippocampal place cells. J. Cogn. Neurosci. 24, 1031–1054. doi:10.1162/jocn_a_00200
Puryear, C. B., Kim, M. J., and Mizumori, S. J. Y. (2010). Conjunctive encoding of movement and reward by ventral tegmental area neurons in the freely navigating rodent. Behav. Neurosci. 124, 234–247. doi:10.1037/a0018865
Puryear, C. B., King, M., and Mizumori, S. J. Y. (2006). Specific changes in hippocampal spatial codes predict spatial working memory performance. Behav. Brain Res. 169, 168–175. doi:10.1016/j.bbr.2005.12.007
Ranck, J. B. Jr. (1973). Studies on single neurons in dorsal hippocampal formation and septum in unrestrained rats. I. Behavioral correlates and firing repertoires. Exp. Neurol. 41, 461–531. doi:10.1016/0014-4886(73)90290-2
Robinson, T. E., and Kolb, B. (2004). Structural plasticity associated with exposure to drugs of abuse. Neuropharmacology 47(Suppl. 1), 33–46. doi:10.1016/j.neuropharm.2004.06.025
Sabatino, M., Ferraro, G., Liberti, G., Vella, N., and La Grutta, V. (1985). Striatal and septal influence on hippocampal theta and spikes in the cat. Neurosci. Lett. 61, 55–59. doi:10.1016/0304-3940(85)90400-8
Samsonovich, A., and McNaughton, B. L. (1997). Path integration and cognitive mapping in a continuous attractor neural network model. J. Neurosci. 17, 5900–5920.
Save, E., Buhot, M. C., Foreman, N., and Thinus-Blanc, C. (1992a). Exploratory activity and response to a spatial change in rats with hippocampal or posterior parietal cortical lesions. Behav. Brain Res. 47, 113–127. doi:10.1016/S0166-4328(05)80118-4
Save, E., Poucet, B., Foreman, N., and Buhot, M. C. (1992b). Object exploration and reactions to spatial and nonspatial changes in hooded rats following damage to parietal cortex or hippocampal formation. Behav. Neurosci. 106, 447–456. doi:10.1037/0735-7044.106.3.447
Save, E., Nerad, L., and Poucet, B. (2000). Contribution of multiple sensory information to place field stability in hippocampal place cells. Hippocampus 10, 64–76. doi:10.1002/(SICI)1098-1063(2000)10:1<64::AID-HIPO7>3.0.CO;2-Y
Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593–1599. doi:10.1126/science.275.5306.1593
Shetty, P. K., Galeffi, F., and Turner, D. A. (2012). Cellular links between neuronal activity and energy homeostasis. Front. Pharmacol. 3:1–14. doi:10.3389/fphar.2012.00043
Siapas, A. G., Lubenov, E. V., and Wilson, M. A. (2005). Prefrontal phase locking to hippocampal theta oscillations. Neuron 46, 141–151. doi:10.1016/j.neuron.2005.02.028
Smith, D. M., and Mizumori, S. J. (2006a). Hippocampal place cells, context, and episodic memory. Hippocampus 16, 716–729. doi:10.1002/hipo.20208
Smith, D. M., and Mizumori, S. J. (2006b). Learning-related development of context-specific neuronal responses to places and events: the hippocampal role in context processing. J. Neurosci. 26, 3154–3163. doi:10.1523/JNEUROSCI.3234-05.2006
Smith, D. M., Wakeman, D., Patel, J., and Gabriel, M. (2004). Fornix lesions impair context-related cingulothalamic neuronal patterns and concurrent discrimination learning. Behav. Neurosci. 118, 1225–1239. doi:10.1037/0735-7044.118.6.1225
Tabuchi, E. T., Mulder, A. B., and Wiener, S. I. (2000). Position and behavioral modulation of synchronization of hippocampal and accumbens neuronal discharges in freely moving rats. Hippocampus 10, 717–728. doi:10.1002/1098-1063(2000)10:6<717::AID-HIPO1009>3.0.CO;2-3
Tanila, H., Shapiro, M. K., and Eichenbaum, H. (1997). Discordance of spatial representation in ensembles of hippocampal place cells. Hippocampus 7, 613–623. doi:10.1002/(SICI)1098-1063(1997)7:6<613::AID-HIPO4>3.3.CO;2-#
Tse, D., Langston, R. F., Kakeyama, M., Bethus, I., Spooner, P. A., Wood, E. R., et al. (2007). Schemas and memory consolidation. Science 316, 76–82. doi:10.1126/science.1135935
Tulving, E. (2002). Episodic memory: from mind to brain. Ann. Rev. Psychol. 53, 1–25. doi:10.1146/annurev.psych.53.100901.135114
Turrigiano, G. G. (1999). Homeostatic plasticity in neuronal networks: the more things change, the more they stay the same. Trends Neurosci. 22, 221–227. doi:10.1016/S0166-2236(98)01341-1
Turrigiano, G. G. (2008). The self-tuning neuron: synaptic scaling of excitatory synapses. Cell 135, 422–435. doi:10.1016/j.cell.2008.10.008
Turrigiano, G. G. (2011). Homeostatic synaptic plasticity: Local and global mechanisms for stabilizing neuronal function. Cold Spring Harb. Perspect. Biol. 4, a005736. doi:10.1101/cshperspect.a005736
Turrigiano, G. G., Leslie, K. R., Desai, N. S., Rutherford, L. C., and Nelson, S. B. (1998). Activity-dependent scaling of quantal amplitude in neocortical neurons. Nature 391, 892–896. doi:10.1038/36103
Turrigiano, G. G., and Nelson, S. B. (2004). Homeostatic plasticity in the developing nervous system. Nat. Rev. Neurosci 5, 97–107. doi:10.1038/nrn1327
van der Meer, M. A., and Redish, A. D. (2009). Low and high gamma oscillations in rat ventral striatum have distinct relationships to behavior, reward, and spiking activity on a learned spatial decision task. Front. Integr. Neurosci. 3:9. doi:10.3389/neuro.07.009.2009
Vanderwolf, C. H. (1969). Hippocampal electrical activity and voluntary movement in the rat. Electroencephalogr. Clin. Neurophysiol. 26, 407–418. doi:10.1016/0013-4694(69)90092-3
Varela, F., Lachaux, J. P., Rodriguez, E., and Martinerie, J. (2001). The brain web: phase synchronization and large-scale integration. Nat. Rev. Neurosci. 2, 229–239. doi:10.1038/35067550
Vida, I., Bartos, M., and Jonas, P. (2006). Shunting inhibition improves robustness of gamma oscillations in hippocampal interneuron networks by homogenizing firing rates. Neuron 49, 107–117. doi:10.1016/j.neuron.2005.11.036
Vinogradova, O. S. (1995). Expression, control, and probably functional significance of the neuronal theta-rhythm. Prog. Neurobiol. 45, 523–583. doi:10.1016/0301-0082(94)00051-I
Whittington, M. A., Traub, R. D., and Jefferys, J. G. (1995). Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature 373, 612–615. doi:10.1038/373612a0
Wiener, S. I. (1996). Spatial, behavioral and sensory correlates of hippocampal CA1 complex spike cell activity: implications for information processing functions. Prog. Neurobiol. 49, 335–361. doi:10.1016/0301-0082(96)00019-6
Wikenheiser, A. M., and Redish, A. D. (2011). Changes in reward contingency modulate the trial-to-trial variability of hippocampal place cells. J. Neurophysiol. 106, 589–598. doi:10.1152/jn.00091.2011
Womelsdorf, T., Fries, P., Mitra, P. P., and Desimone, R. (2006). Gamma-band synchronization in visual cortex predicts speed of change detection. Nature 439, 733–736. doi:10.1038/nature04258
Womelsdorf, T., Schoffelen, J. M., Oostenveld, R., Singer, W., Desimone, R., Engel, A. K., et al. (2007). Modulation of neuronal interactions through neuronal synchronization. Science 316, 1609–1612. doi:10.1126/science.1139597
Wood, E. R., Dudchenko, P. A., and Eichenbaum, H. (1999). The global record of memory in hippocampal neural activity. Nature 397, 613–616. doi:10.1038/17605
Wood, E. R., Dudchenko, P. A., Robitsek, R. J., and Eichenbaum, H. (2000). Hippocampal neurons encode information about different types of memory episodes occurring in the same location. Neuron 27, 623–633. doi:10.1016/S0896-6273(00)00071-4
Yeshenko, O., Guazzelli, A., and Mizumori, S. J. (2004). Context-dependent reorganization of spatial and movement representations by simultaneously recorded hippocampal and striatal neurons during performance of allocentric and egocentric tasks. Behav. Neurosci. 118, 751–769. doi:10.1037/0735-7044.118.4.751
Keywords: hippocampus, prediction errors, prefrontal cortex, dopamine, striatum, memory, decision making
Citation: Mizumori SJY (2013) Context prediction analysis and episodic memory. Front. Behav. Neurosci. 7:132. doi: 10.3389/fnbeh.2013.00132
Received: 15 July 2013; Accepted: 11 September 2013;
Published online: 07 October 2013.
Edited by:
Ekrem Dere, University Pierre and Marie Curie Paris 6, FranceReviewed by:
J. Wayne Aldridge, University of Michigan, USAHoward Eichenbaum, Boston University, USA
Copyright: © 2013 Mizumori. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Sheri J. Y. Mizumori, Laboratory of Neural Systems, Decision Science, Learning and Memory, Department of Psychology, University of Washington, Box 351525, Seattle, WA 98195-1525, USA e-mail: mizumori@uw.edu