Recent evidence obtained in rats suggests that hippocampal neurons become sequentially activated for brief time intervals that are time-locked to task-relevant events while the animals perform reinforcement learning tasks. Similar sequentially activated cells have also been found in the medial prefrontal cortex. It has been suggested that these activity patterns may correspond to goal representations. Recently, a different function has been proposed for these cell assemblies. According to this proposal, these neurons may be viewed as “time encoding” neurons that provide the neural substrate for temporal order memory. This theory was motivated by the observation that hippocampal neurons become sequentially activated during the trace period of an object-odor paired associate task. Here it is pointed out that these neurons also find a functional interpretation within a theory that views such activity patterns as signatures of goal representations. New experiments are proposed to test this theory.
Evidence suggests that hippocampal neurons become activated in a temporal sequence during the 10s-long trace period separating the object stimulus and the odor in a version of Kesner et al.’s (2005) object-odor paired associate task (MacDonald and Eichenbaum, 2009 ). In this task, rats are presented with one of two objects (A or B) for a brief time period. Then, they spend a trace period that lasts ten seconds. After the trace period the rats are presented with a sand cup that contains one of two odors (1 or 2). The sand cup contains a food reward for object-odor pairs A1 and B2, but not for A2 or B1 (Kesner et al., 2005 ). Two theories exist concerning the functional role of the hippocampal neurons that are activated in a temporal sequence during the trace period of this task. One theory suggests that these neurons may be viewed as “time encoding” neurons that provide the neural substrate for temporal order memory (Rolls, 2010 ). Another theory views these cells as an integral component of a goal representation network. Here the latter theory is explained in detail and new experiments are proposed to test it.
According to this theory, sequentially activated hippocampal neurons are the hippocampal branch of a goal representation network that involves the coordinated activity of prefrontal, limbic, basal ganglia and midbrain circuits. The theory proposes that, during reinforcement learning, hippocampal neurons receive inputs from prefrontal goal representations, and that they project information about the reward-predictive value of these goals back to the prefrontal goal representations to mediate a reward-based action selection mechanism (Okatan, 2007a ,b , 2009 ). The theory suggests that, as a result of this prefrontal-hippocampal communication, different cell assemblies are activated in the hippocampus during trips to different goals (Okatan, 2009 ). Using a statistical neural model, the theory argues that, in a task where a cue presentation signals one of N goals, the presentation of the cue triggers N distinct and simultaneous sets of cue-selective cells in the hippocampus, each set being driven by a distinct prefrontal goal representation and competing to become a set of winner cells (Okatan, 2009 ). The winner cells probabilistically influence the subsequent choice behavior of the animal, and the activity of the hippocampal winner cells acquires the reward-predictive value of the associated prefrontal goal representation through training by the mesolimbic dopaminergic input, which serves as a reward-prediction error signal (Okatan, 2007a ,b , 2009 ).
For Kesner et al.’s (2005) task, this theory proposes the following: (i) the presentation of one of the objects A or B on a given trial launches object-specific sequentially activated hippocampal cell assemblies, (ii) with learning, the cell assembly that reliably gets rewarded becomes strengthened such that it reliably wins competitive interactions and dominates other cell assemblies, (iii) this strengthening brings with it odor selectivity, which manifests itself first near the time of reward delivery and, with learning, propagates backward in time toward the cue presentation. Thus, the theory proposes that the cells of such winner hippocampal cell assemblies have object and odor selectivity, and influence the animal’s upcoming response. The results of MacDonald and Eichenbaum (2009) provide support for some of these proposals. In addition, the theory suggests that the activity of cue and response selective cells is modulated by the magnitude of the reward expected or received by the animal (Okatan, 2007a ,b , 2009 ). Preliminary evidence supporting this has been found in a conditional spatial goal selection task on a T-maze, where the magnitude of the reward delivered to the rats was changed within each session (Okatan et al., 2009a ,b ).
In this theory sequentially activated cell assemblies are viewed as signatures of how goals are maintained in the working memory (Fujisawa et al., 2008 ; Pastalkova et al., 2008 ; Okatan, 2009 ). This hypothesis may be tested using a modified version of Kesner et al.’s (2005) task. The structure of the new task would be identical to the original task, except that on some randomly selected trials, x seconds after the onset of the trace period, a light would be turned on, where x, 0 < x < 10 s, would be random. Here, the light signal acts as a “game changer” that indicates that the trial will be aborted at the end of the usual 10-s trace period (i.e. the response period will be skipped), and that a new trial will begin by randomly selecting A or B as the object to be presented. The theory suggests that as soon as the light is turned on, the cell assemblies that are active in the hippocampus and the prefrontal cortex will disappear until the animal’s working memory is loaded with information about the next goal, such as the information provided at the next object presentation. Thus, the theory proposes that the cells whose “episode field” (Pastalkova et al., 2008 ) corresponds to a time point t, x < t < 10 s, will or will not fire depending on whether the light was left off or was turned on, respectively. In a variant of this experiment, the light may predict a switch in the pending reward contingency. In this version of the experiment, turning the light on would be expected to dissolve the cell assembly that corresponds to the now obsolete goal and maintain the cell assembly associated with the newly confirmed goal.
In sum, this theory views the sequentially activated hippocampal cells as the hippocampal component of goal representations within a prefrontal-limbic-basal ganglia-midbrain network for learning in which goals are represented by neural activity patterns that self-organize under the associative modulation mediated by the mesocortical, mesolimbic and mesostriatal dopaminergic input to these brain areas. In this theory, goal representations are neural codes that sit at the top of an information hierarchy in which rapidly formed hippocampal memory traces serve as context-dependent reward-predictive values of the goal representations during reinforcement learning. These theories may help design new experiments to further characterize the functional role played by sequentially activated cell assemblies within and outside the hippocampus.
References
Fujisawa, S., Amarasingham, A., Harrison, M. T., and Buzsáki, G. (2008). Behavior-dependent short-term assembly dynamics in the medial prefrontal cortex. Nat. Neurosci. 11, 823–833.
Kesner, R. P., Hunsaker, M. R., and Gilbert, P. E. (2005). The role of CA1 in the acquisition of an object–trace–odor paired associate task. Behav. Neurosci. 119, 781–786.
MacDonald, C., and Eichenbaum, H. (2009). Hippocampal Neurons Disambiguate Overlapping Sequences of Non-Spatial Events. 101.21. 2009 Neuroscience Meeting Planner. Chicago, IL: Society for Neuroscience (Online).
Okatan, M. (2007a). What does learning-related hippocampal neural activity tell us about hippocampal information processing? Neuroanatomy 6(Suppl. 1), 6. Abstracts, 6th National Congress of Neuroscience, Safranbolu/Karabük.
Okatan, M. (2007b). A Joint Probability Model for Learning-Related Neural Activity and Behavioral Responses that Accounts for Learning and Executive control: An Application of the Temporal Difference Learning Algorithm. 2007 New England Statistics Symposium. Storrs, CT: University of Connecticut.
Okatan, M. (2009). Correlates of reward-predictive value in learning-related hippocampal neural activity. Hippocampus 19, 487–506.
Okatan, M., Owens, C. B., Komorowski, R., and Eichenbaum, H. (2009a). Correlates of reward and non-reward information signaled by hippocampal CA1 neurons during reinforcement learning. Turk. J. Neurol. 15, 139. Abstracts, 8th National Congress of Neuroscience, Abant.
Okatan, M., Owens, C. B., and Eichenbaum, H. (2009b). Reward-Related Prospective and Retrospective Spiking Activity in Hippocampal Neurons During Reinforcement Learning. 101.16. 2009 Neuroscience Meeting Planner. Chicago, IL: Society for Neuroscience (Online).
Pastalkova, E., Itskov, V., Amarasingham, A., and Buzsáki G. (2008). Internally generated cell assembly sequences in the rat hippocampus. Science 321, 1322–1327.
Citation: Okatan M (2010) Hippocampal cell assemblies: time encoding neurons or goal representations? Front. Neural Circuits 4:17. doi: 10.3389/fncir.2010.00017
Received: 13 April 2010;
Accepted: 02 June 2010;
Published online: 23 June 2010
Copyright: © 2010 Okatan. This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: okatan@bu.edu