Closing the Loop Around Neural Systems

Discovering the structure underlying observed data is a recurring problem in machine learning with important applications in neuroscience. It is also a primary function of the brain. When data can be actively collected in the context of a closed action-perception loop, behavior becomes a critical determinant of learning efficiency. Psychologists studying exploration and curiosity in humans and animals have long argued that learning itself is a primary motivator of behavior. However, the theoretical basis of learning-driven behavior is not well understood. Previous computational studies of behavior have largely focused on the control problem of maximizing acquisition of rewards and have treated learning the structure of data as a secondary objective. Here, we study exploration in the absence of external reward feedback. Instead, we take the quality of an agent's learned internal model to be the primary objective. In a simple probabilistic framework, we derive a Bayesian estimate for the amount of information about the environment an agent can expect to receive by taking an action, a measure we term the predicted information gain (PIG). We develop exploration strategies that approximately maximize PIG. One strategy based on value-iteration consistently learns faster than previously developed reward-free exploration strategies across a diverse range of environments. Psychologists believe the evolutionary advantage of learning-driven exploration lies in the generalized utility of an accurate internal model. Consistent with this hypothesis, we demonstrate that agents which learn more efficiently during exploration are later better able to accomplish a range of goal-directed tasks. We will conclude by discussing how our work elucidates the explorative behaviors of animals and humans, its relationship to other computational models of behavior, and its potential application to experimental design, such as in closed-loop neurophysiology studies.

Brain-machine interfaces (BMIs) that can precisely monitor and control neural activity will likely require new hardware with improved resolution and specificity. New nanofabricated electrodes with feature sizes and densities comparable to neural circuits may lead to such improvements. In this perspective, we review the recent development of vertical nanowire (NW) electrodes that could provide highly parallel single-cell recording and stimulation for future BMIs. We compare the advantages of these devices and discuss some of the technical challenges that must be overcome for this technology to become a platform for next-generation closed-loop BMIs.

Defining the connections among neurons is critical to our understanding of the structure and function of the nervous system. Recombinant viruses engineered to transmit across synapses provide a powerful approach for the dissection of neuronal circuitry in vivo. We recently demonstrated that recombinant vesicular stomatitis virus (VSV) can be endowed with anterograde or retrograde transsynaptic tracing ability by providing the virus with different glycoproteins. Here we extend the characterization of the transmission and gene expression of recombinant VSV (rVSV) with the rabies virus glycoprotein (RABV-G), and provide examples of its activity relative to the anterograde transsynaptic tracer form of rVSV. rVSV with RABV-G was found to drive strong expression of transgenes and to spread rapidly from neuron to neuron in only a retrograde manner. Depending upon how the RABV-G was delivered, VSV served as a polysynaptic or monosynaptic tracer, or was able to define projections through axonal uptake and retrograde transport. In animals co-infected with rVSV in its anterograde form, rVSV with RABV-G could be used to begin to characterize the similarities and differences in connections to different areas. rVSV with RABV-G provides a flexible, rapid, and versatile tracing tool that complements the previously described VSV-based anterograde transsynaptic tracer.