Impact Factor 3.877

The world's most-cited Neurosciences journals

Review ARTICLE

Front. Neurosci., 12 July 2016 | https://doi.org/10.3389/fnins.2016.00312

A Review of Control Strategies in Closed-Loop Neuroprosthetic Systems

  • 1Biomedical Engineering and Neuroscience, The MARCS Institute, University of Western Sydney, Sydney, NSW, Australia
  • 2School of Medicine, University of Western Sydney, Sydney, NSW, Australia
  • 3Neuroscience Research Australia, Sydney, NSW, Australia

It has been widely recognized that closed-loop neuroprosthetic systems achieve more favorable outcomes for users then equivalent open-loop devices. Improved performance of tasks, better usability, and greater embodiment have all been reported in systems utilizing some form of feedback. However, the interdisciplinary work on neuroprosthetic systems can lead to miscommunication due to similarities in well-established nomenclature in different fields. Here we present a review of control strategies in existing experimental, investigational and clinical neuroprosthetic systems in order to establish a baseline and promote a common understanding of different feedback modes and closed-loop controllers. The first section provides a brief discussion of feedback control and control theory. The second section reviews the control strategies of recent Brain Machine Interfaces, neuromodulatory implants, neuroprosthetic systems, and assistive neurorobotic devices. The final section examines the different approaches to feedback in current neuroprosthetic and neurorobotic systems.

Introduction

A neuroprosthetic is a device or system that has an interface with the nervous system and supplements or substitutes functionality in the patient's body. For the purpose of this review we have included neuromodulatory systems and brain computer interfaces under the general description of neuroprosthetics, as well as rehabilitation systems such as exoskeletons. The key identifying characteristic of the neuroprosthetic is that it has an interface with the subject's nervous system, as distinct from an implantable devices such as an pacemaker or an insulin pump. Consequently there are a broad range of devices that we consider neuroprosthetics. To date there have been a number of reviews of neuroprosthetic systems. There is significant literature on classification algorithms and detection strategies (Schwartz, 2004; Lotte et al., 2007; Green and Kalaska, 2011; Borton et al., 2014; Morimoto and Kawato, 2015), including comparison and evaluation of the relative strengths of different approaches. However, there are fewer examinations the authors are aware of that investigate the different control approaches that have been implemented in neuroprosthetic settings. Performing such a review is made more difficult due to the small number of studies that have compared different control approaches within the same experiment, and the fact that many neuroprosthetic studies have by necessity been conducted with very small sample sizes, sometimes involving a single subject. Additionally many of the devices examined in this review are experimental or investigational, and are not yet in use in clinical or therapeutic settings (Sun and Morrell, 2014).

When considering these devices in the context of control and feedback it can be helpful to place them along a number of axes (Figure 1) to partition the large variety of systems and approaches. The first axis we have considered is the location of the interface with the nervous system, with the Central Nervous System (CNS) subdivided into the brain and the spinal cord, and then the Peripheral Nervous System (PNS), consisting of afferent and efferent pathways. Along this axis and additional distinction can be drawn between single channel systems that use a single electrode as the interface to the subject's nervous system, and multichannel systems which utilize many parallel channels for interface. Finally the channel can be unidirectional for simplex communication, or bidirectional for half-duplex, or full duplex communication.

FIGURE 1
www.frontiersin.org

Figure 1. Neuroprosthetic Systems. An illustration of the conceptual space of neuroprosthetic devices. Devices can be classified as similar if they provide assistance in the same Modality, have an equivalent level of Invasiveness, or interface with the user's nervous system in the same Location.

Another axis to consider is the invasiveness of the interface. At one end we place noninvasive interfaces such as Electroencephlogram (EEG), Magnetic Resonance Imaging (MRI), and Electromyography (EMG). These interfaces are hampered by a variety of issues, including low spatial resolution and low signal to noise ratios. The advantage they enjoy is that they can be deployed with little risk, which has meant they are a popular platform for neuroprosthetic development. Next are minimally invasive interfaces such as microwire EMG and microneurographic recordings from the PNS. With a more invasive interface there are better quality signals, but still minimal intervention with the subject's body, reducing risk of complications. These interfaces are susceptible to movement, leading to changes in the quality of the recorded signal. The most invasive interfaces require surgical implantation, and come with risks associated with surgery as well as risk of damage to the part of the nervous system. Within this category a distinction can be drawn between the less invasive, such as cuff electrodes around PNS sites and Electrocorticogram (ECog)—these interfaces do not disrupt the blood brain barrier; and the more invasive Multi Electrode Arrays (MEA) in PNS nerves, motor or somatosensory cortex, Deep Brain Stimulators (DBS) to treat Parkinson's Disease with electrodes in the Globus Pallidus, Sub Thalamic Nucleus, or the Pedunculopontine Nucleus. These multichannel interfaces offer high resolution, but comparatively small spatial coverage (Krook-Magnuson et al., 2015). These electrodes come with risks that are still not fully understood. These primarily involve the physical trauma due to the insertion of the electrode, with effects occurring over different timescales. Shortly after insertion there is bleeding and swelling, as well as physical damage to neurons (Fernandez et al., 2014). Over longer timescales the presence of the recording device compromises the blood-brain barrier, allowing ingress of cellular and molecular components from elsewhere in the body (Schwartz, 2004). Additionally the long term stability of the recording site is often compromised, due to immune and mechanical actions on the microelectrodes (Krook-Magnuson et al., 2015). At the furthest extent of this axis are the interfaces that reconfigure the subject's nervous system. Targeted Muscle Reinnervation (TMR) surgically rewires an amputee subject's PNS, by deinnervating muscles that have no biomechanical role after the amputation and redirecting the preserved nerves from the amputated limb to the deinnervated muscles (Kuiken et al., 2009), allowing for high quality EMG recordings as control inputs to a prosthetic. Optogenetic techniques offer a non-electrical interface to neurons, by using light to activate special ion-channels. This technique enables individual neurons to be targeted, which is extremely difficult with cortical microelectrodes, as well as the possibility of selectively activating a class of neurons such as excitatory neurons instead of inhibitory (Krook-Magnuson et al., 2015). However, optogentic approaches require the introduction of genes that encode for light activated ion-channels (Deisseroth, 2011), and the issues associated with this may preclude this technique in humans.

The final axis is the modality of the prosthesis. Sensory neuroprosthetics offer input from artificial sensors, as in a cochlear implant or bionic eye, or modulate sensory input as is the case in neurostimulators for treating chronic pain. Systems to treat motor impairment are frequently referred to as Brain Machine Interfaces (BMI) or Brain Computer Interfaces (BCI) and infer motor intent from the subject in order to control a virtual or physical effector. Cognitive devices modulate the activity of the CNS and include devices such as DBS stimulators for Parkinson's Disease, depression, and hippocampal stimulators for memory.

Neuroprosthetics

The following examples are representative, but not exhaustive, and are placed in Figure 1 to illustrate the wide variety of devices that can be described as having a neuroprosthetic interface. Many of these devices are experimental demonstrations, and not clinically approved interventions. Adjacent to the choice of controller, there are choices about the location and method for acquiring a suitable input signal (Grill et al., 2009; Andersen et al., 2010), and questions about the consequences of long term use of these and similar devices that have not yet been answered.

Peripheral nervous system interfaces are attractive as they are less invasive than the central nervous system alternatives, while still offering a rich source of information for neuroprosthetic control. Inmann and Haugland (2004) used a cuff electrode around the median nerve to record nerve activation due to touch and that input was used to modulate the Functional Electrical Stimulation of the subjects muscles. Horch et al. (2011) demonstrated that human subjects who were provided with intrafascicular electrical stimulation of the median and ulnar nerves derived from sensors on a myoelectric prosthetic limb were able to use the feedback to perform object recognition tasks by digital manipulation. Tan et al. (2014) used non-penetrating cuff electrodes on the median, ulnar and radial nerves of human subjects to provide natural sensations of touch while operating a myoelectric prosthetic allowing for improved performance of manual tasks. Raspopovic et al. (2014) showed that a bidirectional interface with the median and ulnar nerves could be used to provide artificial sensory feedback related to the forces exerted on a sensorized prosthetic limb, and that the artificial sensations allowed the subject to improve their ability to sense characteristics of the objects being manipulated. Targeted Muscle Reinnervation generates a rich high density signal for surface Electromyography (EMG) that enables simultaneous operation of multiple degrees of freedom in a myoelectric prosthetic limb (Kuiken et al., 2009). The tissue serves as a bioamplifier for the nerve signal, allowing a large array of surface electrodes to be deployed on the subject. The array provides a rich signal suitable for pattern recognition, and combined with a high performance prosthetic limb gives the subject an improved experience. As a sensory modality prosthetic, (Dommel et al., 2009) are testing a vision prosthesis for electrical stimulation of the retina. Spinal cord stimulation may be able to generate gait patterns suitable for locomotion in paralyzed patients. Vogelstein et al. (2008) describes the design of a system that is capable of generating primitive locomotion in anesthetized felines. Borton et al. (2014) developed an electrochemical spinal neuroprosthesis to reactivate the circuits in a damaged spinal cord, allowing hindlimb movement sufficient to enable walking in paralyzed rats.

Yanagisawa et al. (2011) used ECog electrodes placed over the sensorimotor cortex of a stroke patient in order to control a supernumerary robotic hand that was able to mimic the hand posture of the subject. Berger et al. (2011) implanted microwire electrodes in the hippocampus of rats and recorded the activity while the animals were trained to complete a memory task. Subsequent stimulation of the electrodes according to the trained model improved the performance of the rats at the cognitive task. Neuromodulators for Deep Brain Stimulation have been used to treat the symptoms of Parkinson's disease and depression (Grahn et al., 2014). Multi Electrode Arrays implanted in motor cortex have been successfully used to acquire signals for the multi degree of freedom control of robotic limbs (Hochberg et al., 2012; Wodlinger et al., 2015). Guggenmos et al. (2013) describe a Brain Machine Brain Interface in rats that utilizes microwire recordings from a premotor area to detect spiking activity leading to stimulation of the somatosensory area. The Activity Dependant Stimulation via the neuroprosthetic prototype enabled rats with injury to the motor area to recover reach and grasp behavior.

This list of devices is not exhaustive, and serves only to illustrate the different axes of Modality, Invasiveness and Location when considering neuroprosthetic designs. For a more thorough description of the available neuroprosthetic devices and interface technologies, the reader is directed to the following excellent reviews (Navarro et al., 2005; Grill et al., 2009; Micera et al., 2010; Ortiz-Catalan et al., 2012). These reviews focus on the interface techniques with the nervous system, and provide a detailed discussion of the limits of current interfaces.

Introduction to Control

The design of a neuroprosthetic varies significantly between different modalities. Because it is a multidisciplinary field often the language used to describe the system can vary between devices. The terms “closed-loop,” “feedback,” and “online” may take on different meanings. Identifying and acquiring a suitable input signal is a non-trivial task (Krook-Magnuson et al., 2015). This makes the fabrication of a substitute system in the case of impairment a complex endeavor. Developing a suitable simplified model for embodiment as an open-loop system is often a first step. But it is not always straightforward; take motor control as example—many models of muscles exist. Highly biophysical models (such as cross-bridge models) of muscles become large systems of non-linear differential equations when describing whole muscles or limbs (Ionescu and De Keyser, 2006). Similarly modeling the individual neurons that act as part of the Basal Ganglia, in a complex network of interconnections across the CNS (Broccard et al., 2014), rapidly becomes an intractable problem when developing an open-loop model for DBS to treat Parkinson's Disease. As a consequence the Basal Ganglia has often been modeled internally by a neurologist (Hosain et al., 2014), rather than explicitly within the DBS device, with stimulation parameters adjusted by the clinician observing the patient's symptoms instead of in response to a model. It is also possible that the control problem for a given neuroprosthetic has more than a single loop that needs to be considered, possibly due to the interaction of different physiological systems and different timescales (Houk, 1988). In this case control may best be achieved by a hierarchy of controllers, or a series of adaptive controllers that can be tuned at different stages of design (McFarland et al., 2006).

Nervous System Control

All parts of the human body have evolved to operate by extremely complex closed-loop control. Different subsystems, such as the cardiovascular system or the immune system operate under closed-loop control, with sensors and effectors operating at micro (LeDuc et al., 2011) and macroscopic scales (Houk, 1988). Sensory organs can be directly connected to the nervous system, such as stretch receptors, or they can be indirectly coupled by messenger systems such as hormone signaling. Command of these systems can be voluntary, or have a voluntary component, but they may also be completely automatic. Substituting or supplementing the performance of an element of the body is the aim of a neuroprosthetic device. Achieving this involves the fabrication of an effector, such as a prosthetic limb, that can replicate at least a subset of the body's functionality. But there is no function without control, so it is also necessary to model, and potentially integrate with, the different control loops within the human body. There has been some good success with simpler systems, possibly due to the high level state abstraction of the control within the nervous system (Holinski et al., 2013), as in the control of the hand. By examining the joint angles for fingers during different hand postures (grasping different objects) using principal component analysis it has been shown that the first 3 principal components can account for 90% of the variability. However, the grasp posture data describes the hand only in the final state, it does not describe the trajectory the fingers took to achieve the position around the object. When joint angle data was recorded continuously from subjects performing natural hand movements, 8–9 principal components were needed to describe 90% of the variance (Danziger, 2014). Thus, classifying hand states, and transitioning between fixed postures in a prosthetic is a more straightforward task than attempting the dexterous control of individual fingers (Aggarwal et al., 2011). Although we can treat a robotic effector as part of the body and nervous system of the operator, all current techniques for recording neural activity involve the projection of the high dimensional space of hundreds to thousands of neurons down through the recording electrode array to the much lower dimensionality of the end point of the effector (Carmena, 2013).

Feedforward Control

Feedforward or open-loop control generates a command for the plant that is expected to produce the correct output. However, there is no measurement of the output from the plant, and hence no measurement of error, so the controller has no mechanism to modulate a command (Houk, 1988). A block diagram of open-loop control is shown as Figure 2. Implicit within open-loop control is the assumption of a perfectly described system that can be used to generate a control. Leaving aside the difficulties in creating a perfect model of any system, open-loop approaches do not take noise or measurement error into account.

FIGURE 2
www.frontiersin.org

Figure 2. Feedforward and Feedback Control. Feedforward or open-loop control is shown here in the solid line. The controller generates a command that is applied to the system, or Plant. In response to the command the system performs an action at the Output. Closed-loop or feedback control is achieved by the inclusion of the Sensor component, shown here as the dashed line. The Sensor measures the Output enabling the Controller to assess the error and adjust the next Input to the Plant.

Feedback Control

Feedback, or closed-loop control requires the inclusion of sensors in the system under control. The feedback controller generates a command for the plant, and the sensors measure the output of the plant in response to the command. If a measurement, such as the angle of a joint differs from the expected output, then the error signal can be used by the feedback controller to modify the generated commands. There are many mathematical approaches that can be used to modify feedback controller output (Crago et al., 1996).

Adaptive Control

Adaptive control can be applied to both feedforward and feedback controllers. By using sensors to measure the input and output of the system adaptive control strategies seek to adjust the controller in response to perturbations in the environment or the controlled system (Crago et al., 1996). Adaptive controllers enable the development of a control strategy without requiring complete knowledge of the system being controlled, however, as a consequence adaptive controllers are rarely optimal.

Internal Model Control

Internal Model Control (IMC) is an approach to feedback controller design that incorporates a model of the system that is being controlled (García et al., 1989). The model can be developed based only on the relationship between the inputs and outputs of the system, or alternatively a partial model or complete model of the system can be utilized (LeDuc et al., 2011). At each time step the internal model is evaluated forward to a horizon, offering a prediction of the system behavior in response to the controller's input, and the control inputs are evaluated against a cost function to find the optimum command to be executed at the next time step (Pan et al., 2015). A block diagram illustrates IMC as Figure 3.

FIGURE 3
www.frontiersin.org

Figure 3. Internal Model Control. The inclusion of a model of the Plant allows for the Controller to incorporate some of the dynamics of the system into the control policy.

Classification

A Classifier breaks a system into discrete states, and maps a relationship between an input and a system state (Schwartz, 2004). Classifiers can be supervised or unsupervised. Supervised systems are trained on an input-output relationship in a data set, and attempt to generalize the relationship to new data, while unsupervised systems attempt to partition or cluster the dataset.

Actor-Critic

The Actor-Critic architecture separates the control policy from the evaluation of the action. The Actor component of the systems choses a policy, which affects the state of the system. The Critic component assesses the state of the system in accordance with a cost function, and provides the evaluation to the Actor (Mahmoudi and Sanchez, 2011). The goal based evaluation differs from the error signal of other control approaches, and does not require that the Actor has a model of the system's behavior (Mahmoudi et al., 2013).

Control Algorithms

Table 1 summarizes the Control Algorithms.

TABLE 1
www.frontiersin.org

Table 1. Summary of Control Strategies.

Control policies can be implemented by classification, model independent, or model based approaches (Kameneva et al., 2015).

Control Policies with Independent Models

Consider the system state to be independent at each time step.

Bang-Bang Control

Also referred to as On-Off control, in this scheme when a threshold for a measured variable is crossed a program is activated. Although simple this control scheme has been used successfully to automate tasks that have previously required human intervention, such as the delivery of cortical electrical stimulation after ECog seizure detection (Peters et al., 2001), or the mapping of stimulus thresholds in high electrode count implanted neurostimulators (Wilder et al., 2009).

Finite State Machine

A State Machine is a model of a system. It can be considered a more complex implementation of Bang-Bang control. The measurement of a system value, combined with the modeled system's current state triggers an action and a state transition (Markovic et al., 2014). If the modeled system is periodic, such as gait during walking, then it can be possible to have transitions due to timing (Holinski et al., 2013), in which case the neuroprosthetic enables state transitions in response to deviants from the periodic behavior, such as starting or stopping the gait.

Population Vector Algorithm

The biomechanics of the arm make motor control a difficult problem. Additionally the mechanisim of control within the motor cortex remains an open question. Is the cortical representation in an area such as the activity recorded from M1 encoding the lengthening or shortening of individual muscles (Schwartz, 2004), or is it representing the kinematics of movement (Ajemian et al., 2008)? In either case there is evidence for a forward and inverse model representation of motor control existing within the brain (Andersen et al., 2010; Green and Kalaska, 2011). The Population Vector Algorithm (PVA) is a popular method to decode neural activity recorded from cortical MEA in the motor cortex for the control of a robotic effector, or cursor in a 2D or 3D space. This control algorithm rests on the observation that different neurons have directional preferences—they have higher spike rates for movements in some directions (Shpigelman et al., 2009). Individual neurons do not offer enough specificity to be useful, but a large enough population of neurons, recorded from simultaneously, can be used to determine the intended direction and movement velocity by linear regression. Neuroprosthetic control can then be performed by relying on a “targeting” strategy of decoding the end point trajectory apparent in each cell's activity.

In Helms Tillery et al. (2003) a non-human primate's BMI was extracting an X, Y, and Z signal for the end effector on the robotic arm and the other degrees of freedom of the arm were under the control of the robotic device. In Hatsopoulos et al. (2005) a human participant with 128 electrode array implanted in the precentral gyrus was able to achieve 2D cursor control on a laptop.

Initial implementations of PVA utilized cortical recordings made while the participant watched cursor movement, or the movement of a limb. However, although a PVA decoder created in this way may show good performance in offline testing, the closed-loop performance will not be better, and may be worse (Chase et al., 2009).

Control Policies with Dynamic Models

There are methods that rely on having a model of the dynamic system describing the parameter the neuroprosthetic is controlling. This might be a model of the spiking behavior of a region of the CNS that is to be modulated by a DBS, or a model of the kinematics of the trajectory of a cursor in a motor BMI, or the kinematics of the gait of an exoskeleton. Models can be linear or non-linear. Using the model, and an error signal, and modeling feedback the next state can be predicted using a variety of methods described below. These approaches are iterative, and work well as part of a closed-loop as they represent the process of the subject modifying the input control signal in response to the feedback signal.

A simple linear state model for velocity can be represented as:

xt=Axt1+wt1yt=Cxt1+qt1

where xt is the velocity as a 3 dimensional vector at time step t, A is a matrix of parameters describing the trajectory and wt−1 is a noise term. The second equation describes the measurement model of the neural activity, yt. C is a matrix of parameters that relate velocity to the neural activity and qt is a noise term. This model can be expanded with the inclusion of an additional term to model the input of the control signal on the system state as follows:

xt=Axt1+But1+ wt1

where ut is the control signal at time step t, and B is a matrix of parameters describing the trajectory. The task of the following algorithms is to predicte the state in order to generate an error, the difference between the predicted and observed state, which can be used to adjust the control signal.

There are a number of assumptions within the model, including that the sensory feedback to the subject of the current state xt is error free and instantaneous (Shanechi and Carmena, 2013). The matrix B is tightly coupled to a particular task, making it difficult for the subject to use the control input ut to drive the neuroprosthetic if the task order changes during use (Matlack et al., 2014). Williams et al. (2013) points to the utility in including both a “hold” state and a “rest” state in the design of tasks for motor BMI decoders, and which is often not included as part of the model state. Finally, Hogri et al. (2015) illustrate a clear box modeling approach (LeDuc et al., 2011) where a simplified cerebellar microcircuit is implemented as a VLSI chip and interfaced to anesthetized rats.

Kalman Filter

The Kalman Filter (KF) is a recursive optimal estimator and is good at extracting signal from noisy measurements. It has been widely deployed in industrial automation and control systems engineering for over half a century. In its original form the state and measurement models needed to be linear. The extended Kalman Filter (EKF) models non-linear processes where a linearization has been performed, and implementations such as the Unscented Kalman Filter (Li et al., 2009) can utilize non-linear models. A number of variants have been proposed for neuroprosthetic closed-loop control, including SmoothBatch (Orsborn et al., 2012) and ReFIT (Gilja et al., 2012) which capture elements of the neuroprosthetic task in the model and updates the decoder parameters during the operation of the system. Updating the decoder in this manner is referred to as closed-loop decoder adaption (CLDA). In Dangi et al. (2014) Recursive Maximum Likelihood is used as part of CLDA to continuously adjust the KF. This is probably useful because the recording of the neural signal may be non-stationary due to factors (electrode drift, movement artifacts, external noise) as well as the fact that the subject may have changes in attention during the operation of the device, and the learning process may change the parameters for error and modeling (Chase et al., 2009). When examining various parameters that can be tuned in the decoder, (Cunningham et al., 2011) determined that bin width has a large impact on the performance of the KF, and should be optimized. Potentially due to the subjects ability to interact with the closed-loop system, shorter bin widths of 25–50 ms provide improved performance over longer bin widths.

Point Process Filters

The activity of individual neurons in the ensemble can be modeled as point processes with each spike being an event, which enables the filter to respond much more rapidly then methods that rely on binned spike counts or estimates of instantaneous firing rates (Li et al., 2009).

Reinforcement-Learning

In Actor-Critic architectures two coupled systems work together with complimentary models of the task. The two systems adapt using a Reinforcement Learning approach (DiGiovanna et al., 2009). The user of the system supplies a signal indicating success or failure to the Critic, which supplies a training signal to the Actor to allow adaption (Mahmoudi et al., 2008). By trial and error the Actor interacts with the environment, and the Critic's feedback rewards successful actions. The Actor-Critic approach may also be well-suited to neuroprosthetic control in a real world usage scenario where the task and associated trajectory varies from moment to moment, and achieving the goal may be the only reinforcement signal available (DiGiovanna et al., 2009). Mahmoudi et al. (2008) describes a neuroprosthetic for a Sprague Dawley rat with 32 electrodes implanted in primary motor cortex (M1)—symmetrically, 16 electrodes per hemisphere. This was used to control a robotic arm, which the rat used to press levers. Meanwhile the Critic component is implemented as a computer agent that adapts via the Reinforcement Learning paradigm based on the rewards the rat user receives, and the rat user is learning to modulate its neural activation modifying the directional tuning of the units in M1. An extension to this approach involves extracting the goal success signal directly from the subject. By recording from the Nucleus Accumbens in the ventral striatum of rats, an area believed to associate sensory perception with motor tasks, (Mahmoudi and Sanchez, 2011) were able to use the rat subject's internal representation of goal success as the evaluative feedback signal to the Critic component.

PID Feedback Control

The Proportional-Integral-Derivative (PID) controller is an extremely common and widely deployed controller in closed-loop systems. The three terms—proportional, integral, and derivative are calculated from the plant's response to the input and are summed to generate the error signal (Åström and Wittenmark, 1990). Chaos control approaches, such as delayed feedback control, utilize the dynamics of the system to modify the control input. By taking advantage of the chaotic system sensitivity to perturbation, system state can be changed with minimal cost. In Vlachos et al. (2016) the use of delayed feedback control enables closed loop control of a seizure model (a spiking neural network) and the recovery of the non-seizure dynamics, while in Slutzky et al. (2003) seizure activity induced in rat hippocampal slice preparations was moderately controlled.

Control Policies with Classifiers

Classifiers don't need a model of the system instead they attempt to determine a relationship between a set of measurements and a given state. Some classifiers can be sensitive to changes in the data they use to determine the classes (Lotte et al., 2007), such as Artificial Neural Networks whereas Linear Discriminant Analysis is more robust in the face of changes to data used to train the classifier.

Linear Discriminant Analysis

Linear discriminant analysis (LDA), and other related techniques, are statistical methods that find the features in the measurement of a signal that indicate the probability that it belongs to a given class (Mika et al., 1999). The assignment to a class can be used to trigger a neuroprosthetic intervention, such as the detection via EEG of a motor command and the subsequent activation of an ankle exoskeleton (Xu et al., 2014).

Artificial Neural Network

The Artificial Neural Network is a data driven approach to classification that in contrast to LDA and other statistical methods does not rely on the assumption of the underlying probability distribution of the system (Zhang, 2000). ANNs are organized in layers, with nodes or neurons connected typically in an input, hidden and output layer structure (Figure 4). There are numerous topologies, but among the most popular is the Multi Layer Perceptron (MLP), a three layer feedforward network. ANNs are trained with the presentation of input data that has been identified as belonging to an output class, and a learning rule is applied to adjust the weights on the connections between the nodes, of which back propagation is the most well-known.

FIGURE 4
www.frontiersin.org

Figure 4. Artificial Neural Network. An illustration of a typical ANN topology. An input layer projects to a single hidden layer, which connects to the output layer. Common variations include additional hidden layers and recurrent connections.

ANNs have been used to predict end-point gait parameters from the EMG recorded from the neuromuscular activation of subjects with Spina Bifida (SB; Chang et al., 2009), and to achieve realtime dexterous control of a myoelectric prosthetic hand from cortical recordings of rhesus monkeys (Aggarwal et al., 2008). Echostate Neural Networks are a Recurrent Neural Network (Sussillo et al., 2012) that have been used in non-human primates for a motor BMI, and have been able to outperform the Kalman Filter.

Support Vector Machines

The Support Vector Machine (SVM) is a supervised machine learning approach that can perform classification or regression. The SVM identifies a hyperplane that separates classes within the data, by non-linearly projecting data points into a higher dimensional space (Tapson et al., 2013).

Linear Solutions to Higher Dimensional Interlayers

Linear Solutions of Higher Dimensional Interlayers (LSHDI) are a class of networks that have some similarity in architecture to ANNs, with an input, hidden, and output layer. They are distinct due to the much larger hidden layer, the random generation of the weights on between the input and the hidden layer, and the linear response of the output layer (Tapson et al., 2013). The Neural Engineering Framework (NEF) builds systems out of networks that have LSHDI characteristics (Eliasmith and Anderson, 2004). The NEF has been used to design Spiking Neural Networks that implements the Kalman Filter as part of a cortical motor BMI (Dethier et al., 2011). The Synaptic Kernel Inverse Method (SKIM) is an LSHDI network for spiking input (Tapson et al., 2013) that can perform both classification and regression.

Feedback

There can be more than one feedback loop in the neuroprosthetic system (Broccard et al., 2014). Feedback can be the visual observation of the robotic effector as it is in many motor BMIs, allowing the operator to modulate their neural activity before it is decoded. Or in the case of a DBS neuroprosthetic the feedback signal may be acquired from recording electrodes implanted alongside the stimulating electrodes, in which case the feedback signal is returned directly to the device (Herron and Chizeck, 2014). Bidirectional interfaces, via the PNS or the CNS, enable the transformed signals of sensors on the robotic effector to be transmitted into the operators nervous system and interpreted as a sensory percept (Armiger et al., 2013). Feedback can also be from sensors on the robotic effector directly to the controller, bypassing the operator (Broccard et al., 2014; Markovic et al., 2014). And finally, reversing the direction of information, natural sensors can be used as a source of feedback for the controller (Holinski et al., 2013; Mendez et al., 2013; Wright et al., 2015) and the brain can be used as the source of the error signal (Mahmoudi and Sanchez, 2011).

Discussion

We have reviewed the control policies employed by recent neuroprosthetic systems. For the purposes of this review we included motor BMIs, assistive devices, neuromodulatory systems, and other devices with an interface into a subject's nervous system. Many of the devices reported on in the literature are being developed in non-human primate or other animal models, only a subset have been tested in human subjects. There is a frequent conflation of detection and classification approaches with digital or Bang-Bang control within the literature.

Although closed-loop systems have been demonstrated experimentally there remain significant limits on our ability to describe the activity in the brain, and consequently develop control policies to respond to that activity. Simulation of cortical activity (Ehrens et al., 2015; Sandler et al., 2015; Vlachos et al., 2016), the use of experimental platforms (Keren and Marom, 2014), and the use of animal models has enabled the development of a wide range of neuroprosthetic systems. However, the appropriate method to transition these systems in human subjects is not clear. Among the difficulties is the body's response to chronic implantation of microelectrodes (Fernandez et al., 2014), the appropriate place to collect a signal (Krook-Magnuson et al., 2015), and the possibility that longterm attempts to control a cortical system may compromise some of the desirable behavior (Keren and Marom, 2014).

Comparisons of control policies across different modalities, interfaces and levels of invasiveness are difficult. Even within a given neuroprosthetic category it can be difficult to perform a comparison due to the wide variety of task designs, different subject training regimes and varying reported metrics; (Koyama et al., 2010; Sussillo et al., 2012) are rare exceptions. Examination of Table 1 reveals that the motor BMI discipline appears to have explored the widest variety of control policies, and that the use of the Kalman Filter as part of a closed-loop system has broad support. Improvements to the traditional Kalman Filter to allow non-linear models of neural activity combined with its ability to be implemented in real-time continue to make it an attractive approach.

Closed-loop motor BMI systems have had significant success with closed-loop decoder adaption (CLDA; Shanechi and Carmena, 2013), supporting the use of closed-loop control. However, the CLDA approach has identified two distinct strategies in motor neuroprosthetics—decoding vs. learning. The decoding approach aims to read the natural motor plan whereas the learning approach monitors the changing neural activity as the brain learns to operate the prosthetic. It is not known at this stage if a similar duality of strategies will be applicable in other modalities. The difficulty in specifying a model for use in many of the control policies previously describes arises from our continued uncertainty about specific action of many of the components of the nervous system. It has been observed that users of Cochlear Implants have improved speech recognition performance after completing training with the device (Doucet et al., 2006), which may argue for a learning interpretation. An important caveat for the learning approach is that the neuroprosthetic system must be stable as regards the interface and transform of the input signal to the effector output, to allow the subject the opportunity to develop the “prosthetic motor memory” necessary for skillful operation (Carmena, 2013).

Neuroprosthetic development of closed-loop systems has been driven in part by the concern that the risks involved in highly invasive interfaces need to be mitigated by a strong case for the therapeutic benefit. Patient abandonment for upper limb prosthetics is high, with many wearers ceasing upper limb prosthetic use within 12 months of receipt of the device. Concerns cited by users are weight, appearance and difficulty of use (Biddiss and Chau, 2007). Extrapolating to more invasive systems, it may be difficult to argue the cost benefit if patient dissatisfaction is very high. Although these devices cannot be abandoned in the same manner as a detachable prosthetic limb, there is some suggestive research indicating an unwillingness to participate in experimental trials, which may suggest that the perceived benefit of neuroprosthetic systems by the target patient populations remains low (Illes et al., 2011). By improving control we can offer improved functionality and increased therapeutic benefit.

Author Contributions

JW performed the review and drafted the manuscript. JT, AS, and VM revised the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Agashe, H. A., and Contreras-Vidal, J. L. (2011). Reconstructing hand kinematics during reach to grasp movements from electroencephalographic signals. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2011, 5444–5447. doi: 10.1109/iembs.2011.6091389

PubMed Abstract | CrossRef Full Text | Google Scholar

Aggarwal, V., Kerr, M., Davidson, A. G., Davoodi, R., Loeb, G. E., Schieber, M. H., et al. (2011). “Cortical control of reach and grasp kinematics in a virtual environment using musculoskeletal modeling software,” in 2011 5th International IEEE/EMBS Conference on Neural Engineering (NER) (Cancún), 388–391. doi: 10.1109/NER.2011.5910568

CrossRef Full Text

Aggarwal, V., Singhal, G., He, J., Schieber, M. H., and Thakor, N. V. (2008). Towards closed-loop decoding of dexterous hand movements using a virtual integration environment. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2008, 1703–1706. doi: 10.1109/iembs.2008.4649504

PubMed Abstract | CrossRef Full Text | Google Scholar

Ajemian, R., Green, A., Bullock, D., Sergio, L., Kalaska, J., and Grossberg, S. (2008). Assessing the function of motor cortex: single-neuron models of how neural response is modulated by limb biomechanics. Neuron 58, 414–428. doi: 10.1016/j.neuron.2008.02.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Andersen, R. A., Hwang, E. J., and Mulliken, G. H. (2010). Cognitive neural prosthetics. Annu. Rev. Psychol. 61, 169–190, C161–C163. doi: 10.1146/annurev.psych.093008.100503

PubMed Abstract | CrossRef Full Text | Google Scholar

Åström, K. J., and Wittenmark, B. (1990). Computer-Controlled Systems: Theory and Design. Upper Saddle River, NJ: Prentice-Hall International.

Google Scholar

Armiger, R. S., Tenore, F. V., Katyal, K. D., Johannes, M. S., Mahklin, A., Natter, M. L., et al. (2013). Enabling closed-loop control of the Modular Prosthetic Limb through haptic feedback. Johns Hopkins APL Tech. Dig. 31, 345–353.

Google Scholar

Berger, T. W., Hampson, R. E., Song, D., Goonawardena, A., Marmarelis, V. Z., and Deadwyler, S. A. (2011). A cortical neural prosthesis for restoring and enhancing memory. J. Neural Eng. 8:046017. doi: 10.1088/1741-2560/8/4/046017

PubMed Abstract | CrossRef Full Text | Google Scholar

Biddiss, E. A., and Chau, T. T. (2007). Upper limb prosthesis use and abandonment: a survey of the last 25 years. Prosthet. Orthot. Int. 31, 236–257. doi: 10.1080/03093640600994581

PubMed Abstract | CrossRef Full Text | Google Scholar

Borton, D., Bonizzato, M., Beauparlant, J., DiGiovanna, J., Moraud, E. M., Wenger, N., et al. (2014). Corticospinal neuroprostheses to restore locomotion after spinal cord injury. Neurosci. Res. 78, 21–29. doi: 10.1016/j.neures.2013.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Broccard, F. D., Mullen, T., Chi, Y. M., Peterson, D., Iversen, J. R., Arnold, M., et al. (2014). Closed-loop brain-machine-body interfaces for noninvasive rehabilitation of movement disorders. Ann. Biomed. Eng. 42, 1573–1593. doi: 10.1007/s10439-014-1032-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Carmena, J. M. (2013). Advances in neuroprosthetic learning and control. PLoS Biol. 11:e1001561. doi: 10.1371/journal.pbio.1001561

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, C. L., Jin, Z., Chang, H. C., and Cheng, A. C. (2009). From neuromuscular activation to end-point locomotion: an artificial neural network-based technique for neural prostheses. J. Biomech. 42, 982–988. doi: 10.1016/j.jbiomech.2009.03.030

PubMed Abstract | CrossRef Full Text | Google Scholar

Chase, S. M., Schwartz, A. B., and Kass, R. E. (2009). Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms. Neural Netw. 22, 1203–1213. doi: 10.1016/j.neunet.2009.05.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Cisotto, G., Pupolin, S., Silvoni, S., Cavinato, M., Agostini, M., and Piccione, F. (2013). “Brain-computer interface in chronic stroke: an application of sensorimotor closed-loop and contingent force feedback,” in IEEE International Conference on Communications (Budapest), 4379–4383. doi: 10.1109/ICC.2013.6655255

CrossRef Full Text

Corbett, E. A., Kording, K. P., and Perreault, E. J. (2013). Real-time evaluation of a noninvasive neuroprosthetic interface for control of reach. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 674–683. doi: 10.1109/TNSRE.2013.2251664

PubMed Abstract | CrossRef Full Text | Google Scholar

Corbett, E. A., Kording, K. P., and Perreault, E. J. (2014). Dealing with target uncertainty in a reaching control interface. PLoS ONE 9:e86811. doi: 10.1371/journal.pone.0086811

PubMed Abstract | CrossRef Full Text | Google Scholar

Crago, P. E., Lan, N., Veltink, P. H., Abbas, J. J., and Kantor, C. (1996). New control strategies for neuroprosthetic systems. J. Rehabil. Res. Dev. 33, 158–172.

PubMed Abstract | Google Scholar

Cunningham, J. P., Nuyujukian, P., Gilja, V., Chestek, C. A., Ryu, S. I., and Shenoy, K. V. (2011). A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces. J. Neurophysiol. 105, 1932–1949. doi: 10.1152/jn.00503.2010

PubMed Abstract | CrossRef Full Text | Google Scholar

Dangi, S., Gowda, S., Héliot, R., and Carmena, J. M. (2011). “Adaptive Kalman filtering for closed-loop Brain-Machine Interface systems,” in 2011 5th International IEEE/EMBS Conference on Neural Engineering (NER) (Cancún), 609–612. doi: 10.1109/NER.2011.5910622

CrossRef Full Text

Dangi, S., Gowda, S., Moorman, H. G., Orsborn, A. L., So, K., Shanechi, M., et al. (2014). Continuous closed-loop decoder adaptation with a recursive maximum likelihood algorithm allows for rapid performance acquisition in brain-machine interfaces. Neural Comput. 26, 1811–1839. doi: 10.1162/NECO_a_00632

PubMed Abstract | CrossRef Full Text | Google Scholar

Danziger, Z. (2014). A reductionist approach to the analysis of learning in brain-computer interfaces. Biol. Cybern. 108, 183–201. doi: 10.1007/s00422-014-0589-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Deisseroth, K. (2011). Optogenetics. Nat. Methods 8, 26–29. doi: 10.1038/nmeth.f.324

PubMed Abstract | CrossRef Full Text | Google Scholar

Dethier, J., Nuyujukian, P., Eliasmith, C., Stewart, T., Elassaad, S. A., Shenoy, K. V., et al. (2011). A brain-machine interface operating with a real-time spiking neural network control algorithm. Adv. Neural Inf. Process. Syst. 2011, 2213–2221.

PubMed Abstract | Google Scholar

Dethier, J., Nuyujukian, P., Ryu, S. I., Shenoy, K. V., and Boahen, K. (2013). Design and validation of a real-time spiking-neural-network decoder for brain-machine interfaces. J. Neural Eng. 10:036008. doi: 10.1088/1741-2560/10/3/036008

PubMed Abstract | CrossRef Full Text | Google Scholar

DiGiovanna, J., Mahmoudi, B., Fortes, J., Principe, J. C., and Sanchez, J. C. (2009). Coadaptive brain-machine interface via reinforcement learning. IEEE Trans. Biomed. Eng. 56, 54–64. doi: 10.1109/TBME.2008.926699

PubMed Abstract | CrossRef Full Text | Google Scholar

Dommel, N. B., Wong, Y. T., Lehmann, T., Dodds, C. W., Lovell, N. H., and Suaning, G. J. (2009). A CMOS retinal neurostimulator capable of focussed, simultaneous stimulation. J. Neural Eng. 6:035006. doi: 10.1088/1741-2560/6/3/035006

PubMed Abstract | CrossRef Full Text | Google Scholar

Doucet, M. E., Bergeron, F., Lassonde, M., Ferron, P., and Lepore, F. (2006). Cross-modal reorganization and speech perception in cochlear implant users. Brain 129, 3376–3383. doi: 10.1093/brain/awl264

PubMed Abstract | CrossRef Full Text | Google Scholar

Ehrens, D., Sritharan, D., and Sarma, S. V. (2015). Closed-loop control of a fragile network: application to seizure-like dynamics of an epilepsy model. Front. Neurosci. 9:58. doi: 10.3389/fnins.2015.00058

PubMed Abstract | CrossRef Full Text | Google Scholar

Eliasmith, C., and Anderson, C. H. (2004). Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems. Cambridge, MA: MIT press.

Google Scholar

Fernandez, E., Greger, B., House, P. A., Aranda, I., Botella, C., Albisua, J., et al. (2014). Acute human brain responses to intracortical microelectrode arrays: challenges and future prospects. Front. Neuroeng. 7:24. doi: 10.3389/fneng.2014.00024

PubMed Abstract | CrossRef Full Text | Google Scholar

García, C. E., Prett, D. M., and Morari, M. (1989). Model predictive control: theory and practice—A survey. Automatica 25, 335–348. doi: 10.1016/0005-1098(89)90002-2

CrossRef Full Text | Google Scholar

Gilja, V., Nuyujukian, P., Chestek, C. A., Cunningham, J. P., Yu, B. M., Fan, J. M., et al. (2012). A high-performance neural prosthesis enabled by control algorithm design. Nat. Neurosci. 15, 1752–1757. doi: 10.1038/nn.3265

PubMed Abstract | CrossRef Full Text | Google Scholar

Golub, M. D., Yu, B. M., Schwartz, A. B., and Chase, S. M. (2014). Motor cortical control of movement speed with implications for brain-machine interface control. J. Neurophysiol. 112, 411–429. doi: 10.1152/jn.00391.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Gowda, S., Orsborn, A. L., and Carmena, J. M. (2012). Parameter estimation for maximizing controllability of linear brain-machine interfaces. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2012, 1314–1317. doi: 10.1109/embc.2012.6346179

PubMed Abstract | CrossRef Full Text | Google Scholar

Grahn, P. J., Mallory, G. W., Khurram, O. U., Berry, B. M., Hachmann, J. T., Bieber, A. J., et al. (2014). A neurochemical closed-loop controller for deep brain stimulation: toward individualized smart neuromodulation therapies. Front. Neurosci. 8:169. doi: 10.3389/fnins.2014.00169

PubMed Abstract | CrossRef Full Text | Google Scholar

Green, A. M., and Kalaska, J. F. (2011). Learning to move machines with the mind. Trends Neurosci. 34, 61–75. doi: 10.1016/j.tins.2010.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Grill, W. M., Norman, S. E., and Bellamkonda, R. V. (2009). Implanted neural interfaces: biochallenges and engineered solutions. Annu. Rev. Biomed. Eng. 11, 1–24. doi: 10.1146/annurev-bioeng-061008-124927

PubMed Abstract | CrossRef Full Text | Google Scholar

Guggenmos, D. J., Azin, M., Barbay, S., Mahnken, J. D., Dunham, C., Mohseni, P., et al. (2013). Restoration of function after brain damage using a neural prosthesis. Proc. Natl. Acad. Sci. U.S.A. 110, 21177–21182. doi: 10.1073/pnas.1316885110

PubMed Abstract | CrossRef Full Text | Google Scholar

Hatsopoulos, N., Mukand, J., Polykoff, G., Friehs, G., and Donoghue, J. (2005). Cortically controlled brain-machine interface. Conf. Proc. IEEE Eng. Med. Biol. Soc. 7, 7660–7663. doi: 10.1109/iembs.2005.1616286

PubMed Abstract | CrossRef Full Text | Google Scholar

Helms Tillery, S. I., Taylor, D. M., and Schwartz, A. B. (2003). “The general utility of a neuroprosthetic device under direct cortical control,” in Engineering in Medicine and Biology Society, 2003. Proceedings of the 25th Annual International Conference of the IEEE (Cancún), 2043–2046. doi: 10.1109/IEMBS.2003.1280137

CrossRef Full Text

Herron, J., and Chizeck, H. J. (2014). “Prototype closed-loop deep brain stimulation systems inspired by Norbert Wiener,” in 2014 IEEE Conference on Norbert Wiener in the 21st Century (21CW) (Boston, MA), 1–6. doi: 10.1109/NORBERT.2014.6893922

CrossRef Full Text

Hochberg, L. R., Bacher, D., Jarosiewicz, B., Masse, N. Y., Simeral, J. D., Vogel, J., et al. (2012). Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375. doi: 10.1038/Nature11076

PubMed Abstract | CrossRef Full Text | Google Scholar

Hogri, R., Bamford, S. A., Taub, A. H., Magal, A., Del Giudice, P., and Mintz, M. (2015). A neuro-inspired model-based closed-loop neuroprosthesis for the substitution of a cerebellar learning function in anesthetized rats. Sci. Rep. 5:8451. doi: 10.1038/srep08451

PubMed Abstract | CrossRef Full Text | Google Scholar

Holinski, B. J., Everaert, D. G., Mushahwar, V. K., and Stein, R. B. (2013). Real-time control of walking using recordings from dorsal root ganglia. J. Neural Eng. 10:056008. doi: 10.1088/1741-2560/10/5/056008

PubMed Abstract | CrossRef Full Text | Google Scholar

Horch, K., Meek, S., Taylor, T. G., and Hutchinson, D. T. (2011). Object discrimination with an artificial hand using electrical stimulation of peripheral tactile and proprioceptive pathways with intrafascicular electrodes. IEEE Trans. Neural Syst. Rehabil. Eng. 19, 483–489. doi: 10.1109/TNSRE.2011.2162635

PubMed Abstract | CrossRef Full Text | Google Scholar

Hosain, M. K., Kouzani, A., and Tye, S. (2014). Closed loop deep brain stimulation: an evolving technology. Australas. Phys. Eng. Sci. Med. 37, 619–634. doi: 10.1007/s13246-014-0297-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Houk, J. C. (1988). Control strategies in physiological systems. FASEB J. 2, 97–107.

PubMed Abstract | Google Scholar

Hu, J., Si, J., Olson, B. P., Clement, R. S., and He, J. (2003). “Decoding motor cortical spike trains for brain machine interface applications,” in Engineering in Medicine and Biology Society, 2003. Proceedings of the 25th Annual International Conference of the IEEE, 2071–2074. doi: 10.1109/IEMBS.2003.1280144

CrossRef Full Text | Google Scholar

Illes, J., Reimer, J. C., and Kwon, B. K. (2011). Stem cell clinical trials for spinal cord injury: readiness, reluctance, redefinition. Stem Cell Rev. 7, 997–1005. doi: 10.1007/s12015-011-9259-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Inmann, A., and Haugland, M. (2004). Functional evaluation of natural sensory feedback incorporated in a hand grasp neuroprosthesis. Med. Eng. Phys. 26, 439–447. doi: 10.1016/j.medengphy.2004.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Iolov, A., Ditlevsen, S., and Longtin, A. (2014). Stochastic optimal control of single neuron spike trains. J. Neural Eng. 11:046004. doi: 10.1088/1741-2560/11/4/046004

PubMed Abstract | CrossRef Full Text | Google Scholar

Ionescu, C., and De Keyser, R. (2006). Activating paralyzed muscles using adaptive control and a neuroprosthetic technique. Automat. Quality Test. Robotics 2006 IEEE Int. Conf. 1, 43–47. doi: 10.1109/AQTR.2006.254494

CrossRef Full Text | Google Scholar

Iturrate, I., Grizou, J., Omedes, J., Oudeyer, P. Y., Lopes, M., and Montesano, L. (2015). Exploiting task constraints for self-calibrated brain-machine interface control using error-related potentials. PLoS ONE 10:e0131491. doi: 10.1371/journal.pone.0131491

PubMed Abstract | CrossRef Full Text | Google Scholar

Jarosiewicz, B., Masse, N. Y., Bacher, D., Cash, S. S., Eskandar, E., Friehs, G., et al. (2013). Advantages of closed-loop calibration in intracortical brain-computer interfaces for people with tetraplegia. J. Neural Eng. 10:046012. doi: 10.1088/1741-2560/10/4/046012

PubMed Abstract | CrossRef Full Text | Google Scholar

Kameneva, T., Abramian, M., Zarelli, D., Nesic, D., Burkitt, A. N., Meffin, H., et al. (2015). Spike history neural response model. J. Comput. Neurosci. 38, 463–481. doi: 10.1007/s10827-015-0549-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Keren, H., and Marom, S. (2014). Controlling neural network responsiveness: tradeoffs and constraints. Front. Neuroeng. 7:11. doi: 10.3389/fneng.2014.00011

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, S. P., Simeral, J. D., Hochberg, L. R., Donoghue, J. P., and Black, M. J. (2008). Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neural Eng. 5, 455–476. doi: 10.1088/1741-2560/5/4/010

PubMed Abstract | CrossRef Full Text | Google Scholar

Koyama, S., Chase, S. M., Whitford, A. S., Velliste, M., Schwartz, A. B., and Kass, R. E. (2010). Comparison of brain-computer interface decoding algorithms in open-loop and closed-loop control. J. Comput. Neurosci. 29, 73–87. doi: 10.1007/s10827-009-0196-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Krook-Magnuson, E., Gelinas, J. N., Soltesz, I., and Buzsaki, G. (2015). Neuroelectronics and biooptics: closed-loop technologies in neurological disorders. JAMA Neurol. 72, 823–829. doi: 10.1001/jamaneurol.2015.0608

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuiken, T. A., Li, G., Lock, B. A., Lipschutz, R. D., Miller, L. A., Stubblefield, K. A., et al. (2009). Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. JAMA 301, 619–628. doi: 10.1001/jama.2009.116

PubMed Abstract | CrossRef Full Text | Google Scholar

LeDuc, P. R., Messner, W. C., and Wikswo, J. P. (2011). How do control-based approaches enter into biology? Annu. Rev. Biomed. Eng. 13, 369–396. doi: 10.1146/annurev-bioeng-071910-124651

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Z., O'Doherty, J. E., Hanson, T. L., Lebedev, M. A., Henriquez, C. S., and Nicolelis, M. A. (2009). Unscented Kalman filter for brain-machine interfaces. PLoS ONE 4:e6243. doi: 10.1371/journal.pone.0006243

PubMed Abstract | CrossRef Full Text | Google Scholar

Lotte, F., Congedo, M., Lecuyer, A., Lamarche, F., and Arnaldi, B. (2007). A review of classification algorithms for EEG-based brain-computer interfaces. J. Neural Eng. 4, R1–R13. doi: 10.1088/1741-2560/4/2/R01

PubMed Abstract | CrossRef Full Text | Google Scholar

Mahmoudi, B., Digiovanna, J., Principe, J. C., and Sanchez, J. C. (2008). Neuronal tuning in a brain-machine interface during Reinforcement Learning. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2008, 4491–4494. doi: 10.1109/iembs.2008.4650210

PubMed Abstract | CrossRef Full Text | Google Scholar

Mahmoudi, B., Pohlmeyer, E. A., Prins, N. W., Geng, S., and Sanchez, J. C. (2013). Towards autonomous neuroprosthetic control using Hebbian reinforcement learning. J. Neural Eng. 10:066005. doi: 10.1088/1741-2560/10/6/066005

PubMed Abstract | CrossRef Full Text | Google Scholar

Mahmoudi, B., and Sanchez, J. C. (2011). A symbiotic brain-machine interface through value-based decision making. PLoS ONE 6:e14760. doi: 10.1371/journal.pone.0014760

PubMed Abstract | CrossRef Full Text | Google Scholar

Marathe, A. R., and Taylor, D. M. (2013). Decoding continuous limb movements from high-density epidural electrode arrays using custom spatial filters. J. Neural Eng. 10:036015. doi: 10.1088/1741-2560/10/3/036015

PubMed Abstract | CrossRef Full Text | Google Scholar

Markovic, M., Dosen, S., Cipriani, C., Popovic, D., and Farina, D. (2014). Stereovision and augmented reality for closed-loop control of grasping in hand prostheses. J. Neural Eng. 11:046001. doi: 10.1088/1741-2560/11/4/046001

PubMed Abstract | CrossRef Full Text | Google Scholar

Marzullo, T. C., Lehmkuhle, M. J., Gage, G. J., and Kipke, D. R. (2010). Development of closed-loop neural interface technology in a rat model: combining motor cortex operant conditioning with visual cortex microstimulation. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 117–126. doi: 10.1109/TNSRE.2010.2041363

PubMed Abstract | CrossRef Full Text | Google Scholar

Matlack, C., Haddock, A., Moritz, C. T., and Chizeck, H. J. (2014). Motor cortical decoding performance depends on controlled system order. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2014, 2553–2556. doi: 10.1109/embc.2014.6944143

PubMed Abstract | CrossRef Full Text | Google Scholar

McCreadie, K. A., Coyle, D. H., and Prasad, G. (2014). Is sensorimotor BCI performance influenced differently by mono, stereo, or 3-D auditory feedback? IEEE Trans. Neural Syst. Rehabil. Eng. 22, 431–440. doi: 10.1109/TNSRE.2014.2312270

PubMed Abstract | CrossRef Full Text | Google Scholar

McFarland, D. J., Krusienski, D. J., and Wolpaw, J. R. (2006). Brain–computer interface signal processing at the Wadsworth Center: mu and sensorimotor beta rhythms. Prog. Brain Res. 159, 411–419. doi: 10.1016/s0079-6123(06)59026-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Mendez, A., Sawan, M., Minagawa, T., and Wyndaele, J. J. (2013). Estimation of bladder volume from afferent neural activity. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 704–715. doi: 10.1109/TNSRE.2013.2266899

PubMed Abstract | CrossRef Full Text | Google Scholar

Micera, S., Carpaneto, J., and Raspopovic, S. (2010). Control of hand prostheses using peripheral information. IEEE Rev. Biomed. Eng. 3, 48–68. doi: 10.1109/RBME.2010.2085429

PubMed Abstract | CrossRef Full Text | Google Scholar

Mika, S., Rätsch, G., Weston, J., Schölkopf, B., and Mũllers, K.-R. (1999). “Fisher discriminant analysis with kernels,” in Neural Network Signal Processing IX (Madison, WI), 41–48.

Morimoto, J., and Kawato, M. (2015). Creating the brain and interacting with the brain: an integrated approach to understanding the brain. J. R. Soc. Interface 12:20141250. doi: 10.1098/rsif.2014.1250

PubMed Abstract | CrossRef Full Text | Google Scholar

Nagasako, J. M., Fenwick, C. M., Kennedy, I. L., Marion, K. R., Zizzi, M. G., Riofrio, J. A., et al. (2015). “Closed-loop electromyography controller for augmented hand exoskeleton gripping,” in 41st Annual Northeast Biomedical Engineering Conference (NEBEC) (Troy, NY), 1–2. doi: 10.1109/NEBEC.2015.7117233

CrossRef Full Text

Navarro, X., Krueger, T. B., Lago, N., Micera, S., Stieglitz, T., and Dario, P. (2005). A critical review of interfaces with the peripheral nervous system for the control of neuroprostheses and hybrid bionic systems. J. Peripher. Nerv. Syst. 10, 229–258. doi: 10.1111/j.1085-9489.2005.10303.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Olson, B. P., Si, J., Hu, J., and He, J. (2005). Closed-loop cortical control of direction using support vector machines. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 72–80. doi: 10.1109/TNSRE.2004.843174

PubMed Abstract | CrossRef Full Text | Google Scholar

Olson, B., and Si, J. (2010). Evidence of a mechanism of neural adaptation in the closed loop control of directions. Int. J. Intelligent Comput. Cybernetics 3, 5–23. doi: 10.1108/17563781011028523

CrossRef Full Text | Google Scholar

Orsborn, A. L., Dangi, S., Moorman, H. G., and Carmena, J. M. (2012). Closed-loop decoder adaptation on intermediate time-scales facilitates rapid BMI performance improvements independent of decoder initialization conditions. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 468–477. doi: 10.1109/TNSRE.2012.2185066

PubMed Abstract | CrossRef Full Text | Google Scholar

Orsborn, A. L., Moorman, H. G., Overduin, S. A., Shanechi, M. M., Dimitrov, D. F., and Carmena, J. M. (2014). Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393. doi: 10.1016/j.neuron.2014.04.048

PubMed Abstract | CrossRef Full Text | Google Scholar

Ortiz-Catalan, M., Branemark, R., Hakansson, B., and Delbeke, J. (2012). On the viability of implantable electrodes for the natural control of artificial limbs: review and discussion. Biomed. Eng. Online 11:33. doi: 10.1186/1475-925X-11-33

PubMed Abstract | CrossRef Full Text | Google Scholar

Pan, H., Ding, B., Zhong, W., Kumar, G., and Kothare, M. V. (2015). “Designing closed-loop brain-machine interfaces with network of spiking neurons using MPC strategy,” in 2015 American Control Conference (ACC) (Chicago, IL), 2543–2548. doi: 10.1109/ACC.2015.7171117

CrossRef Full Text

Peters, T. E., Bhavaraju, N. C., Frei, M. G., and Osorio, I. (2001). Network system for automated seizure detection and contingent delivery of therapy. J. Clin. Neurophysiol. 18, 545–549. doi: 10.1097/00004691-200111000-00004

PubMed Abstract | CrossRef Full Text | Google Scholar

Raspopovic, S., Capogrosso, M., Petrini, F. M., Bonizzato, M., Rigosa, J., Di Pino, G., et al. (2014). Restoring natural sensory feedback in real-time bidirectional hand prostheses. Sci. Transl. Med. 6:222ra219. doi: 10.1126/scitranslmed.3006820

PubMed Abstract | CrossRef Full Text

Sandler, R. A., Song, D., Hampson, R. E., Deadwyler, S. A., Berger, T. W., and Marmarelis, V. Z. (2015). Hippocampal closed-loop modeling and implications for seizure stimulation design. J. Neural Eng. 12:056017. doi: 10.1088/1741-2560/12/5/056017

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwartz, A. B. (2004). Cortical neural prosthetics. Annu. Rev. Neurosci. 27, 487–507. doi: 10.1146/annurev.neuro.27.070203.144233

PubMed Abstract | CrossRef Full Text | Google Scholar

Shanechi, M. M., and Carmena, J. M. (2013). “Optimal feedback-controlled point process decoder for adaptation and assisted training in brain-machine Interfaces (Providence),” in 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER) (San Diego, CA), 653–656. doi: 10.1109/NER.2013.6696019

CrossRef Full Text

Shanechi, M. M., Chemali, J. J., Liberman, M., Solt, K., and Brown, E. N. (2013). A brain-machine interface for control of medically-induced coma. PLoS Comput. Biol. 9:e1003284. doi: 10.1371/journal.pcbi.1003284

CrossRef Full Text | Google Scholar

Shanechi, M. M., Orsborn, A., Moorman, H., Gowda, S., and Carmena, J. M. (2014). High-performance brain-machine interface enabled by an adaptive optimal feedback-controlled point process decoder. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2014, 6493–6496. doi: 10.1109/embc.2014.6945115

PubMed Abstract | CrossRef Full Text | Google Scholar

Shpigelman, L., Lalazar, H., and Vaadia, E. (2009). “Kernel-ARMA for hand tracking and brain-machine interfacing during 3D motor control,” in Advances in Neural Information Processing Systems (Red Hook, NY: Curran Associates, Inc.), 1489–1496.

Slutzky, M. W., Cvitanovic, P., and Mogul, D. J. (2003). Manipulating epileptiform bursting in the rat hippocampus using chaos control and adaptive techniques. IEEE Trans. Biomed. Eng. 50, 559–570. doi: 10.1109/TBME.2003.810701

PubMed Abstract | CrossRef Full Text | Google Scholar

So, K., Dangi, S., Orsborn, A. L., Gastpar, M. C., and Carmena, J. M. (2014). Subject-specific modulation of local field potential spectral power during brain-machine interface control in primates. J. Neural Eng. 11:026002. doi: 10.1088/1741-2560/11/2/026002

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, F. T., and Morrell, M. J. (2014). Closed-loop neurostimulation: the clinical experience. Neurotherapeutics 11, 553–563. doi: 10.1007/s13311-014-0280-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Sussillo, D., Nuyujukian, P., Fan, J. M., Kao, J. C., Stavisky, S. D., Ryu, S., et al. (2012). A recurrent neural network for closed-loop intracortical brain-machine interface decoders. J. Neural Eng. 9:026027. doi: 10.1088/1741-2560/9/2/026027

PubMed Abstract | CrossRef Full Text | Google Scholar

Tan, D. W., Schiefer, M. A., Keith, M. W., Anderson, J. R., Tyler, J., and Tyler, D. J. (2014). A neural interface provides long-term stable natural touch perception. Sci. Transl. Med. 6:257ra138. doi: 10.1126/scitranslmed.3008669

PubMed Abstract | CrossRef Full Text | Google Scholar

Tapson, J. C., Cohen, G. K., Afshar, S., Stiefel, K. M., Buskila, Y., Wang, R. M., et al. (2013). Synthesis of neural networks for spatio-temporal spike pattern recognition and processing. Front. Neurosci. 7:153. doi: 10.3389/fnins.2013.00153

PubMed Abstract | CrossRef Full Text | Google Scholar

Vlachos, I., Deniz, T., Aertsen, A., and Kumar, A. (2016). Recovery of dynamics and function in spiking neural networks with closed-loop control. PLoS Comput. Biol. 12:e1004720. doi: 10.1371/journal.pcbi.1004720

PubMed Abstract | CrossRef Full Text | Google Scholar

Vogelstein, R. J., Tenore, F., Guevremont, L., Etienne-Cummings, R., and Mushahwar, V. K. (2008). A silicon central pattern generator controls locomotion in vivo. IEEE Trans. Biomed. Circuits Syst. 2, 212–222. doi: 10.1109/TBCAS.2008.2001867

PubMed Abstract | CrossRef Full Text | Google Scholar

Wilder, A. M., Hiatt, S. D., Dowden, B. R., Brown, N. A., Normann, R. A., and Clark, G. A. (2009). Automated stimulus-response mapping of high-electrode-count neural implants. IEEE Trans. Neural Syst. Rehabil. Eng. 17, 504–511. doi: 10.1109/TNSRE.2009.2029494

PubMed Abstract | CrossRef Full Text | Google Scholar

Williams, J. J., Rouse, A. G., Thongpang, S., Williams, J. C., and Moran, D. W. (2013). Differentiating closed-loop cortical intention from rest: building an asynchronous electrocorticographic BCI. J. Neural Eng. 10:046001. doi: 10.1088/1741-2560/10/4/046001

PubMed Abstract | CrossRef Full Text | Google Scholar

Wodlinger, B., Downey, J. E., Tyler-Kabara, E. C., Schwartz, A. B., Boninger, M. L., and Collinger, J. L. (2015). Ten-dimensional anthropomorphic arm control in a human brain-machine interface: difficulties, solutions, and limitations. J. Neural Eng. 12:016011. doi: 10.1088/1741-2560/12/1/016011

PubMed Abstract | CrossRef Full Text | Google Scholar

Wright, J., Macefield, V. G., van Schaik, A., and Tapson, J. (2015). “Decoding force from multiunit recordings from the median nerve,” in 2015 IEEE International Conference on Rehabilitation Robotics (ICORR) (Singapore), 956–960. doi: 10.1109/ICORR.2015.7281327

CrossRef Full Text | Google Scholar

Xu, R., Jiang, N., Mrachacz-Kersting, N., Lin, C., Asin Prieto, G., Moreno, J. C., et al. (2014). A closed-loop brain-computer interface triggering an active ankle-foot orthosis for inducing cortical neural plasticity. IEEE Trans. Biomed. Eng. 61, 2092–2101. doi: 10.1109/TBME.2014.2313867

PubMed Abstract | CrossRef Full Text | Google Scholar

Xu, Z., Guan, C., So, R. Q., Ang, K. K., and Toe, K. K. (2015). “Motor cortical adaptation induced by closed-loop BCI,” in 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER) (Montpellier), 21–24. doi: 10.1109/NER.2015.7146550

CrossRef Full Text

Yanagisawa, T., Hirata, M., Saitoh, Y., Goto, T., Kishima, H., Fukuma, R., et al. (2011). Real-time control of a prosthetic hand using human electrocorticography signals. J. Neurosurg. 114, 1715–1722. doi: 10.3171/2011.1.JNS101421

PubMed Abstract | CrossRef Full Text | Google Scholar

Yeom, H. G., Kim, J. S., and Chung, C. K. (2014). High-accuracy brain-machine interfaces using feedback information. PLoS ONE 9:e103539. doi: 10.1371/journal.pone.0103539

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, G. P. (2000). Neural networks for classification: a survey. IEEE Trans. Syst. Man Cybernet. Part C (Applications and Reviews) 30, 451–462. doi: 10.1109/5326.897072

CrossRef Full Text | Google Scholar

Zhang, Y., Schwartz, A. B., Chase, S. M., and Kass, R. E. (2012). Bayesian learning in assisted brain-computer interface tasks. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2012, 2740–2743. doi: 10.1109/EMBC.2012.6346531

PubMed Abstract | CrossRef Full Text | Google Scholar

Appendix: Literature Search Methodology

To generate a list of papers for the review we performed a search on Scopus using the following search criteria, restricting the results to articles or conference proceedings:

Topic = (neuroprosthetic OR neurorobotic OR exoskeleton OR neurostimulator OR (brain AND interface)) AND Topic = (control OR controller AND (scheme OR algorithm OR strategy)) AND Topic = (closed-loop OR closedloop). The results were limited to Articles and Conference Proceedings from 1990 to 2015.

With the above search we obtained 147 papers from Scopus. We excluded papers that were from unrelated fields, as well as papers that described the design or fabrication of an exoskeleton without reference to a control system, or that described a control strategy used by an exoskeleton or rehabilitative device without a nervous system interface. We excluded papers that discussed computing architectures for experimental design, or detailed the fabrication of electronics. Once duplicates and excluded papers were removed we were left with 85 articles.

The papers were divided into five categories based on their primary topics.

Control Strategy

Papers mainly describing the design of an algorithm. Papers comparing the performance of algorithms by a particular neuroprosthetic. Papers describing a change to the control loop such as the addition of feedback or the use of additional signal processing.

Neuroprosthetic Design

Papers describing a neuroprosthetic system, including interface site, electrode fabrication, sensors, architecture and algorithms.

Device Testing

Papers that describe the performance of an experimental neuroprosthetic. Papers that describe an experimental neuroprosthetic in a non-human test subject. Papers that compare the performance of a neuroprosthetic in a patient with an existing clinical therapy.

Simulators

Papers that describe a simulation environment for development or testing of neuroprosthetics, or papers that describe a model of the nervous system for a neuroprosthetic device or control algorithm to interact with.

Reviews

Papers that are reviews of neuroprostheses, algorithms or interfaces.

Keywords: neuroprosthetics, control theory, closed-loop, brain-machine interface, feedback

Citation: Wright J, Macefield VG, van Schaik A and Tapson JC (2016) A Review of Control Strategies in Closed-Loop Neuroprosthetic Systems. Front. Neurosci. 10:312. doi: 10.3389/fnins.2016.00312

Received: 07 March 2016; Accepted: 21 June 2016;
Published: 12 July 2016.

Edited by:

Michela Chiappalone, Istituto Italiano di Tecnologia, Italy

Reviewed by:

Paolo Del Giudice, Italian National Institute of Health, Italy
Dennis J. McFarland, Wadsworth Center for Laboratories and Research, USA

Copyright © 2016 Wright, Macefield, van Schaik and Tapson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: James Wright, james.wright@uws.edu.au