A neuroprosthetic is a device or system that has an interface with the nervous system and supplements or substitutes functionality in the patient's body. For the purpose of this review we have included neuromodulatory systems and brain computer interfaces under the general description of neuroprosthetics, as well as rehabilitation systems such as exoskeletons. The key identifying characteristic of the neuroprosthetic is that it has an interface with the subject's nervous system, as distinct from an implantable devices such as an pacemaker or an insulin pump. Consequently there are a broad range of devices that we consider neuroprosthetics. To date there have been a number of reviews of neuroprosthetic systems. There is significant literature on classification algorithms and detection strategies (Schwartz, 2004; Lotte et al., 2007; Green and Kalaska, 2011; Borton et al., 2014; Morimoto and Kawato, 2015), including comparison and evaluation of the relative strengths of different approaches. However, there are fewer examinations the authors are aware of that investigate the different control approaches that have been implemented in neuroprosthetic settings. Performing such a review is made more difficult due to the small number of studies that have compared different control approaches within the same experiment, and the fact that many neuroprosthetic studies have by necessity been conducted with very small sample sizes, sometimes involving a single subject. Additionally many of the devices examined in this review are experimental or investigational, and are not yet in use in clinical or therapeutic settings (Sun and Morrell, 2014).

When considering these devices in the context of control and feedback it can be helpful to place them along a number of axes (Figure 1) to partition the large variety of systems and approaches. The first axis we have considered is the location of the interface with the nervous system, with the Central Nervous System (CNS) subdivided into the brain and the spinal cord, and then the Peripheral Nervous System (PNS), consisting of afferent and efferent pathways. Along this axis and additional distinction can be drawn between single channel systems that use a single electrode as the interface to the subject's nervous system, and multichannel systems which utilize many parallel channels for interface. Finally the channel can be unidirectional for simplex communication, or bidirectional for half-duplex, or full duplex communication.

Another axis to consider is the invasiveness of the interface. At one end we place noninvasive interfaces such as Electroencephlogram (EEG), Magnetic Resonance Imaging (MRI), and Electromyography (EMG). These interfaces are hampered by a variety of issues, including low spatial resolution and low signal to noise ratios. The advantage they enjoy is that they can be deployed with little risk, which has meant they are a popular platform for neuroprosthetic development. Next are minimally invasive interfaces such as microwire EMG and microneurographic recordings from the PNS. With a more invasive interface there are better quality signals, but still minimal intervention with the subject's body, reducing risk of complications. These interfaces are susceptible to movement, leading to changes in the quality of the recorded signal. The most invasive interfaces require surgical implantation, and come with risks associated with surgery as well as risk of damage to the part of the nervous system. Within this category a distinction can be drawn between the less invasive, such as cuff electrodes around PNS sites and Electrocorticogram (ECog)—these interfaces do not disrupt the blood brain barrier; and the more invasive Multi Electrode Arrays (MEA) in PNS nerves, motor or somatosensory cortex, Deep Brain Stimulators (DBS) to treat Parkinson's Disease with electrodes in the Globus Pallidus, Sub Thalamic Nucleus, or the Pedunculopontine Nucleus. These multichannel interfaces offer high resolution, but comparatively small spatial coverage (Krook-Magnuson et al., 2015). These electrodes come with risks that are still not fully understood. These primarily involve the physical trauma due to the insertion of the electrode, with effects occurring over different timescales. Shortly after insertion there is bleeding and swelling, as well as physical damage to neurons (Fernandez et al., 2014). Over longer timescales the presence of the recording device compromises the blood-brain barrier, allowing ingress of cellular and molecular components from elsewhere in the body (Schwartz, 2004). Additionally the long term stability of the recording site is often compromised, due to immune and mechanical actions on the microelectrodes (Krook-Magnuson et al., 2015). At the furthest extent of this axis are the interfaces that reconfigure the subject's nervous system. Targeted Muscle Reinnervation (TMR) surgically rewires an amputee subject's PNS, by deinnervating muscles that have no biomechanical role after the amputation and redirecting the preserved nerves from the amputated limb to the deinnervated muscles (Kuiken et al., 2009), allowing for high quality EMG recordings as control inputs to a prosthetic. Optogenetic techniques offer a non-electrical interface to neurons, by using light to activate special ion-channels. This technique enables individual neurons to be targeted, which is extremely difficult with cortical microelectrodes, as well as the possibility of selectively activating a class of neurons such as excitatory neurons instead of inhibitory (Krook-Magnuson et al., 2015). However, optogentic approaches require the introduction of genes that encode for light activated ion-channels (Deisseroth, 2011), and the issues associated with this may preclude this technique in humans.

The final axis is the modality of the prosthesis. Sensory neuroprosthetics offer input from artificial sensors, as in a cochlear implant or bionic eye, or modulate sensory input as is the case in neurostimulators for treating chronic pain. Systems to treat motor impairment are frequently referred to as Brain Machine Interfaces (BMI) or Brain Computer Interfaces (BCI) and infer motor intent from the subject in order to control a virtual or physical effector. Cognitive devices modulate the activity of the CNS and include devices such as DBS stimulators for Parkinson's Disease, depression, and hippocampal stimulators for memory.

The following examples are representative, but not exhaustive, and are placed in Figure 1 to illustrate the wide variety of devices that can be described as having a neuroprosthetic interface. Many of these devices are experimental demonstrations, and not clinically approved interventions. Adjacent to the choice of controller, there are choices about the location and method for acquiring a suitable input signal (Grill et al., 2009; Andersen et al., 2010), and questions about the consequences of long term use of these and similar devices that have not yet been answered.

Peripheral nervous system interfaces are attractive as they are less invasive than the central nervous system alternatives, while still offering a rich source of information for neuroprosthetic control. Inmann and Haugland (2004) used a cuff electrode around the median nerve to record nerve activation due to touch and that input was used to modulate the Functional Electrical Stimulation of the subjects muscles. Horch et al. (2011) demonstrated that human subjects who were provided with intrafascicular electrical stimulation of the median and ulnar nerves derived from sensors on a myoelectric prosthetic limb were able to use the feedback to perform object recognition tasks by digital manipulation. Tan et al. (2014) used non-penetrating cuff electrodes on the median, ulnar and radial nerves of human subjects to provide natural sensations of touch while operating a myoelectric prosthetic allowing for improved performance of manual tasks. Raspopovic et al. (2014) showed that a bidirectional interface with the median and ulnar nerves could be used to provide artificial sensory feedback related to the forces exerted on a sensorized prosthetic limb, and that the artificial sensations allowed the subject to improve their ability to sense characteristics of the objects being manipulated. Targeted Muscle Reinnervation generates a rich high density signal for surface Electromyography (EMG) that enables simultaneous operation of multiple degrees of freedom in a myoelectric prosthetic limb (Kuiken et al., 2009). The tissue serves as a bioamplifier for the nerve signal, allowing a large array of surface electrodes to be deployed on the subject. The array provides a rich signal suitable for pattern recognition, and combined with a high performance prosthetic limb gives the subject an improved experience. As a sensory modality prosthetic, (Dommel et al., 2009) are testing a vision prosthesis for electrical stimulation of the retina. Spinal cord stimulation may be able to generate gait patterns suitable for locomotion in paralyzed patients. Vogelstein et al. (2008) describes the design of a system that is capable of generating primitive locomotion in anesthetized felines. Borton et al. (2014) developed an electrochemical spinal neuroprosthesis to reactivate the circuits in a damaged spinal cord, allowing hindlimb movement sufficient to enable walking in paralyzed rats.

Yanagisawa et al. (2011) used ECog electrodes placed over the sensorimotor cortex of a stroke patient in order to control a supernumerary robotic hand that was able to mimic the hand posture of the subject. Berger et al. (2011) implanted microwire electrodes in the hippocampus of rats and recorded the activity while the animals were trained to complete a memory task. Subsequent stimulation of the electrodes according to the trained model improved the performance of the rats at the cognitive task. Neuromodulators for Deep Brain Stimulation have been used to treat the symptoms of Parkinson's disease and depression (Grahn et al., 2014). Multi Electrode Arrays implanted in motor cortex have been successfully used to acquire signals for the multi degree of freedom control of robotic limbs (Hochberg et al., 2012; Wodlinger et al., 2015). Guggenmos et al. (2013) describe a Brain Machine Brain Interface in rats that utilizes microwire recordings from a premotor area to detect spiking activity leading to stimulation of the somatosensory area. The Activity Dependant Stimulation via the neuroprosthetic prototype enabled rats with injury to the motor area to recover reach and grasp behavior.

This list of devices is not exhaustive, and serves only to illustrate the different axes of Modality, Invasiveness and Location when considering neuroprosthetic designs. For a more thorough description of the available neuroprosthetic devices and interface technologies, the reader is directed to the following excellent reviews (Navarro et al., 2005; Grill et al., 2009; Micera et al., 2010; Ortiz-Catalan et al., 2012). These reviews focus on the interface techniques with the nervous system, and provide a detailed discussion of the limits of current interfaces.

The design of a neuroprosthetic varies significantly between different modalities. Because it is a multidisciplinary field often the language used to describe the system can vary between devices. The terms “closed-loop,” “feedback,” and “online” may take on different meanings. Identifying and acquiring a suitable input signal is a non-trivial task (Krook-Magnuson et al., 2015). This makes the fabrication of a substitute system in the case of impairment a complex endeavor. Developing a suitable simplified model for embodiment as an open-loop system is often a first step. But it is not always straightforward; take motor control as example—many models of muscles exist. Highly biophysical models (such as cross-bridge models) of muscles become large systems of non-linear differential equations when describing whole muscles or limbs (Ionescu and De Keyser, 2006). Similarly modeling the individual neurons that act as part of the Basal Ganglia, in a complex network of interconnections across the CNS (Broccard et al., 2014), rapidly becomes an intractable problem when developing an open-loop model for DBS to treat Parkinson's Disease. As a consequence the Basal Ganglia has often been modeled internally by a neurologist (Hosain et al., 2014), rather than explicitly within the DBS device, with stimulation parameters adjusted by the clinician observing the patient's symptoms instead of in response to a model. It is also possible that the control problem for a given neuroprosthetic has more than a single loop that needs to be considered, possibly due to the interaction of different physiological systems and different timescales (Houk, 1988). In this case control may best be achieved by a hierarchy of controllers, or a series of adaptive controllers that can be tuned at different stages of design (McFarland et al., 2006).

Table 1 summarizes the Control Algorithms.

Control policies can be implemented by classification, model independent, or model based approaches (Kameneva et al., 2015).

There can be more than one feedback loop in the neuroprosthetic system (Broccard et al., 2014). Feedback can be the visual observation of the robotic effector as it is in many motor BMIs, allowing the operator to modulate their neural activity before it is decoded. Or in the case of a DBS neuroprosthetic the feedback signal may be acquired from recording electrodes implanted alongside the stimulating electrodes, in which case the feedback signal is returned directly to the device (Herron and Chizeck, 2014). Bidirectional interfaces, via the PNS or the CNS, enable the transformed signals of sensors on the robotic effector to be transmitted into the operators nervous system and interpreted as a sensory percept (Armiger et al., 2013). Feedback can also be from sensors on the robotic effector directly to the controller, bypassing the operator (Broccard et al., 2014; Markovic et al., 2014). And finally, reversing the direction of information, natural sensors can be used as a source of feedback for the controller (Holinski et al., 2013; Mendez et al., 2013; Wright et al., 2015) and the brain can be used as the source of the error signal (Mahmoudi and Sanchez, 2011).

We have reviewed the control policies employed by recent neuroprosthetic systems. For the purposes of this review we included motor BMIs, assistive devices, neuromodulatory systems, and other devices with an interface into a subject's nervous system. Many of the devices reported on in the literature are being developed in non-human primate or other animal models, only a subset have been tested in human subjects. There is a frequent conflation of detection and classification approaches with digital or Bang-Bang control within the literature.

Although closed-loop systems have been demonstrated experimentally there remain significant limits on our ability to describe the activity in the brain, and consequently develop control policies to respond to that activity. Simulation of cortical activity (Ehrens et al., 2015; Sandler et al., 2015; Vlachos et al., 2016), the use of experimental platforms (Keren and Marom, 2014), and the use of animal models has enabled the development of a wide range of neuroprosthetic systems. However, the appropriate method to transition these systems in human subjects is not clear. Among the difficulties is the body's response to chronic implantation of microelectrodes (Fernandez et al., 2014), the appropriate place to collect a signal (Krook-Magnuson et al., 2015), and the possibility that longterm attempts to control a cortical system may compromise some of the desirable behavior (Keren and Marom, 2014).

Comparisons of control policies across different modalities, interfaces and levels of invasiveness are difficult. Even within a given neuroprosthetic category it can be difficult to perform a comparison due to the wide variety of task designs, different subject training regimes and varying reported metrics; (Koyama et al., 2010; Sussillo et al., 2012) are rare exceptions. Examination of Table 1 reveals that the motor BMI discipline appears to have explored the widest variety of control policies, and that the use of the Kalman Filter as part of a closed-loop system has broad support. Improvements to the traditional Kalman Filter to allow non-linear models of neural activity combined with its ability to be implemented in real-time continue to make it an attractive approach.

Closed-loop motor BMI systems have had significant success with closed-loop decoder adaption (CLDA; Shanechi and Carmena, 2013), supporting the use of closed-loop control. However, the CLDA approach has identified two distinct strategies in motor neuroprosthetics—decoding vs. learning. The decoding approach aims to read the natural motor plan whereas the learning approach monitors the changing neural activity as the brain learns to operate the prosthetic. It is not known at this stage if a similar duality of strategies will be applicable in other modalities. The difficulty in specifying a model for use in many of the control policies previously describes arises from our continued uncertainty about specific action of many of the components of the nervous system. It has been observed that users of Cochlear Implants have improved speech recognition performance after completing training with the device (Doucet et al., 2006), which may argue for a learning interpretation. An important caveat for the learning approach is that the neuroprosthetic system must be stable as regards the interface and transform of the input signal to the effector output, to allow the subject the opportunity to develop the “prosthetic motor memory” necessary for skillful operation (Carmena, 2013).

Neuroprosthetic development of closed-loop systems has been driven in part by the concern that the risks involved in highly invasive interfaces need to be mitigated by a strong case for the therapeutic benefit. Patient abandonment for upper limb prosthetics is high, with many wearers ceasing upper limb prosthetic use within 12 months of receipt of the device. Concerns cited by users are weight, appearance and difficulty of use (Biddiss and Chau, 2007). Extrapolating to more invasive systems, it may be difficult to argue the cost benefit if patient dissatisfaction is very high. Although these devices cannot be abandoned in the same manner as a detachable prosthetic limb, there is some suggestive research indicating an unwillingness to participate in experimental trials, which may suggest that the perceived benefit of neuroprosthetic systems by the target patient populations remains low (Illes et al., 2011). By improving control we can offer improved functionality and increased therapeutic benefit.

JW performed the review and drafted the manuscript. JT, AS, and VM revised the manuscript.