Neurorobotic Models of Neurological Disorders: A Mini Review

Modeling is widely used in biomedical research to gain insights into pathophysiology and treatment of neurological disorders but existing models, such as animal models and computational models, are limited in generalizability to humans and are restricted in the scope of possible experiments. Robotics offers a potential complementary modeling platform, with advantages such as embodiment and physical environmental interaction yet with easily monitored and adjustable parameters. In this review, we discuss the different types of models used in biomedical research and summarize the existing neurorobotics models of neurological disorders. We detail the pertinent findings of these robot models which would not have been possible through other modeling platforms. We also highlight the existing limitations in a wider uptake of robot models for neurological disorders and suggest future directions for the field.


INTRODUCTION
Modeling is widely used in contemporary biomedical research to gain insights into pathophysiology and treatment of neurological disorders. Models may share physiological, behavioral, or other characteristics with humans yet allow for manipulations that are not possible in human studies, and allow faster data collection. They have contributed to our knowledge in a variety of fields by improving our understanding of pathophysiology, from genetics and molecular biology to systemic neural disturbance and behaviors, and guide the search for therapeutic solutions (Ericsson et al., 2013). However, the use of animal models for predicting the effectiveness of treatment strategies in clinical trials has remained controversial due to (1) their translational failure, which may be explained in part by methodological flaws in animal studies and critical disparities between the animal models and the clinical trials testing treatment strategies (van der Worp et al., 2010), and (2) the guiding principles for ethical use of animals in research, in particular the "Replacement, Reduction, and Refinement" principle (Russell and Burch, 1960). Furthermore, there is increased focus in ascertaining the impact of social interactions on neurological disorders (Kennedy and Adolphs, 2012). The rise of computational modeling and robotics may complement and overcome some of these limitations. This review aims to expand and summarize all published robot models designed to mimic prevalent neurological disorders by detailing their methodologies, experimental paradigms, and translational outcomes. It further aims to contextualize robotics within the various modeling platforms; and act as a reference guide to allow further development of the approach.

Computational Neural Models
Computational neural models can focus on specific parts of the nervous system in isolation, or account for changes to a single neural population in a complex multilevel system, as well as extract highly-detailed data that are difficult to obtain from animal models on causative mechanism, neural dynamics, and treatments (Humphries et al., , 2018. Nevertheless, computational neural models are limited by being a partial implementation of a complete system. For example, fully modeling the human brain is not yet computationally tractable due its high complexity architecture-86 billion neurons running in parallel each with an average of 7,000 connections to other neurons 1 (Richardson, 2017).
Computational models can reveal behaviors that are beyond the scope of non-computational ones. For example when characterizing visual attentional deficits (i.e., neglect) resulting from stroke during viewing a straight line, Mozer et al. (1997)'s computational model showed that neglect was related to the total line length, despite the patient not apprehending the entire line. Additionally, electrophysiological data from animal models, such as monkey single-cell recordings during a visual attention task (e.g., Lanyon and Denham, 2004), can be used to implement computational models. Due to the ability to adjust neural function, computational models are able to explain symptoms of disorders using simple and neurologically plausible principles, such as the prediction error minimization mechanism to explain autism symptoms even if the underlying pathophysiology has not yet been identified (Lawson et al., 2014).
Parkinson's Disease (PD) mechanisms are frequently contextualized by network models, allowing easy translation to computational modeling (Albin et al., 1989;McGregor and Nelson, 2019). A variety of computational models of PD exist mostly focusing on the basal ganglia circuit (Humphries et al., 2018). For example, Frank (2005) designed a neural network model of the basal ganglia during learning and execution of a cognitive-weather prediction task. This model showed that the basal ganglia modulates the execution of actions that are being considered in the frontal cortex, including working memory updating, through phasic changes in dopamine, and that reduced dynamic range of dopamine can explain both PD and dopamine-overdose through a single underlying dysfunction. Dovzhenok and Rubchinsky (2012) and Caligiore et al. (2019) explored PD motor symptoms using a model of basal gangliathalamo-cortical loops. Dovzhenok and Rubchinsky (2012) utilized a simple model of a single neuron representing each area, whereas Caligiore et al. (2019) represented the areas with neural populations and introduced multiple interventions representing different dopaminergic dysfunctions. In another experiment, Kumaravelu et al. (2016) used a similar cortex-basal ganglia-thalamus network in the 6-OHDA lesioned rat model of PD to simulate different modes of deep brain stimulation.

Neurorobotic Models
Embodiment of computational neural models in a robot allows for direct sensory-action closed-loop interactions with a real environment as well as with environmental noise, improving the generalizability of experiments to real world scenarios. This can be advantageous for both understanding human cognition (e.g., Chaminade and Cheng, 2009;Di Nuovo et al., 2013) and neurological or psychiatric disorders, particularly because the latter are often characterized by behavior and interaction (e.g., Smith and Gasser, 2005;Asada et al., 2009). For example, nearly all psychiatric and many neurological disorders diagnoses are made clinically, without reliable imaging or biomarker tests (WHO, 1993), thus robotic models are natural candidates to study them by embodying the symptoms as a starting point, in comparison to purely computational models that rely on neuropathological knowledge to construct. One suitable candidate for neurorobotics studies is PD because many of the symptoms seen relate to environmental interactions (Poewe et al., 2017). Robot models can both inform biological data and be hypothesis-generating. Robotics allows for precise implementation of theoretical models and consequential controlled manipulation that is not possible in animal models because of ethical reasons or limited methodology. Robot models increase the replicability of experiments and avoid the issues of subject fatigue allowing for both more and quicker experiments, as well as avoiding the effects of fatigue on performance. Therefore, robot models use resources efficiently and minimize animal studies.

Neurorobotic Models Using Virtual Simulation
Computational models have been implemented into virtual robot simulations, thus bypassing the actual construction of the robot but compromising by having an equally simulated environment. These have included more simplified machines such as simulating a 2D robot leg using a neural system with a hierarchical central pattern generator to model leg weakness as a symptom of stroke to explore rehabilitation strategies (Ichimura and Yamazaki, 2019). Another study simulated a two-link robot arm using a basal ganglia neuron model to examine the relation of neuron potential strength and gradient with symptoms in PD through varying dopamine (Connolly et al., 2000). This model found that heterogeneity of striatal neuron potentials may be responsible for replicated dyskinetic symptoms, such as bradykinesia, rigidity, and tremor.
More complex robots have been also used in this context but with different approaches. Pio-Lopez et al. (2016) investigated the concept of active inference-a mechanism of minimizing variation in perceptual prediction-using a 7-degrees of freedom arm of a PR2 robot. They manipulated visual and proprioceptive noise to investigate prediction errors; one finding suggested that the failure of sensory attenuation would subvert movements, similarly to bradykinesia in PD. Another experiment simulated bipolar-affective disorder (BPAD) and Alzheimer's disease in a NAO robot, to investigate the role of emotion, specifically pleasure, and arousal, in memory (Allan et al., 2015). Researchers used Naïve Bayes Data Mining to train the robot then simulated a RoboCup match penalty kick using the Webots simulator. They found that emotional deregulation after a certain stress threshold in the BPAD model whereas the Alzheimer's disease model, which only memorized events of high emotional value, maintained a relatively high level of emotion throughout the experiment.
In some cases, computational neural models have been proposed to investigate disorders implicitly. For example, by modeling synaptic plasticity such as the wheeled robot of Aguilera et al. (2015). They evolved the model using a genetic algorithm, and controlled it by a network of Kuramoto oscillators with homeostatic plasticity to develop behavioral task preferences mediated by sensorimotor patterns. Similarly, Caligiore et al. (2014) used a simulated iCub robot with a transfer expert reinforcement learning (TERL) neural network to investigate sensorimotor learning during childhood development and how simulated "damage" to the network may have implications for autistic spectrum disorder (ASD).

NEUROROBOTICS MODELS OF NEUROLOGICAL DISORDERS
Here we discuss and summarize the existing neurorobotic models of prevalent neurological disorders and the experiments conducted with them (see Table 1 for a summary).

Parkinson's Disease (PD) and
Huntington's Disease (HD) Yiping et al. (2010) presented a model imitating PD and HD, another basal ganglia disorder resulting in hyperkinetic movements. The authors simulated dopaminergic neuron death in a two-channel behavior selection model of the basal ganglia, constructed with leaky-integrator artificial neurons, and optimized with a genetic algorithm. This was embedded into the control architecture of a Lego robot. The robot would alternate between wander, avoid, and play behaviors in an open space. The PD robot did not perform any behavior in almost half of the experiment and its behavior sequence separated into short fragments, compared to normal control. The HD robot, instead, had the tendency to execute any of the behaviors even if the respective importance value was small, therefore simulating the behavior patterns of the two diseases in a real-world environment demonstrating the viability of the modeling platform. However, ultimately, the model is an abstraction by substituting behavior selection for the symptoms of hypo/hyperkinesia; this limitation can be overcome by linking the neural controller to musculoskeletal architecture and incorporating further brain regions such as the motor cortex in order to mimic the physical disease symptoms.

Unilateral Spatial Neglect
Unilateral spatial neglect is a disorder of processing and perceiving stimuli, caused not by a loss of sensation but stroke or tumor, on one side of the visual field often contralateral to the damaged cerebral hemisphere, typically the posterior parietal cortex. It is a heterogeneous condition due to the variety of associated lesion locations and may required typespecific rehabilitation as a result. To explore this hypothesis, Conti et al. (2016) embedded an artificial neural network that controlled the spatial attention of an iCub robot. They replicated a human experiment by having the robot perform a spatial exploration task of exploring cubes in four different orientations using proprioceptive stimuli, followed by rehabilitation sessions. They used stochastic gradient descent with momentum to train the neural network, then "pruned" neural links to simulate lesions, and re-applied the training to simulate rehabilitation. Furthermore, they modeled specialization of the right hemisphere by incorporating a plasticity mechanism into some trials of the experiment. Their findings were in line with those in human studies, validating the use of a robot model with environmental interaction in this disorder, but additionally they were able to hypothesize novel plasticity mechanisms of the right hemisphere in spatial specialization. Their model too included a degree of abstraction by simplifying the neural controller to hemispheres rather than specific cortical regions therefore limiting anatomical correlation with humans. Nevertheless, the latter also confers an advantage by allowing study of an isolated function whereas in a complex brain other functions would be affected too by the lesion thus making it difficult to investigate in other models. Future work could also implement additional stimuli such as vision and touch.

Obsessive-Compulsive Disorder (OCD)
Obsessive-compulsive disorder is characterized by repetitive intrusive thoughts resulting in the need to perform certain actions repeatedly.  conceptualized OCD as a decision-making disorder and designed a robot based on a cybernetic and signal attenuation model within a motivationbased robot controller with an internal signal deficit (faulty interoception). The Elisa-3, an Arduino-based robot, performed a free-roam task requiring decisions to satisfy its needs-energy, integrity, and integument-by utilizing resources using three behaviors-groom (with a damage penalty), feed, and avoid. To simulate OCD, the perceived target values for integument were increased, resulting in increased grooming with many robots "dying" as a result of energy falling to zero whilst grooming and an average worse level of well-being. Interestingly, mildly raised target values improved average well-being.
The authors also further developed the model by implementing "stress"-both internal, related to its needs, and external, due to confinement . Also, rather than changing the target values to simulate OCD, this model would express OCD behaviors resulting from high "stress." They found a positive feedback loop which maintained high stress levels after the robot was confined, due to a consequential increase in internally generated stress because of not meeting its needs. Additionally, they tried different type of intervention to mimic response prevention therapy with opposite stress response depending on the type of intervention. Similar to Conti et al. (2016)'s model previously, the ability to monitor internal parameters through a robot model led to their findings above. These models are abstract without an artificial neural network but allow investigation of OCD, where the pathophysiology has not been ascertained; future work can incorporate an artificial neural network and simulate  pharmacological treatments to investigate the underlying neural correlates.

Schizophrenia
Schizophrenia is a psychiatric disorder characterized by symptoms of hallucination, delusion, and disorganized thought. Yamashita and Tani (2012) created a model using a SONY QRIO humanoid robot controlled by a hierarchical continuous time recurrent neural network with a multiple time scale recurrent neural network component. For more models of Schizophrenia, please check the survey paper of Lanillos et al. (2020). The robot performed a manipulation task following rules associated with object position using proprioception and vision, which generated forward predictions of the two stimuli, learned through a back-propagation through time algorithm. Schizophrenia was simulated through degrees of functional disconnection in the hierarchical network by adding random noise. Mild disconnection led to a spontaneous intermittent increase of aberrant prediction errors which affected the goal-orientation of the robot, whereas severe disconnection resulted in cataplectic and stereotypical behaviors, similarly to what is seen in patients. Translationally, they hypothesized that prediction error signals may be indistinguishable in a patient with mild schizophrenia from those generated from real stimuli and resulting in a prodromal delusional mood, demonstrating an advantage of neurorobotics modeling. This model manages to explore a non-anatomically localized network-wide neural process in a real-world environment, difficult to achieve using other models; further work could take advantage of internal monitoring to aim to concurrently simulate other schizophrenia symptoms such as delusions and hallucinations to assess whether a similar mechanism is responsible. Other models also use neural networks but focus on specific symptoms, like deficit in the recognition (Cohen and Billard, 2016).

Autism Spectrum Disorder (ASD)
ASD encompasses a range of neurodevelopment disorders affecting mostly social interaction and presenting restricted, repetitive patterns of behavior. Focusing on behavioral inflexibility, Idei et al. (2017Idei et al. ( , 2018) used a NAO robot to investigate the effects of sensory precision on adaptive behavior. They implemented a stochastic-continuous time recurrent neural network with parametric bias, which is a more complex architecture compared to the first approaches (Cohen, 1994) . For more models of ASD, please check the survey paper of Lanillos et al. (2020). The robot participated in an adaptive interaction ball-playing task using vision and proprioception with a human experimenter. It learned four different behaviors using the back-propagation through time algorithm. The level of sensory precision was increased or decreased by changing estimated sensory variance and prediction error signal level was analyzed. Both changes resulted in similar inflexible behavior patterns, such as repetition and freezing seen in patients with ASD, but due to different processes at the network level. Translationally, their findings suggest that externally-observed behavioral classification of psychiatric disorders may be limited for understanding etiology, supporting the use of neurorobotics models. This study also modeled a network-wide neural process; as a developmental disorder it would be important to simulate and assess the impact of this error mechanism on cognitive development.

Other Robot Models With Implications for Neurological Disorders
Robotic experiments not directly modeling a specific pathology can nevertheless inform pathophysiology. For example, neural networks have been used in cognitive neurorobotics to understand human cognitive functions, such as memory (Shibata Alnajjar et al., 2013) or language (Marocco et al., 2010), and therefore implications for cognitive dysfunction may be inferred.
An early basal ganglia model by Prescott et al. (2006) using the Khepera I robot in an behavioral task of collecting and consuming food modulated by motivations, such as hunger, showed that behavioral disintegration can occur when confronted with multiple high salience alternatives, with implications for the symptomatology, specifically bradykinesia, in PD. Using a similar task, Lewis and Cañamero (2016) used a NAO robot to investigate the interaction between pleasure, perception, and motivation and found that high pleasure improved viability and flexibility in adaptive behavior, with implications for obsessive-compulsive disorder (OCD). Krichmar (2013) used a CarlRoomba robot with a neural network that modeled neurotransmitters in an open-field test to demonstrate that high serotonin increased anxiety whereas high dopamine led to increased curiosity and risk-taking, which is seen in gambling addiction following long term dopaminergic treatment in PD. Conversely, low dopamine led to a more withdrawn behavior, like in PD. In a more complex model, Lones et al. (2016) used a Koala II robot with a hormone-driven epigenetic mechanism that controlled cognitive development and a novel neural network with learning capabilities to interact with environments of different levels of sensorimotor experience. They showed that the sensory-deprived robot performed worse at adaptation and learning, with potential implications for developmental disorders and rehabilitation. Lastly, neurorobotic models have also been used to investigate mental simulations, such as Seepanomwan et al. (2015) who used an iCub robot with FIGURE 1 | Neurorobotics model workflow of neurological disorders. The EMBODIMENT of the COMPUTATIONAL MODEL in a robot allows the investigation of the link between brain-body-environment with resultant modification of internal parameters via online learning. Additionally, real-time operation allows co-monitoring of internal parameters and external behavior. The model can be designed to simulate pathophysiology or be backwards designed through monitoring the behavior of the robot to mimic symptoms. Monitoring behavior and internal parameters result in hypothesis generation, translation to clinical research, and minimization of animal use. The model is robust, unfatiguable and easily modifiable, which allows for replicable and scalable experiments that efficiently use resources. a biologically-plausible decision-making mechanism to do a mental rotation task, which may be used to investigate lesions to different parts of the cortex.

DISCUSSION
The models above demonstrate the utility of robotics in investigating neurological disorders, particularly the ability to monitor both internal parameters and external behavior with the environment. They can generate insights and translational outputs by: manipulating internal parameters; simulating more abstract systemic functions, such as prediction; and monitoring internal states in conjunction with external behavior (Figure 1). Some experiments also demonstrate how these models can be used to guide therapeutics and rehabilitation. Further experiments can be designed, tailored to utilize these advantages by modeling scenarios and measuring parameters beyond the scope of other models. For example, investigating the relationships between neural plasticity, environmental interaction, movement, and cognition in neurological disorders such as PD (Humphries et al., 2019). However, the heterogeneity of robot designs and underlying artificial neural networks employed above highlights a weakness of this modeling platform due to lack of overlap between separate models. Therefore, there is a need to standardize robotic modeling of disorders to improve generalizability of experiments to humans and the interaction between disordered and normal function.  provide a 12-step 3-stage design framework for creating robot models of neurological disorders and The Neurorobotics Platform of the Human Brain Project aims to simplify the workflow and reduce the level of the required programming skills by allowing researchers to design and run basic experiments in neurorobotics using simulated robots and simulated environments linked to computational neural models that are abstraction of the brain structure and function (Falotico et al., 2017). Some of those simplified models may be suitable for specific analyses but might represent limitations as neural controllers. Nevertheless, increased dialogue between the fields of neurorobotics and computational modeling is required to improve the biological-plausibility of the neural controllers of robots. In conclusion, few neurological disorders have yet been modeled using the neurorobotics approach, each with varying methodologies, however they demonstrate that neurorobotics is a valid modeling platform for a spectrum of different pathologies with unique advantages and potential for translational output.