Memory Prosthesis: Is It Time for a Deep Neuromimetic Computing Approach?

Memory loss, one of the most dreaded afflictions of the human condition, presents considerable burden on the world's health care system and it is recognized as a major challenge in the elderly. There are only a few neuromodulation treatments for memory dysfunctions. Open loop deep brain stimulation is such a treatment for memory improvement, but with limited success and conflicting results. In recent years closed-loop neuroprosthesis systems able to simultaneously record signals during behavioral tasks and generate with the use of internal neural factors the precise timing of stimulation patterns are presented as attractive alternatives and show promise in memory enhancement and restoration. A few such strides have already been made in both animals and humans, but with limited insights into their mechanisms of action. Here, I discuss why a deep neuromimetic computing approach linking multiple levels of description, mimicking the dynamics of brain circuits, interfaced with recording and stimulating electrodes could enhance the performance of current memory prosthesis systems, shed light into the neurobiology of learning and memory and accelerate the progress of memory prosthesis research. I propose what the necessary components (nodes, structure, connectivity, learning rules, and physiological responses) of such a deep neuromimetic model should be and what type of data are required to train/test its performance, so it can be used as a true substitute of damaged brain areas capable of restoring/enhancing their missing memory formation capabilities. Considerations to neural circuit targeting, tissue interfacing, electrode placement/implantation, and multi-network interactions in complex cognition are also provided.

Memory loss, one of the most dreaded afflictions of the human condition, presents considerable burden on the world's health care system and it is recognized as a major challenge in the elderly. There are only a few neuromodulation treatments for memory dysfunctions. Open loop deep brain stimulation is such a treatment for memory improvement, but with limited success and conflicting results. In recent years closed-loop neuroprosthesis systems able to simultaneously record signals during behavioral tasks and generate with the use of internal neural factors the precise timing of stimulation patterns are presented as attractive alternatives and show promise in memory enhancement and restoration. A few such strides have already been made in both animals and humans, but with limited insights into their mechanisms of action. Here, I discuss why a deep neuromimetic computing approach linking multiple levels of description, mimicking the dynamics of brain circuits, interfaced with recording and stimulating electrodes could enhance the performance of current memory prosthesis systems, shed light into the neurobiology of learning and memory and accelerate the progress of memory prosthesis research. I propose what the necessary components (nodes, structure, connectivity, learning rules, and physiological responses) of such a deep neuromimetic model should be and what type of data are required to train/test its performance, so it can be used as a true substitute of damaged brain areas capable of restoring/enhancing their missing memory formation capabilities. Considerations to neural circuit targeting, tissue interfacing, electrode placement/implantation, and multi-network interactions in complex cognition are also provided.
Keywords: deep learning, neuromimetic architecture, neuromimetic computing, closed loop stimulation, memory implants Memory is important in our lives. It is our brain's filing system. Without memory we are unable to remember our past experiences and our loved ones, yet be able to think about the future. Without memory we cannot learn anything. Loss of ability to remember is one of the most dreaded afflictions of the human condition and presents considerable and rising social and economic costs on the world's health and social care systems in the context of the increasing aging of the world's population. Brain disorders such as Alzheimer's disease (AD) and Traumatic Brain Injury (TBI) lead to profound memory deficits and are recognized as major challenges and one of the most important causes of disability in the elderly.
Direct deep-brain stimulation (DBS) has emerged in the last decade as a neuromodulation technique to treat memory dysfunctions (Hu et al., 2009;Arrieta-Cruz et al., 2010;Laxton et al., 2010;Stone et al., 2011;Boggio et al., 2012;Lyketsos et al., 2012;Suthana et al., 2012;Fell et al., 2013;Hardenacke et al., 2013;Hescham et al., 2013aHescham et al., , 2015Lee D. J. et al., 2013;Suthana and Fried, 2014;Sweet et al., 2014;Lee et al., 2015;Sankar et al., 2015;Zhang et al., 2015;Jacobs et al., 2016;Lozano et al., 2016;Rezai et al., 2016), but with limited success and contradicting results. A review of all DBS studies is beyond the scope of this article. Interested readers should refer to Bick and Eskandar (2016); Khan et al. (2019); Curot et al. (2017); Ezzyat and Rizzuto (2018) for excellent extensive reviews of the effects of DBS on all memory-related brain areas. Below I briefly review a few of these conflicting studies. In one study DBS at 50Hz applied to human entorhinal cortex (EC) enhanced spatial memory, while hippocampal stimulation did not affect performance (Suthana et al., 2012), whereas in another study DBS at 50 Hz application to both human EC and hippocampus (HC) disrupted spatial and verbal memory (Jacobs et al., 2016). In both studies DBS was applied during the encoding phase, and recall performance was tested when stimulation was off. In another study when 50 Hz DBS was applied between the encoding and recall periods in the left medial temporal lobe (MTL) of patients, then memory recall was impaired (Merkow et al., 2017). Direct electrical stimulation at 50 Hz in HC, parahippocampal regions, prefrontal cortex and lateral temporal cortex (LTC) found that high gamma activity induced by word presentation was decreased in regions where stimulation decreased memory performance, and increased in LTC where memory enhancement was observed (Kucewicz et al., 2018). In other studies, memory impairment was observed when both hippocampi were stimulated simultaneously (Lacruz et al., 2010), but the type of impairment depended on which hippocampus was stimulated (Coleshill et al., 2004). Theta-burst micro-stimulation with physiologic level currents in the right EC during learning significantly improved memory specificity for novel portraits as well as recognition of previously-viewed photos, but not for similar lures (Titiz et al., 2017). On the other hand, theta-burst stimulation of human MTL resulted in spatial memory retrieval impairment (Kim et al., 2018). Theta-burst stimulation in amygdala or fornix (FX) in humans led to visuospatial memory enhancement (Miller et al., 2015;Inman et al., 2018). Chronic DBS at 130-450 Hz for several months showed no significant or subtle improvement in memory (Velasco et al., 2007;McLachlan et al., 2010;Boëx et al., 2011;Miatton et al., 2011). Bilateral 20Hz DBS of nucleus basalis of Meynert (NBM) showed memory improvement when stimulation was applied at an earlier stage of dementia and a younger age cohort (Kuhn et al., 2015). Bilateral DBS of anterior thalamic nucleus (ATN) of an epilepsy patient cohort showed greater subjective memory impairment when the stimulation was on and improved word fluency and verbal memory (Fisher et al., 2010;Oh et al., 2012).
Similar conflicting results have been observed in animal studies. Intermittent stimulation in NBM in adult monkeys enhanced working memory, but continuous stimulation led to memory impairment (Liu et al., 2017). EC stimulation in rats promoted neurogenesis in dentate gyrus and enhanced spatial memory in a water maze task in a manner dependent on neurogenesis (Stone et al., 2011). Chronic DBS in Alzheimer's disease (AD) mice improved performance in Morris water maze task with AD-DBS mice spending more time at the novel object and location than with AD-no stimulation mice (Mann et al., 2018). EC, FX, and region CA1 stimulation during a spatial memory study restores performance in a rat scopolamine injection dementia model (Hescham et al., 2013b(Hescham et al., , 2015, whereas in another study DBS of EC and FX showed significant HC-dependent spatial memory improvement in Morris water maze than in ATN DBS (Zhang et al., 2015). HC-independent recognition memory was also enhanced by EC and FX DBS, but not with ATN DBS (Zhang et al., 2015). Lowcurrent stimulation of rostral intralaminar thalamic nuclei in rats just prior to memory retrieval in a delayed match-to-sample task improved performance, whereas high-current stimulation impaired it (Mair and Hembrook, 2008).
These conflicting results are due to methodological differences across human and animal studies including but not limited to details in participants (age, cognitive, and neurologic abnormalities), animal species (rats, mice, monkeys), behavioral task design, electrode characteristics (e.g., electrode geometry), electrode placements (location), stimulation parameters (amplitude, impedance, frequency, duration, charge density), timing of stimulation (during encoding phase, during retrieval phase, in-between encoding, and retrieval), mode of stimulation (intermittent, chronic, continuous) and statistical analysis methods (Montgomery and He, 2016;Suthana et al., 2018). Open-loop DBS generates only pre-programmed high frequency electrical stimulations without being able to receive feedback from the current brain state. Because of its therapeutic effectiveness, clinical innervations have so far preceded the scientific understanding of its mechanisms of action (McIntyre et al., 2004).
Future advances in memory prosthesis technology should thus address fundamental questions on its therapeutic mechanisms of action. They should also be closed-loop (i.e., receive feedback from the current brain state), capable of online selfadaptation to time-varying environments, and amenable to lowpower hardware implementations for memory restoration and rehabilitation (Senova et al., 2018). They should be able to simultaneously record neural signals during behavioral tasks and then with the use of internal factors of the neural state determine the precise timing of stimulation (e.g., stimulating at a particular phase of an ongoing endogenous neural oscillation), or make the decision whether to stimulate at all (Hampson et al., 2013;Deadwyler et al., 2017;. Developments toward the latter direction have already been attempted (Berger et al., 2008(Berger et al., , 2011Deadwyler et al., 2017;Ezzyat et al., 2017. The Ramp project (Ramp project) 1 examined the efficacy of a biohybrid architecture of tightly coupled natural and neuromorphic hardware neurons. CoroNet (Coronet FP7 project) 2 developed the scientific and technological foundations for future "bio-hybrid" devices that will combine biological and artificial nervous tissues. DARPA's RAM project (DARPA RAM project) 3 aims to develop and test a wireless, fully implantable neural-interface medical device for human clinical use. The Human Brain Project (Human Brain Project) 4 although not directly contributing in the biohybrid/implant direction, it indirectly contributes to it with its neuromorphic hardware (1Mio cores Spinnaker machine) and brain simulation platform.
The first stride toward a closed-loop implantable memory prosthesis system was conducted by Berger et al. (Song et al., 2009;Berger et al., 2010Berger et al., , 2011Hampson et al., 2012) as an artificial bridge between the chemically lesioned CA3 and CA1 synaptic connections in a rat's hippocampus, when the animal was trained to perform a delayed non-matched sample (DNMS) task. The chip consisted of three components: (1) a recording multi-electrode array (MEA), (2) a very large scale integration (VLSI) implemented multi-input multi-output (MIMO) prediction model of neural activity based on the recorded neural signals, and (3) a stimulating MEA driven by the MIMO predicted neural activities. The MIMO predicted spiking neural activity was based on five electrophysiological mechanisms: (i) a feedforward process transforming the input MEA recorded spike train to a synaptic potential, (ii) a feedback process generating an after-potential caused by the output spike, (iii) an intrinsic neuronal noise, (iv) a subthreshold potential dynamics, and (v) a threshold function to generate each output spike. When the chip was tested against the damaged CA3-CA1 connection in the lesioned rat, the animal was able to successfully perform the DNMS task with a success rate of over 90% (the success rate for a lesioned rat without the prosthetic device was <50%), demonstrating the chip as a viable memory enhancement device. A second stride toward memory improvement by the chip was made by the same group in non-human primates trained in a delayed match-to-sample (DMS) task (Deadwyler et al., 2017). Despite the chip's successes, it had several limitations. First, it was tested against a single behavioral task on a welltrained animal. That meant the model was "trained" to perform a single input-output mapping. Furthermore, the model was non-adaptive (hard-wired), unable to improve its performance through experience according to a prescribed learning rule. Initial attempts toward the latter direction have been recently made by the same group by incorporating a phenomenological spike timing-dependent plasticity (STDP) rule in an updated MIMO model (Song et al., 2014). However, its synaptic plasticity rule was far too simplistic to capture the complex molecular and biochemical dynamics of synaptic plasticity in vivo (Froemke and Dan, 2002;Froemke et al., 2005;Wang et al., 2005). Both MIMO models were completely blind to the CA3 circuit memory computations and processes during their therapeutic courses of action.
A third stride toward a closed-loop memory enhancement/restoration stimulation system was recently made by Ezzyat et al. (2017 using a machine learning (ML) approach. A set of stimulation-free trials with neural data and labels indicating memory performance was collected from 25 neurosurgical patients undergoing clinical monitoring for epilepsy while they participated in a delayed free recall memory task. A multivariate classifier model was then trained to discriminate patterns of neural activity during encoding for each particular participant. The resulting weight codes from training were then used during testing to map features of iEEG activity to an output probability value, which in turn generated appropriate stimulation patterns during a later word recall phase. Improved memory recall performance was demonstrated particularly when stimulation was timed to periods of poor memory function. Despite its memory improvement success, the closed-loop stimulation system was completely "blind" to the neurobiology of learning and memory offering no insights into the biophysical mechanisms of action of DBS stimulation of the human lateral MTL when participants perform a memory recall task.
Below, I provide few guidelines on how to construct such a system. I propose that a computational deep (multi-layered) neuromimetic circuit approach empowered with biophysically realistic learning rules mimicking the neural dynamics of memory related circuits amenable to neuromorphic VLSI hardware driven by in-vivo MEA recordings, able to decode memory engrams and stimulate memory related populations of neurons should be adopted to move forward the memory prosthesis research. Model components (nodes, synapses, connectivity) should have to mimic the operations of real neurons, synapses and circuits. Several strides toward this direction have already been made (Cutsuridis and Wenneckers, 2009;Cutsuridis et al., 2010bCutsuridis et al., , 2011Cutsuridis and Hasselmo, 2012;Schneider et al., 2012;Pendyam et al., 2013;Bezaire et al., 2016;Sanjay and Krothapalli, 2019;Yu et al., 2019). One such stride was the Cutsuridis et al. (2010b) microcircuit model of region CA1 dynamics in encoding and retrieval of memories. The study explored the functional roles of somatic, axonic and dendritic inhibition during these processes. It showed how theta modulated inhibition separated encoding and retrieval of memories in the hippocampus into two functionally independent processes. The study predicted: (1) somatic inhibition allowed generation of dendritic calcium spikes that promoted synaptic LTP, while minimizing cell output, (2) proximal dendritic inhibition controlled both cell output and suppressed dendritic calcium spikes, thus preventing LTP, and (3) distal dendritic inhibition removed interference from spurious memories during recall. Some of the Cutsurdis et al. study's predictions have been recently verified by experimental studies (Siegle and Wilson, 2014). The model should also be empowered with biophysically realistic learning rules (LTP/LTD/STDP, branch potentiation, clustered plasticity, metaplasticity, error driven Hebbian learning, etc) mimicking the processes and operations of synaptic plasticity across different timeframes (ms, seconds, minutes, hours, days, longer) in neural cells (Kastellakis et al., 2015(Kastellakis et al., , 2016Li et al., 2016). Once the model's neural dynamics has been extensively validated against experimental data from multiple levels of detail (molecular, synaptic, cellular, dendritic, micro-, meso-and macro-circuit), thus casting it as a faithful representation of a real human/animal tissue (memory circuit), then the model should be trained with real MEA recording and stimulation data from humans or animals while they are performing memory-related behavioral tasks and with verified memory restoration/enhancement effects. Deficits should be in the encoding and/or retrieval of declarative memories (or specific types of declarative memories). Behavioral memory tasks should assess performance metrics across various timeframes (hours, days, weeks, or longer) testing different memory specificities (e.g., memory of an object, event, or context in which it occurs, or high-level semantics of sets of objects/events, or an association of an object and an event linked to one another in a memory occurring either simultaneously or in a temporal sequence). MEA data should be split in training, cross-validation and testing datasets. Model's performance must be tested across individual participants and/or the whole participant population and it must be able to retain functionality across time, situational contexts, and/or experimental settings (tasks). Model robustness and generalization should be validated within and across individual human participant and/or animal and should be demonstrated by the ability of the model to restore memory function when applied to different human participants/animals and in different situational contexts.
Once the model has been computationally trained and its performance have been extensively tested across individuals, experimental settings, memory types and situational contexts, then its structure and weight codes can be transferred to a neuromorphic chip to be implanted or interfaced with indwelling probes for recording and stimulation of human and/or animal neural activity. At this point a number of other outstanding technical difficulties need to be overcome and questions to be answered: • Electrode Placement and Implantation: The exact placement and trajectory path for the recording and simulation electrodes is of paramount importance to any successful implantable neuroprosthesis system. Any slight deviation from the optimal path to the target due to lead migration or misplacement may result in adverse effects such as hemorrhage, seizures, abnormal sensations, etc or tissue damage (Edwards et al., 2017). Electrode location thus must be adjusted to maximize therapeutic effects, while minimize adverse ones (Edwards et al., 2017). Intra/post-operative imaging (e.g., MRI or CAT) scans can confirm electrode placement (Edwards et al., 2017). • Neural Circuit Targeting: The electrical field generated by a DBS macroelectrode affects the three-dimensional geometry of the surrounding to the electrode neural processes (i.e., axons and dendrites) (McIntyre et al., 2004). Knowing the anatomical distribution of the DBS electric field and controlling its shape is of utmost importance to maximize the therapeutic effect of stimulation, minimize its adverse effects, and get a deeper understanding of the DBS mechanisms of action (Klooster et al., 2016;Edwards et al., 2017). Electrode design (size, diameter, number of contacts) and directional steering is an active experimental and theoretical research area (Klooster et al., 2016). Mathematical models using finite difference or finite element methods model the electric field induced in the brain during DBS as a function of different stimulation parameters and delineate the effects the electric field has on the neural tissue. The importance of specific conductivities, encapsulation layers and steering toward the stimulation target are some of the main focuses of these studies (Wei and Grill, 2005;Johnson and McIntyre, 2008;Vasques et al., 2009;Schmidt and van Rienen, 2012a,b;Lempka and McIntyre, 2013). Recently developed neural probes have provided precision in shaping the electrical field generated during stimulation (Klooster et al., 2016). One such probe is the "SureSTIM" (Martens et al., 2011), a 64 disc-shaped electrode array arranged in 16 equallyspaced rows, which allows for both long-term stimulation and local field potential recording, while diminishes the induction of adverse effects by stimulating tissue beyond the stimulation target. • Neural Tissue Interfacing and Longevity: Brain-chip interfaces allow for chips and nerve tissue to establish a close physical interaction thus allowing the transfer of information in one or both directions (Vassanelli et al., 2012). Major operations, like cognition including memory, are sustained by the concurrent activity of a large number of neurons in complex neural networks located in several interconnected brain structures. To better understand neural circuit operations and to develop powerful brain-machine interfaces, then an interface between a semiconductor chip or an ensemble of chips and the neural tissue of a living animal allowing for bi-directional communication (not only to record but also to control neuronal activity) and high-spatiotemporal resolution sampling of a large number of neurons over the networks, and simultaneously from multiple regions of the brain is needed (Vassanelli et al., 2012). Usually small CMOS chips featuring stimulation and recording sites integrated at high-density implanted in one or in several brain areas, either independently or simultaneously, can lead to an unprecedented control of neuronal activity in the mammalian brain (CyberRat ICT 2007 project) 5 Obtaining such high spatiotemporal resolution enables to explore and control brain information processing with unprecedented detail. The chips are either directly implanted into the tissue or connected through leads that reside permanently in the brain. Wireless transmission is desired to simplify chips connectivity with the monitoring system and to remove interference with animals' movements (Vassanelli et al., 2012). Several bottlenecks are usually faced: power dissipation induced heat generation of the chips, biocompatibility and mechanical-electrical stability, particularly for chronic implantation in the freely behaving animal, chip implantation (and chip design) to match at best the 2D architecture of the array with the 3D architecture of the neuronal networks in the brain while limiting to the minimum tissue damage (Vassanelli, 2018).
• Multi-Network Interactions in Complex Cognition: For a long time, it was hypothesized that DBS worked either via functional ablation by suppressing or inhibiting the structure being stimulated or via activation of the stimulated structure (McIntyre et al., 2004). It is currently accepted that DBS changes network-wide oscillations and there may be coherence between cortical and subcortical brain signals (Wagle Shukla and Okun, 2012;Lee H. et al., 2013). Are these changes though due to a widespread DBS electric field affecting circuits/areas/regions well beyond the stimulated one (global effects) or due to a localized electric field affecting only the DBS brain circuit/region/area, which in turn drives other connected with it brain circuits/regions/areas (local effects)? A notable study on uncovering the mechanisms of whole-brain dynamics of deep brain stimulation has shown that DBS shifts global brain dynamics of patients toward a healthy regime with the effect more pronounced in specific brain areas (Saenger et al., 2017). Higher communicability and coherence in brain areas were measured when DBS was on than then it was off (Saenger et al., 2017).
Overall, to accelerate progress in memory prosthesis technologies then a closed-loop deep neuromimetic circuit computing approach empowered with biophysically realistic learning rules mimicking the neural dynamics of memory related circuits amenable to neuromorphic VLSI hardware driven by in-vivo MEA recordings, able to decode memory engrams and stimulate memory related populations of neurons should be adopted. Such software novelties along with multimodal neuroimaging, electrophysiological and electrochemical monitoring technologies and innovative neural probe engineering advances (e.g., SureSTIM) could then act as true substitutes (bridges) of damaged memory-related brain areas capable of restoring/enhancing their missing memory formation capabilities as well as deciphering their mechanisms of action.

DATA AVAILABILITY
No datasets were generated or analyzed for this study.

AUTHOR CONTRIBUTIONS
The author confirms being the sole contributor of this work and has approved it for publication.

ACKNOWLEDGMENTS
Authors would like to thank Stefanos Kollias, Jonathan Erichsen and the two reviewers for comments on earlier versions of the manuscript.