Spoken Digit Classification by In-Materio Reservoir Computing With Neuromorphic Atomic Switch Networks

Introduction

Speech recognition is a seminal task in the field of artificial intelligence and natural language processing. Typical algorithmic approaches to speech recognition break apart sections of raw speech data and bin them into hidden Markov models manipulating Markov chains. While effective, these approaches are more computationally intensive than some recently developed neural network models, which may prove a more suitable compute framework for handling increasingly larger data sets (Schatz and Feldman, 2018; Mustafa et al., 2019; Deshmukh, 2020). Artificial Neural Networks (ANNs) have also been a promising avenue for more efficient speech recognition tasks which offer the benefit of being trained for natural language processing and are believed to be a more suitable candidate for handling the varied complexity of each person’s unique voice and accent. Implementation of ANNs in modern computing hardware remains computationally burdensome and often requires access to and utilization of high-performance computing clusters. A suitable hardware architecture for local execution of complex tasks such as natural language processing must be able to process dynamic, temporal data in real-time while remaining energy efficient. Memristive materials have been identified as strong candidate for such applications as they offer an opportunity to alleviate the bus latency between memory and processing elements in traditional von Neumann architectures while also performing in-memory computation with reduced power consumption (Ielmini and Wong, 2018). The nonlinear character of memristors, resulting from the underlying physics of the material itself, is essential for enabling simultaneous storage of data (memory) and performance of complex tasks with it (processing) through a relatively new technique known as evolution in-materio (Miller and Downing, 2002; Harding and Miller, 2009; Miller et al., 2014; Dale et al., 2017).

The growing field of evolution in-materio computing has sought to optimize computational architectures via evolutionary (search) algorithms (Harding and Miller, 2009; Dale et al., 2017). The materials and architectures employed vary with the desired facet of computation, but ideally these materials are computationally and energetically efficient at employing a litany of machine learning based algorithms. Utilizing a single hardware element capable of exhibiting both memory and processing alleviates the burden of busing information between two separate hardware components, reducing latency in computation (Mustafa et al., 2019). The most robust currently known architecture that combines the aforementioned elements is the mammalian brain, which has been both a foundation and inspiration toward the development of architectures which can efficiently process multi-input, chaotic, and/or time-varying (temporal) datasets.

This work focuses on the class of neuromorphic computing devices known as Atomic Switch Networks (ASN), comprising a highly interconnected network of memristive nanowire junctions as shown schematically in Figure 1. Ongoing efforts to develop memristive hardware for neuromorphic computing include not only ASNs, but also patterned crossbar arrays, and nanoparticle clusters (Moon et al., 2019; Du et al., 2017; Alibart et al., 2013; Sattar et al., 2013; Tappertzhofen et al., 2012). ASN-based devices provide a physical system with structure and functional dynamics reminiscent of the mammalian brain (Srinivasa and Cruz-Albrecht, 2012; Avizienis et al., 2012; Türel et al., 2004; Calimera et al., 2013) that has previously been employed as a computational material for applications in Reservoir Computing (RC) (Lukoševičius and Jaeger, 2009; Schrauwen et al., 2007; Snyder et al., 2012; Du et al., 2017; Goudarzi et al., 2014; Sillin et al., 2013; Fu et al., 2020). The atomic switch is a nanoscale electroionic element consisting of a Metal-Insulator-Metal (MIM) junction whose properties can be manipulated via a time-dependent input signal (Zhu et al., 2020; Kuncic et al., 2020; Manning and et al., 2018; Manning et al., 2017). Individual atomic switches have been shown to produce memristive, nonlinear responses, exhibiting both short and long-term memory as well as quantized conductance (Sattar et al., 2013; Tappertzhofen et al., 2012; Terabe et al., 2005; Hasegawa et al., 2010). For electrochemical metallization memristors filament growth is dominated by cation transport through the insulating medium as shown in Figure 1 and has been experimentally observed in-situ (Guo et al., 2007; Yang et al., 2012; Sun et al., 2019). These properties render atomic switches and other memristive systems as ideal circuit elements for use within a network architecture that can serve as a dynamic physical reservoir used to solve complex computational tasks, including speech recognition and natural language processing (Kan et al., 2021; Zhong et al., 2021).

FIGURE 1

FIGURE 1. Schematic diagram of an AgI-based ASN device, from nanowire junction to chip. (A) initial high resistance state of the system. (B) filament formation process under an applied bias (C) completed silver filament short circuits between overlapping nanowires (low resistance state). Yellow-gray represents AgI. Dark-gray represents Ag. Filament formation occurs as a gapless junction between Ag nanowires. (D) SEM image of the interconnected nanowire (scale bar = 20 um). (E) Optical image of microelectrode array at center of the ASN device (scale bar = 360 um). (F) Optical image of a complete 16-electrode ASN device (scale bar = 5 mm).

RC provides a framework for computing complex functions using a dynamical system as a “reservoir” (Lukoševičius and Jaeger, 2009; Hashmi et al., 2011; Lukoševičius et al., 2012; Sillin et al., 2013). The RC framework is ideal for the processing of dynamic, temporal real-time signals and can be used in many of the same situations as recurrent feed-forward neural networks. RC also offers advantages such as fault-tolerance and the capacity for learning (Hashmi et al., 2011; Stieg et al., 2014). Passing a time varying input through a dynamic reservoir produces a higher dimensional representation of the signal through nonlinear transformation, where different points on the reservoir are measured and linearly combined to reproduce an arbitrary output signal as shown in Figure 2. Training is only performed on the linear readout coefficients (voltage readouts are shown in Figure 3 and in Supplementary Figure S2 demonstrating a reproducible response over time); the reservoir dynamics themselves are generally considered fixed. Limiting training to the weights between the reservoir and output layer alleviates the need to use gradient-descent based methods, greatly minimizing the associated computational burden.

FIGURE 2

FIGURE 2. Overview of a traditional software-based reservoir (top) in contrast to the ASN acting as an in-materio reservoir (bottom), in which 13 MFCC’s are sequentially delivered to the ASN in the form of a time-varying voltage to a single electrode. Utilizing physical nodes enables hardware acceleration at a lower power cost. Simultaneous, real-time voltage measurements are carried out at each of the remaining 14 electrodes and provided to an output layer for regression analysis.

FIGURE 3

FIGURE 3. (A) The workflow for RC-based speech recognition using ASN-based devices involved encoding and separation of raw audio data - spoken digits - data into overlapping windows, each of which was converted into 13 MFCCs. Individual MFCCs were arranged to minimize input thrashing and then delivered as input voltage to a single electrode of the ASN device. Output data, in the form of voltage traces, was collected at all remaining electrodes. (B) The raw FSDD audio signal of “Jackson” speaking the digit zero and its subsequent conversion to a voltage signal (C). The resultant 14 voltage recordings and their unique responses are overlaid in (D) with additional detail provided in Supplementary Material.

As an alternative to simulation-driven RC, in-materio RC leverages material complexity for computational purposes (Teuscher, 2017; Konkoli et al., 2018; Tanaka et al., 2019; Nakajima, 2020). Whereas early implementations of RC simply utilized a body of a liquid acting as the dynamic reservoir, more recent works harnessed the intrinsic properties of complex physical systems, including ASNs, as the basis for a computation (Lukosevicius, 2011; Lukoševičius et al., 2012; Snyder et al., 2012; Goudarzi et al., 2014; Fu et al., 2020). Software RC has historically been demonstrated as a suitable method for a litany of complex tasks including pattern classification, signal generation and temporal based logic tasks (Tanaka et al., 2019). Hardware based approaches to RC commonly leverage photonic interactions or memristor dynamics, though photonic systems aren’t performing computations in-materio in contrast to memristors (Vandoorne et al., 2010; Tanaka et al., 2019). In-materio approaches to traditional RC have recently garnered attention as potential candidates to accelerate compute times while achieving higher power efficiency. Recent in-materio studies have demonstrated high accuracy in time-series analysis (Moon et al., 2019; Zhong et al., 2021), handwritten digit identification (Midya et al., 2019) and biosignal processing (Kudithipudi et al., 2016).

Computational neural models such as the perceptron and support vector machine can also be used as reservoirs; however, long convergence times can be a drawback depending on the task. Material-based reservoirs have the benefit of efficiently performing these tasks in-situ, enabling low-power, on-chip computing (Loppacher et al., 2003; Kuzum et al., 2012; Bürger et al., 2015). This alternative approach offers the opportunity to employ neural networks and machine learning algorithms offline, without the need to access servers, clusters and other high-performance computing infrastructures.

ASNs have been shown to represent uniquely suitable class of materials for implementation of hardware-based RC, namely complex network architectures with the requisite material complexity (Avizienis et al., 2012; Stieg et al., 2014; Nayak et al., 2010). These self-organized systems offer a unique opportunity to produce highly interconnected memristive networks, where a density of atomic switch junctions of up to 10⁸/cm³⁰ has been previously reported. The fabrication scheme, based on electroless deposition, produces a diverse ensemble of silver nanowires with varying lengths, widths and thereby junction dimensions. This structural diversity in the material substrate imparts a distribution of operational characteristics that improves the capacity to perform non-linear transformations of input signals.

Herein, we report the use of a new memristive material, silver iodide (AgI), as the functional element in the ASN framework (Liang et al., 2007; Tappertzhofen et al., 2012; Cai et al., 2013). Silver iodide can be robustly prepared in a brief vapor phase reaction of iodine vapor with silver nanowires at room temperature in contrast to the lengthy formation times at elevated temperatures of previously reported silver sulfides. This promising material provides voltage-controlled resistance in both the bulk and when integrated into crossbar architectures, rendering it suitable as a memristive material for RC applications which require non-linear transformations and quantized conductance states (Stieg et al., 2014). This work expands the catalog of investigated ASN materials by fabricating and testing AgI for non-linear, temporal computation through the classification of spoken digits.

Methods

Device Fabrication

The substrate for ASN devices, a multielectrode array enabling spatiotemporal stimulation and monitoring, was fabricated using standard thermally oxidized (500 nm) silicon wafers as the base substrate. A 16-electrode grid of Pt (150 nm) was patterned by photolithography and deposited using a negative photoresist (AZ NLOF 2020) onto a Cr or Ti wetting layer (5 nm). Liftoff was induced overnight in N-methyl-2-pyrrolidone (NMP) at 60°C. Point contact electrodes were prepared using a patterned insulating layer of SU-8 (400 nm) which was soft baked (90°C), exposed to UV, post exposure baked (90°C), developed for 3 min, and hard baked at 180°C for 30 min. An array of copper (300 nm) seed sites with 5 × 5 μm spacing in a grid were patterned onto inner point contact electrodes and deposited onto AZ NLOF 2020 via metal evaporation at 3 nm/s followed by lift-off overnight in NMP (60°C). The resultant device platforms consist of a stack of Si/SiO2/Cr/Pt-electrodes/SU-8/Cu-posts (Supplementary Figure S1) and were stored in inert atmosphere until bottom-up silver nanoarchitecture construction (Sillin et al., 2013; Demis et al., 2016).

This substrate was placed into a 50 mm solution of silver nitrate (AgNO₃) for 30–60 min. Silver nanowires formed through an electroless deposition reaction involving the reduction of silver and the oxidation of copper through the following reaction:

$C{u}_{\left(s\right)}^0+2A{g}_{\left(aq\right)}^{+}\to C{u}_{\left(aq\right)}^{+2}+2A{g}_{\left(s\right)}^0$

The ordered copper posts (5 × 5 µm) directed a density-controlled formation of interconnected silver nanowires, whereby each ASN exhibited a unique structure determined by the bottom-up fabrication of metal cations. Subsequent silver iodide was formed in a nitrogen purged and sealed glass chamber with the ASN chip suspended over a small iodine pellet. Two different experimental techniques, one under ambient conditions (5 min exposure time) and the other with added heat (30°C, 2–3 min exposure time) were employed with both techniques successfully iodizing the silver nanowires.

$2A{g}_{\left(s\right)}+I_{2\left(g\right)}\to 2Ag{I}_{\left(s\right)}$

UV-Vis and XPS samples were prepared using transparent silver thin films (20 nm). These films were deposited on glass cover slides via a silver target in a Hummer 6.2 sputter system at 15 mA from Anatech Ltd. (Hayward, CA, United States) under an argon vacuum environment (80 mtorr).

Material Characterization

Optical and scanning electron microscopy (SEM) were used to characterize the as-fabricated structure of the nanowire network. SEM images were acquired using the JEOL JSM-7500F. X-Ray photoelectron (XPS) and UV-VIS spectroscopy were employed using transparent Ag thin film substrates with Ag as a control. Absorbance spectra of thin films were collected using the HP 8453 spectrophotometer. XPS spectra were obtained on an AXIS Ultra DLD XPS instrument from Kratos Analytical. The X-ray source was Al Kα at 1,486.6 eV. Survey (1,200 eV) and high-resolution scans were integrated over 4 and 16 sweeps, respectively.

Electrical Characterization

Characterization of ASN devices involves the spatially defined stimulation and monitoring of electrical activity throughout the network in the form of current and voltage traces. All input-output signals were generated/acquired using a purpose-built software package developed in Labview in conjunction with dedicated hardware manufactured by National Instruments. A data acquisition card (DAQ) (model PXIe-6368) was used to deliver input signals routed through a shielded connector box (model SCB-68A) to the ASN device. A source measurement unit (model PXIe-4141) was used to measure current flow through the ASN at user-selected electrodes, where acquired and applied signals were routed using a 16 × 32 switch matrix terminal block (model TB-2642B). Voltage traces were simultaneously monitored at all 16 electrodes using the DAQ card. All components were housed in a National Instruments chassis (model PXIe-1078) with an embedded controller.

Prior to any FSDD output signals, each ASN was driven through an initialization (activation) process in which the electrodes were sequentially stimulated with 7 Hz triangle waves. This process was repeated with increasing voltages (0.01–1 V) to realize switching patterns within the network. The switch matrix was employed in conjunction with the DAQ to calculate the resistance of every electrode combination prior to and after initialization, where successful activation was characterized by a sharp reduction in the network-wide parallel resistance as compared to the virgin metal system. Current-voltage and voltage-voltage measurements utilized triangle wave outputs from the DAQ card. The FSDD signal outputs were also produced by the DAQ card at selected electrode locations via the switch matrix.

Reservoir Computing

The AgI ASN’s were evaluated for their potential RC applications through three different tests: non-temporal logic operations, temporal logic operations and recall of previous inputs and spoken digit classification. The non-linear XOR task was chosen for all logic operations and the assessment of the networks temporal properties as described in the Supplementary Information.

Spoken digit classification was implemented in AgI ASN devices via RC using the FSDD. The task was not performed using raw audio data, but rather using Mel-Frequency Cepstrum Coefficients (MFCCs) of the data, similar to previously reported techniques. Each 8 kHz wave-format sound file from the FSDD was zero-padded up to 1 s of recording length and then converted into MFCCs using the “python_speech_features” Python package. Mel-frequency cepstrum is a short-term power spectrum of the sound waves, using a linear cosine transform of a log power spectrum and is a nonlinear mel scale of frequency that approximates the human auditory response better than standard linear spacing of frequency components.

Default settings were used, resulting in an array of MFCCs where each 25 ms window of signal was parameterized by 13 MFCCs. Windows were offset by 10 ms, resulting in 1,287 total coefficients. To reduce device thrashing, the resulting MFCC array was flattened and fed to the network one at a time. The entire temporal sequence of the lowest-frequency coefficient was passed first, then the next-lowest-frequency coefficient’s values, and so on. The resulting 1,287 Hz signal (shown in Figure 2) was sent to an input electrode, 14 electrodes were measured, and another electrode was grounded. Both the input and 14 read electrodes were recorded at 1 kHz. For RC, the resulting voltage streams were sampled at the end of sub-windows of computation, and the entire collection of sampled recordings was linearly regressed to indicate which digit was spoken (see Figure 4). Twelve unique spoken digit recordings were used, characterized by two speakers, saying three digits, two unique times. The FSDD speakers were “Jackson” and “Theo”, the digits spoken were zero, one, or two, and the first two instances of each digit were used. As a baseline, regressions were performed on only the input electrode’s voltage reading (“input only” mode) as well as on the full electrode suite of the input electrode and the 14 readout electrodes (“reservoir” mode).

FIGURE 4

FIGURE 4. UV-Vis spectra of silver thin films before and after iodization under ambient (λ max = 433 nm) and heated to 30°C (λ max = 424 nm). The presence of surface plasmon resonance in the blank silver samples suggests the thin films are discontinuous small islands of metal formed during the sputtering process.

Results and Discussion

Material and Device Characterization

Silver nanowire networks like those shown in Figure 1B were reliably produced based on previously developed protocols. The network functionalization process requires conversion of silver nanowire junctions to silver iodide. The protocol for the formation of silver iodide was validated using UV-Vis and X-ray Photoelectron Spectroscopies (XPS). Figure 4 provides representative visible absorption spectra of as-prepared Ag and AgI thin films. Ag thin films prepared by desktop sputtering exhibited a Surface Plasmon Resonance (SPR), suggesting the presence of silver islands within the film (Bharathi Mohan et al., 2007). These results are in line with previous reports which have demonstrated that silver exposed to iodine decreases SPR intensity coupled with a buildup of excitons. An absorbance peak around 420 nm has been previously reported and longer exposure to iodine at ambient temperature yielded a red-shifted maximum, which has been associated with the formation of larger AgI particles (Bharathi Mohan et al., 2007; Gnanavel and Sunandana, 2008). XPS results shown in Figure 5 confirmed the presence of characteristic peaks for iodide 3d_5/2 and 3d_3/2 core level energies previously reported in metal iodides at binding energies of 620 and 631 electron volts (eV) which are absent in silver control samples (Kato and et al., 2015). While both functionalization protocols successfully produced AgI, the heated method was used for all ASN devices due to quicker sublimation of solid iodine.

FIGURE 5

FIGURE 5. XPS spectra of the iodine 3d_5/2 and 3d_3/2 core levels in silver-based ASN devices exposed to (sample, red) and not exposed to (control, black) iodization procedures. The two peaks at 620 and 631 eV correspond to the expected I⁻ bands for I_3/d7.

To confirm the viability of AgI networks as a physical substrate for in-materio RC, the spatially distributed nonlinear characteristics of the ASN were examined. Voltage traces acquired at each of the 14 measurement electrodes enabled the analysis of Lissajous plots (V-V) as shown in Figure 6. AgI devices demonstrated distributed nonlinear dynamics throughout the entirety of the nanowire network as a consequence of their highly interconnected nature, where a stable and reproducible nonlinear transformation of the input signal was observed. Different switching regimes emerge throughout the network (Figure 6) demonstrating different dynamics dominating spatial regions, suggesting there is a combination of switching dominated (blue, green plots) and capacitance dominated (red, pink) regions distributed throughout the network under an applied bias at any given electrode combination. The switching for mechanism for AgI junctions is accepted to arise from the formation metallic filaments between the insulating material classifying them as electrochemical metallization cells (Guo et al., 2007; Yang et al., 2012; Sun et al., 2019; Yang et al., 2013). The memristive properties of individual AgI junctions have been well characterized by Tappertzhofen et al. (2012), Sánta et al. (2020) and clearly demonstrate pinched hysteresis in their I-V curves. The unique dynamics observed in ASNs are the result of coupled memristive switching events among many interconnected junctions, where measurements at a point electrode capture the dynamics of an ensemble of memristive elements rather than a single one. Consequently, Lissajous plots of ASN device operation do not commonly produce the characteristic pinched hysteresis loops associated with individual memristive junctions. This capacity for the non-linear transformation of time-varying signals and temporal datasets renders the AgI nanowire network ideal for the performance RC-based speech recognition tasks.

FIGURE 6

FIGURE 6. Representative normalized Lissajous plots of all 16 electrodes measured simultaneously using a 7 Hz triangle waveform swept from –1 to +1 V over the course of 23 s with the grounded electrode (top left) and input signal (top right) recorded. Different colors correspond to different emergent dynamics spatially distributed throughout the network. The first sweep can be seen in all plots as indicated by the black arrow. The network demonstrated a spatially diverse system with reproducible, non-linear behavior distributed throughout the networks.

AgI Atomic Switch Networks-Based Reservoir Computing

AgI nanowire networks were evaluated for their RC potential in spoken digit recognition as shown schematically in Figure 4. To effectively benchmark the value of the nanowire network in the performance of a spoken digit classification task, linear regression was performed in two ways. First, linear regression of the input voltages only - defined as “Input Only”-was carried out in the absence of the physical reservoir. Second, the full reservoir system–defined as “Reservoir” - employed regression of both the input signal and all device outputs. Inclusion of the input signal allows the regression to more accurately discern correlations between the transformed output signals and the input itself. FSDD digits encoded as MFCCs and passed to the network as a temporal sequence at 1,287 Hz were successfully classified as shown in Figure 7. A sufficient number of training examples were found to stabilize the reservoir’s behavior, and evaluating testing data on only a single array of readout coefficients was found to be valid (Scharnhorst et al., 2017).

FIGURE 7

FIGURE 7. (Left) Performance of the spoken digit classification task using AgI nanowire networks for in-materio RC to tap the temporal sequence of spoken digit MFCCs at N different points and regressing to identify the digit spoken. Mean accuracy and standard deviation clearly shows that the “Reservoir” readout method avoided overfitting and improved task performance as compared to using the “Input Only” mode (Right). The input signal amplitude (voltage) was observed to have minimal. impact on accuracy, indicating the potential for maintaining task performance under low-power operation of AgI ASNs.

The target function was regressed by dividing the measured electrode data into N segments and using the last data point from each segment, this post-processing of voltage traces is done offline. For the “input only” mode, this means that N = 80 used 80 values in the regression. For the “reservoir” mode, this means that N = 80 used 80 × 15 = 1,200 values in the regression. To determine the accuracy at each N value, 12-fold cross-validation was employed using 11 of the audio files as training data and the 12th audio file as testing data. Each file was delivered to the device multiple times on a loop, aggregating far more than twelve tests to compute the accuracy. Nonetheless, there were only 12 unique data streams used. As a result, this problem suffered from significant overfitting, indicated by the “input only” results decreasing in accuracy as more points were used for the regression. This overfitting manifested as significant noise in the accuracy; N = 100 might give an accuracy as high as 100%, while N = 101 would give an accuracy of 54%. To account for this, the space of points N tested was divided into windows of size 25, and the average and standard deviation of accuracy within this window is shown in Figure 7. For instance, the mean and standard deviation shown at N = 100 indicate the statistics for N ∈ {88, ..., 112}. The ASN reservoir also demonstrated highly accurate results across a wide range of input voltages (0.5–10 V), suggesting potential utility of these devices for low-power applications.

These results clearly demonstrate the added stability provided by the ASN reservoir, evidenced by consistent accuracy at higher points of regression in the reservoir. The ASN’s robustness and versatility was demonstrated by its capability to discern spoken digits when stimulated by both high and low voltage signals without a significant loss in accuracy. The ASN also provided a moderate benefit in accuracy, even before the input-only lines began overfitting. The lack of overfitting on the reservoir lines could be interpreted as a side-effect of the temporal, non-linear properties of the reservoir. This is corroborated by the fact that the reservoir lines achieved higher accuracy than the input only lines, a phenomenon that could not be achieved without non-linear or temporal behavior. Rather than relying on a stream of individual values, each of which has some noise associated, the reservoir readout mode could rely on 15 such streams. Assuming the noise on each electrode is somewhat independent, averaging these channels could have significantly reduced noise.

Conclusion

Neuromorphic nanowire networks such as the ASN represent a burgeoning class of material architectures whose dynamical nature makes them uniquely suited to serve as physical substrates for hardware-based, in-materio computing. While the ever-increasing demands for computational capacity and complexity continue to challenge even the most advanced computing architectures, dynamical in-memory compute platforms such as the ASN may provide an alternative solution that is scalable, energy-efficient, adaptive, and capable of processing complex, time-varying data without the need for pre-programming or remote intervention. Expanding the catalog of memristive materials amenable to production of ASN-based devices, and thereby the diversity of network dynamics available for task performance, further increases their potential utility as a platform technology for next-generation computing applications. The new AgI-based ASN devices reported here served as a dynamic, memristive reservoir for the nonlinear transformation of temporal data and demonstrated the capacity to reliably classify spoken digits with high accuracy across a wide range of input voltages. Combined with the relative ease and low cost of the fabrication process, these AgI nanowire networks represent both a new material system that is ripe for future study and an opportunity to further develop the concept of in-materio computing toward real-world applications.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

SL and WW contributed equally to this work. AS, WW, and KS conceived and designed the experiments. SL, WW, KS, and CD performed experiments and analyzed data. All authors discussed the results and contributed to preparation of the manuscript. S.L, WW, KS, and AS wrote the manuscript. CT and JG reviewed and edited the manuscript.

Funding

This work was partially supported by the World Premier International Center for Materials Nanoarchitectonics (MANA) at the National Institute for Materials Science (Tsukuba, Japan). This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No (NSF grant number), and Semiconductor Research Corp. under Grant No (2015,209,024). The views expressed are those of the author(s) and do not reflect the official policy or position of the Department of Defense or the United States Government. Approved for public release, distribution is unlimited.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors gratefully acknowledge past group members: Cristina Martin-Olmos, Henry Sillin and Audrius V. Avizienis for their foundation work in chip fabrication, LabView programming and crystal growth, respectively. Physical ASN chips were fabricated in the UCLA Nanofabrication Laboratory at the California Nanosystems Institute (CNSI). Instruments used in this study were operated and maintained by the Molecular Instrumentation Center at the Department of Chemistry and Biochemistry at UCLA.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnano.2021.675792/full#supplementary-material

References

Alibart, F., Zamanidoost, E., and Strukov, D. B. (2013). Pattern Classification by Memristive Crossbar Circuits Using Ex Situ and In Situ Training. Nat. Commun. 4, 2072. doi:10.1038/ncomms3072

PubMed Abstract | CrossRef Full Text | Google Scholar

Avizienis, A. V., Sillin, H. O., Martin-Olmos, C., Shieh, H. H., Aono, M., Stieg, A. Z., et al. (2012). Neuromorphic Atomic Switch Networks. PLoS One 7–e42772. doi:10.1371/journal.pone.0042772

PubMed Abstract | CrossRef Full Text | Google Scholar

Bharathi Mohan, D., Sreejith, K., and Sunandana, C. S. (2007). Surface Plasmon-Exciton Transition in Ultra-thin Silver and Silver Iodide Films. Appl. Phys. B 89, 59–63. doi:10.1007/s00340-007-2768-6

CrossRef Full Text | Google Scholar

Bürger, J., Goudarzi, A., Stefanovic, D., Stefanovic, D., and Teuscher, C. (2015). Computational Capacity and Energy Consumption of Complex Resistive Switch Networks. AIMS Mater. Sci. 2, 530–545. doi:10.3934/matersci.2015.4.530

CrossRef Full Text | Google Scholar

Cai, K., Sun, J., Li, B., and Zhou, J. (2013). Hysteretic Current-Voltage Characteristics and Memristive Behaviors in AgI Nano-Particles Assembly. ECS J. Solid State. Sci. Technol. 2, N6–N10. doi:10.1149/2.017301jss

CrossRef Full Text | Google Scholar

Calimera, A., Macii, E., and Poncino, M. (2013). The Human Brain Project and Neuromorphic Computing. Funct. Neurol. 28, 191–196. doi:10.11138/FNeur/2013.28.3.191

PubMed Abstract | CrossRef Full Text | Google Scholar

Dale, M., Miller, J. F., Stepney, S., and Adamatzky, A. (2017). Reservoir Computing as a Model for In-Materio Computing. Adv. Unconv. Comput. 1, 533–571. doi:10.1007/978-3-319-33924-5_22

CrossRef Full Text | Google Scholar

Demis, E. C., Aguilera, R., Scharnhorst, K., Aono, M., Stieg, A. Z., and Gimzewski, J. K. (2016). Nanoarchitectonic Atomic Switch Networks for Unconventional Computing. Jpn. J. Appl. Phys. 55, 1102B2. doi:10.7567/jjap.55.1102b2

CrossRef Full Text | Google Scholar

Deshmukh, A. M. (2020). Comparison of Hidden Markov Model and Recurrent Neural Network in Automatic Speech Recognition. Ejers 5, 958–965. doi:10.24018/ejers.2020.5.8.2077

CrossRef Full Text | Google Scholar

Du, C., Cai, F., Zidan, M. A., Ma, W., Lee, S. H., and Lu, W. D. (2017). Reservoir Computing Using Dynamic Memristors for Temporal Information Processing. Nat. Commun. 8, 1–10. doi:10.1038/s41467-017-02337-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Fu, K., Zhu, R., Loeffler, A., Hochstetter, J., Diaz-Alvarez, A., Stieg, A., Gimzewski, J., Nakayama, T., and Kuncic, Z. (2020). “Reservoir Computing with Neuromemristive Nanowire Networks,” in Proceedings of the International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers Inc). doi:10.1109/IJCNN48605.2020.9207727

CrossRef Full Text | Google Scholar

Gnanavel, M., and Sunandana, C. S. (2008). “Optical Absorption and Photoluminescence in Ultra Thin Silver and Silver Iodide Films,” in 2008 IEEE PhotonicsGlobal at Singapore, doi:10.1109/IPGC.2008.4781352

CrossRef Full Text | Google Scholar

Goudarzi, A., Lakin, M. R., and Stefanovic, D. (2014). Reservoir Computing Approach to Robust Computation Using Unreliable Nanoscale Networks. Berlin, Germany: Springer. arXiv.org. doi:10.1145/2770287.2770315

CrossRef Full Text

Guo, X., Schindler, C., Menzel, S., and Waser, R. (2007). Understanding the Switching-Off Mechanism in Ag+ Migration Based Resistively Switching Model Systems. Appl. Phys. Lett. 91, 133513. doi:10.1063/1.2793686

CrossRef Full Text | Google Scholar

Harding, S., and Miller, J. F. (2009). “Evolution in Materio,” in Encyclopedia Of Complexity And Systems Science 3220–3233. New York: Springer, 3220–3233. doi:10.1007/978-0-387-30440-3_190

CrossRef Full Text | Google Scholar

Hasegawa, T., Ohno, T., Terabe, K., Tsuruoka, T., Nakayama, T., Gimzewski, J. K., et al. (2010). Learning Abilities Achieved by a Single Solid-State Atomic Switch. Adv. Mater. 22, 1831–1834. doi:10.1002/adma.200903680

PubMed Abstract | CrossRef Full Text | Google Scholar

Hashmi, A., Berry, H., Temam, O., and Lipasti, M. (2011). “Automatic Abstraction and Fault Tolerance in Cortical Microachitectures,” in Proceedings of the 38th Annual International Symposium on Computer Architecture, San Jose, CA, June 4–8, 2011. doi:10.1145/2000064.2000066

CrossRef Full Text

Ielmini, D., and Wong, H. S. P. (2018). In-memory Computing with Resistive Switching Devices. Nat. Electron. 1, 333–343. doi:10.1038/s41928-018-0092-2

CrossRef Full Text | Google Scholar

Kan, S., Nakajima, K., Takeshima, Y., Asai, T., Kuwahara, Y., and Akai-Kasaya, M. (2021). Simple Reservoir Computing Capitalizing on the Nonlinear Response of Materials: Theory and Physical Implementations. Phys. Rev. Appl. 15, 024030. doi:10.1103/physrevapplied.15.024030

CrossRef Full Text | Google Scholar

Kato, Y., Ono, L. K., Lee, M. V., Wang, S., Raga, S. R., and Qi, Y. (2015). Silver Iodide Formation in Methyl Ammonium Lead Iodide Perovskite Solar Cells with Silver Top Electrodes. Adv. Mater. Inter. 2, 1500195. doi:10.1002/admi.201570065

CrossRef Full Text | Google Scholar

Konkoli, Z., Nichele, S., Dale, M., and Stepney, S. (2018). “Reservoir Computing with Computational Matter,” in Natural Computing Series, Springer-Verlag, 269–293. doi:10.1007/978-3-319-65826-1_14

CrossRef Full Text | Google Scholar

Kudithipudi, D., Saleh, Q., Merkel, C., Thesing, J., and Wysocki, B. (2016). Design and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing. Front. Neurosci. 9, 61907. doi:10.3389/fnins.2015.00502

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuncic, Z., Kavehei, O., Zhu, R., Loeffler, A., Fu, K., Hochstetter, J., et al. (2020). Neuromorphic Information Processing with Nanowire Networks, in Institute of Electrical and Electronics Engineers (IEEE), 1–5. doi:10.1109/iscas45731.2020.9181034

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuzum, D., Jeyasingh, R. G. D., Yu, S., and Wong, H.-S. P. (2012). Low-Energy Robust Neuromorphic Computation Using Synaptic Devices. IEEE Trans. Electron. Devices 59, 3489–3494. doi:10.1109/TED.2012.2217146

CrossRef Full Text | Google Scholar

Liang, X. F., Chen, Y., Shi, L., Lin, J., Yin, J., and Liu, Z. G. (2007). Resistive Switching and Memory Effects of AgI Thin Film. J. Phys. D: Appl. Phys. 40, 4767–4770. doi:10.1088/0022-3727/40/16/004

CrossRef Full Text | Google Scholar

Loppacher, C. H., Guggisberg, M., Pfeiffer, O., Meyer, E., Bammerlin, M., Lüthi, R., et al. (2003). Direct Determination of the Energy Required to Operate a Single Molecule Switch. Phys. Rev. Lett. 90, 4. doi:10.1103/physrevlett.90.066107

CrossRef Full Text | Google Scholar

Lukoševičius, M., and Jaeger, H. (2009). Reservoir Computing Approaches to Recurrent Neural Network Training. Comput. Sci. Rev. 3, 127–149.

Google Scholar

Lukoševičius, M., Jaeger, H., and Schrauwen, B. (2012). Reservoir Computing Trends. KI-Künstliche Intelligenz 26, 365–371. doi:10.1007/s13218-012-0204-5

Google Scholar

Lukosevicius, M. (2011). Reservoir Computing and Self-Organized Neural Hierarchies. Bremen, Germany: IRC-Library, Information Resource Center der Jacobs University Breme.

Manning, H. G., Biswas, S., Holmes, J. D., and Boland, J. J. (2017). Nonpolar Resistive Switching in Ag@TiO2 Core-Shell Nanowires. ACS Appl. Mater. Inter. 9, 38959–38966. doi:10.1021/acsami.7b10666

PubMed Abstract | CrossRef Full Text | Google Scholar

Manning, H. G., et al. (2018). Emergence of Winner-Takes-All Connectivity Paths in Random Nanowire Networks. Nat. Commun. 9, 1–9. doi:10.1038/s41467-018-05517-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Midya, R., Wang, Z., Asapu, S., Zhang, X., Rao, M., Song, W., et al. (2019). Reservoir Computing Using Diffusive Memristors. Adv. Intell. Syst. 1, 1900084. doi:10.1002/aisy.201900084

CrossRef Full Text | Google Scholar

Miller, J. F., and Downing, K. (2002). “Evolution in Materio: Looking beyond the Silicon Box,” in Proceedings - NASA/DoD Conference On Evolvable Hardware, EH Vols 2002-January. Piscataway, NJ: Institute of Electrical and Electronics Engineers Inc., 167–176.

Google Scholar

Miller, J. F., Harding, S. L., and Tufte, G. (2014). Evolution-in-materio: Evolving Computation in Materials. Evol. Intelligence, 7 49–67.

Google Scholar

Moon, J., Ma, W., Shin, J. H., Cai, F., Du, C., Lee, S. H., et al. (2019). Temporal Data Classification and Forecasting Using a Memristor-Based Reservoir Computing System. Nat. Electron. 2, 480–487. doi:10.1038/s41928-019-0313-3

CrossRef Full Text | Google Scholar

Mustafa, M. K., Allen, T., and Appiah, K. (2019). A Comparative Review of Dynamic Neural Networks and Hidden Markov Model Methods for Mobile On-Device Speech Recognition. Neural Comput. Applic 31, 891–899. doi:10.1007/s00521-017-3028-2

CrossRef Full Text | Google Scholar

Nakajima, K. (2020). Physical Reservoir Computing-An Introductory Perspective. Jpn. J. Appl. Phys. 59, 060501. doi:10.35848/1347-4065/ab8d4f

CrossRef Full Text | Google Scholar

Nayak, A., Tamura, T., Tsuruoka, T., Terabe, K., Hosaka, S., Hasegawa, T., et al. (2010). Rate-Limiting Processes Determining the Switching Time in a Ag2S Atomic Switch. J. Phys. Chem. Lett. 1, 604–608. doi:10.1021/jz900375a

CrossRef Full Text | Google Scholar

Sánta, B., Molnár, D., Haiber, P., Gubicza, A., Szilágyi, E., Zolnai, Z., et al. (2020). Nanosecond Resistive Switching in Ag/AgI/PtIr Nanojunctions. Beilstein J. Nanotechnol. 11, 92–100. doi:10.3762/bjnano.11.9

PubMed Abstract | CrossRef Full Text | Google Scholar

Sattar, A., Fostner, S., and Brown, S. A. (2013). Quantized Conductance and Switching in Percolating Nanoparticle Films. Phys. Rev. Lett. 111, 136808. doi:10.1103/physrevlett.111.136808

PubMed Abstract | CrossRef Full Text | Google Scholar

Scharnhorst, K., Woods, W., Teuscher, C., Stieg, A., and Gimzewski, J. (2017). “Non-Temporal Logic Performance of an Atomic Switch Network,” in 2017 Proceedings of the IEEE/ACM International Symposium on Nanoscale Architectures (Institute of Electrical and Electronics Engineers Inc.). NANOARCH 2017 133–138doi:10.1109/NANOARCH.2017.8053728

CrossRef Full Text | Google Scholar

Schatz, T., and Feldman, N. H. (2018). “Neural Network vs. HMM Speech Recognition Systems as Models of Human Cross-Linguistic Phonetic Perception,” in Proceedings of the Conference on Cognitive Computational Neuroscience http://kaldi-asr.org/. doi:10.32470/ccn.2018.1240-0

CrossRef Full Text | Google Scholar

Schrauwen, B., Defour, J., Verstraeten, D., and Van Campenhout, J. (2007). The Introduction of Time-Scales in Reservoir Computing, Applied to Isolated Digits Recognition. Proc. 15th Eur. Symp. Artif. Neural Networks., 471–479. doi:10.1007/978-3-540-74690-4_48

CrossRef Full Text | Google Scholar

Sillin, H. O., Aguilera, R., Shieh, H. H., Avizienis, A. V., Aono, M., Stieg, A. Z., et al. (2013). A Theoretical and Experimental Study of Neuromorphic Atomic Switch Networks for Reservoir Computing. Nanotechnology 24. doi:10.1088/0957-4484/24/38/384004

CrossRef Full Text | Google Scholar

Snyder, D. R., Goudarzi, A., and Teuscher, C. (2012). Computational Capabilities of Random Automata Networks for Reservoir Computing. College Park, MD: Phys Rev E Stat Nonlin Soft Matter Phys. arXiv.org. doi:10.7551/978-0-262-31050-5-ch035

CrossRef Full Text

Srinivasa, N., and Cruz-Albrecht, J. M. (2012). Neuromorphic Adaptive Plastic Scalable Electronics: Analog Learning Systems. IEEE Pulse 3, 51–56. doi:10.1109/mpul.2011.2175639

PubMed Abstract | CrossRef Full Text | Google Scholar

Stieg, A. Z., Avizienis, A. V., Sillin, H. O., Martin-Olmos, C.., Lam, M-L., Aono, M., et al. (2014). Self-organized Atomic Switch Networks. Jpn. J. Appl. Phys. 53. doi:10.7567/jjap.53.01aa02

CrossRef Full Text | Google Scholar

Sun, W., Gao, B., Chi, M., Xia, Q., Yang, J. J., Qian, H., et al. (2019). Understanding Memristive Switching via In Situ Characterization and Device Modeling. Nat. Commun. 10, 3453. doi:10.1038/s41467-019-11411-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., et al. (2019). Recent Advances in Physical Reservoir Computing: A Review. Neural Netw. 115, 100–123. doi:10.1016/j.neunet.2019.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Tappertzhofen, S., Valov, I., and Waser, R. (2012). Quantum Conductance and Switching Kinetics of AgI-Based Microcrossbar Cells. Nanotechnology 23, 145703. doi:10.1088/0957-4484/23/14/145703

PubMed Abstract | CrossRef Full Text | Google Scholar

Terabe, K., Hasegawa, T., Nakayama, T., and Aono, M. (2005). Quantized Conductance Atomic Switch. Nature 433, 47–50. doi:10.1038/nature03190

PubMed Abstract | CrossRef Full Text | Google Scholar

Teuscher, C. (2017). The Weird, the Small, and the Uncontrollable: Redefining the Frontiers of Computing. Computer 50, 52–58. doi:10.1109/mc.2017.3001242

CrossRef Full Text | Google Scholar

Türel, Ö., Lee, J. H., Ma, X., and Likharev, K. K. (2004). Neuromorphic Architectures for Nanoelectronic Circuits. Int. J. Circ. Theor. Appl. 32, 277–302. doi:10.1002/cta.282

CrossRef Full Text | Google Scholar

Vandoorne, K., Fiers, M., Verstraeten, D., Schrauwen, B., Dambre, J., and Bienstman, P. (2010). “Photonic Reservoir Computing: A New Approach to Optical Information Processing,” in 2010 12th International Conference on Transparent Optical Networks. doi:10.1109/ICTON.2010.5548990

CrossRef Full Text | Google Scholar

Yang, J. J., Strukov, D. B., and Stewart, D. R. (2013). Memristive Devices for Computing. Nat. Nanotech 8, 13–24. doi:10.1038/nnano.2012.240

PubMed Abstract | CrossRef Full Text | Google Scholar

Yang, Y., Guo, P., Gaba, S., Chang, T., Pan, X., and Lu, W. (2012). Observation of Conducting Filament Growth in Nanoscale Resistive Memories. Nat. Commun. 3, 1–8. doi:10.1038/ncomms1737

CrossRef Full Text | Google Scholar

Zhong, Y., Tang, J., Li, X., Gao, B., Qian, H., and Wu, H. (2021). Dynamic Memristor-Based Reservoir Computing for High-Efficiency Temporal Signal Processing. Nat. Commun. 12, 1–9. doi:10.1038/s41467-020-20692-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhu, R., Hochstetter, J., Loeffler, A., Diaz-Alvarez, A., Stieg, A., Gimzewski, J., et al. (2020). “Harnessing Adaptive Dynamics in Neuro-Memristive Nanowire Networks for Transfer Learning,” in 2020 International Conference on Rebooting Computing (ICRC). doi:10.1109/icrc2020.2020.00007

CrossRef Full Text | Google Scholar