Modeling and Experimental Demonstration of a Hopfield Network Analog-to-Digital Converter with Hybrid CMOS/Memristor Circuits

The purpose of this work was to demonstrate the feasibility of building recurrent artificial neural networks with hybrid complementary metal oxide semiconductor (CMOS)/memristor circuits. To do so, we modeled a Hopfield network implementing an analog-to-digital converter (ADC) with up to 8 bits of precision. Major shortcomings affecting the ADC's precision, such as the non-ideal behavior of CMOS circuitry and the specific limitations of memristors, were investigated and an effective solution was proposed, capitalizing on the in-field programmability of memristors. The theoretical work was validated experimentally by demonstrating the successful operation of a 4-bit ADC circuit implemented with discrete Pt/TiO2−x/Pt memristors and CMOS integrated circuit components.


INTRODUCTION
Recurrent artificial neural networks are an important computational paradigm capable of solving a number of optimization problems (Hopfield, 1984;Tank and Hopfield, 1986). One classic example of such networks is a Hopfield analog-to-digital converter (Tank and Hopfield, 1986;Lee and Sheu, 1989;Smith and Portmann, 1989). Although such a circuit may be of little practical use, and inferior, for example, to similar-style feed forward-type ADC implementations (Chigusa and Tanaka, 1990), it belongs to a broader constrained optimization class of networks which minimize certain preprogrammed energy functions and have several applications in control and signal processing (Tank and Hopfield, 1986). The Hopfield network ADC circuit also represents an important bridge between computational neuroscience and circuit design, and an understanding of the potential shortcomings of such a relatively simple circuit is therefore important for implementing more complex recurrent neural networks.
An example of a 4-bit Hopfield network ADC is shown in Figure 1 (Tank and Hopfield, 1986). The originally proposed network consists of an array of linear resistors (also called weights or synapses) and four peripheral inverting amplifiers (neurons). Each neuron receives currents from the input and reference lines and from all other neurons via corresponding synapses. The analog input voltage V S is converted to the digital code V 3 V 2 V 1 V 0 , i.e., by first forcing all neuron outputs to zero (Lee and Sheu, 1989) and then letting the system evolve to the appropriate stationary state.
To understand how the Hopfield network performs the ADC operation, let us first describe its electrical behavior. Assuming leakage-free neurons with infinite input and zero output impedances, the dynamic equation governing the system evolution of the input voltage U j of the j-th neuron is described as: where g(·) is a neuron activation function, C is the neuron's input capacitance, T ij is a conductance of the synapse connecting the output of the i-th neuron with the input of the j-th neuron, while are the corresponding effective offset input current and effective input conductance for the j-th neuron. HereV R is a reference voltage, while T R and T S are conductances of reference and input weights, respectively ( Figure 1A). Note that neuron input U i can be either positive or negative, but the output of the neuron is either zero or positive. The inverted outputs of the neurons, which are fed back to the network, are therefore either negative or zero. One activation function suitable for such mapping is the sigmoid function 1/(1+exp[-U]). Neuron output needs to be inverted to keep the feedback weights positive and thus to allow physical implementation with passive devices, such as resistors 1 . Alternatively, the Hopfield network operation can be described by an energy function. The evolution of the dynamic system described by Equation (2) is equivalent to a minimization of the energy function: (5) where the last term can be neglected for very steep transfer functions (Hopfield, 1984). In Tank and Hopfield (1986), showed that a 4-bit ADC task (Equation 1) can be described by the following energy function: Here the first term tends to satisfy Equation (1), while the second tends to force each digital output V i to be either "0" or "1." After rearranging the terms in Equation (6) and comparing the result with Equation (5), the appropriate weights for performing the ADC task are: In the Hopfield ADC network, the number of synapses grows quadratically with the number of neurons. Compact implementation of the synapses is therefore required if such circuits are to be practical. This is certainly challenging to achieve with conventional CMOS technology, because, according to Equation (7), it requires analog weights with a relatively large dynamic range, i.e., in the order of 2 2N , where N is the bit precision. Weights can be stored digitally, but this approach comes with a large overhead (Moopenn et al., 1990). On the other hand, analog CMOS implementations of the synapses have to cope with the mismatch issues often encountered in CMOS circuits (Indeveri et al., 2011). Consequently, several attempts have been made to implement synapses with alternative, nonconventional technologies. In some of the early implementations of Hopfield networks, weights were realized as corresponding thin film (Jackel et al., 1987) or metal line (Graf et al., 1986;Schwartz et al., 1987) conductance values, patterned using e-beam lithography and reactive-ionetching. The main limitation of these approaches was that the weights were essentially one-time programmable, with rather crude accuracy. A much more attractive solution was very recently demonstrated in Eryilmaz et al. (2014), which describes a Hopfield network implementation with synapses based on phase change memory paired with conventional field-effect transistors. That work, together with other recent advances in device technologies (Wu et al., 2012;Zhang et al., 2012) revived interest in the theoretical modeling of recurrent neural networks based on hybrid circuits (Waser et al., 2009;Strukov and Kohlstedt, 2012;Lehtonen et al., 2014;Rakkiyappan et al., 2014;Walls and Likharev, 2014). This paper explores the implementation of synapses with an emerging, very promising type of memory devices, namely metal-oxide resistive switching devices ("memristor") (Wu et al., 2012;Zhang et al., 2012). In the next section we discuss the general implementation details of the Hopfield network ADC, including the memristor devices which were utilized in the experimental setup. This is followed by a theoretical analysis of the considered hybrid circuits' sensitivity to certain representative sources of non-ideal behavior and discussion of a possible solution to such problems. The theoretical results were validated with SPICE simulations (Section Simulation Results) and experimental work (Section Experimental Results). The paper concludes with a Discussion section. It should be noted that preliminary experimental results, without any theoretical analysis, were reported earlier in Gao et al. (2013a), where we first presented a Hopfield network implementation with metaloxide memristors. The only other relevant experimental work on memristor-based Hopfield networks that we are aware of was published recently in Hu et al. (2015). However, the network demonstrated in Hu et al. (2015) was based on 9 memristors whereas the circuit presented in this work involves 16.

MATERIALS AND METHODS FOR HOPFIELD NETWORK IMPLEMENTATION WITH HYBRID CIRCUITS
Following on from our earlier works (Alibart et al., 2013;Gao et al., 2013b;Merrikh-Bayat et al., 2014), we here consider the implementation of a hybrid CMOS/memristive circuit (Figure 1). In this circuit, density-critical synapses are implemented with Pt/TiO 2−x /Pt memristive devices, while neurons are implemented by CMOS circuits.
In their simplest form, memristors are two-terminal passive elements, the conductance of which can be modulated reversibly by applying electrical stress. Due to the simple structure and ionic nature of their memory mechanism, metal-oxide memristors have excellent scaling prospects, often combined with fast, low energy switching and high retention (Strukov and Kohlstedt, 2012). Many metal oxide based memristors can also be switched continuously, i.e., in analog manner, by applying electrical bias (current or voltage pulses) with gradually increasing amplitude and/or duration. Figure 2A shows typical continuous switching I-Vs for the considered Pt/TiO 2−x /Pt devices . The devices were implemented in "bone-structure" geometry with an active area of ∼1 µm 2 using the atomic layer deposition technique. An evaporated Ti/Pt bottom electrode (5 nm/25 nm) was patterned by conventional optical lithography on a Si/SiO 2 substrate (500 µm/200 nm, respectively). A 30 nm TiO 2 switching layer was then realized by atomic layer deposition at 200 • C using Titanium Isopropoxide (C 12 H 28 O 4 Ti) and water as precursor and reactant, respectively. A Pt/Au electrode (15 nm/25 nm) was evaporated on top of the TiO 2 blanket layer, and the device was finally rapidly annealed at 500 • C in an N 2 and N 2 +O 2 atmosphere for 5 min to improve the crystallinity of the TiO 2 material. Details of the fabrication and characterization of the considered memristors are given in Alibart et al. (2012).
After programming the memristors to the desired resistance, it was important for their state to remain unchanged during operation of the Hopfield network, so to prevent any disturbance the voltage drop across them was always kept within the |V|≤ 0.2 V "disturb-free" range .
The static I-V characteristics (i.e., those within disturbfree regime) for several different memory states are shown in Figure 2B. To assist SPICE simulation, the experimental I-V curves at small biases were fitted by the following static equation with a single memory state G: The need to keep the voltage drop across memristive devices small also affects neuron design. A simple leaky operational amplifier (op-amp) integrator could be sufficient to implement neuron functionality, but ensuring disturb-free operation with such a design is not easy. This issue was resolved by implementing neurons with three op-amps connected in series ( Figure 1B). The first op-amp was an inverting amplifier which held virtual ground even if the neuron's output was saturated. The second op-amp was an open loop amplifier implementing a sign-like activation function. The field effect transistor in the negative feedback of this op-amp was initially turned on to force the neuron's outputs to zero (i.e., to set into initial state before computing output) and then turned off during network convergence. The last op-amp inverted the signal and ensured that the neuron output was within the −0.2 V ≤ V ≤ 0 voltage range. Note that since the neuron bandwidth was mainly determined by the input capacitance of the second amplifier, and the other sources of parasitic capacitance could be neglected for simplicity, the capacitive load of the second amplifier (Figure 1B) was effectively a neuron input capacitance ( Figure 1A).
Assuming ideal op-amps and no possibility of saturation by the first and last amplifiers, the dynamic equation for this neuron design can be written as: where g() is a transfer function of the second op-amp (see Appendix for more details on derivation). For a very steep transfer function, the second term in the right hand part of Equation (9a) can be neglected (Hopfield, 1984). The network is then described by the original energy function (Equation 5) and the weights are proportional to those defined in Equation (7), i.e., where the additional coefficient 5 is due to the reduced, i.e., 0.2 V, output voltage corresponding to digital "1" in the considered circuit [as opposed to output voltage 1 V assumed in the original ADC energy function in Equation (6) for ADC and the weights in Equation (7) derived from that energy function]. The physical implementation of this Hopfield network ADC posed a number of additional challenges. However, it should first be mentioned that variations in neuron delay and input capacitances, which may result in oscillatory behavior and the settling in of false energy minima (Lee and Sheu, 1989;Smith and Portmann, 1989), were not a problem in our case thanks to the slow operating speed, which was enforced to reduce capacitive coupling. The specific problems regarding the considered implementation were offsets in virtual ground, resulting from the voltage offsets (u o ) and limited gain (A) of the op-amps ( Figure 1B). Another, somewhat less severe, problem was the nonlinear conductance of the memristive devices (defined via parameter β-, see Equation 8). In the Appendix it is shown how limited gain and non-zero offset result in an additional constant term I 0 in dynamical equation (Equation A7), which can be factored into the reference weights as follows: The Hopfield network with practical, non-ideal neurons can still therefore be approximated by the original energy equation and it should be possible to circumvent the effects of limited gain and voltage offset by fine-tuning the reference weights. This idea was verified via SPICE modeling and experimental work, as described in the next section.

Simulation Results
Using Equation (8) for the memristors and SPICE models for the IC components, in the next series of simulations we studied how particular non-ideal behavior affects differential (DNL) and integral (INL) nonlinearities in ADC circuits (van de Plassche, 2003). Figure 3A shows INL and DNL as a function of the open loop DC gain, which was varied simultaneously for all three op-amps, assuming ideal memristors with β = 0 and no voltage offset. Note that in this simulation, the gain-bandwidth product (GBP) was increased proportionally to the open loop DC gain, and was equal to 3 MHz at A DC = 2 × 10 5 . Because the circuit operated at about 1.5 KHz, the effective gain A ≈ A DC /100 for all simulations (and also for the experimental work discussed below). Figure 3B shows the impact of the  voltage offset on DNL and INL (simulated as an offset on the ground nodes), which was varied simultaneously for all three opamps. Finally, Figure 3C shows the effect of I-V nonlinearity, which was varied by changing constant β in Equation (8), assuming all other parameters of the network to be close to ideal, i.e., that the voltage offset u o = 0 and the open loop DC gain A DC = 10 6 . Note that for β > 0, the memristor weights were chosen in such a way that the conductance of the device at −0.2V matched the corresponding values prescribed by Equation (10). The results shown in Figure 3 confirm the significant individual contribution of the considered sources of non-ideal behavior on the ADC's performance. Figure 4A shows the simulation results considering all these factors together for the specific values u o = 3 mV, β = 1, A DC = 2 × 10 5 , and GBP = 3 MHz, which are representative of the experimental setup. The gain and voltage offset values were taken from the specifications of the discrete IC op-amps used in the experiment. Clearly, the ADC output is distorted and contains numerous errors, with the largest contribution to INL being due to finite gain (Figure 3). Figures 4B, 5 show the simulation results with new values for the reference weights calculated according to Equation (11) for the 4-bit and 8-bit ADCs, respectively. The results shown in these figures confirm that non-ideal behavior in op-amps, such as limited gain and voltage offsets, can be efficiently compensated by fine-tuning memristors.

Experimental Results
The simulation results were also validated experimentally by implementing a 4-bit Hopfield network ADC in a breadboard setup consisting of Pt/TiO 2−x /Pt memristive devices and discrete IC CMOS components ( Figure 6A). The memristor chips were assembled in standard 40-pin DIP packages by wire-bonding 20 standalone memristive devices. Because input voltage range is 0 ≤ V S ≤ V max S = 3.0 V, the weights T S were realized with regular resistors 2 . The discrete memristors and other IC components were then connected as shown in Figure 1 with external wires.
The memristors implementing feedback and reference weights were first tuned ex-situ using a previously developed algorithm  to the values defined by Equation (10). The ex-situ tuning for each memristor was performed individually before the devices were connected in a circuit. This was done to simplify the experiment and it is worth mentioning that in general, it should be possible to tune memristors after they are connected in the crossbar circuit, as it was experimentally demonstrated by our group for standalone devices connected in crossbar circuits (Alibart et al., 2013;Gao et al., 2013c) and integrated passive crossbar circuits (Prezioso et al., 2015a,b).
As was discussed in Sections Materials and Methods for Hopfield Network Implementation with Hybrid Circuits and Results, limited gain and voltage offsets of operational amplifiers can be compensated by adjusting reference weights according to Equations (11,A12). To demonstrate in-field configurability of memristors, the reference weights were fine-tuned in-situ. In particular, reference weights were adjusted to ensure correct outputs at four particular input voltages, when V S is equal to 1/16, 1/8, 1/4, and 1/2 of its maximum value. The tuning is performed first for V S = 1/16 V max S , for which the correct operation of ADC assumes that the least significant output bit V 0 flips from 0 to 1 (corresponding to voltage 0.2 V in our case), which is ensured by fine-tuning reference weight T R0 . Similarly, the output bit V 1 should flip from 0 to 1 when V S = 1/8 V max S , which is ensured by fine-tuning reference weight T R1 and so on. Because we started fine-tuning from the least significant output, it is sufficient to fine-tune only one corresponding reference weight at a time for a particular input voltage, which greatly simplified in-situ tuning procedure. Also, the direction of adjustment was always straightforward to determine due to monotonic dependence of the input voltage at which a particular output bit flips from 0 to 1, on the corresponding reference weight (Equation 11).

DISCUSSION
The network parameters for the experimental work are summarized in Table 1. Although there were a few A/D conversion errors in the experimental work (Figure 6), the results are comparable with the simulations of the optimized network, and much better than those obtained for the unoptimized network. The experimental results for the unoptimized network were significantly worse in comparison with the simulation, and are not shown in this paper. It is worth mentioning that for the considered memristors drift of conductive state over time was negligible due to highly nonlinear switching kinetics specific to these devices (Alibart et al., , 2013Prezioso et al., 2015a). In principle, for other types of memristors with inferior retention properties it should be possible to occasionally fine-tune memristor state to cope with conductance drift. A related issue might be measurement noise upon reading the state of the memristor, e.g., due to the fluctuations in the device conductance over time, which is sometimes observed as random telegraph noise (Gao et al., , 2013bPrezioso et al., 2015b). Such noise can be tolerated by performing quasi DC read measurements, however, the downside would be potentially much slower tuning process.
To conclude, in this work we investigated hybrid CMOS/metal-oxide-memristor circuit implementation of a Hopfield recurrent neural network performing analog-to-digital conversion tasks. We showed that naïve implementation of such networks, with weights prescribed by the original theory, produces many conversion errors, mainly due to the nonideal behavior of the CMOS components in the integrated circuit. We then proposed a method of adjusting weights in the Hopfield network to overcome the non-ideal behavior of the network components and successfully validated this technique experimentally on a 4-bit ADC circuit. The ability to fine-tune the conductances of memristors in a circuit was essential for implementing the proposed technique. In our opinion, the work carried out proved to be an important milestone and its results will be valuable for implementing more practical large-scale recurrent neural networks with CMOS/memristor circuits. Experimental research into CMOS/memristor neural networks is still very scarce and, to the best of our knowledge, the demonstrated Hopfield network is the most complex network of its type reported to date. From a broader perspective, this paper demonstrates one of the main advantages of utilizing memristors in analog circuits, namely the feasibility of fine-tuning memristors after fabrication to overcome variations in analog circuits.

AUTHOR CONTRIBUTIONS
XG and FMB performed simulation work. FMB, XG, and LG performed the experimental demo. LG, BH, and FA fabricated devices. DS supervised the project. All discussed the results.

ACKNOWLEDGMENTS
This work was supported by NSF grant CCF-1028378 and by the Air Force Office of Scientific Research (AFOSR) under MURI grant FA9550-12-1-0038, and by Spanish grant TEC2012-37868-C04-01 (BIOSENSE) (with support from the European Regional Development Fund).