Partial quantisation scheme for optimising the performance of hopfield network

The ideal Hopfield network would be able to remember information and recover the missing information based on what has been remembered. It is expected to have applications in areas such as associative memory, pattern recognition, optimisation computation, parallel implementation of VLSI and optical devices, but the lack of memory capacity and the tendency to generate pseudo-attractors make the network capable of handling only a very small amount of data. In order to make the network more widely used, we propose a scheme to optimise and improve its memory and resilience by introducing quantum perceptrons instead of Hebbian rules to complete its weight matrix design. Compared with the classical Hopfield network, our scheme increases the threshold of each node in the network while training the weights, and the memory space of the Hopfield network changes from being composed of the weight matrix only to being composed of the weight matrix and the threshold matrix together, resulting in a dimensional increase in the memory capacity of the network, which greatly solves the problem of the Hopfield network’s memory The problem of insufficient memory capacity and the tendency to generate pseudo-attractors was solved to a great extent. To verify the feasibility of the proposed scheme, we compare it with the classical Hopfield network in four different dimensions, namely, non-orthogonal simple matrix recovery, incomplete data recovery, memory capacity and model convergence speed. These experiments demonstrate that the improved Hopfield network with quantum perceptron has significant advantages over the classical Hopfield network in terms of memory capacity and recovery ability, which provides a possibility for practical application of the network.


Introduction
Machine learning [1] is an important branch of artificial intelligence and a way to achieve artificial intelligence, i.e. machine learning is used as a means to solve problems in artificial intelligence. It is a multi-disciplinary discipline involving probability theory, statistics convex optimisation, complexity theory and many other disciplines. Machine learning algorithms are a class of algorithms that analyse existing data to obtain a certain pattern and use this pattern to make predictions about unknown data. It has been used with great success in very many fields, including medicine [2], biology [3], chemistry [4], physics [5][6][7][8] and mathematics [9]. Machine learning has proven to be one of the most successful ways to explore the field of artificial intelligence.
Perceptron [10] is a two-classification linear classification model, which aims to find the hyperplane that divides the training data linearly. Its biggest feature is that it is easy to implement. Suppose the training data set is D (x ϱ ,ŷ ϱ ) m ϱ 1 , wherex ϱ ⊆ R m ,ŷ ϱ ∈ {+1, −1}. The perceptron model is: Whereŵ andx are the model parameters of the perceptron, w ∈ R m is called weight or weight vector, and b ∈ R is called bias. w ·x represents the inner product ofŵ andx. The sign function is a symbolic function: The linear equationŵ ·x + b 0 is a hyperplane in the characteristic space, whereŵ is the normal vector of the hyperplane and b is the intercept of the hyperplane. The hyperplane can divide the feature space into two parts, and the point above the hyperplane conformsŵ ·x + bP0, otherwise, it conformsŵ ·x + b < 0. The model of the classic perceptron and its application to classification is illustrated in Figure 1. Quantum information is a new discipline developed based on quantum physics and information technology, which mainly includes two fields: quantum communication and quantum computing. Quantum communication focuses on quantum cryptography [11,12], quantum teleportation [13][14][15][16], and quantum direct communication [17], while quantum computing focuses on algorithms that fit quantum properties [18][19][20][21][22][23]. This is an extremely active field, as it has the potential to disrupt classical informatics, communication technologies, and computing methods.
Quantum perceptron belongs to quantum machine learning algorithms [24,25], which is the quantum counterpart of the classical perceptron model. Kapoor proved that quantum computation can provide significant improvements in the computational and statistical complexity of the perceptron model [26]; Schuld proposed a scalable quantum perceptron based on quantum Fourier transform [27], which can be used as a component of other more advanced networks [28]; Tacchino proposed a quantum perceptron model that can run on nearterm quantum processing hardwar [29]. Currently, quantum perceptron models are in the exploratory stage and there is no absolute authority on them. In our work, the quantum perceptron model based on the quantum phase estimation algorithm [27] proposed by Schuld is used. The inverse quantum Fourier transform and the gradient descent algorithm on a classical computer are used to train the weight matrix of the perceptron.
Hopfield network (HNN) are single-layer full feedback network [30], which are characterised by the fact that the output x i of any neuron is fed back to all neurons x j as output by connecting the weights w ij . The network usually uses Hebbian rule [31] for the design of the weight matrix. Hebbian rule is simpler but useful for the design of the weight matrix in HNN. However, sometimes the Hebbian rule cannot find an exact weight matrix, even though such a matrix exists [32]. This is because the rule does not incorporate the thresholds of the HNN into the training, which can result in attractors producing ranges of attraction domains that overlap each other or even appear to overwrite. And if the vectors to be stored are closer to each other, their probability of error is higher.
Considering that the weight matrix designed by the Hebbian rule is not enough to support the HNN to accomplish various practical tasks, we propose an improvement scheme, which will use the quantum perceptron instead of the Hebbian rule for the design of the HNN weights, Firstly, the weights and thresholds of the Hopfield network are mapped into the weight matrix of the quantum perceptron, and each node of the HNN is used as the input vector, and the weight matrix of the quantum perceptron is passed through the quantum The final weight matrix of the quantum perceptron is the weight matrix and threshold matrix of the HNN. The improved HNN has more memory storage space than the Hebbian rule because it has an additional threshold matrix to assist in storage, and can store the memorised information better. Moreover, due to the more accurate weight information, it is also easier to reach the steady state when iterating the HNN, thus the resilience and model convergence speed are significantly improved. Currently, the most widespread use of HNNs is for information recovery and information matching. Our improved HNN has been simulated and analysed to provide a huge improvement over the classical HNN in both information recovery and information matching, which makes the improved HNN more usable than the classical HNN, which is expected to provide more applications for HNNs in more fields, such as playing a greater role in virus information identification, human brain simulation, and error correction of quantum noise [33].
In Section 2, we describe in detail the HNN model, the Hebbian rule, the quantum Fourier transform and the quantum phase estimation algorithm used in this paper; Section 3 describes in detail the theory of our approach, including the Frontiers in Physics frontiersin.org correspondence between the HNN and the perceptron model, the quantum perceptron model and how to use the quantum perceptron model for training the HNN weights and thresholds; Section 4 presents our simulation Section 4 presents our experimental analysis, in which we design experiments to verify the feasibility of our proposed scheme and its improvement and advantages over the classical scheme; Section 5 concludes the paper and provides predictions and analysis of the future of our proposed scheme.

Hopfield network
HNN are multi-input, thresholded, binary nonlinear dynamic systems. The excitation function of the neuron is usually a step function, and the value of the neuron is −1, 1, or 0,1. When the value is 0 or −1, the current neuron is in the inhibited state, and when the value is 1, the current neuron is in the activated state. The HNN is a single layer neural network in which all neuron nodes are connected to other neuron nodes. There is no self-feedback between the nodes, forming a complete graph model. A neuron node in the inhibited state will enter the activated state when the stimulus exceeds a set threshold, i.e. it will jump from 0 or −1 to 1.
Each node in a HNN has the same function, and the output of a single node corresponds to the final state of that node, denoted by x i , with the states of all nodes forming the state of the network The topology and mode of operation is shown in Figure 2. The network enters a steady state and produces an output when the rate of change of the energy function of the network, ΔE = 0 or when a preset upper limit of iterations is reached. The energy function and the rate of change of the energy function are as follows.
where W x ij is the weight matrix, X x i { } is the network state and θ θ i { } is the threshold matrix.

Hebbian rule
The Hebbian rule describes the basic principle of synaptic plasticity, that is, continuous and repeated stimulation from presynaptic neurons to postsynaptic neurons can increase the efficiency of synaptic transmission.
The Hebbian rule is the oldest and simplest neuron learning rule. Here is the description equantion of the Hebbian rule: Where w ij is the connection weight from neuron j to neuron i, p is the number of training modes, and x z i is the i input of neuron k. In the HNN, Hebbian rules can be used to design weight matrices: Here w ii = 0, which means that there is no self-feedback between nodes. The equantion is rewritten as follows: Where I is the unit matrix and X is the system state of HNN.

HNN attractor and pseudo attractor
Considering that the Hopfield network has M samples of X m , then: Frontiers in Physics frontiersin.org Because of n > M, therefore: According to Eq. 9: when a given sample, X m is the ideal attractor and produces a certain attractor domain around it, which will be "captured" by the attractor in the attractor domain. However, the condition that the given samples are orthogonal to each other is too harsh, which eventually leads to the attraction domain of some points outside the samples, which are regarded as pseudo attractors of the HNN.

Quantum Fourier transform
The quantum Fourier transform is an efficient quantum algorithm for the Fourier transform of quantum amplitudes. The quantum Fourier transform is not the classical counterpart of the Fourier transform and does not speed up the Fourier transform process on classical data, but it can perform an important task-phase estimation, i.e. estimating the eigenvalues of the You operator under certain conditions. The matrix representation of the quantum Fourier transform is as follows: Where, ω e 2πi N cos 2π N + i sin 2π N . In the classical Fourier transform, the transformation takes the following form: The mathematical form of the quantum Fourier transform is similar to the mathematical representation of the discrete Fourier transform [34]. It is an operator defined on a set of standard orthogonal bases |0〉, |1〉/|N − 1〉 with the following action: An arbitrary quantum state action can be expressed as: where the amplitude y k The transform itself does not have much obvious value, but it is an important component subalgorithm of the quantum phase estimation algorithm. The quantum Fourier transform corresponds to the quantum line diagram (omitting the Figure 3 illustrates the quantum circuit of the quantum Fourier transform. Frontiers in Physics frontiersin.org

Qunamtum phase estimation algorithm
The quantum phase estimation algorithm is the key to many quantum algorithms [6,35], and its role is to estimate the phase in the eigenvalues of the eigenvectors corresponding to the You matrix. The quantum circuit for quantum phase estimation is shown in Figure 4. The algorithm uses two registers, the first of which contains τ quantum bits with initial state |0〉. The value of τ depends on the number of bits desired to be accurately estimated and the desired success rate. The second register has an initial state of |x n 〉. The essence of the process is the ability to perform the inverse Fourier transform: where state |φ〉 is the estimated value of φ.

Correspondence between perceptron models and HNN
Firstly, we will discuss HNN with the range restricted to cells with non-zero thresholds and a step function as the threshold function, which is by far the most common form of HNN. Secondly two consensus needs to be established: 1) the units in this network are perceptrons. 2) The perceptron can determine the weights and thresholds of the network for the problem to be learned. Focus on consensus i): Based on the definitions of HNN and perceptual machines above, it is clear that the unit in a HNN is a perceptual machine.
Focus on consensus ii): Consider a HNN with n cells, where W is the weight matrix of n × n, such that θ i denotes the threshold of the cell i and the state of the network is X. If one wants this network to reach a steady state, it means that the following n inequalities must be satisfied: Since it has no self-feedback, only the n (n − 1)/2 non-zero entries of the weight matrix W and the n thresholds of the cells appear in these inequalities. Let u denote the vector of n + n (n + 1)/2 dimension whose components are the non-diagonal elements of the weight matrix w ij (i < j) and the n threshold minus signs. The vector u is given by the following equation: u w 12 , w 13 , . . . , w 1n , w 23 , w 24 , . . . , w 2n , . . . , w n−1n , −θ 1 , . . . , −θ n ( ) The vector x is transformed into n auxiliary vectors v 1 , v 2 , v 3 , . . . , v n of dimension n + n (n + 1)/2 given by the expression: Eq. 15 can be rewritten in the following form: Eq. 18 shows that the solution to the original problem is found by computing the linear separation of vectors z i . The vectors belonging to the positive half-space are those with sgn(x i ) 1, and those belonging to the negative half-space are those with sgn(x i ) −1. This problem can be solved using perceptron learning, which allows us to calculate the weight vector v required for linear separation and from this to derive the weight matrix W with the threshold matrix θ. Figure 5 shows the correspondence between the HNN and the perceptron model.

Obtaining parameter information using quantum perception
The connection between the HNN and the perceptron model was described above. It is now clarified how the design of the HNN weight matrix can be carried out using the quantum perceptron. Firstly, σ = (v, u) is input to the quantum perceptron model as an initial parameter and the model update rule for the quantum perceptron is as follows: From the above equation, it can be deduced that |σ〉 is an eigenvector of the matrix U and e 2πiΔϕh(u,v) is the corresponding eigenvalue. By picking the appropriate value of t in the quantum perceptron, the inverse Fourier transform by: It is possible to obtain a value of, which is very close to the true phase, and also becomes closer to the true phase as the value of t becomes larger. Combining Eq. 19 gives: Therefore the value of σ = (v, u) can be obtained byφ′.According to [], it can be known that in the perceptron model, its weight update rule: where Y q sgn(u(ξ)σ q ), η is the learning rate. However, when training with a perceptron, it is difficult to guarantee the separability of the data. Therefore, our perceptron model is trained using the delta rule, i.e. a gradient descent algorithm to search the space of possible weight vectors in order to find the best-fitting sample weight vector. The process is implemented with the aid of a classical computer. Its weight update rule is expressed in the same form as (Eq. 25), except that Y q = u(ξ)σ q .

FIGURE 3
Quantum circuits for quantum Fourier transform.

FIGURE 4
Quantum circuits for quantum phase estimation.

FIGURE 5
HNN and perceptual model transformation relationship.

FIGURE 6
Non-orthogonal simple matrix memory test.
Frontiers in Physics frontiersin.org

Computational complexity analysis
We analyze the computational complexity of the HNN in two steps. 1) Analysis of the lift rate of the data to be trained after conversion of the HNN to the perceptron model. 2) The computational complexity required to complete the weight parameters by means of the quantum phase estimation algorithm. First we analyse i), any HNN with n nodes satisfying the requirements of Section 3.1 can be converted into a perceptron model with n (n − 1)/2 weight parameters.
For ii), we analyze here two different algorithms for finding the weight parameters, namely the gradient descent-based algorithm and the Grover fast weight finding algorithm. The time complexity of the gradient descent-based algorithm is mainly  controlled by the number of steps ε accuracy, i.e. O ∝ ε 2 ; the time complexity of finding the parameters using the Grover algorithm can reach O(n) under certain conditions. It is clear from this analysis that the final computational complexity is O(n ϒ ), regardless of the algorithm used. However, quantum machine learning is able to process information using quantum effects, as in this paper, where we input the training set as a superposition of feature vectors into a quantum perceptron model that can be processed simultaneously, and this process is not affected by the size of the model. The value of this process is small when the model size is small, and becomes more apparent as the model size increases and becomes the most important part of determining the computational complexity.

Emulation analysis
The two most important applications of HNN are data matching and data recovery, which correspond to the accuracy of the HNN's weight matrix and memory capacity respectively. The convergence speed of the HNN model is extremely important in both data matching and data recovery. To this end, we designed three experiments, namely a non-orthogonal simple matrix recovery test, a Random binary-based incomplete matrix recovery test, and a memory capacity test based on the recognizability of QR codes, to compare the effectiveness of our proposed improved HNN with that of the classical HNN, and finally we added a model convergence speed comparison experiment to compare the performance differences between the models.
Our simulation analysis is based on the pennylane open source framework. The framework has embedded transition algorithms between quantum and classical algorithms as well as parameter optimisation algorithms, eliminating the need for us to package the parameters and design the optimisation algorithms separately. With this framework, the measured and calculated weight parameters are directly updated iteratively by means of a gradient descent algorithm, and the relevant information is fed back into the quantum algorithm to update the perceptron weights. Using this as a basis, we have designed the following simulation experiments.

Result
In the non-orthogonal simple matrix memory test, we demonstrated that our proposed solution can effectively cope with the memory confusion caused by non-orthogonal simple matrices; in the fragmented data recovery test, we demonstrated that our proposed QP-HNN has an average recovery rate improvement of 30.6% and a maximum of 49.1% in the effective interval compared with Hebbian rule-Hopfield network (HR-HNN), making it more practical. In the memory stress test based on QR code recognisability, our proposed QP-HNN is 2.25 times more effective than HR-HNN.

Non-orthogonal simple matrix memory test
The non-orthogonal simple matrix memory test is set up for the Hebbian rule in the classical HNN, as one of the prerequisites for the design of the weight matrix using the Hebbian rule is that the input vectors must be orthogonal to each other, and if they do not satisfy orthogonality, the designed weight matrix may be incorrect. We demonstrate the impact of this deficiency using two non-orthogonal 3D row vectors X v = [0, 1, 0] and X ϑ = [1, 1, 1] as the input matrices for HR-HNN and QP-HNN, as shown in Figure 6 Where   QuantumPerceptron, ClassicalPerceptron, and HebbianRule to design the weight matrices respectively. A matrix M bri was selected from M ι and generated M bri ′ M bri .1/3 of the data in M bri ′ was inverted to simulate data residuals, and this matrix was used as the input matrix for the network to test the recovery rate of the above three HNN. The example model is shown in Figure 7, and its recovery rate with different numbers of binary matrices memorised is shown in Figure 8.
From Figure 8, it can be seen that the resilience of the HR-HNN network decreases rapidly as ι becomes larger ι means that the orthogonality between the matrices in M ι decreases, and consequently, memory confusion ensues. The resilience of the network basically fails at ι = 20 and is completely lost at ι = 30; the ClassicalPerceptron-Hopfield (CP-HNN) network is highly similar to the QP-HNN network in terms of resilience and has excellent robustness in the first and middle stages of ι growth because the network also trains the threshold This is equivalent to increasing the error tolerance space and mitigating errors due to the non-orthogonality of the vectors in the matrix. As can be seen from the diagram, the network is still very resilient at ι = 20. However, as ι increases, the fault tolerance space becomes saturated and the resilience decreases rapidly until it fails.

Memory capacity test based on the recognizability of QR codes
In order to visualise the memory capacity of the models, the differences between the models are presented using QR codes, which have different levels of fault tolerance and represent the number of error pixels that can be tolerated in the QR code. For our tests we have used the L level of fault tolerance, which allows for a maximum of 7% of incorrect pixels.
The QR code q 1 is generated and stored in the "Successful Identification", generating a QR code set Q q n , n 2, 3, 4, 5 . . . the information in q n is an irregular string of numbers generated by a random number generator, a randomly selected m -QR code from Q is used as the interfering QR code, and q 1 is involved in the design work of HR-HNN and QP-HNN weight matrices. After 100 tests and statistical processing, the output matrix of HR-HNN can be successfully recognised when m ≤ 4; QP − Hop output matrix can be successfully recognised when, m ≤ 8. In Figure 9 we show a comparison of these two HNNs in terms of memory capacity.

HNN recovery rate test
The usability of HNN is also affected by the number of iterations required for the model to converge, which in turn is affected by the completeness of the weights, threshold information and input data. Therefore, building on the previous subsection, we further investigate the number of iterations required for q′ to recover to the stateq where information can be correctly identified for different numbers of interfering QR codes, as shown in subplot a and subplot b in Figure 10. Subplot c shows the difference in the number of iterations required for q′ to recover toq with the same amount of information. As can be seen from the figure, QP-HNN possesses a significant advantage over HR-HNN for the q′ toq process, and this advantage becomes more pronounced as m grows. Table 1 counts the recovery capacity limit of the HNN when the preset upper limit of 30,000 iterations is reached, where HR-HNN reaches the memory limit at m = 4, i.e. at m = 5, q′ cannot recover toq even if the number of iterations is increased, while in QP-HNN, the memory limit occurs at m = 8.

Conclusion
We improve the original HNN weight design method by using a quantum perceptron instead of the Hebbian rule. The improved QP-HNN can better handle non-orthogonal matrices, and its information memory and recovery capabilities as well as model convergence speed are significantly improved compared to HR-HNN. It also opens up the possibility of further expanding the scope of applications in areas such as virus information recognition, human brain simulation, and error correction of quantum noise.
Our improved scheme is based on the quantum perceptron model proposed that we can input all the data to be processed into the model simultaneously by transforming and preparing them into quantum entangled states. The current model used is still the quantum-classical computing model, where the optimal weighting parameters are found by a classical computer, but Kapoor et al. have shown that the weighting parameters can be found much faster using the Grover algorithm, considerably increase the efficiency of finding the weight parameters to compensate for the extra time consumed in its determination of the weights compared to the Hebbian rule. Currently, corresponding quantum models of HNNs already exist, and the combination of quantum perceptrons and quantum HNNs is also destined to be more desirable in pure quantum computers than in classical HNNs.
Frontiers in Physics frontiersin.org

Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.