Multi-user motion recognition using sEMG via discriminative canonical correlation analysis and adaptive dimensionality reduction

1. Introduction

Surface electromyography (sEMG) signals are electrical signals that occur on the surface of human skin when the motor unit motion potential of a motor-associated muscle propagates along the muscle fibers (Vigotsky et al., 2018; Medved et al., 2020). These signals contain a wealth of information including muscle contraction force and joint torque (Disselhorst-Klug et al., 2009). In recent years, extensive research has been conducted into human–robot collaborative robots, teleoperated surgical robots, rehabilitation robots, wearable monitoring devices, and medical diagnosis based on sEMG signals (Ghassemi et al., 2019; Li et al., 2020; Luo et al., 2020a, 2021; Qi and Aliverti, 2020; Su et al., 2020, 2022; Qi et al., 2021). In particular, for stroke patients and patients with limb muscle injuries, a rehabilitation method in which the healthy side drives the affected side (an exoskeleton robot drives the limb on the affected side by recognizing the movements of the healthy side) can accelerate neurological remodeling and rehabilitation. However, people in these groups cannot provide a large dataset for training classifier models because of their limited physical fitness (Fang et al., 2020). Moreover, muscle strength, amount of subcutaneous fat, skin impedance, the fixed position of the electrodes, the degree of muscle fatigue, and limb posture all differ significantly among different users. All of this makes it difficult for new users to fit models trained on other users (Lobov et al., 2018).

Owing to these problems, there has been much research on various aspects of sEMG, including signal preprocessing, feature extraction, feature optimization, and classification (Elamvazuthi et al., 2015; Bi et al., 2019; Simao et al., 2019). For instance, Pan et al. (2018) developed a general sEMG interface for continuous prediction of coordinated movements between the palmar fingers and wrist flexion/extension. This model can be customized based on the musculoskeletal model of the individual user, and it can fit multiple users, including upper-limb amputees. User-generic musculoskeletal models capture generalized relationships between neuromuscular signals and human-generated limb movements. The successful application of this model reportedly relies on recording sEMG signals of specific muscles. This means that the patient must determine the exact location of the upper extremity muscles, which poses some challenges in practical applications for upper extremity rehabilitation patients. Xue et al. (2021) proposed a new framework called CCA-OT to handle the problem of multi-user gesture recognition. The CCA-OT framework reduces the differences in the distribution of the feature matrix. However, it was reported that the data distribution differed significantly between the training and testing sets of the framework. The framework incorporates a new feature dimension of 45 dimensions, so there is room for further optimization. Matsubara and Morimoto (2013) proposed a bilinear model comprising (i) user-related and (ii) motion-related linear factors; it extracts user-independent features from users and classifies them with a support vector machine (SVM) classifier, and the classification recognition rate indicated that the model outperformed non-multi-user methods. However, the dimensions of the style and content variables were chosen by trial and error, which added to the difficulty of the experiment, and the performance of the model was affected significantly by the electrode placement offset. Despite some progress, considerable challenges remain in applying these findings to clinical implementation (Samuel et al., 2019; Jarque-Bou et al., 2021).

Herein, a novel multi-user sEMG framework is proposed, based on discriminative canonical correlation analysis (DCCA) and adaptive dimensionality reduction (ADR), to reduce the variability of human sEMG, eliminate complex redundant information, improve recognition rates, and even reduce the training time required for classifier models. First, we propose an ADR optimization method based on the derived DCCA algorithm and design an ADR–DCCA architecture suitable for multiple-user action recognition. Then, the ADR–DCCA framework is verified to be superior to other CCA extensions through comparative experiments. Finally, the ADR-DCCA framework is used for an upper limb rehabilitation task in which the healthy side of the user drives the affected side. The contributions of this work can be summarized as follows.

• The DCCA algorithm is applied to the variability of human sEMG to project the views of two sEMGs of different users into a low-dimensional uniform style space. In this space, the intra-class correlation of the new features is guaranteed to be maximized and the inter-class correlation is minimized.

• The ADR optimization method is proposed; this is the main contribution of this work. This method selects the most suitable dimension for motion recognition from the new features, which effectively reduces the redundant information.

The framework presented here can further extract the common information of training users and new users, effectively reduce personality information, and find the most suitable dimension for new users' action classification. New users can adapt to the sEMG interface using only a few sets of actions. In addition, the framework can be used to build an accurate model for a new user when two channels of the sEMG signal are missing. It is important for the patient to quickly obtain a model of the healthy side to drive the affected side. A multi-user contralateral-driven rehabilitation system is shown in Figure 1.

FIGURE 1

Figure 1. A multi-user health-side-driven rehabilitation system with an ADR-DCCA framework. The sEMG signals are collected from the healthy side arm of a new user and processed accordingly in the ADR-DCCA framework. The recognition is done by an upper limb exoskeleton robot.

The structure of this paper is as follows. Section 2 describes the CCA and DCCA methods and then introduces the multi-user sEMG motion recognition framework. Section 3 describes the experimental data acquisition scheme. Section 4 shows the effectiveness of the proposed multi-user sEMG interface through multiple sets of comparative tests and analyses. Finally, Section 5 concludes the paper and discusses possible extensions and future directions.

2. Methods

CCA technique, which was introduced by Hotelling in 1936 (Hotelling, 1992), is a multivariate statistical method for studying the correlation between two sets of variables. It quantifies the association between two sets of variables and transforms the analysis of the correlation into an analysis between linear combinations of the two sets of variables. CCA has been widely used in the fields of data mining, machine learning, signal processing, biomedical engineering, healthcare data analysis, genetics, etc. However, to our knowledge, researchers have not yet addressed the problem of processing new feature dimensions after projection. Here, we develop an ADR-DCCA framework by combining the values of the fitness functions of SVM.

2.1. Theory of discriminative canonical correlation analysis

In our framework, an improved CCA method maps the sEMG features of different individuals into the same low-dimensional space to train a classifier model, which improves the applicability and generalization ability of the model to some extent. Considering the properties of the training user feature matrix and the new user calibration feature matrix, we define two column vectors, $X={\left(\begin{array}{c}{X}_1^{(1)},\cdots ,X_n^{(1)},\cdots \cdots ,X_1^{(c)},\cdots ,X_n^{(c)}\end{array}\right)}^T$and $Y={\left(\begin{array}{c}{Y}_1^{(1)},\cdots ,Y_n^{(1)},\cdots \cdots ,Y_1^{(c)},\cdots ,Y_n^{(c)}\end{array}\right)}^T$. Here, c is the motion category; n is the number of samples of the same type of motion; p and q are the number of features of a single sample in X and Y, respectively; and X and Y are standardized by column and expressed as

$\begin{array}{ll}X=\left[\begin{array}{c}{X_1^{(1)}}^T\\ ⋮\\ {X_n^{(1)}}^T\\ ⋮\\ ⋮\\ {X_1^{(c)}}^T\\ ⋮\\ {X_n^{(c)}}^T\end{array}\right]=\left[\begin{array}{ccc}{X_{11}^{(1)}}^T& \cdots & {X_{1p}^{(1)}}^T\\ ⋮& \ddots & ⋮\\ {X_{n1}^{(1)}}^T& \cdots & {X_{np}^{(1)}}^T\\ ⋮& ⋮\\ ⋮& ⋮\\ {X_{11}^{(c)}}^T& \cdots & {X_{1p}^{(c)}}^T\\ ⋮& \ddots & ⋮\\ {X_{n1}^{(c)}}^T& \cdots & {X_{np}^{(c)}}^T\end{array}\right],& (1)\end{array}$

$\begin{array}{ll}Y=\left[\begin{array}{c}{Y_1^{(1)}}^T\\ ⋮\\ {Y_n^{(1)}}^T\\ ⋮\\ ⋮\\ {Y_1^{(c)}}^T\\ ⋮\\ {Y_n^{(c)}}^T\end{array}\right]=\left[\begin{array}{ccc}{Y_{11}^{(1)}}^T& \cdots & {Y_{1q}^{(1)}}^T\\ ⋮& \ddots & ⋮\\ {Y_{n1}^{(1)}}^T& \cdots & {Y_{nq}^{(1)}}^T\\ ⋮& ⋮\\ ⋮& ⋮\\ {Y_{11}^{(c)}}^T& \cdots & {Y_{1q}^{(c)}}^T\\ ⋮& \ddots & ⋮\\ {Y_{n1}^{(c)}}^T& \cdots & {Y_{nq}^{(c)}}^T\end{array}\right].& (2)\end{array}$

The basic idea of CCA is to find the variables $A={\left[a_1,\cdots ,a_p\right]}^T$ and $B={\left[b_1,\cdots ,b_q\right]}^T$ that maximize the correlation coefficient between the linear combinations U_i = XA_i and V_i = YB_i, A_i ∈ A, B_i ∈ B, U_i ∈ U, V_i ∈ V(Hardoon et al., 2004; Sun et al., 2005). We choose the pair with the highest correlation coefficient in the linear combinations as the first set of canonical variables, the pair with the second-highest correlation coefficient as the second group of canonical variables, and so on until the correlation between X and Y is extracted (Sun et al., 2010); the correlation coefficient between canonical variables is called the canonical correlation coefficient. The optimization problem for U_i and V_i is given by

$\begin{array}{ll}arg{\displaystyle \underset{A,B}{max}}\rho =corr(U,V)=\frac{cov(U,V)}{\sqrt{D(U)}\sqrt{D(V)}}.& (3)\end{array}$

To simplify the expression, we let cov(X, Y) = C₁₂, cov(X, X) = C₁₁, cov(Y, Y) = C₂₂, and we add the qualification $D(U)=A^T{C}_{11}A=1,D(V)=B^T{C}_{22}B=1$ to limit the occurrence of linear transformations of variables A and B. Then, the optimization function is

$\begin{array}{c}arg{\displaystyle \underset{A,B}{max}}\rho =corr(U,V)\\ =\frac{A^T{C}_{12}B}{\sqrt{A^T{C}_{11}A}\sqrt{B^T{C}_{22}B}}\\ =A^T{C}_{12}B\\ \mathrm{such}\mathrm{that}(\text{s}.\text{t}.)A^T{C}_{11}A=1,B^T{C}_{22}B=1.& (4)\end{array}$

Using Lagrange multipliers, we convert the above optimization problem into a conditional extreme-value problem. By solving Equation (4), it is easy to obtain

$\begin{array}{ll}{C_{11}}^{-1}{C}_{12}{C_{22}}^{-1}{C}_{21}A-{\lambda }^{2}A=0,& (5)\end{array}$

$\begin{array}{ll}{C_{22}}^{-1}{C}_{21}{C_{11}}^{-1}{C}_{12}B-{\lambda }^{2}B=0.& (6)\end{array}$

From Equations (5) and (6), we know that ${C_{11}}^{-1}{C}_{12}{C_{22}}^{-1}{C}_{21}$ and ${C_{22}}^{-1}{C}_{21}{C_{11}}^{-1}{C}_{12}$ have the same eigenroots λ², with A and B as the corresponding eigenvectors. The feature roots are arranged from largest to smallest, and the feature vectors are arranged in order of the corresponding feature roots. The feature vector that corresponds to the largest feature root is transformed m times according to the qualification to obtain the first set of A₁, B₁.

After that, we keep solving for the second set of canonical variables. Find variables A₂ and B₂ that maximize the correlation between U₂ = XA₂ and V₂ = YB₂ under the qualification $D(U_2)={A_2}^T{C}_{11}{A}_2=1,D(V_2)={B_2}^T{C}_{22}{B}_2=1$. The first pair of canonical variables has been extracted, so the second pair should be extracted without the information in the first pair (orthogonal to the first pair). We add a new qualification, $cov(U_1,U_2)={A_1}^2{C}_{11}{A}_2=0,cov(V_1,V_2)={B_1}^2{C}_{22}{B}_2=0$, with which we obtain

$\begin{array}{ll}{C_{11}}^{-1}{C}_{12}{C_{22}}^{-1}{C}_{21}{A}_2-{\lambda }^2{A}_2=0,& (7)\end{array}$

$\begin{array}{ll}{C_{22}}^{-1}{C}_{21}{C_{11}}^{-1}{C}_{12}{B}_2-{\lambda }^2{B}_2=0.& (8)\end{array}$

It is clear that the results A₂, B₂ of Equations (7) and (8) are the same A, B as those obtained from Equations (5) and (6). Therefore, the eigenvector corresponding to the next-largest eigenvalue is the one that we seek. Continuing in this way, it is easy to obtain the rth pair of eigenvectors, where r < min(p, q).

DCCA is based on CCA, which finds the largest intra-class correlation while minimizing the correlation across classes. DCCA can effectively improve the robustness of the generated features, thereby improving the performance of the whole system (Gatto and dos Santos, 2017). We define the vector$e_{nk}=\left[0_1^{(1)},\cdots ,0_n^{(1)},\cdots \cdots ,1_1^{(k)},\cdots ,1_n^{(k)},\cdots \cdots ,0_1^{(c)},\cdots ,0_n^{(c)}\right]$, k ∈ (1, c), and then the intra-class correlation matrix is

$\begin{array}{ll}{C}_w={\displaystyle \sum _{k=1}^c}{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}{X_i}^{(k)T}{Y_j}^{(k)}={\displaystyle \sum _{k=1}^c}{(e_{nk}X)}^T(e_{nk}Y).& (9)\end{array}$

We also define the row vector 1_n = [1, ⋯ , 1]_{1 × (c×n)}; then, the inter-class correlation matrix is

$\begin{array}{l}{C}_b={\displaystyle \sum _{k1=1}^c}{\displaystyle \sum _{\genfrac{}{}{0ex}{}{k2=1}{k2\ne k1}}^c}{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}{X_i}^{(k1)T}{Y_j}^{(k2)}\\ ={\displaystyle \sum _{k1=1}^c}{\displaystyle \sum _{k2=1}^c}{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}{X_i}^{(k1)T}{Y_j}^{(k2)}-{\displaystyle \sum _{k=1}^c}{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}{X_i}^{(k)T}{Y_j}^{(k)}\\ ={(1_{n}X)}^T(1_{n}Y)-{\displaystyle \sum _{k=1}^c}{X}^T{e_{nk}}^T{e}_{nk}Y\\ =-{\displaystyle \sum _{k=1}^c}{X}^T{e_{nk}}^T{e}_{nk}Y.& (10)\end{array}$

As both the X and Y datasets are column normalized, we have ${(1_{n}X)}^T(1_{n}Y)=0$. DCCA can be described as solving

$\begin{array}{l}\underset{A,B}{\max }\left(A^T{C}_{w}B-A^T{C}_{b}B\right)=\underset{A,B}{\max }\left({\displaystyle \sum _{k=1}^c{A}^T}{X}^T{e}_{nk}{}^T{e}_{nk}YB\right)\\ \text{ s}.\text{t}.A^T{X}^{T}XA=1,B^T{Y}^{T}YB=1.& (11)\end{array}$

Given the specificity of e_nk, it is easy to prove that

$\begin{array}{ll}\underset{A,B}{\max }\left({\displaystyle \sum _{k=1}^c{A}^T}{X}^T{e}_{nk}^T{e}_{nk}YB\text{}\right)=\underset{A,B}{\max }\left(A^T{X}^T{\displaystyle \sum _{k=1}^c\text{}}\left(e_{nk}{}^T{e}_{nk}\right)YB\text{}\right).& (12)\end{array}$

Therefore, the optimization function can be written as

$\begin{array}{l}\underset{A,B}{\max }\left(A^T{X}^{T}c{\displaystyle \sum _{k=1}^c\left(e_{nk}{}^T{e}_{nk}\right)}YB\right)\\ \text{s}.\text{t}.A^T{X}^{T}XA=1,B^T{Y}^{T}YB=1,& (13)\end{array}$

and using Lagrangian multipliers to tackle Equation (13) gives

$\begin{array}{ll}\begin{array}{c}{C_{11}}^{-1}{C}_w{C_{22}}^{-1}{C_w}^{T}A-{\lambda }^{2}A=0,\\ {C_{22}}^{-1}{C_w}^T{C_{11}}^{-1}{C}_{w}B-{\lambda }^{2}B=0.\end{array}& (14)\end{array}$

Solving this system of equations shows that ${C_{11}}^{-1}{C}_w{C_{22}}^{-1}{C_w}^T$ and ${C_{22}}^{-1}{C_w}^T{C_{11}}^{-1}{C}_w$ have the same characteristic root λ² and that A and B are the corresponding eigenvectors, which is the same conclusion as that obtained with CCA.

2.2. Multi-user sEMG interface with ADR and DCCA

The above derivation leads to the conclusion that DCCA seeks a pair of linear transformations or non-linear changes to maximize the intra-class correlation of two sets of features while ensuring the minimum inter-class correlation. The new features extracted by DCCA in different categories of the same feature set are statistically uncorrelated; thus, redundant information in different categories of features is eliminated. The proposed architecture is able to train classifier models with less data than traditional pattern recognition methods. The training data maintains a somewhat linear or non-linear relationship with the new test data, which helps new rehabilitation users to obtain accurate models faster. To the best of our knowledge, its use in developing an sEMG interface for multi-user is novel. We mapped the DCCA theory into the sEMG signal feature matrix and designed the framework shown in Figure 2, in which DCCA analyzes the training user feature matrix X and the new user calibration feature matrix Y1 (which is used to tune the parameters of the framework). The parameters that give strong correlation between the training user features and the new user features are obtained, which in this case are two independent matrices, namely, A and B. Then, new training features are obtained by A interacting with training user features X.

FIGURE 2

Figure 2. A multi-user surface electromyography (sEMG) motion-recognition framework was designed using discriminative canonical correlation analysis (DCCA) and adaptive dimensionality reduction (ADR).

In this framework, the dimension of A is changed continuously according to the fitness function information provided by the SVM until the best model is found. Specifically, a₁ denotes the first column of A, a₂ denotes the first two columns of A, and so on; a_l (1 ≤ l ≤ r) denotes the first l columns of A. Thus, we can easily obtain the matrix U_l of different dimensions under the action of a_l. By defining the SVM fitness function f(t), the value of the fitness function f(U_l) can be obtained. The ADR problem can be expressed as an optimization problem for the following functions:

$\begin{array}{ll}argmax\left\{f(U_1),f(U_2),\dots ,f(U_r)\right\}.& (15)\end{array}$

Finally, the new testing features are obtained based on the new user feature matrix Y2 and the matrix B_l.

3. Signal acquisition

In the experiments described here, a customized Delsys wireless sEMG acquisition device was used for multichannel acquisition of sEMG signals. This device is a distributed contact electrode, which has the advantages of flexible placement and wireless convenience and is not affected by the lack of channels. The sampling rate of each sEMG channel of this device is 2,000 Hz with 16-bit resolution. The sEMG signals were displayed and stored in a PC client using sEMG-recorder software developed in-house, and the data for each action were saved in a separate .csv file.

The experiments involved eight participants (called N1 to N8) who were in good physical condition and had healthy limbs. The participants comprised seven men and one woman between the ages of 23 and 27 years; they participated voluntarily and were informed of the subject matter and signed an informed-consent form prior to participating in the experiments. The sEMG signals were recorded from the subject's right arm. Based on human physiology and upper-limb muscle distribution, we chose the deltoid (channel 1), pectoralis major (channel 2), biceps brachii (channel 3), triceps brachii (channel 4), flexor carpi radialis (channel 5), extensor carpi radialis (channel 6), extensor carpi ulnaris (channel 7), and flexor carpi ulnaris (channel 8). Position tracking was performed to determine the exact positions of the electrodes according to the criteria developed by the SENIAM (Surface ElectroMyoGraphy for the Non-Invasive Assessment of Muscles) European concerted action in the Biomedical Health and Research Program (BIOMED II) of the European Union (Hermens et al., 2000). The accurate muscle positioning is shown in Figure 3. Prior to data acquisition, the subjects' electrode pasting locations were cleared to prevent interference from hair; the skin at the relevant locations was then wiped with alcohol, and the electrodes were disinfected and finally pasted onto the prescribed muscle positions.

FIGURE 3

Figure 3. A map of precise locations of muscles for upper-limb movements.

During acquisition, each participant used no more than 80% of their maximum voluntary contraction to maintain control of their movements (Cheng et al., 2018). To ensure that the force used by the subject in performing the motion was within the required range, each subject was evaluated for force using a high-definition haptic device from QUANSER prior to data collection. During the recording process, the subject was asked to perform the following 12 upper-limb motor movements to the best of their ability with moderate constant force contractions: (i) palm extension, (ii) fist clenching, (iii) wrist abduction, (iv) wrist adduction, (v) wrist extension, (vi) wrist flexion, (vii) elbow flexion, (viii) elbow extension, (ix) elbow flexion and shoulder flexion, (x) elbow flexion and shoulder extension, (xi) shoulder horizontal adduction, and (xii) shoulder horizontal abduction (see Figure 4). When the subject heard a beep from the computer, they performed the specified movement, holding the final position for 3 s and then resting for 3 s before the next movement. Each subject completed the 12 movements 100 times, and in total 9,600 sets of experimental data were recorded from the eight participants. Considering that participants might become fatigued during sEMG collection, data were collected for no more than 5 min at a time.

FIGURE 4

Figure 4. Upper-limb movements defined in present study: 1—palm extension; 2—fist clenching; 3—wrist abduction; 4—wrist adduction; 5—wrist extension; 6—wrist flexion; 7—elbow flexion; 8—elbow extension; 9—elbow flexion and shoulder flexion; 10—elbow flexion and shoulder extension; 11—shoulder horizontal adduction; 12—shoulder horizontal abduction.

4. Experiments and analysis

In this section, we describe how we evaluated the recognition capability of the proposed ADR-DCCA framework to show its feasibility. The preprocessing, feature extraction, and classification processes were carried out in the sEMG signal-processing pipeline.

In multichannel sEMG studies, preprocessing is necessary, mainly to reduce noise (Brunelli et al., 2015). In sEMG signals, this consists mainly of system noise, artifacts, industrial frequency interference, and channel cross-talk (De Luca et al., 2010). In the experiments, we used the wavelet algorithm, chose the sym5 wavelet function with five decomposition layers, and used henrsure to de-baseline and drift the sEMG signal of each channel. The signal was filtered with a fourth-order Butterworth filter to improve the signal-to-noise ratio and to keep the signal in the range of 20–500 Hz while removing external noise and artifacts.

Feature extraction is an important step in signal processing, as it reduces the amount of data while also extracting useful features into low-dimensional data. As EMG signals are known to suffer from lack of smoothness, windowing was performed on the sEMG signals after pre-processing (Ashraf et al., 2021). The window length was 300 samples with a time period of 150 ms, and the window shift was 50 samples with a time period of 25 ms (Luo et al., 2020b). After adding windows to the sEMG data, features including mean absolute value, root mean square, variance (Ahsan et al., 2011), maximum mean absolute value (MMAV), maximum root mean square (MRMS), maximum variance (MVAR) (MMAV, MRMS, and MVAR are improved features by the authors), fourth-order power spectral density, and power spectrum estimation (Zhang et al., 2011; Khushaba, 2014). The dimensionality of the extracted features was 64 features per motion, i.e., 8 channels × 8 features.

Based on the data gathered as described in Section 3, the experiments reported in this section used a fully separate design method for the training and testing sets. That is, one of the eight users was selected as a new user, and the remaining 7 people's data were treated as training user data. We call the process after selecting the new user a trial. There were eight trials in total. According to the seven training users' data, up to seven recognition rates can be determined for each new user. We calculated the average recognition rate for each trial. The above method was used for both Experiment 1 and Experiment 2. Experiment 1 involved (i) testing the present dataset using the CCA-OT framework proposed by Xue et al. (2021), (ii) testing the present dataset using the CCA framework proposed by Khushaba (2014), (iii) testing the present dataset after adding ADR optimization to the framework of (ii), and (iv) testing the present dataset using the proposed ADR-DCCA method. For Experiment 2, Experiment 1 was repeated but with two channels removed randomly from the user test set.

4.1. Experiment 1

4.1.1. User-independent CCA-OT framework

One of the eight participants was selected as a new user, while the other seven were selected as training users. The tests were performed according to the CCA-OT analysis method described by Xue et al. (2021), and the recognition rates for the eight trials are given in Table 1.

TABLE 1

Table 1. Experimental results for user-independent CCA-OT framework.

4.1.2. User-independent CCA-without-ADR framework

Testing was carried out according to the CCA method described by Khushaba (2014), which reduces the differences in sEMG signal properties when the same motion is performed by different people. The recognition rates for the eight trials are given in Table 2.

TABLE 2

Table 2. Experimental results for user-independent CCA-without-ADR framework.

4.1.3. User-independent CCA-with-ADR framework

Following the CCA method described by Khushaba (2014), we performed ADR optimization of the framework. The ADR optimization method chooses the most suitable reconstructed features for user classification. The recognition rates for the eight trials are given in Table 3.

TABLE 3

Table 3. Experimental results for user-independent CCA-with-ADR framework.

4.1.4. User-independent DCCA-with-ADR framework

Using the method proposed herein, data from the training users and new user were fed into the ADR-DCCA framework. The user characteristics were rebuilt into low-dimensional features independent of individual styles via parameter pair optimization, and the user features overcame individual differences. This framework has the advantages of that of Khushaba (2014) while also minimizing the inter-class correlation of different motion features and adaptively selecting new feature dimensions that are most beneficial to the classification performance; this reduces classifier training time and improves classification accuracy. The recognition rates for the eight trials are given in Table 4.

TABLE 4

Table 4. Experimental results for user-independent DCCA-with-ADR framework.

In Experiment 1, the dataset used in this study was tested with four methods, respectively. For each method, results were recorded for eight trials denoted N1 to N8. For the different methods, we plotted the mean values of each trial as histograms. In Figure 5, histograms of the same color show the average recognition accuracy for each test user with the same method, and error bars represent the standard deviation of this user's motion recognition rate on different training sets.

FIGURE 5

Figure 5. Histogram of average accuracies of 12 campaigns classified by four methods for eight users.

4.2. Experiment 2

The sEMG data of all test users were reduced randomly by two channels and then tested again as in Experiment 1 to verify the robustness of the method proposed herein. Tables 5–8 record the recognition results of eight trials (N1 to N8) for each method under 4 methods, and the average recognition rate of each trial was calculated. As in Experiment 1, for the different methods we plotted the average result of each test as a histogram. In Figure 6, histograms in the same color show the average recognition accuracy for each test user under the same method, and error bars represent the standard deviation of this user's motion recognition rate on different training sets.

TABLE 5

Table 5. Experimental results for user-independent CCA-OT framework (the sEMG data of all test users were reduced randomly by two channels).

TABLE 6

Table 6. Experimental results for user-independent CCA-without-ADR framework (the sEMG data of all test users were reduced randomly by two channels).

TABLE 7

Table 7. Experimental results for user-independent CCA-with-ADR framework (the sEMG data of all test users were reduced randomly by two channels).

TABLE 8

Table 8. Experimental results for user-independent DCCA-with-ADR framework (the sEMG data of all test users were reduced randomly by two channels).

FIGURE 6

Figure 6. Histogram of average accuracies of 12 campaigns classified by four methods for eight users (randomly reducing the user test set by two channels).

4.3. Results and discussion

For this subsection, we performed preprocessing and feature extraction and extracted a feature set of 64 features per movement ([6 time-domain features + 2 frequency-domain features] × 8 channels). For the classification, we chose the well-known SVM LIBSVM (Chang and Lin, 2011), as Khushaba (2014) reported that a global SVM performs better in this type of action recognition. In our experimental approach, the training and calibration sets were completely independent, i.e., the training user and the new user were different individuals, and the experimental results were obtained through multiple cross-validations. Khushaba (2014) and Xue et al. (2021) performed CCA (i) between a training feature matrix and an expert feature matrix and (ii) between a test feature matrix and an expert feature matrix. In the present framework, DCCA is used to directly extract the correlation between the training user feature matrix and the new user feature matrix, thereby eliminating the intermediate link. The correlation between the training feature matrix and the test feature matrix is enhanced, and the distribution differences between classes are strengthened. The adaptive dimensionality-reduction method eliminates the redundant information in the high-dimensional features and retains the low-dimensional features that are highly correlated between the new user and the training users. High classification accuracy and good robustness were achieved in classifying 12 categories of upper-limb movements in eight subjects. In Experiment 1, the motion recognition rate of the ADR-DCCA framework proposed in this paper was 92.23; this was 2.89, 4.28, and 2.72 higher than the average recognition rates of the CCA-OT framework, the conventional CCA framework, and the ADR-CCA framework, respectively. In particular, compared with the 26–126 dimensions in the technique described by Khushaba (2014), we used 5–8 dimensions of features that were obtained by fusing 64 dimensions of features, thereby achieving higher accuracy in motion recognition. In Experiment 2, the identification accuracies of the CCA-OT framework, the conventional CCA framework, the ADR-CCA framework, and the ADR-DCCA framework were 85.01, 81.09, 84.81, and 88.75%, respectively. The experimental results show that the framework described herein maintains high classification accuracy even after being deprived of data from two channels, which indicates that the framework is robust. Moreover, the accuracy of motion classification was maintained at 88.75% in the test involving the random removal of two channels, despite the poor performance of the subjects compared with the full-channel subjects. The random channel deletion feature in the proposed framework provides a possible solution for the rehabilitation of muscle-deficient patients and motion recognition of missing channel users.

The training dataset and calibration dataset were passed through the CCA and DCCA algorithms, respectively, to obtain two sets of parameters V. These two sets of V were used to construct each user's test feature matrix. After projection, two new sets of test feature matrices were obtained. The data in the top three columns of the most favorably identified features were plotted as a three-dimensional scatter plot. Figures 7A–D show the post-projection features observed by the CCA method from four directions, and Figures 7E–H show the same for the DCCA method. Compared with the plots for the CCA method, those for the DCCA method have more cohesive color blocks more cohesive and show lower intra-class variance while maintaining good inter-class separability, which is beneficial to classification performance. The results of the three-dimensional scatter plots also verify our hypothesis on the performance of DCCA.

FIGURE 7

Figure 7. Three-dimensional scatter plots of three new features reconstructed using the CCA framework (A–D) and DCCA framework (E–H). The 12 different-colored balls represent the different motions of the new user (the black ball at the bottom is the coordinate rotation mark).

We also analyzed how the number of samples in the calibration process affects the classification accuracy, as well as the direct relationship among the number of samples and the new feature dimension. In this case, we tested the performance of the ADR-CCA and ADR-DCCA frameworks by selecting a certain number of samples from the calibration dataset. As shown in Figure 8, the accuracy of the classification significantly improved and the dimensionality of the features effectively decreased as the size of the calibration dataset increased. In addition, we performed experiments in which the classifier was trained directly with calibration data, where both calibration and test data were obtained from the same human body. Figure 8 shows the recognition results obtained when training the classifier with the calibration data in these experiments. Directly training the classifier with the calibration data using 3–4 sets of calibration data clearly resulted in better performance compared with the CCA method. Unfortunately, the classification performance of all three methods did not meet the application requirements. However, the proposed method ADR-DCCA method had the highest classification accuracy at 5–10 sets of calibration data. Smaller calibration datasets were chosen because this study was intended to target specific groups such as rehabilitation patients. Here, 3–10 sets of corrected data were selected for testing, and a good classification effect could be achieved with eight sets. The experimental results indicate that the ADR-DCCA framework has advantages over other CCA extension frameworks for multi-user sEMG interfaces.

FIGURE 8

Figure 8. Relationships between sample size and classification accuracy and between sample size and new feature dimensions. ADR-DCCA and ADR-CCA denote the classification accuracy of the ADR-DCCA architecture and ADR-CCA with different numbers of calibration sets, respectively. ADR-DCCA-FD and ADR-CCA-FD are the corresponding feature dimensions under classification accuracy. TCCD indicates the classification accuracy when training the classifier model directly with the calibration dataset.

5. Conclusion

In order to solve the problem of stroke patients or patients with limb muscle injuries being unable to quickly obtain a suitable contralateral model for rehabilitation training, a suitable multi-user sEMG motion-recognition framework is presented here. First, the training dataset and the new user calibration dataset are projected into the same style space. New features for training users and new users are recreated based on the parameters obtained after the projection. Second, the dimensionality of the new features is optimized iteratively using an adaptive dimensionality-reduction optimization method to obtain the dimensionality of the system features that are most suitable for motion recognition. Finally, a model is constructed that is suitable for new users and shows good classification performance in user test set classification. The experimental results indicate that the DCCA algorithm is effective at extracting the relevant information between the new user test set and the training user feature set, and that the ADR method effectively reduces the redundant information between the two datasets. These are both important factors in the improvement in motion recognition performance obtained with the framework. This framework also better addresses the problem of large differences in muscle-generated sEMG signals when different people perform the same movement and enables new users to engage in contralateral-driven rehabilitation patterns more quickly. The accuracy of motion classification was maintained at 88.75% in the test involving the random removal of two channels, despite the poor performance of the subjects compared with the full-channel subjects. The random channel deletion feature in the proposed framework provides a possible solution for the rehabilitation of muscle-deficient patients and motion recognition of missing channel users. Finally, we tested the proposed method in the rehabilitation laboratory, and the accuracy of action recognition reached 90.52%. In conclusion, our proposed ADR-DCCA method has important potential applications in the field of rehabilitation.

Although this work implemented sEMG-based action recognition for rehabilitation movements of upper limb exoskeleton robots and glove robots, the experiments only considered human variation and channel loss in sEMG action recognition. In fact, there are many sources of uncertainty in sEMG action recognition, including fatigue differences and electromagnetic interference. In addition, the sEMG-based information is limited. In future work, we will continue to improve the proposed framework and integrate EEG signals and acceleration-sensing information to enable it to cope with more non-ideal situations, thereby improving the comprehensive performance of action recognition.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Qufu Normal University Biomedical Ethics Committee. The patients/participants provided their written informed consent to participate in this study. The animal study was reviewed and approved by Qufu Normal University Biomedical Ethics Committee. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

JW conceived the study, designed and conducted the experiments, performed the data analysis, and drafted and revised the manuscript. DC and YW provided guidance on the algorithms in the study and analyzed the experimental results. All authors contributed to manuscript revision and read and approved the submitted version.

Funding

This work was partially supported by the National Natural Science Foundation of China (Grant 62073187), the Major Scientific and Technological Innovation Project in Shandong Province (Grant 2019JZZY011111), and the Natural Science Foundation of Shandong Province (Grant ZR2022MF236).

Acknowledgments

The authors thank YW and DC for their help and support, and the College of Engineering of Qufu Normal University for providing the experimental platform.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ahsan, M. R., Ibrahimy, M. I., and Khalifa, O. O. (2011). “Electromygraphy (EMG) signal based hand gesture recognition using artificial neural network (ANN),” in 2011 4th International Conference on Mechatronics (ICOM) (Kuala Lumpur), 1–6. doi: 10.1109/ICOM.2011.5937135

CrossRef Full Text | Google Scholar

Ashraf, H., Waris, A., Gilani, S. O., Kashif, A. S., Jamil, M., Jochumsen, M., et al. (2021). Evaluation of windowing techniques for intramuscular EMG-based diagnostic, rehabilitative and assistive devices. J. Neural Eng. 18, 016017. doi: 10.1088/1741-2552/abcc7f

PubMed Abstract | CrossRef Full Text | Google Scholar

Bi, L., Feleke, A. G., and Guan, C. (2019). A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration. Biomed. Signal Process. Control 51, 113–127. doi: 10.1016/j.bspc.2019.02.011

CrossRef Full Text | Google Scholar

Brunelli, D., Tadesse, A. M., Vodermayer, B., Nowak, M., and Castellini, C. (2015). “Low-cost wearable multichannel surface EMG acquisition for prosthetic hand control,” in 2015 6th International Workshop on Advances in Sensors and Interfaces (IWASI) (Gallipoli: IEEE), 94–99. doi: 10.1109/IWASI.2015.7184964

CrossRef Full Text | Google Scholar

Chang, C.-C., and Lin, C.-J. (2011). LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 1–27. doi: 10.1145/1961189.1961199

CrossRef Full Text | Google Scholar

Cheng, J., Wei, F., Li, C., Liu, Y., Liu, A., and Chen, X. (2018). Position-independent gesture recognition using sEMG signals via canonical correlation analysis. Comput. Biol. Med. 103, 44–54. doi: 10.1016/j.compbiomed.2018.08.020

PubMed Abstract | CrossRef Full Text | Google Scholar

De Luca, C. J., Donald Gilmore, L., Kuznetsov, M., and Roy, S. H. (2010). Filtering the surface EMG signal: movement artifact and baseline noise contamination. J. Biomech. 43, 1573–1579. doi: 10.1016/j.jbiomech.2010.01.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Disselhorst-Klug, C., Schmitz-Rode, T., and Rau, G. (2009). Surface electromyography and muscle force: limits in sEMG–force relationship and new approaches for applications. Clin. Biomech. 24, 225–235. doi: 10.1016/j.clinbiomech.2008.08.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Elamvazuthi, I., Zulkifli, Z., Ali, Z., Khan, M. A., Parasuraman, S., Balaji, M., et al. (2015). Development of electromyography signal signature for forearm muscle. Proc. Comput. Sci. 76, 229–234. doi: 10.1016/j.procs.2015.12.347

CrossRef Full Text | Google Scholar

Fang, C., He, B., Wang, Y., Cao, J., and Gao, S. (2020). EMG-centered multisensory based technologies for pattern recognition in rehabilitation: state of the art and challenges. Biosensors 10, 85. doi: 10.3390/bios10080085

PubMed Abstract | CrossRef Full Text | Google Scholar

Gatto, B. B., and dos Santos, E. M. (2017). “Discriminative canonical correlation analysis network for image classification,” in 2017 IEEE International Conference on Image Processing (ICIP) (Beijing), 4487–4491. doi: 10.1109/ICIP.2017.8297131

CrossRef Full Text | Google Scholar

Ghassemi, M., Triandafilou, K., Barry, A., Stoykov, M. E., Roth, E., Mussa-Ivaldi, F. A., et al. (2019). Development of an EMG-controlled serious game for rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 283–292. doi: 10.1109/TNSRE.2019.2894102

PubMed Abstract | CrossRef Full Text | Google Scholar

Hardoon, D. R., Szedmak, S., and Shawe-Taylor, J. (2004). Canonical correlation analysis: an overview with application to learning methods. Neural Comput. 16, 2639–2664. doi: 10.1162/0899766042321814

PubMed Abstract | CrossRef Full Text | Google Scholar

Hermens, H. J., Freriks, B., Disselhorst-Klug, C., and Rau, G. (2000). Development of recommendations for SEMG sensors and sensor placement procedures. J. Electromyogr. Kinesiol. 10, 361–374. doi: 10.1016/S1050-6411(00)00027-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Hotelling, H. (1992). Relations Between Two Sets of Variates. New York, NY: Springer. doi: 10.1007/978-1-4612-4380-9_14

CrossRef Full Text | Google Scholar

Jarque-Bou, N. J., Sancho-Bru, J. L., and Vergara, M. (2021). A systematic review of EMG applications for the characterization of forearm and hand muscle activity during activities of daily living: results, challenges, and open issues. Sensors 21, 3035. doi: 10.3390/s21093035

PubMed Abstract | CrossRef Full Text | Google Scholar

Khushaba, R. N. (2014). Correlation analysis of electromyogram signals for multiuser myoelectric interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 745–755. doi: 10.1109/TNSRE.2014.2304470

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, K., Zhang, J., Wang, L., Zhang, M., Li, J., and Bao, S. (2020). A review of the key technologies for sEMG-based human-robot interaction systems. Biomed. Signal Process. Control 62, 102074. doi: 10.1016/j.bspc.2020.102074

CrossRef Full Text | Google Scholar

Lobov, S., Krilova, N., Kastalskiy, I., Kazantsev, V., and Makarov, V. A. (2018). Latent factors limiting the performance of sEMG-interfaces. Sensors 18, 1122. doi: 10.3390/s18041122

PubMed Abstract | CrossRef Full Text | Google Scholar

Luo, J., Huang, D., Li, Y., and Yang, C. (2021). Trajectory online adaption based on human motion prediction for teleoperation. IEEE Trans. Autom. Sci. Eng. 1–5. doi: 10.1109/TASE.2021.3111678

CrossRef Full Text | Google Scholar

Luo, J., Lin, Z., Li, Y., and Yang, C. (2020a). A teleoperation framework for mobile robots based on shared control. IEEE Robot. Autom. Lett. 5, 377–384. doi: 10.1109/LRA.2019.2959442

CrossRef Full Text | Google Scholar

Luo, J., Liu, C., Feng, Y., and Yang, C. (2020b). A method of motion recognition based on electromyographic signals. Adv. Robot. 34, 976–984. doi: 10.1080/01691864.2020.1750480

CrossRef Full Text | Google Scholar

Matsubara, T., and Morimoto, J. (2013). Bilinear modeling of EMG signals to extract user-independent features for multiuser myoelectric interface. IEEE Trans. Biomed. Eng. 60, 2205–2213. doi: 10.1109/TBME.2013.2250502

PubMed Abstract | CrossRef Full Text | Google Scholar

Medved, V., Medved, S., and Kova, I. (2020). Critical appraisal of surface electromyography (sEMG) as a taught subject and clinical tool in medicine and kinesiology. Front. Neurol. 11, 560363. doi: 10.3389/fneur.2020.560363

PubMed Abstract | CrossRef Full Text | Google Scholar

Pan, L., Crouch, D. L., and Huang, H. (2018). Myoelectric control based on a generic musculoskeletal model: Toward a multi-user neural-machine interface. IEEE Trans. Neural Syst. Rehabil. Eng. 26, 1435–1442. doi: 10.1109/TNSRE.2018.2838448

PubMed Abstract | CrossRef Full Text | Google Scholar

Qi, W., and Aliverti, A. (2020). A multimodal wearable system for continuous and real-time breathing pattern monitoring during daily activity. IEEE J. Biomed. Health Inform. 24, 2199–2207. doi: 10.1109/JBHI.2019.2963048

PubMed Abstract | CrossRef Full Text | Google Scholar

Qi, W., Ovur, S. E., Li, Z., Marzullo, A., and Song, R. (2021). Multi-sensor guided hand gesture recognition for a teleoperated robot using a recurrent neural network. IEEE Robot. Autom. Lett. 6, 6039–6045. doi: 10.1109/LRA.2021.3089999

CrossRef Full Text | Google Scholar

Samuel, O. W., Asogbon, M. G., Geng, Y., Al-Timemy, A. H., Pirbhulal, S., Ji, N., et al. (2019). Intelligent emg pattern recognition control method for upper-limb multifunctional prostheses: advances, current challenges, and future prospects. IEEE Access 7, 10150–10165. doi: 10.1109/ACCESS.2019.2891350

CrossRef Full Text | Google Scholar

Simao, M., Mendes, N., Gibaru, O., and Neto, P. (2019). A review on electromyography decoding and pattern recognition for human-machine interaction. IEEE Access 7, 39564–39582. doi: 10.1109/ACCESS.2019.2906584

CrossRef Full Text | Google Scholar

Su, H., Hu, Y., Karimi, H. R., Knoll, A., Ferrigno, G., and De Momi, E. (2020). Improved recurrent neural network-based manipulator control with remote center of motion constraints: experimental results. Neural Netw. 131, 291–299. doi: 10.1016/j.neunet.2020.07.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Su, H., Qi, W., Schmirander, Y., Ovur, S. E., Cai, S., and Xiong, X. (2022). A human activity-aware shared control solution for medical human–robot interaction. Assem. Autom. 42, 388–394. doi: 10.1108/AA-12-2021-0174

CrossRef Full Text | Google Scholar

Sun, L., Ji, S., and Ye, J. (2010). Canonical correlation analysis for multilabel classification: a least-squares formulation, extensions, and analysis. IEEE Trans. Pattern Anal. Mach. Intell. 33, 194–200. doi: 10.1109/TPAMI.2010.160

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, Q.-S., Zeng, S.-G., Liu, Y., Heng, P.-A., and Xia, D.-S. (2005). A new method of feature fusion and its application in image recognition. Pattern Recogn. 38, 2437–2448. doi: 10.1016/j.patcog.2004.12.013

CrossRef Full Text | Google Scholar

Vigotsky, A. D., Halperin, I., Lehman, G. J., Trajano, G. S., and Vieira, T. M. (2018). Interpreting signal amplitudes in surface electromyography studies in sport and rehabilitation sciences. Front. Physiol. 8, 985. doi: 10.3389/fphys.2017.00985

PubMed Abstract | CrossRef Full Text | Google Scholar

Xue, B., Wu, L., Wang, K., Zhang, X., Cheng, J., Chen, X., et al. (2021). Multiuser gesture recognition using sEMG signals via canonical correlation analysis and optimal transport. Comput. Biol. Med. 130, 104188. doi: 10.1016/j.compbiomed.2020.104188

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, F., Li, P., Hou, Z.-G., Chen, Y., Xu, F., Hu, J., et al. (2011). “SEMG feature extraction methods for pattern recognition of upper limbs,” in 2011 International Conference on Advanced Mechatronic Systems (Zhengzhou), 222–227.

Google Scholar