AUTHOR=Zhang Xiaodong , Li Hanzhe , Dong Runlin , Lu Zhufeng , Li Cunxin 

TITLE=Electroencephalogram and surface electromyogram fusion-based precise detection of lower limb voluntary movement using convolution neural network-long short-term memory model

JOURNAL=Frontiers in Neuroscience

VOLUME=Volume 16 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.954387

DOI=10.3389/fnins.2022.954387

ISSN=1662-453X

ABSTRACT=The EEG and sEMG fusion have been widely used in detection of human movement intention for human-robot interaction, but the internal relationship of EEG and sEMG signal is not clear so that their fusion still has some shortcomings. A precise fusion method of EEG and sEMG using CNN-LSTM model will be investigated to detect lower limb voluntary movement in this paper. At first, the EEG and sEMG signals processing of each stage was analyzed, so that the response time difference between EEG and sEMG can be estimated to detect lower limb voluntary movement, and it can be calculated by the symbolic transfer entropy. Secondly, the data fusion and feature of EEG and sEMG was both used for obtaining data matrix of model, and a hybrid CNN-LSTM was established for the EEG and sEMG based decoding model of lower limb voluntary movement, so that the estimated value of time difference was about 24 ~ 26 ms, and the calculated value was between 25-45 ms. Finally, the offline experimental results showed that the accuracy of data fusion was significantly higher than feature fusion-based accuracy in 5-fold cross-validation, and the average accuracy of EEG and sEMG data fusion was more than 95%, the improved the average accuracy for eliminating the response time difference between EEG and sEMG was about 0.7±0.26% in data fusion. In the meantime, the online average accuracy of data fusion-based CNN-LSTM was more than 87% in all subjects. The above results demonstrated that the time difference had an influence on the EEG and sEMG fusion to detect lower limb voluntary movement, and the proposed CNN-LSTM model can achieve a high performance. This work provides a stable and reliable basis for human-robot interaction of lower limb exoskeleton.