Multimodal Fusion Technologies and Applications in the Context of Neuroscience

62.8K
views
99
authors
17
articles
Editors
7
Impact
Loading...
Original Research
08 September 2022
Article Cover Image
5,888 views
29 citations
Original Research
16 August 2022

Pandemic-related sleep disorders affect human physical and mental health. The artificial intelligence (AI) based sleep staging with multimodal electrophysiological signals help people diagnose and treat sleep disorders. However, the existing AI-based methods could not capture more discriminative modalities and adaptively correlate these multimodal features. This paper introduces a multimodal attention network (MMASleepNet) to efficiently extract, perceive and fuse multimodal features of electrophysiological signals. The MMASleepNet has a multi-branch feature extraction (MBFE) module followed by an attention-based feature fusing (AFF) module. In the MBFE module, branches are designed to extract multimodal signals' temporal and spectral features. Each branch has two-stream convolutional networks with a unique kernel to perceive features of different time scales. The AFF module contains a modal-wise squeeze and excitation (SE) block to adjust the weights of modalities with more discriminative features and a Transformer encoder (TE) to generate attention matrices and extract the inter-dependencies among multimodal features. Our MMASleepNet outperforms state-of-the-art models in terms of different evaluation matrices on the datasets of Sleep-EDF and ISRUC-Sleep. The implementation code is available at: https://github.com/buptantEEG/MMASleepNet/.

3,846 views
29 citations
9,023 views
16 citations
Scatter diagrams of clinical scales and MEP parameters on the affected side of stroke survivors. (A) MEP latency versus Fugl-Meyer assessment upper extremity scales. (B) CMCT of MEP versus Fugl-Meyer assessment upper extremity scales. (C) MEP latency versus ARAT. (D) CMCT of MEP versus ARAT. ARAT, Action Research Arm Test; CMCT, central motor conduction time.
Original Research
01 June 2022

Objective: Increased muscle co-contraction of the agonist and antagonist muscles during voluntary movement is commonly observed in the upper limbs of stroke survivors. Much remain to be understood about the underlying mechanism. The aim of the study is to investigate the correlation between increased muscle co-contraction and the function of the corticospinal tract (CST).

Methods: Nine stroke survivors and nine age-matched healthy individuals were recruited. All the participants were instructed to perform isometric maximal voluntary contraction (MVC) and horizontal task which consist of sponge grasp, horizontal transportation, and sponge release. We recorded electromyography (EMG) activities from four muscle groups during the MVC test and horizontal task in the upper limbs of stroke survivors. The muscle groups consist of extensor digitorum (ED), flexor digitorum (FD), triceps brachii (TRI), and biceps brachii (BIC). The root mean square (RMS) of EMG was applied to assess the muscle activation during horizontal task. We adopted a co-contraction index (CI) to evaluate the degree of muscle co-contraction. CST function was evaluated by the motor-evoked potential (MEP) parameters, including resting motor threshold, amplitude, latency, and central motor conduction time. We employed correlation analysis to probe the association between CI and MEP parameters.

Results: The RMS, CI, and MEP parameters on the affected side showed significant difference compared with the unaffected side of stroke survivors and the healthy group. The result of correlation analysis showed that CI was significantly correlated with MEP parameters in stroke survivors.

Conclusion: There existed increased muscle co-contraction and impairment in CST functionality on the affected side of stroke survivors. The increased muscle co-contraction was correlated with the impairment of the CST. Intervention that could improve the excitability of the CST may contribute to the recovery of muscle discoordination in the upper limbs of stroke survivors.

4,831 views
18 citations
Recommended Research Topics
Frontiers Logo

Frontiers in Computational Neuroscience

Advancements in Smart Diagnostics for Understanding Neurological Behaviors and Biosensing Applications
Edited by Saad Arif, Muhammad Zia Ur Rehman, Zohaib Mushtaq
17.8K
views
47
authors
9
articles
Frontiers Logo

Frontiers in Neuroscience

Neural Interface for Cognitive Human-Robot Interaction and Collaboration
Edited by Hong Cheng, Zeng-Guang Hou, Xiaodong Guo, Mitsuhiro Hayashibe, Zhan Li
59K
views
57
authors
11
articles
Frontiers Logo

Frontiers in Neuroscience

Currents in Biomedical Signals Processing - Methods and Applications
Edited by Aleksandra Dagmara Kawala-Sterniuk, Mariusz Pelc, Radek Martinek, Grzegorz Marcin Wójcik
75.5K
views
51
authors
11
articles
Frontiers Logo

Frontiers in Neurorobotics

Deep Learning Techniques Applied to Affective Computing
Edited by Shiqing Zhang, Xiaopeng Hong, Xiaobai Li, Zhen Cui, Wenming Zheng
63K
views
48
authors
11
articles
Frontiers Logo

Frontiers in Neuroscience

Neural Signals Acquisition and Intelligent Analysis
Edited by Xiaomin Yang, Yin Tian, Gwanggil Jeon, Yu Pang
24.6K
views
52
authors
10
articles