Real-World Evaluation of Deep Learning Decoders for Motor Imagery EEG-based BCIs

Pierre Sedi Nzakuna¹Emanuele D'Auria¹Vincenzo Paciello^1*Vincenzo Gallo¹Ernest N. Kamavuako^2,3Aimé Lay-Ekuakille^4,5Kyandoghere KYAMAKYA^6,7*

• ¹University of Salerno, Fisciano, Italy
• ²King's College London, London, United Kingdom
• ³University of Kindu, Kindu, Democratic Republic of Congo
• ⁴Universita del Salento, Lecce, Italy
• ⁵ISPT-Kin, Kinshasa, Democratic Republic of Congo
• ⁶Universitat Klagenfurt, Klagenfurt am Wörthersee, Austria
• ⁷Universite de Kinshasa, Kinshasa, Democratic Republic of Congo

Motor Imagery (MI) Electroencephalography (EEG)-based control in online Brain-Computer Interfaces demands issuing decisions on short temporal context. However, most published Deep Learning (DL) EEG decoders are introduced and validated offline on public datasets with longer windows, leaving their real-time suitability uncertain. We investigate this gap by testing 10 representative DL decoders—covering convolutional neural networks (CNNs), filter-bank CNNs, temporal-convolutional networks (TCNs), attention and Transformer hybrids—under a soft real-time protocol with 2-second windows. We quantify performance with accuracy, sensitivity, precision, miss-as-neutral (MANR), false-alarm rate (FAR), information-transfer rate (ITR), and workload. To relate behavior to physiology, we examine lateralization indices, C3 vs C4 mu-band power, and MI-versus-Neutral topographic contrasts. Our study reveals ranking shifts between offline and online BCI operation and a marked rise in inter-subject variability. Compact spectro-temporal CNN backbones with lightweight temporal context (TCN or dilations) sustain performance more consistently under short-time windows, whereas deeper attention and Transformer stacks are more sensitive to subject and session differences. This work provides a reproducible benchmark and actionable guidance for online-first EEG decoder design and calibration that remains reliable under real-world, short-time constraints.