AUTHOR=Singh Avinash Kumar , Bianchi Luigi 

TITLE=Encoding temporal information in deep convolution neural network

JOURNAL=Frontiers in Neuroergonomics

VOLUME=Volume 5 - 2024

YEAR=2024

URL=https://www.frontiersin.org/journals/neuroergonomics/articles/10.3389/fnrgo.2024.1287794

DOI=10.3389/fnrgo.2024.1287794

ISSN=2673-6195

ABSTRACT=A recent development in deep learning techniques has attracted attention to the decoding and classification of electroencephalogram (EEG) signals. Despite several efforts utilizing different features in EEG signals, a significant research challenge is to use time-dependent features in combination with local and global features. There have been several efforts to remodel the deep learning convolution neural networks (CNNs) to capture time-dependency information. Thesefeatures are usually either hand-crafted features such as power ratios or splitting data into smallersized windows related to specific properties such as peak at 300ms. However, these approaches partially solve the problem but simultaneously hinder CNN's capability to learn from unknown information that might be present in the data. Other approaches, like recurrent neural networks, are very suitable for learning time-dependent information from EEG signals in the presence of unrelated sequential data. To solve this, we have proposed an encoding kernel (EnK), a novel time encoding approach, which uniquely introduces time decomposition information during vertical convolution operation in CNN. The encoded information lets CNN learn time-dependent features in addition to local and global features. We performed extensive experiments on several EEG datasets-physical-human robot collaboration (pHRC), P300 visual-evoked potentials, motor imagery (MI), movement-related cortical potentials (MRCP), and DEAP. EnK outperforms the state-of-art by up to 6.5% reduction in mean-squared error (mse) and 9.5% improvement in F1-score compared on average for all datasets together compared to base models. These results support our approach and show a high potential to improve the performance of physiological and non-physiological data. Moreover, the EnK can be applied to virtually any deep learning architecture with minimal effort.