Region-of-interest estimation using convolutional neural network and long short-term memory for functional near-infrared spectroscopy data
-
1
Doshisha University, Graduate School of Life and Medical Sciences, Japan
-
2
Doshisha University, Faculty of Life and Medical Sciences, Japan
-
3
DENSO CORPORATION, Japan
Introduction
In recent years, functional near-infrared spectroscopy (fNIRS) has attracted attention as a noninvasive functional neuroimaging technology. fNIRS visualizes brain activity by measuring the hemodynamic responses of oxy- and deoxy-hemoglobin (Hb) associated with neural behavior. fNIRS allows us to identify a cortical activation or brain regions associated with a given stimulus by analyzing the time courses of oxy- and deoxy-Hb.
However, since fNIRS signals often contain noises (e.g., motion-related artifacts and psychological noises including heartbeat), it is not easy to extract meaningful brain activation. Furthermore, comparison of the raw fNIRS data between subjects should not be done because fNIRS detects only the relative change in oxy- or deoxy-Hb. To perform a group analysis, baseline calibration is also needed. However, there is no established way for preprocessing of fNIRS signals.
The purpose of this study was to estimate the brain regions associated with a given task or stimulus by automatically extracting features of cortical activities from the fNIRS data. We proposed a novel feature extraction method for the fNIRS signals whose temporal and spatial characteristics were considered.
Method
Deep learning methods have been mostly used for classification of multi-dimensional data, however, in this study, we focused on another aspect of the deep learning methodology regarding determination of region-of-interest (ROI) associated with a given task or stimuli.
In our proposed approach, a group classifier is constructed from all subject fNIRS data using supervised learning. The group classifier is constructed for all channels of a fNIRS measurement system, and a group label is supervised during each learning process. After the learning is completed, the classification accuracy using only a single channel is compared among all channels, and the channel whose classification accuracy has better performance is extracted as the critical ROI for group classification.
Moreover, we proposed a new deep learning algorithm which is a fusion of two algorithms, convolutional neural network (CNN) [1] and long short-term memory (LSTM) [2]. Although both algorithms can automatically perform feature extraction, CNN preserves the spatial information on input data during learning, and LSTM stores the temporal information. Taking advantage of these two algorithms, our proposed algorithm basically consisted of five layers, input, convolution, LSTM, pooling, and output layer. We can identify the ROI because neuron units of the input layer are associated with the fNIRS probes placed on participants' head.
Experiments
To examine the effectiveness of our approach, we tried to extract the ROIs related to working memory. Cerebral blood flow during N-back (N=2, 3) task, which was often used to assess the working memory, was measured using fNIRS. 30 healthy male subjects (average age: 23.3 ± 1.5 years, right-handed) and 5 healthy female subjects (average age: 21.7 ± 0.52 years, right-handed) participated in the experiment. The fNIRS probes were placed according to the International 10–20 system. Using the fNIRS data obtained, our classifier is trained to classify the input data as either “2-back” or “3-back.”
Results and Discussion
The average percentage of correct answers in 2-back (low degree of difficulty) and 3-back (high degree of difficulty) tasks were 90.2 ± 8.98% and 84.3 ± 8.87%, respectively. It was shown to be significantly different by Wilcoxon signed-rank test (p <0.01).
Using this fNIRS data, our classifier achieved the classification accuracy of 91.4 ± 1.49%. Moreover, with a comparison of single-channel classification accuracy for all channels, we successfully extracted left dorsolateral prefrontal cortex (DLPFC) and anterior prefrontal cortex (APFC) as task-related ROIs.
DLPFC is activated in a number of working memory task and cognitive task, and is also known to play a key role in cognitive control and adaptation of a strategy to improve the task performance. In particular, the left DLPFC is said to be activated in verbal working memory task [3].
APFC is the area where highly abstract information is processed. It has also been reported that APFC and DLPFC had activated in a dual-task situation [4]. Furthermore, activation of DLPFC and APFC is associated with the difficulties of N-back problems [5][6]. These observations suggest that the ROIs estimated by the proposed method are reasonable. Consequently, our proposed method has been shown to be useful for a brain function analysis of fNIRS data.
References
1.Y.LeCun：Gradient-based learning applied to document recognition, Proceedings of the IEEE, Vol. 86, pp. 2278-2324 (1998)
2.S.Hochreiter：Long short-term memory, Neural computation, Vol. 9, pp. 1735-1780 (1997)
3.Edward E.Smith, J.Jonides：Working memory：A view from neuroimaging, Cognitive psychology, Vol. 33, pp. 5-42 (1997)
4.R.Narender, Adrian M.Owe：Anterior pre- frontal cortex: insights into function from anatomy and neuroimaging, Nature Reviews Neuroscience, Vol. 5, pp. 184-194 (2004)
5.Adrian M.Owen：N-Back Working Memory Paradig：A Meta-Analysis of Normative Functional Neuroimaging Studies, Human Brain Mapping, Vol. 25, pp. 46-59 (2005)
6.C.Katrin: Increased neural activity during high working memory load predicts low relapse risk in alcohol dependence, Addiction biology, Vol. 19, pp. 402- 414 (2014)
Keywords:
functional near-infrared spectroscopy (fNIRS),
working memory,
n-back task,
deep learning,
Convolutional Neural Networks (CNN),
Long Short-Term Memory
Conference:
Neuroinformatics 2016, Reading, United Kingdom, 3 Sep - 4 Sep, 2016.
Presentation Type:
Poster
Topic:
Neuroimaging
Citation:
Tamaki
T,
Hiwa
S,
Hachisuka
K,
Okuno
E and
Hiroyasu
T
(2016). Region-of-interest estimation using convolutional neural network and long short-term memory for functional near-infrared spectroscopy data.
Front. Neuroinform.
Conference Abstract:
Neuroinformatics 2016.
doi: 10.3389/conf.fninf.2016.20.00027
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
28 Apr 2016;
Published Online:
18 Jul 2016.
*
Correspondence:
Mr. Takaya Tamaki, Doshisha University, Graduate School of Life and Medical Sciences, Kyoto, Japan, ttamaki@mis.doshisha.ac.jp