DATA REPORT article
Sec. Neural Technology
Volume 14 - 2020 | https://doi.org/10.3389/fnins.2020.579353
Open Access Multimodal fNIRS Resting State Dataset With and Without Synthetic Hemodynamic Responses
- 1Neurophotonics Center, Biomedical Engineering, Boston University, Boston, MA, United States
- 2Department of Speech, Language and Hearing Sciences, Boston University, Boston, MA, United States
Functional Near-Infrared Spectroscopy (fNIRS) is an optical neuroimaging technology that has rapidly gained momentum within the last decades (Boas et al., 2014; Scholkmann et al., 2014; Yücel et al., 2017). It is a non-hazardous and non-invasive optical brain imaging technique that uses near-infrared light to measure local cortical concentration changes of oxygenated and deoxygenated hemoglobin (HbO2/HbR), which are associated with brain metabolism (Villringer and Chance, 1997; Ferrari and Quaresima, 2012). fNIRS has been considered a cost-effective and mobile alternative for functional Magnetic Resonance Imaging in conventional neuroscientific research. It is very suitable—and thus increasingly being used—for single trial analysis and Brain Computer Interface (BCI) applications (Matthews et al., 2008; Hong et al., 2018) as a single modality or along with Electroencephalography (EEG). While EEG and fNIRS signal processing is essential to increase the contrast to noise ratio (CNR) of measured brain responses, the nature of the signals and processing methods differ greatly. Hemodynamic brain responses in fNIRS are usually masked by local and systemic physiological confounding signals, for instance from superficial (scalp) blood flow, low frequency oscillations (Mayer waves), motion and breathing (Elwell et al., 1999; Yücel et al., 2016; von Lühmann et al., 2019). New and increasingly complex and powerful statistical methods are being developed that aim to remove the confounding factors in the signal, improve CNR and increase the detection/classification accuracy of hemodynamic responses. An objective way of validating the power of these novel methods and comparing them with the existing ones is to use an fNIRS dataset which has all the confounding signals but also a known hemodynamic brain response. One solution for this problem is to generate realistic fNIRS ground truth data by modeling a hemodynamic response function (HRF) on top of real resting state data (Gagnon et al., 2012; von Lühmann et al., 2019, 2020a,b). This approach can be used as a good approximation for realistic fNIRS signals with evoked responses, for which the ground truth is available. Generating such data is comparatively straight forward but requires prior knowledge in fNIRS signal characteristics as well as experience in fNIRS signal processing. Moreover, the use of short-separation fNIRS measurements and additional physiological signals, such as accelerometer or photoplethysmography (PPG), has been shown to enable methods that yield improved CNR (Yücel et al., 2015; von Lühmann et al., 2020a), but there are only few openly available multimodal fNIRS datasets (Shin et al., 2017, 2018) and even fewer multimodal datasets that include sufficient resting state periods to enable the approach described above. Thus, as a remedy, here we provide such a multimodal dataset with (and without) added synthetic HRF ground truth, short-separation fNIRS measurements, accelerometer, and other physiological measurement, for the data science community in order to facilitate the validation of novel methods. We also provide a simple code example to enable customization and modification of the HRF ground truth in the data.
The resting state data consists of two subsets: Dataset I, with 5 min resting state data from 14 participants and Dataset II, with 10 min resting state data from 14 participants. The data details follow and are summarized in Figure 1.
Figure 1. Summary of dataset metadata. Red dots are fNIRS emitters, blue dots are fNIRS detectors. LS, Long-Separation; SS, Short-Separation; PPG, Photoplethysmography; RESP, Respiration; BP, Blood Pressure; ACCEL, Accelerometer. HRF with 20, 50, and 100 % of +0.66 | −0.23 μM peak HbO2/HbR amplitude. Sensitivity profile in log10 dB.
Participants and Demographics
Dataset I consists of recordings from 14 healthy participants (age: 21 ± 2 years; 11 male/3 female) and Dataset II from 14 healthy participants (age: 32 ± 19 years; 7 male/6 female/1 not reported) with no neurological or psychological disorders.
Participants were seated in a comfortable chair with room light on and were asked to look at a fixation cross on a black screen ~50 cm in front of them. A 5-min resting state data (Dataset I) or a 10-min resting state data (Dataset II) were recorded from each participant.
fNIRS data were acquired using a multichannel continuous wave fNIRS system (CW6, TechEn Inc. MA, USA) operating at 690 and 830 nm wavelengths. The system is an optical imager with 32 frequency encoded lasers (half at 690 and half at 830 nm) and 32 avalanche photo-diode detectors. The light is carried from the system to the head probe and back via optical fiber bundles. fNIRS data were acquired at a sample rate of 50 Hz.
DATASET I: Optode Array and Auxiliary Measurements
Both head optode arrays were designed utilizing AtlasViewer software (Aasted et al., 2015) (Figure 1). The optode array for Dataset I consisted of an elastic cap (EasyCap, Herrsching, Germany) with 8 sources, 12 long-separation detectors (~3 cm apart from the source) and 2 short-separation detectors (~1 cm apart from the source) providing, in total, 26 long-separation and 2 short-separation channels covering the occipital lobe.
Systemic physiological changes and head motions of the participants were simultaneously recorded along with the fNIRS data using an MP160 data acquisition and analysis system (BIOPAC Systems Inc., Goleta, CA). The pulse waveform was recorded using a PPG100C amplifier and TSD200 PPG pulse transducer placed on the participant's right index finger (BIOPAC Systems Inc., Goleta, CA). Respiration data was collected via measuring the abdominal (or thoracic) expansion and contraction using a RSP100C amplifier and a TSD201 respiration transducer (respiration belt) (BIOPAC Systems Inc., Goleta, CA) around the participant's chest. The blood pressure waveform was recorded using a DA100C amplifier and a TSD110 pressure transducer (BIOPAC Systems Inc., Goleta, CA) placed on the participant's right thumb. Head motions in x, y, z directions were collected using an accelerometer (ADXL335, Analog Devices Inc., Norwood, MA) secured on the head with a headband.
DATASET II: Optode Array and Auxiliary Measurements
The optode array for Dataset II consisted of an elastic cap (EasyCap, Herrsching, Germany) with 16 sources, 24 long-separation detectors (~3 cm apart from the source) and 8 short-separation detectors (~1 cm apart from the source) providing, in total, 48 long-separation and 8 short-separation channels covering the head from frontal to parietal regions bilaterally.
Head motions of the participants in x, y, z directions were simultaneously recorded along with the fNIRS data using a 3-axis accelerometer (ADXL335, Analog Devices Inc., Norwood, MA) secured on the head with a headband.
Adding Synthetic HRF to the fNIRS Data
In the documented data repository, we provide the acquired resting state data with and without synthetic HRF as well as the scripts used for the generation of the data to enable users to alter and re-generate ground truth HRF according to their needs. We generate synthetic HRFs with three different amplitudes using a gamma function with a time-to-peak of 6 s and a total duration of 16.5 s. The shape of this synthetic HRF is also depicted in Figure 2A. The three amplitudes are provided as percentages (100/50/20%) of a typical average amplitude of a task-evoked HRF (Huppert et al., 2006) and simulate varying degrees of CNR in the data: The (100%) amplitude is equal to +1% | −2% change from baseline intensity at 690 nm | 830 nm leading to an HRF peak amplitude of +0.66 | −0.23 μM for HbO2 | HbR, respectively with a differential pathlength factor of 6 (Delpy et al., 1988; Boas et al., 2004) for a 30 mm source-detector separation. For each participant in the two datasets, all resting state data is divided into windows of 20 s length. The HRFs are added in the intensity domain at a random onset (0–3.5 s) within each 20 s window for a randomly selected half of all available long separation channels after pruning with a 5 dB SNR threshold. This results in an average of 15 | 38 trials per participant and HRF amplitude in each resting state Dataset I | II.
Figure 2. Overview of data quality and baseline analysis for Datasets I and II. (A) Across-trial recovered HRF estimated with tCCA GLM using Homer3 for all three amplitudes (100, 50, 20%) (participant 33 from Dataset I and participant 98 from Dataset II - the first participant in each data set). Red: HbO2, Blue: HbR, Black dashed: Ground Truth. (B) Mean SNR in dB across channels for all participants. Whiskers indicate standard deviation. (C) Mean Motion Ratio across channels for all participants which indicates the ratio of motion-contaminated data to the whole data. (D) Example time course of multimodal data in Dataset I. LS, Long-separation channel; SS, Short-separation channel; AccelX|Y|Z, Accelerometer; PPG, Photoplethysmogram; BP, Blood Pressure; RESP, Respiration.
Data Structure and Format
Both datasets are presented in the Shared Near Infrared File Format V1.0 Specification (snirf), which is based on the HDF5 file format (https://github.com/fNIRS/snirf). SnirfClass function loads the snirf object into the MATLAB environment. Table 1 provides the list of variables in the current dataset snirf object. The main fields of interest are: the data field which has the fNIRS raw signal at each channel and relevant information, the probe field which has optode array information, the aux field which has all the auxiliary measurements and their details, and the stim field which has the experimental paradigm information. Please note that, while snirf.aux(2)/(3)/(4) have AccelX, AccelY, and AccelZ measurements for both datasets, Dataset I has PPG, blood pressure (BP), RESP at snirf.aux(5), (6), and (7), respectively in addition to these.
Baseline Analysis and Data Quality Assessment
Baseline analysis and data quality assessment was performed using the openly available Homer3 toolbox (https://github.com/BUNPC/Homer3) (Huppert et al., 2009). HRFs were recovered from the augmented resting state data using the processing stream provided in the repository (tCCA_xmpl_procStream_Gauss_noHPF.cfg under “code” folder). This processing stream includes 0.5 Hz zero phase low pass filter with an effective order of 6, conversion to HbO2 and HbR using the modified Beer-Lambert Law (Delpy et al., 1988; Boas et al., 2004), and subsequent HRF estimation with the temporally embedded General Linear Model (tCCA GLM) approach using short-distance channels and a polynomial drift term for nuisance regression and Gaussian basis functions for the HRF regressor (von Lühmann et al., 2020a). Figure 2A exemplifies the resulting HRF estimates in one augmented channel for all three amplitudes from participant 33 from Dataset I and participant 98 from Dataset II. Data quality is provided for each participant as across-channel average of the Signal to Noise Ratio (SNR) in Figure 2B and as the mean motion ratio across channels in Figure 2C. Channel SNR is calculated as 20 × log10 of the mean over std. of the raw intensity signal. The motion ratio is calculated as the ratio between the cumulative time of segments in the data that were considered to be confounded by motion artifacts, as identified by the Homer2 function hmrMotionArtifactByChannel (with tMotion = 0.5, tMask = 0.5, STDEVthresh = 20, AMPthresh = 5), to the total acquisition time. Figure 2D displays a typical segment of all available signals (z-scored) in the first participant in Dataset I. Long and short-separation fNIRS channels exhibit typical low frequency components and cardiac pulsation, which is also present in the PPG and BP measurement.
We reported a multimodal fNIRS resting state dataset from 28 participants, that we provide with and without added synthetic HRF ground truth at three different amplitudes. We include the script used for the generation of these data to enable users to adapt this approach to their own needs. The availability of multiple auxiliary biosignals, such as motion (accelerometer) and PPG in the data, can be used to explore and extend existing multimodal fNIRS-based signal processing approaches (von Lühmann et al., 2019, 2020a). Resting fNIRS data with added known HRF enables the validation of novel processing methods for single trial HRF detection and BCI as well as more general artifact rejection and preprocessing approaches and their comparison with existing methods. This can also be useful for methods that tackle challenges such as non-stationarities in the amplitude and time to peak of hemodynamic responses to a stimulus.
Data Availability Statement
Publicly available datasets were analyzed in this study. This data and the code for HRF ground-truth generation can be found here: https://www.nitrc.org/frs/?group_id=1071.
The studies involving human participants were reviewed and approved by Boston University IRB. The participants provided their written informed consent to participate in this study.
XL and NG collected the first and second datasets, respectively. AL and MY analyzed the data, prepared the figures, and wrote the report. DB reviewed the report. All authors confirmed the final version of the report.
This work was funded by a research contract under Facebook's Sponsored Academic Research Agreement and in part by NIH R24NS104096.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors declare that this study received funding from Facebook. The funder was involved in the decision to submit the paper for publication.
Aasted, C. M., Yücel, M., A., Cooper, R. J., Dubb, J., Tsuzuki, D., Becerra, L., et al. (2015). Anatomical guidance for functional near-infrared spectroscopy: AtlasViewer tutorial. Neurophotonics 2:020801. doi: 10.1117/1.NPh.2.2.020801
Boas, D. A., Dale, A. M., and Franceschini, M. A. (2004). Diffuse optical imaging of brain activation: approaches to optimizing image sensitivity, resolution, and accuracy. Neuroimage 23, 275–288. doi: 10.1016/j.neuroimage.2004.07.011
Boas, D. A., Elwell, C. E., Ferrari, M., and Taga, G. (2014). Twenty years of functional near-infrared spectroscopy: introduction for the special issue. Neuroimage 85, 1–5. doi: 10.1016/j.neuroimage.2013.11.033
Delpy, D. T., Cope, M., van der Zee, P., Arridge, S., Wray, S., and Wyatt, J. (1988). Estimation of optical pathlength through tissue from direct time of flight measurement. Phys. Med. Biol. 33:1433. doi: 10.1088/0031-9155/33/12/008
Elwell, C. E., Springett, R., Hillman, E., and Delpy, D. T. (1999). Oscillations in cerebral haemodynamics. Implications for functional activation studies. Advances in. Experimental Exp. Medicine and. Biology Biol. vol. 471, 57–65. doi: 10.1007/978-1-4615-4717-4_8
Ferrari, M., and Quaresima, V. (2012). A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application. Neuroimage 63, 921–935. doi: 10.1016/j.neuroimage.2012.03.049
Gagnon, L., Cooper, R. J., Yücel, M. A., Perdue, K. L., Greve, D. N., and Boas, D. A. (2012). Short separation channel location impacts the performance of short channel regression in NIRS. Neuroimage 59, 2518–2528. doi: 10.1016/j.neuroimage.2011.08.095
Hong, K.-S., Khan, M. J., and Hong, M. J. (2018). Feature extraction and classification methods for hybrid fNIRS-EEG brain-computer interfaces. Front. Hum. Neurosci. 12:246. doi: 10.3389/fnhum.2018.00246
Huppert, T. J., Diamond, S. G., Franceschini, M. A., and Boas, D. A. (2009). HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl. Opt. 48, 280–298. doi: 10.1364/AO.48.00D280
Huppert, T. J., Hoge, R. D., Diamond, S. G., Franceschini, M. A., and Boas, D. A. (2006). A temporal comparison of BOLD, ASL, and NIRS hemodynamic responses to motor stimuli in adult humans. Neuroimage 29, 368–382. doi: 10.1016/j.neuroimage.2005.08.065
Scholkmann, F., Kleiser, S., Metz, A. J., Zimmermann, R., Mata Pavia, J., Wolf, U., et al. (2014). A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology. Neuroimage 85, 6–27. doi: 10.1016/j.neuroimage.2013.05.004
Shin, J., von Lühmann, A., Blankertz, B., Kim, D.-W. W., Jeong, J., Hwang, H.-J. J., et al. (2017). Open access dataset for EEG+ NIRS single-trial classification. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 1735–1745. doi: 10.1109/TNSRE.2016.2628057
Shin, J., Von Lühmann, A., Kim, D. W., Mehnert, J., Hwang, H. J., and Müller, K. R. (2018). Data descriptor: simultaneous acquisition of EEG and NIRS during cognitive tasks for an open access dataset. Sci. Data 5:180003. doi: 10.1038/sdata.2018.3
von Lühmann, A., Boukouvalas, Z., Müller, K. R., and Adal,i, T. (2019). A new blind source separation framework for signal analysis and artifact rejection in functional near-infrared spectroscopy. Neuroimage 200, 72–88. doi: 10.1016/j.neuroimage.2019.06.021
von Lühmann, A., Li, X., Müller, K. R., Boas, D. A., and Yücel, M. A. (2020a). Improved physiological noise regression in fNIRS: a multimodal extension of the General Linear Model using temporally embedded Canonical Correlation Analysis. Neuroimage 208:116472. doi: 10.1016/j.neuroimage.2019.116472
von Lühmann, A., Ortega-Martinez, A., Boas, D. A., and Yücel, M. A. (2020b). Using the general linear model to improve performance in fnirs single trial analysis and classification: a perspective. Front. Hum. Neurosci. 14, 1–1730. doi: 10.3389/fnhum.2020.00030
Yücel, M. A., Selb, J., Aasted, C. M., Lin, P.-Y., Borsook, D., Becerra, L., et al. (2016). Mayer waves reduce the accuracy of estimated hemodynamic response functions in functional near-infrared spectroscopy. Biomed. Opt. Express 7, 3078–3088. doi: 10.1364/BOE.7.003078
Yücel, M. A., Selb, J., Aasted, C. M., Petkov, M. P., Becerra, L., Borsook, D., et al. (2015). Short separation regression improves statistical significance and better localizes the hemodynamic response obtained by near-infrared spectroscopy for tasks with differing autonomic responses. Neurophotonics 2:035005. doi: 10.1117/1.NPh.2.3.035005
Keywords: multimodal, open access, synthetic HRF, resting, fNIRS
Citation: von Lühmann A, Li X, Gilmore N, Boas DA and Yücel MA (2020) Open Access Multimodal fNIRS Resting State Dataset With and Without Synthetic Hemodynamic Responses. Front. Neurosci. 14:579353. doi: 10.3389/fnins.2020.579353
Received: 02 July 2020; Accepted: 19 August 2020;
Published: 29 September 2020.
Edited by:Ian Daly, University of Essex, United Kingdom
Reviewed by:Vassiliy Tsytsarev, University of Maryland, College Park, United States
Wei-Peng Teo, Nanyang Technological University, Singapore
Copyright © 2020 von Lühmann, Li, Gilmore, Boas and Yücel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.