Impact Factor 3.209

The 1st most cited journal in Psychology

Methods ARTICLE

Front. Hum. Neurosci., 16 October 2013 | https://doi.org/10.3389/fnhum.2013.00676

Fusion of fNIRS and fMRI data: identifying when and where hemodynamic signals are changing in human brains

  • 1Bioimaging Core, Faculty of Health Sciences, University of Macau, Macau SAR, China
  • 2Department of Bio and Brain Engineering, KAIST, Daejeon, South Korea

In this study we implemented a new imaging method to fuse functional near infrared spectroscopy (fNIRS) measurements and functional magnetic resonance imaging (fMRI) data to reveal the spatiotemporal dynamics of the hemodynamic responses with high spatiotemporal resolution across the brain. We evaluated this method using multimodal data acquired from human right finger tapping tasks. And we found the proposed method is able to clearly identify from the linked components of fMRI and fNIRS where and when the hemodynamic signals are changing. In particular, the estimated associations between fNIRS and fMRI will be displayed as time varying spatial fMRI maps along with the fNIRS time courses. In addition, the joint components between fMRI and fNIRS are combined together to generate full spatiotemporal “snapshots” and movies, which provides an excellent way to examine the dynamic interplay between hemodynamic fNIRS and fMRI measurements.

Introduction

Functional near infrared spectroscopy (fNIRS) offers unsurpassed high temporal resolution and provides quantitative hemodynamic information for both oxyhemoglobin (HbO2) and deoxyhemoglobin (HbR), which plays an important role in the in vivo study of cognitive processing in the human brain (Jobsis, 1977; Cope and Delpy, 1988; Hoshi, 2003; Singh et al., 2005; Huppert et al., 2009; Ye et al., 2009; Yuan et al., 2010a,b; Brunno et al., 2011; Egetemeir et al., 2011; Gagnon et al., 2012; Yuan, in press). Similar to its fNIRS counterpart, functional magnetic resonance imaging (fMRI) is also a non-invasive imaging method that measures the hemodynamic responses to even-related neural activity with excellent spatial resolution and low temporal resolution. To take advantages of the complementary information from these two imaging modalities, two broad methods for fNIRS and fMRI/MRI integration have been developed for different clinical cases: (1) “spatial constraint,” in which spatial information from fMRI/MRI images are utilized to aid diffuse optical imaging based on fNIRS measurements (Carpenter et al., 2007; Ferradal et al., 2013); (2) “temporal correlation,” where fMRI bold signals are processed to generate the correlation with HbO2 and HbR concentration changes converted from fNIRS recordings (Cui et al., 2011; Tak et al., 2011; Gagnon et al., 2012). However, so far the linking between HbO2/HbR signals and fMRI spatial maps has not been extensively investigated, which represents one of the main challenges for fNIRS and fMRI fusion. Therefore, it is crucial for us to develop new imaging techniques to reveal the connections between these two measurements so that we are able to examine the dynamic interplay between space and time of hemodynamic responses with high spatiotemporal resolution.

Interestingly joint independent component analysis (jICA) method has been developed to compute the linked temporally independent event-related potential (ERP) components and spatial independent fMRI components, which enables inferences to be made using estimated associations between fMRI sources and ERP electromagnetic sources (Calhoun et al., 2006a,b; Sui et al., 2009). However, what has not been tried is a joint estimation of the temporal parts of fNIRS waveforms and the spatial maps of fMRI images. In this study, the jICA is extended to identify the spatiotemporal decompositions composed of fMRI spatial components indicating where the hemodynamic signals are changing and fNIRS components indicating when the hemodynamic signals are changing. The fNIRS-fMRI fusion method will involve calculating for given stimuli, the connection between the time-locked fNIRS waveforms and fMRI activation maps for all participants or different measurement sections from a single subject. It is anticipated that the results derived from jICA could be visualized by computing spatiotemporal “snapshot,” which provides an effective way to examine the dynamic interplay between fNIRS and fMRI hemodynamic sources.

Methods

In terms of jICA, the joint spatial and temporal independences of fNRIS and fMRI are assumed to satisfy the following generative model for the data (Calhoun et al., 2006a,b),

XfNIRS=ASfNIRS,XfMRI=ASfMRI(1)

in which XfNIRS is the group data from the chromophore concentration change of HbO2 or HbR for n subjects/n sections of a single subject and XfNIRS = [XfNIRS1, XfNIRS2, … XfNIRSn]T, XfMRI is the group data from fMRI for n subjects/n sections of a single subject and XfMRI = [XfMRI1, XfMRI2, … XfMRIn]T, SfMRI are the fMRI sources and SfMRI = [SfMRI1, SfMRI2, … SfMRIn]T, and SfNIRS are the fNIRS sources and SfNIRS = [SfNIRS1, SfNIRS2, … SfNIRSn]T. The shared lined mixing matrix A are written,

A=[a11a12a1na21a22a2n....an1an2ann](2)

It is note Equation 1 can be rewritten as a single matrix equation,

[X1fNIRSX1fMRIX2fNIRSX2fMRIXnfNIRSXnfMRI]=A[S1fNIRSS1fMRIS2fNIRSS2fMRISnfNIRSSnfMRI](3)

We employ the infomax ICA method for jICA of Equation 3, which utilizes a gradient ascent iteration algorithm to maximize the entropy of the output of a single layer neural network (Bell and Sejnowski, 1995). The resulting updated equation for the algorithm to calculate the shared unmixing matrix W (i.e., the inversion of A), the fused independent fNIRS sources ufNIRS and fMRI sources ufMRI is as follows,

Δw=η{I2yfNIRS(ufNIRS)T2yfMRI(ufMRI)T}W(4)

in which yfNIRS = g(ufNIRS), yfMRI = g(ufMRI), ufNIRS = WXfNIRS, ufMRI = WXfMRI, and g(x) = 1/(1+ex) is the non-linearity in the neural network. The initial value for W, W(0) is a matrix composed of random vectors.

The jICA method generally doesn't provide us the details on how the components between fNIRS and fMRI interact with one another. To achieve this, the spatiotemporal “snapshots” of the most significant components are generated in two ways (Calhoun et al., 2006a,b). First we need to calculate the linear combination of the fMRI components weighted by their joint fNIRS time courses for a specific point in time. If the n spatial (fMRI) and temporal (fNIRS) components are given by S = [s1 s2sn], and T = [t1 t2tn], where ti is a T × 1 vector containing the fNIRS time points and si is a V × 1 vector represents the V brain voxels, the fMRI movie is computed as MfMRI = |T| × ST. It is noted the absolute value is utilized here because the joint components are fused using a single parameter. And a change in the amplitude of the fMRI component is directly linked to the change in the fNIRS component by this parameter. Meanwhile the fNIRS movie is estimated by MfNIRS = T × |S|T, in which the time course for a given fMRI voxel is computed.

Results

Behavior Tasks and fNIRS-fMRI Recordings

The fNIRS tests are implemented with a block design for a right finger tapping task. The experiment is performed with a 24-chnnel fNIRS system (Oxymon MKIII, Artinis), which has 8 sources, 8 detectors, and 24 channels. In this system, two continuous wave lights at wavelengths 781 and 856 nm are emitted at each source fiber. In the case of block design for right finger tapping tasks, the onset time for the first trigger was at 42 s, then followed by a 21 s period of activation alternated with a 30 s period of rest. This was repeated 10 times for the subject. As such, the total recording time was 552 s. During the task period, subject was instructed to perform a finger flexion and extension action repeatedly. Data segmentation, which is also known as epoching in signal processing, is utilized to chop up the continuous fNIRS data into small time periods. The general way to do this is to extract segments surrounding the event codes from the experiments, e.g., from −35 s prior to the event onset until 35 s after the event code in this study. The original photon density datasets could be downloaded from (http://bisp.kaist.ac.kr/NIRS-SPM/Sample_data). The converted ΔHbO2 and ΔHbR measurements (the unitless differential path length factor DPF = 4; sampling rate: 9.75 Hz) are filtered, segmented and plotted in Figures 1A,B, respectively. The configurations of 24 channels located on the scalp are also provided in Figure 1C. We found channels 8–12 are very unique because their locations are close to the left motor cortex so that they are able to reveal the hemodynamic responses from both the right finger tapping stimuli and other events. As a result, signal averaging is implemented for the datasets from these five channels to generate the representative ΔHbO2/ΔHbR measurements with reduced noise. In particular, five-section (runs 2, 4, 6, 8, and 10) measurements are selected from the representative ten-run fNIRS recordings for further fusion analysis since they show the best signal noise ratio during the stimulus processing.

FIGURE 1
www.frontiersin.org

Figure 1. (A) The data review for ΔHbO2 measurements; (B) the data review for ΔHbR measurements; (C) Channel configurations along the scalp for right finger tapping tasks; (D) the section averaged fMRI image (left side), ΔHbO2 results (middle) and ΔHbR results(right side). The axes (bottom middle and bottom right) illustrate the time scale, in ms, whereas the scale for the middle and right columns records chromophore concentrations in μM.

For fMRI recordings, the fiber length was 10 m to connect the optodes in the MR scanner to the NIRS instrument in the MR control room. A 3.0 T MRI system (ISOL, Republic of Korea) was used to measure the BOLD response. During the blocked task paradigm, the echo planar imaging (EPI) sequence was used with TR/TE = 3000/35 ms, flipangle = 80°, 35 slices, and 4 mm slice thickness (Ye et al., 2009). The fMRI data could be downloaded partly from the same website for the same subject or requested from the KAIST lab. Based on the datasets provided, we generated five mean MRI images which are considered as five-section fMRI measurements (Calhoun et al., 2001; Ye et al., 2009).

Then we will combine the five-section averaged fNIRS recordings in temporal domain and five mean fMRI images in space to extract the joint independent components for hemodynamic fusion. It should be noted here the mean fMRI images are not generated from the raw fMRI images at each TR. As such, the fMRI image for each section may not accurately match its correlated fNIRS measurements in terms of recording times, which may have some influence on the final results. However, the influences should not be that significant since we use mean data from each section for fusion. The five-section averaged fMRI image and ΔHbO2/ΔHbR results are presented in Figure 1D for references.

Results of fNIRS-fMRI Fusion

The computed joint components are provided in Figures 2, 3, in which the spatial components and regions of brain activity from the fMRI maps are plotted on the right while the correlated temporal joint components along with the mean ΔHbO2/ΔHbR signals are given on the left. We can see from Figures 2, 3 that the time courses of temporal ΔHbO2/ΔHbR components correspond well to different peaks on the mean ΔHbO2/ΔHbR signals. Interestingly, when put together the fMRI maps and fNIRS signals, we found the five spatial joint components correlate very well with the five temporal joint components. For example, the first negative peak of ΔHbO2 in Figure 2A basically identifies the physiology noise such as visual activity and eye blinks. Also visible for this waveform of the second temporal joint component in Figure 2A is a late positive peak. During the positive peak, the hemodynamic activity is mainly found in the left motor cortex. Then the joint components from ΔHbO2/ΔHbR and fMRI are combined together to generate full spatiotemporal movies to more clearly display the dynamic interplay between the fNIRS and fMRI measurements. In particular, two movies are created to show the spatiotemporal dynamics of the fMRI and ΔHbO2 fusion, in which Movie 1 displays the hemodynamic responses of right finger tapping events (composite spatiotemporal joint components 2, 4, and 5) while Movie 2 basically reveals brain activity from physiology and body movement(composite joint components 1 and 3). The spatiotemporal hemodynamic changes derived from fMRI and ΔHbR fusion are also recorded by Movies 3, 4 for right finger tapping tasks (composite joint component 1 and 3) and physiology and body movement noise (composite joint component 2, 4, and 5), respectively.

FIGURE 2
www.frontiersin.org

Figure 2. jICA decomposition of ΔHbO2 and fMRI joint data for right finger tapping tasks: three components were found to be significantly correlated with finger tapping tasks [Independent components (ICs) 2, 4, and 5] while two were well-correlated with the physiology and body movement noise (ICs 1 and 3). Each of these components is shown in a separate panel in the figure: IC2 (A), IC4 (B), IC5 (C), IC1 (D) and IC3 (E). The fMRI maps are thresholded at |Z| > 1.5 for display purposes. The averaged event-related ΔHbO2 time course is shown in yellow (the same for all figures) and the ΔHbO2 component is plotted in cyan. Positive (orange) and negative (blue) Z-values are shown in the image. The axes (bottom) illustrate the time scale, in ms, whereas the scale (left) records chromophore concentrations in μM for the figures on the left column. The scale for the figures on the right column shows the bold signal intensity. LM, Left primary motor cortex; SMA, Supplementary motor area; P, Parietal cortex; V, Visual cortex; BP, Body movement/physiology noise.

FIGURE 3
www.frontiersin.org

Figure 3. jICA decomposition of ΔHbR and fMRI joint data for right finger tapping tasks: two components were found to be significantly correlated with right finger tapping task (ICs 1 and 3) while three were well-correlated with the physiology and body movement noise (ICs 2, 4, and 5). Each of these components is shown in a separate panel in the figure: IC1 (A), IC3 (B), IC2 (C), IC4 (D) and IC5 (E). The fMRI maps are thresholded at |Z| > 1.5 for display purposes. The averaged event-related HbR time course is shown in yellow (same for all figures) and the ΔHbR component is plotted in cyan. Positive (orange) and negative (blue) Z-values are shown in the image. The axes (bottom) illustrate the time scale, in ms, whereas the scale (left) records chromophore concentrations in μM for the figures on the left column. The scale for the figures on the right column shows the bold signal intensity. LM, Left primary motor cortex; SMA, Supplementary motor area; P, Parietal cortex; V, Visual cortex; BP, Body movement/physiology noise.

Figure 4 shows seven “snapshots” snipped from Movie 1 produced from fused ΔHbO2 and fMRI data for finger tapping tasks. On the left of the “snapshots” a linear sum of the fMRI maps weighted by their respective ΔHbO2 time courses are provided while the mean ΔHbO2 waveforms combined the estimated ΔHbO2 components are plotted on the right of each “snapshot.” Figure 5 plots five “snapshots” captured from fused the fMRI and ΔHbO2 data, which basically reveals the spatiotemporal responses of the physiology and body movement noise. Likewise, the “snapshots” from Movie 3, 4 produced from fused fMRI and ΔHbR components are given in Figures 6, 7 for finger tapping tasks and physiology and body movement noise, respectively.

FIGURE 4
www.frontiersin.org

Figure 4. ΔHbO2 components and fMRI “snapshots”: on the left of each window is shown a linear combination of the fMRI maps that are well-correlated with finger tapping tasks (composite ICs 2, 4, and 5), weighted by the ΔHbO2 part of the components at a specific point in time. On the right of each window is shown the estimated ΔHbO2 components that are correlated with right finger tapping tasks. The time courses for IC2 (in blue), IC4 (in green) and IC5 (in red) are also plotted on the right of each window. Such a display provides a dynamic way to visualize the brain activity at different time points: −20943 ms (A), −11217 ms (B), −8210 ms (C), −5559 ms (D), −2729 ms (E), 2044 ms (F) and 10884 ms (G).

FIGURE 5
www.frontiersin.org

Figure 5. ΔHbO2 components and fMRI “snapshots”: on the left of each window is shown a linear combination of the fMRI maps that are well-correlated with physiology and body movement noise (composite ICs 1 and 3), weighted by the ΔHbO2 part of the components at a specific point in time. On the right of each window is shown the estimated ΔHbO2 components that are correlated physiology and body movement noise. The time courses for IC1 (in blue) and IC3 (in green) are also plotted on the right of each window. Such a display provides a dynamic way to visualize how the noise affects the brain activity at different time points: −20584 ms (A), −12806 ms (B), 984 ms (C), 5757 ms (D) and 11062 ms (E).

FIGURE 6
www.frontiersin.org

Figure 6. ΔHbR components and fMRI “snapshots”: on the left of each window is shown a linear combination of the fMRI maps that are well-correlated with finger tapping tasks (composite ICs 1 and 3), weighted by the ΔHbR part of the components at a specific point in time. On the right of each window is shown the estimated ΔHbR components that are correlated with right finger tapping tasks. The time courses for IC1 (in blue) and IC3 (in green) are also plotted on the right of each window. Such a display provides a dynamic way to visualize the brain activity at different time points: −13460 ms (A), −10452 ms (B), −1082 ms (C), 1747 ms (D), 4045 ms (E), 9527 ms (F) and 13768 ms (G).

FIGURE 7
www.frontiersin.org

Figure 7. ΔHbR components and fMRI “snapshots”: on the left of each window is shown a linear combination of the fMRI maps that are well-correlated with physiology and body movement noise (composite ICs 2, 4, and 5), weighted by the ΔHbR part of the components at a specific point in time. On the right of each window is shown the estimated ΔHbR components that are correlated physiology and body movement noise. The time courses for IC2 (in blue), IC4 (in green) and IC5 (in red) are also plotted on the right of each window. Such a display provides a dynamic way to visualize how the noise affects the brain activity at different time points: −14518 ms (A), −9391 ms (B), −3734 ms (C), 7052 ms (D) and 13946 ms (E).

Discussion

We show, for the first time, a spatiotemporal reconstruction of the human brain's responses with high temporal and spatial resolution by fusing together the fNIRS signals and fMRI images. The jICA is a very key and unique technique for connecting hemodynamic fNIRS temporal signal and hemodynamic fMRI spatial information according to the joint constraints of temporal and spatial independences. Such direct correlation further provides a very valuable tool to localize non-invasively those high resolution structures that underlines temporally well-resolved fNIRS responses. Meanwhile, deep brain structures, which are not readily detectable with scalp fNIRS alone, can now be identified by the fusion technique when aided by the correlated spatial components from fMRI.

It is observed from Figure 2 that the first and third ΔHbO2-fMRI joint components basically identifies body movement and negative eye blink activities in pre-frontal lobe while the second, fourth and fifth ΔHbO2-fMRI joint components are fused with hemodynamic activity in left primary motor cortex (PMC), supplementary motor area (SMA), and motor association cortex such as parietal cortex. In particular, the fifth joint component basically identifies the neural activity in the SMA and parietal cortex. These observations are further validated by Movies 1, 2, which show the dynamic interplay between space and time of ΔHbO2-fMRI hemodynamic responses. It is noted Movie 1 displays the spatiotemporal changes of neural activation patterns for finger tapping tasks. Further, we can see from Figure 3 that the second, fourth, and fifth ΔHbR-fMRI joint components basically reveal body movement and other physiology noise while the first and third ΔHbR-fMRI joint components identify the patterns of brain activity in the left motor cortex and motor association cortex. In particular, the first ΔHbR-fMRI joint component reveals brain activity in SMA and parietal cortex. These spatiotemporal findings from fused ΔHbR-fMRI data are further processed to generate Movies 3, 4, in which we can see the hemodynamic changes for a finger tapping task with high temporal and spatial resolution, and we can also see clearly how physiology and body movement noise corresponds to the neural activity. Importantly, we can observe from Figures 2, 3 as well as Movies 1, 3 that strong brain activity mainly occurs in the left PMC, which validates stimuli in the right finger tapping task will yield the activations in the left PMC.

Alternatively, we can see from Figure 4 the estimated ΔHbO2 time courses (i.e., linear combinations of the significant ΔHbO2 components) at specific positions in the cerebral cortex. The first “snapshot” in Figure 4A is associated with physiology and body movement noise while the second to fourth “snapshots” show the ΔHbO2 concentration change during onset and stimulus processing of right finger tapping, which are validated by the spatial maps of fMRI image on the left. We can further observe from Figure 4 that the fifth “snapshot” captures the maximized spatiotemporal hemodynamic responses while the sixth and seventh ones show the decreased trends of neural activity. So if we examine the different “snapshots” in Figure 4, the spatiotemporal dynamics of the right finger tapping are revealed. In addition, the negative and positive peaks of the estimated ΔHbO2 time courses in Figure 5 directly correspond to the physiology and body movement noise though partly they capture the weak and negative hemodynamic responses of right finger tapping in the case of strong body movement.

The ΔHbR-fMRI “snapshots” in Figure 6 identify SMA and parietal cortex with early peak hemodynamic responses. Later responses occurring in left PMC show peaks of the ΔHbR responses during stimulus processing. In particular, “snapshots” 2–7 display the hemodynamic changes in detail with high spatiotemporal resolution for finger tapping task. Figure 7 provides the portions of estimated ΔHbR time courses at specific positions of fMRI images, in which we observe the peaks basically corresponding to the activities of physiology noise, eye blinks, and body movement.

In summary, we have used a combination of hemodynamic fNIRS and fMRI data to visualize the neural activation patterns with high spatiotemporal resolution. The fusion techniques have shown the potential by using the joint hemodynamic data to identify unique neural information that cannot be revealed in either technique alone. In particular, we implemented a joint decomposition of fNIRS and fMRI data, which was linked or fused by a common mixing parameter. The present method does not involve the solution of inverse problem for diffuse optical imaging or involve the use of the threshold for the fMRI data. However, we do need to decompose the combined data into specific components by using jICA, which are composed of fNIRS and fMRI portions. Separating the data into joint components will provide us a useful way to examine component specific differences among different patient groups or different stimulus tasks.

We have to point out the present investigation has significant limitations though we are encouraged that the results we have found seem meaningful and valid. For example, we only validate data from a single subject even without accurate measurements from fMRI. In future we would like to examine our method to incorporate different subject groups and complex neural stimuli as well as multiple time points acquired from multiple channels of fNIRS systems.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This research was supported by SRG Grant from University of Macau in Macau. In particular, we thank Prof. Jong Chui Ye at KAIST for providing us the fNIRS/fMRI datasets for our initial methodology study.

Supplementary Material

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnhum.2013.00676/abstract

Movie 1. ΔHbO2 and fMRI fusion movie for finger tapping tasks.

Movie 2. ΔHbO2 and fMRI fusion movie for physiology and body movement noise.

Movie 3. ΔHbR and fMRI fusion movie for finger tapping tasks.

Movie 4. ΔHbR and fMRI fusion movie for physiology and body movement noise.

References

Bell, A. J., and Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159. doi: 10.1162/neco.1995.7.6.1129

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Brunno, S., Luciano, G., Konstantinos, P., Pietro, S., Massimiliano, M., and Simone, C. (2011). An exploratory fNRIS study with immersive virtual reality: a new method for technical implementation. Front. Hum. Neurosci. 5:176. doi: 10.3389/fnhum.2011.00176

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Calhoun, V., Adali, T., Pearlson, G., and Kiehl, K. (2006a). Neuronal chronometry of target detection: fusion of hemodynamic and event-related potential data. Neuroimage 30, 544–553. doi: 10.1016/j.neuroimage.2005.08.060

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Calhoun, V., Adali, T., and Liu, J. (2006b). “A feature-based approach to combine functional MRI, structural MRI and EEG brain imaging data,” in Proceedings of the 28th IEEE EMBS Annual International Conference (New York, NY), 3672–3675.

Pubmed Abstract | Pubmed Full Text

Calhoun, V. D., Adali, T., Pearlson, G. D., and Pekar, J. J. (2001). A Method for making group inferences from functional MRI data using independent component analysis. Hum. Brain Mapp. 14, 140–151. doi: 10.1002/hbm.1048

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Carpenter, C., Pogue, B., Jiang, S., Dehghani, H., Wang, X., and Paulsen, K. D. (2007). Image-guided optical spectroscopy provides molecular-specific information in vivo: MRI-guided spectroscopy of breast cancer hemoglobin, water, and scatter size. Opt. Lett. 32, 933–935. doi: 10.1364/OL.32.000933

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Cope, M., and Delpy, D. T. (1988). System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination. Med. Biol. Eng. Comput. 26, 289–294. doi: 10.1007/BF02447083

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Cui, X., Bray, S., Bryant, D., Glover, G. H., and Reiss, A. L. (2011). A quantitative comparison of NIRS and fMRI across multiple cognitive tasks. Neuroimage 54, 2808–2821. doi: 10.1016/j.neuroimage.2010.10.069

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Egetemeir, J., Stenneken, P., Koehler, S., Fallgatter, A. J., and Hermann, M. J. (2011). Exploring the neural basis of real-life joint action: measuring brain activation during joint table setting with functional near-infrared spectroscopy. Front. Hum. Neurosci. 5:95. doi: 10.3389/fnhum.2011.00095

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ferradal, S. L., Eggebrecht, A. T., Hassanpour, M., Snyder, A. Z., and Culver, J. P. (2013). Atlas-based head modeling and spatial normalization for high-density diffuse optical tomography: in vivo validation against fMRI. Neuroimage. doi: 10.1016/j.neuroimage.2013.03.069. [Epub ahead of print].

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gagnon, L., Yucel, M., Dehaes, M., Cooper, R., Perdue, K., Selb, J., et al. (2012). Quantification of the cortical contributions to the NIRS signal over the motor cortex using concurrent NIRS-fMRI measurements. Neuroimage 59, 3933–3940. doi: 10.1016/j.neuroimage.2011.10.054

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hoshi, Y. (2003). Functional near-infrared optical imaging: utility and limitations in human brain mapping. Psychophysiology 40, 511–520. doi: 10.1111/1469-8986.00053

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Huppert, T., Diamond, S., Franceschini, M., and Boas, D. (2009). Homer: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl. Opt. 48, 280–298. doi: 10.1364/AO.48.00D280

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jobsis, F. F. (1977). Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. Science 198, 1264–1267. doi: 10.1126/science.929199

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Singh, A. K., Okamoto, M., Dan, H., and Dan I Jurcak, V. (2005). Spatial registration of multichannel multi-subject fNIRS data to MNI space without MRI. Neuroimage 27, 842–851. doi: 10.1016/j.neuroimage.2005.05.019

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sui, J., Adali, T., Pearlson, G., Clark, V., and Calhoun, V. (2009). A method for accurate group difference detection by constraining the mixing coefficients in an ICA framework. Hum. Brain Mapp. 30, 2953–2970. doi: 10.1002/hbm.20721

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tak, S., Yoon, S. J., Jang, J., Yoo, K., Jeong, Y., and Ye, J. C. (2011). Quantitative analysis of hemodynamic and metabolic change in subcortical vascular dementia using simultaneous near-infrared spectroscopy and fMRI measurements. Neuroimage 55, 176–184. doi: 10.1016/j.neuroimage.2010.11.046

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ye, J. C., Tak, S., Jang, K. E., and Jung, J. (2009). NIRS-SPM: statistical parametric mapping for near-infrared spectroscopy. Neuroimage 44, 428–447. doi: 10.1016/j.neuroimage.2008.08.036

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yuan, Z. (in press). Spatiotemporal and time-frequency analysis of fNIRS brain signals using independent component analysis. J. Biomed. Opt.

Yuan, Z., Zhang, Q., Sobel, E., and Jiang, H. (2010a). Image-guided optical spectroscopy in diagnosis of osteoarthritis: a clinical study. Biomed. Opt. Express 1, 74–86. doi: 10.1364/BOE.1.000074

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yuan, Z., Zhang, Q., Sobel, E., and Jiang, H. (2010b). High resolution x-ray guided 3D diffuse optical tomography of joint tissues in hand osteoarthritis: morphological and functional assessments. Med. Phys. 37, 4343–4354. doi: 10.1118/1.3467755

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Keywords: fMRI, fNIRS, independent component analysis, multimodal imaging methods, cognitive neurosciences

Citation: Yuan Z and Ye J (2013) Fusion of fNIRS and fMRI data: identifying when and where hemodynamic signals are changing in human brains. Front. Hum. Neurosci. 7:676. doi: 10.3389/fnhum.2013.00676

Received: 02 August 2013; Paper pending published: 31 August 2013;
Accepted: 26 September 2013; Published online: 16 October 2013.

Edited by:

Nobuo Masataka, Kyoto University, Japan

Reviewed by:

Liangzhong Xiang, Stanford University, USA
Yao Sun, Wake Forest Baptist Medical Center, USA

Copyright © 2013 Yuan and Ye. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Zhen Yuan, Bioimaging Core, Faculty of Health Sciences, University of Macau, Av. Padre Tomás Pereira, Taipa, Macau SAR, China e-mail: zhenyuan@umac.mo