The human brain processes a constant stream of sensory information, necessitating advanced filtering and cognitive processing for tasks like recognizing dynamic facial expressions (Wang and Yuan, ). Advances in noninvasive neuroimaging methods, such as electroencephalogram (EEG) (Wang et al., ), magnetic resonance imaging (MRI) (Lu et al., ), and near-infrared spectroscopy (NIRS) (Wang et al., ) have deepened our understanding of how the human brain works over the last decades (Finn et al., ). However, the reliability and reproducibility of the results have been recently questioned and exposed in the spotlight (Cognitive neuroscience at the crossroads, ), highlighting the need to address and clarify the integrity of the research field (Revisiting doubt in neuroimaging research, ).
Hence, the main goal of this Research Topic is to address and explore which factors influence the reliability and reproducibility of brain imaging results and provide practical perspectives and insights to enhance them. We have collected four articles for the Research Topic, whose contributions were discussed as follows.
The first article explored the contributing factors to the brain age index. Brain age is the age gap between chronological age and predicated age from brain imaging data (Cole and Franke, ). There is evidence showing that brain age is associated with common brain disorders (Kaufmann et al., ), and it has been considered as a potential biomarker for various psychiatric disorders (Cole and Franke, ). However, the contribution to brain age from non-brain imaging factors besides neuroimaging data has not been explored. Korbmacher et al. have explored this issue and proposed a Bio-psycho-social model for predicting brain age. The study utilized the UK Biobank data (Miller et al., ) and compared different model configurations. They discovered that the Bio-psycho-social model can partially explain the brain age variance which is comparable to the contribution of the diffusion MRI approach. Additionally, they found large variability in gender and ethnicity differences in brain age. They then suggest that in future studies, the effects of ethnicity, cognitive factors, gender, as well as health and lifestyle factors should be controlled to observe the bio-psycho-social factor impacts on brain age.
The second article touched upon the reliability of the resting-state (rs-)fMRI. Put simply, rs-fMRI captures systematic, non-random variations in brain activation in the absence of a specific task. These activations are indirectly reflected in regional fluctuations in the level of oxygenation of the blood (known as BOLD) (Ogawa et al., ), which has been widely used to explore the functional brain activation (Lu et al., ) and brain networks (Damoiseaux et al., ; Bullmore and Sporns, ; van den Heuvel and Pol, ). There is evidence suggesting that the time-of-day could be associated with reduced global signal fluctuation and functional connectivity (Orban et al., ) alongside the time-of-year effect (Wang et al., ; Zhang et al., ). However, it's still unclear how the time-of-day effect, along with other factors such as age and gender, could sway the results of longitudinal neuroimaging data. Vaisvilaite et al. addressed this issue by leveraging the BETULA dataset, a longitudinal study on aging (Nilsson et al., ), and employing the cross-spectral dynamic causal modeling (DCM) (Friston et al., ), which puts the temporal fluctuations of resting-state BOLD signal into the focus. Specifically, they explored eight DCM models and found that the time-of-day effect together with gender significantly influences the parameters defining the BOLD signal. Therefore, they suggest that when collecting longitudinal neuroimaging datasets, time of day should be considered a covariate in addition to age and gender.
The third article discussed the reliability of the magnetic resonance spectroscopy (MRS) measurement in multi-site/vendor studies. MRS allows for non-invasive investigation of neurometabolic profiles in health and disease, yielding metabolite estimates from a localized volume (Oz et al., ; Wilson et al., ). Widely used in clinical settings (Oz et al., ), MRS has recently been applied to identify IDH-mutated gliomas (Branzoli and Marjanska, ) and explore the association between behavior and MRS-derived neurotransmitters (Li et al., ). It is suggested that the MRS-derived metabolite estimates are reliable within the participants (Wang et al., ). In large-scale clinical studies, a multi-site/vendor design is often favored to achieve the necessary statistical power. However, the impact of different sites and scanners on metabolite estimates is frequently overlooked (Považan et al., ). La et al. investigated how to account for these differences in a pediatric concussion dataset comprising 545 short-TE MRS measurements, collected across six MRI scanners with two MRI vendors at five scanning sites. They used four general linear models and three linear mixed-effects models to investigate site- and vendor effects on the quantitative estimates of five major neuro-metabolites. They found that different analysis strategies of controlling site, vendor, and scanner in MRS data generated different results, from which they advocate that the ComBat harmonization for clinical MRS data (Bell et al., ) should be utilized to remove the site and vendor effects.
The fourth article is even more clinically related, exploring the repeatability of 2D and 4D flow MRI measurement of intracranial blood flow and pulsatility. Arterial stiffening impacts blood vessel compliance and regulation and is considered a contributing factor to various neurological conditions (Poels et al., ). To noninvasively examine arterial stiffening specifically the blood flow velocity, 2D phase-contrast MRI is dominantly used (McCauley et al., ). However, it can only measure a few vessels per scan. On the contrary, the 4D flow MRI can evaluate multiple vessels simultaneously, making it an attractive alternative (Terada et al., ). Yet, the repeatability, reliability, and consistency of 4D flow compared to 2D across intracranial vessels remain uncertain. In the study, Morgan et al. aimed to answer this scientific inquiry. They used various statistical methods including intra-rater reliability, inter-method conformity, and test-retest repeatability to assess the repeatability of the pulsatility index and mean flow among patients with small vessel disease and healthy controls. They found that the 4D flow MRI possesses repeatable and reliable PI measurements and the mean blood flow measurement. Therefore, they have suggested the 4D flow MRI can be confidently used to assess the pulsatility across the major cerebral vessels.
We appreciate the contributions of the authors of these four articles, which hopefully could stimulate and elicit further discussions about the topic: the variability and reliability of brain imaging. Indeed, we have witnessed collective initiatives to enhance the robustness of the neuroimaging results (Korbmacher et al., ), such as best practices in the data analysis (Nichols et al., ; Boudewyn et al., ), guidelines for reporting or publishing (Poldrack et al., ; Keil et al., ; Lin et al., ; Yücel et al., ), and expert consensus (Choi and Kreis, ). We firmly believe by acknowledging the problems we have in neuroimaging studies and adopting the best practices, the neuroimaging results will be more robust and reliable.
Statements
Author contributions
M-YW: Writing—original draft, Writing—review & editing, Conceptualization. HZ: Writing—review & editing, Conceptualization. MY: Writing—review & editing, Conceptualization. KS: Writing—review & editing, Conceptualization.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
BellT. K.GodfreyK. J.WareA. L.YeatesK. O.HarrisA. D. (2022). Harmonization of multi-site MRS data with ComBat. Neuroimage257:119330. 10.1016/j.neuroimage.2022.119330
2
BoudewynM. A.EricksonM. A.WinslerK.RaglandJ. D.YonelinasA.FrankM.et al. (2023). Managing EEG studies: how to prepare and what to do once data collection has begun. Psychophysiology60:e14365. 10.1111/psyp.14365
3
BranzoliF.MarjanskaM. (2020). Magnetic resonance spectroscopy of isocitrate dehydrogenase mutated gliomas: current knowledge on the neurochemical profile. Curr. Opin. Neurol.33, 413–421. 10.1097/WCO.0000000000000833
4
BullmoreE. T.SpornsO. (2009). Complex brain networks: graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci.10, 186–198. 10.1038/nrn2575
5
ChoiI. Y.KreisR. (2021). Advanced methodology for in vivo magnetic resonance spectroscopy. NMR Biomed.34:e4504. 10.1002/nbm.4504
6
Cognitive neuroscience at the crossroads. (2022). Nature608:647. 10.1038/d41586-022-02283-w
7
ColeJ. H.FrankeK. (2017). Predicting age using neuroimaging: innovative brain ageing biomarkers. Trends Neurosci.40, 681–690. 10.1016/j.tins.2017.10.001
8
DamoiseauxJ. S.RomboutsS. A.BarkhofF.ScheltensP.StamC. J.SmithS. M.et al. (2006). Consistent resting-state networks across healthy subjects. Proc. Natl Acad. Sci. USA.103, 13848–13853. 10.1073/pnas.0601417103
9
FinnE. S.PoldrackR. A.ShineJ. M. (2023). Functional neuroimaging as a catalyst for integrated neuroscience. Nature623, 263–273. 10.1038/s41586-023-06670-9
10
FristonK. J.HarrisonL.PennyW. (2003). Dynamic causal modelling. Neuroimage19, 1273–1302. 10.1016/S1053-8119(03)00202-7
11
KaufmannT.van der MeerD.DoanN. T.SchwarzE.LundM. J.AgartzI.et al. (2019). Common brain disorders are associated with heritable patterns of apparent aging of the brain. Nat. Neurosci.22, 1617–1623. 10.1038/s41593-019-0471-7
12
KeilA.DebenerS.GrattonG.JunghöferM.KappenmanE. S.LuckS. J.et al. (2014). Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology51, 1–21. 10.1111/psyp.12147
13
KorbmacherM.AzevedoF.PenningtonC. R.HartmannH.PownallM.SchmidtK.et al. (2023). The replication crisis has led to positive structural, procedural, and community changes. Commun. Psychol.1:3. 10.1038/s44271-023-00003-2
14
LiH.HeiseK.-F.ChalaviS.PutsN. A. J.EddenR. A. E.SwinnenS. P.et al. (2022). The role of MRS-assessed GABA in human behavioral performance. Prog. Neurobiol.212:102247. 10.1016/j.pneurobio.2022.102247
15
LinA.AndronesiO.BognerW.ChoiI.-Y.CoelloE.CudalbuC.et al. (2021). Minimum reporting standards for in vivo magnetic resonance spectroscopy (MRSinMRS): experts' consensus recommendations. NMR Biomed.34:e4484. 10.1002/nbm.4484
16
LuF.LiuP.ChenH.WangM.XuS.YuanZ.et al. (2020). More than just statics: abnormal dynamic amplitude of low-frequency fluctuation in adolescent patients with pure conduct disorder. J. Psychiatr. Res.131, 60–68. 10.1016/j.jpsychires.2020.08.027
17
LuF.WangM.XuS.ChenH.YuanZ.LuoL.et al. (2021). Decreased interhemispheric resting-state functional connectivity in male adolescents with conduct disorder. Brain Imaging Behav.15, 1201–1210. 10.1007/s11682-020-00320-8
18
McCauleyT. R.PeñaC. S.HollandC. K.PriceT. B.GoreJ. C. (1995). Validation of volume flow measurements with cine phase-contrast MR imaging for peripheral arterial waveforms. J. Magn. Reson. Imaging5, 663–668. 10.1002/jmri.1880050608
19
MillerK. L.Alfaro-AlmagroF.BangerterN. K.ThomasD. L.YacoubE.XuJ.et al. (2016). Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nat. Neurosci.19, 1523–1536. 10.1038/nn.4393
20
NicholsT. E.DasS.EickhoffS. B.EvansA. C.GlatardT.HankeM.et al. (2017). Best practices in data analysis and sharing in neuroimaging using MRI. Nat. Neurosci.20, 299–303. 10.1038/nn.4500
21
NilssonL. G.AdolfssonR.BäckmanL.de FriasC.MolanderB.NybergL.et al. (1997). The Betula prospective cohort study: memory, health and aging. Aging Neuropsychol. Cogn.4, 1–32. 10.1080/13825589708256633
22
OgawaS.TankD. W.MenonR.EllermannJ. M.KimS. G.MerkleH.et al. (1992). Intrinsic signal changes accompanying sensory stimulation - functional brain mapping with magnetic-resonance-imaging. Proc. Natl. Acad. Sci. USA.89, 5951–5955. 10.1073/pnas.89.13.5951
23
OrbanC.KongR.LiJ.CheeM. W. L.YeoB. T. T. (2020). Time of day is associated with paradoxical reductions in global signal fluctuation and functional connectivity. Plos Biol.18:e3000602. 10.1371/journal.pbio.3000602
24
OzG.AlgerJ. R.BarkerP. B.BarthaR.BizziA.BoeschC.et al. (2014). Clinical proton MR spectroscopy in central nervous system disorders. Radiology270, 658–679. 10.1148/radiol.13130531
25
PoelsM. M.ZaccaiK.VerwoertG. C.VernooijM. W.HofmanA.van der LugtA.et al. (2012). Arterial stiffness and cerebral small vessel disease the rotterdam scan study. Stroke43, 2637–2642. 10.1161/STROKEAHA.112.676528
26
PoldrackR. A.FletcherP. C.HensonR. N.WorsleyK. J.BrettM.NicholsT. E.et al. (2008). Guidelines for reporting an fMRI study. Neuroimage40, 409–414. 10.1016/j.neuroimage.2007.11.048
27
PovažanM.MikkelsenM.BerringtonA.BhattacharyyaP. K.BrixM. K.BuurP. F.et al. (2020). Comparison of multivendor single-voxel MR spectroscopy data acquired in healthy brain at 26 sites. Radiology295, 171–180. 10.1148/radiol.2020191037
28
Revisiting doubt in neuroimaging research (2022). Nat. Neurosci.25, 833–834. 10.1038/s41593-022-01125-2
29
TeradaM.TakeharaY.IsodaH.WakayamaT.NozakiA. (2022). Technical background for 4D flow MR imaging. Magn. Reson. Med. Sci.21, 267–277. 10.2463/mrms.rev.2021-0104
30
van den HeuvelM. P.PolH. E. H. (2010). Exploring the brain network: a review on resting-state fMRI functional connectivity. Eur. Neuropsychopharmacol.20, 519–534. 10.1016/j.euroneuro.2010.03.008
31
WangM.-Y.YuanA.ZhangJ.XiangY.YuanZ. (2020b). Functional near-infrared spectroscopy can detect low-frequency hemodynamic oscillations in the prefrontal cortex during steady-state visual evoked potential-inducing periodic facial expression stimuli presentation. Vis. Comput. Ind. Biomed. Art3:28. 10.1186/s42492-020-00065-7
32
WangM.-Y.ZhangZ.-M.MiaoX.XiaohongX.YuanZ. (2020a). Electrophysiological evidence of attentional avoidance in sub-clinical individuals with obsessive-compulsive symptoms. IEEE Access8, 91020–91027. 10.1109/ACCESS.2020.2994452
33
WangM. Y.KorbmacherM.EikelandR.CravenA. R.SpechtK. (2024). The intra-individual reliability of H-MRS measurement in the anterior cingulate cortex across 1 year. Hum. Brain Mapp.45:e26531. 10.1002/hbm.26531
34
WangM. Y.KorbmacherM.EikelandR.SpechtK. (2023). “The long night effect on the brain functional organization, in The Organization for Human Brain Mapping 2023 Annual Conference (Montreal).
35
WangM. Y.YuanZ. (2021). EEG decoding of dynamic facial expressions of emotion: evidence from SSVEP and causal cortical network dynamics. Neuroscience459, 50–58. 10.1016/j.neuroscience.2021.01.040
36
WilsonM.AndronesiO.BarkerP. B.BarthaR.BizziA.BolanP. J.et al. (2019). Methodological consensus on clinical proton MRS of the brain: review and recommendations. Magn. Reson. Med.82, 527–550. 10.1002/mrm.27742
37
YücelM. A.LühmannA. V.ScholkmannF.GervainJ.DanI.AyazH.et al. (2021). Best practices for fNIRS publications. Neurophotonics8:012101. 10.1117/1.NPh.8.1.012101
38
ZhangR.Shokri-KojoriE.VolkowN. D. (2023). Seasonal effect-an overlooked factor in neuroimaging research. Transl. Psychiatry13:238. 10.1038/s41398-023-02530-2
Summary
Keywords
neuroimaging, reliability, fMRI, MRS, variability, replication crisis
Citation
Wang M-Y, Zöllner HJ, Yücel MA and Specht K (2024) Editorial: Variability and reproducibility of brain imaging. Front. Psychol. 15:1386948. doi: 10.3389/fpsyg.2024.1386948
Received
16 February 2024
Accepted
04 March 2024
Published
13 March 2024
Volume
15 - 2024
Edited and reviewed by
Sara Palermo, University of Turin, Italy
Updates
Copyright
© 2024 Wang, Zöllner, Yücel and Specht.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Meng-Yun Wang mengyun.wang@uib.no
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.