Quantitative MRI Harmonization to Maximize Clinical Impact: The RIN–Neuroimaging Network

Neuroimaging studies often lack reproducibility, one of the cardinal features of the scientific method. Multisite collaboration initiatives increase sample size and limit methodological flexibility, therefore providing the foundation for increased statistical power and generalizable results. However, multisite collaborative initiatives are inherently limited by hardware, software, and pulse and sequence design heterogeneities of both clinical and preclinical MRI scanners and the lack of benchmark for acquisition protocols, data analysis, and data sharing. We present the overarching vision that yielded to the constitution of RIN-Neuroimaging Network, a national consortium dedicated to identifying disease and subject-specific in-vivo neuroimaging biomarkers of diverse neurological and neuropsychiatric conditions. This ambitious goal needs efforts toward increasing the diagnostic and prognostic power of advanced MRI data. To this aim, 23 Italian Scientific Institutes of Hospitalization and Care (IRCCS), with technological and clinical specialization in the neurological and neuroimaging field, have gathered together. Each IRCCS is equipped with high- or ultra-high field MRI scanners (i.e., ≥3T) for clinical or preclinical research or has established expertise in MRI data analysis and infrastructure. The actions of this Network were defined across several work packages (WP). A clinical work package (WP1) defined the guidelines for a minimum standard clinical qualitative MRI assessment for the main neurological diseases. Two neuroimaging technical work packages (WP2 and WP3, for clinical and preclinical scanners) established Standard Operative Procedures for quality controls on phantoms as well as advanced harmonized quantitative MRI protocols for studying the brain of healthy human participants and wild type mice. Under FAIR principles, a web-based e-infrastructure to store and share data across sites was also implemented (WP4). Finally, the RIN translated all these efforts into a large-scale multimodal data collection in patients and animal models with dementia (i.e., case study). The RIN-Neuroimaging Network can maximize the impact of public investments in research and clinical practice acquiring data across institutes and pathologies with high-quality and highly-consistent acquisition protocols, optimizing the analysis pipeline and data sharing procedures.

Introduction

The identification of early and accurate in vivo non-invasive biological markers-“a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention” (1)-in brain tissue is a crucial endpoint in neuroimaging research (2–6).

Multisite collaboration initiatives allow accruing large-scale quantitative magnetic resonance imaging (qMRI) data paving the way to overcome the current replication crisis in neuroimaging science (7–10) and to data-driven analysis methods, through machine/deep-learning techniques (11, 12), fundamental tools in the identification of reliable neuroimaging biomarkers (13). These initiatives are also important opportunities for sharing technical and scientific knowledge, new ideas, and available resources. Initiatives such as the Alzheimer's Disease Neuroimaging Initiative (ADNI, http://adni.loni.usc.edu/), ESR/EIBALL (https://www.myesr.org/research/european-imaging-biomarkers-alliance-eiball), Quantitative Imaging Biomarkers Alliance (QIBA) (https://www.rsna.org/research/quantitative-imaging-biomarkers-alliance) (14), or Biomedical Informatics Research Network (BIRN) (15, 16) are successful examples of this intention.

Defining qMRI sequences, and standardized procedures for their quality control and for data analysis and sharing within a network of research institutes providing ongoing support according to their specific expertise is a much-needed “conditio sine qua non” if the ultimate aim is to promote the translation of such methods into the clinical context of research hospitals. Within such a network, with expertise that bridges across disciplines (radiology, neurology, physics, computer science, statistics), protocols and standard operating procedures (SOPs) can meet the standards for benchmarking against an MRI technology in constant evolution, with scanners that present a vast heterogeneity in terms of their characteristics (e.g., manufacturer, gradient system, transmitter/receiver coils, sequences, software version).

A recent survey on the current state of neuroimaging biomarker harmonization unmet needs identified different high-level barriers. Amongst them, the lack of guidelines or regulations for the harmonization of data acquisition and analyses, the cost underestimation for infrastructures, the need for qualified and experienced personnel, remain the most important challenging issues (17).

Calls to action to overcome those barriers should include the harmonization of quality controls (QC) procedures to evaluate the performance of the scanners concerning reference values (6, 18, 19) and the harmonization of multivendor state-of-the-art acquisition protocols to guarantee the repeatability and the reproducibility of qMRI measures inter-/intra-scanner (20–24), as well as the setting up of IT infrastructures suitable for data exchange and sharing (25, 26). A network of highly trained personnel, well-integrated with the clinical teams, supporting the implementation across sites, is what can make the real difference in terms of successful clinical impact.

If these issues are well identified and talked about in clinical neuroimaging research settings, they are hardly mentioned in preclinical imaging research (27). In the last few years, the reproducibility of results in preclinical research has shown itself as an important basis for successful clinical trials (27). Unfortunately, only a few multisite studies have been carried on so far. Preclinical multisite harmonization has been limited to a few centers (i.e., 2–3 sites) with the same experimental setup (28, 29) or to the application of shared pipeline analysis to data obtained with different experimental setups (30). Therefore, a multisite coordination that brings together both clinical and technical expertise is a fundamental step for structured large-scale data collection. Moreover, it would be appropriate for this coordination to offer ongoing support for internal and external users. This is a priority not only within clinical research but also for preclinical research and translational studies (31, 32).

The Neuroscience and Neurorehabilitation Network (RIN): RIN–Neuroimaging Network

Under the increasing pressure of the burden and cost of neurological and neuropsychiatric diseases (33, 34), the Italian Ministry of Health in 2017 founded the Neuroscience and Neurorehabilitation Network (RIN) the Italian largest research network in the neuroscience field. RIN drives to collaboration Scientific Institutes of Hospitalization and Care (IRCCSs). One of the fundamental branches of RIN is the RIN–Neuroimaging Network (https://www.reteneuroscienze.it/en/progetti/neuroimaging/), which main ambitious goal is to identify the disease and subject-specific in-vivo neuroimaging biomarkers of diverse neurological and neuropsychiatric conditions. This ambitious goal needs efforts toward increasing the diagnostic and prognostic power of advanced MRI data, requiring, as the first essential step, to specify guidelines and SOPs, to be readily adopted by IRCCS, for data acquisitions, processing, and sharing of disease-specific MRI protocols.

Main Goals

To achieve the main purpose of the RIN–Neuroimaging Network, four initial operational goals have been identified:

Goal 1: definition of shared MRI protocols for the main neurological diseases;

Goal 2: quality controls (QC) on ad hoc phantoms;

Goal 3: harmonization of advanced MRI protocol in clinical and preclinical research;

Goal 4: setting up communication infrastructure and data management.

Methods and Analysis

Design

The initial core of the RIN–Neuroimaging Network, constituted by neuroradiologists, physicists, and engineers of participating IRCCSs, was formalized according to the following criteria:

• To be active in neurology research;

• To have a high field scanner (3T) or ultra-high-field scanner (7T) for clinical or preclinical research or well-known expertise in MRI data analysis and infrastructure management.

All IRCCSs were contacted. Before their formal involvement in RIN–Neuroimaging Network, the technological and clinical specialization of each IRCCS was verified through a detailed survey (Figure 1, Table 1).

FIGURE 1

Figure 1. Listing and geographical distribution of the 23 Italian sites of the RIN–Neuroimaging Network.

TABLE 1

Table 1. Scanner and equipment for each site of RIN–Neuroimaging Network.

The operational goals of the RIN–Neuroimaging Network were pursued through the following work packages (WP): WP1 clinical protocols, WP2 clinical scanners, WP3 preclinical scanner, WP4 infrastructure. Each IRCCS organized its participating staff into working groups (see Figure 2). Every working group reported their activity and WP progress every 6 months during a consensus meeting.

FIGURE 2

Figure 2. Project flow chart and work packages organization.

Interventional Methods

The Network planning was organized into several parallel working phases.

WP1 was conducted by experienced radiologists and was responsible for identifying the biophysical characteristics of an advanced qMRI protocol to be harmonized across the different sites in WP2. Radiologists WP1 focused on identifying the main pathology of interest for the largest number of sites to define guidelines for the acquisition of a minimum standard clinical MRI protocol. Finally, WP1 selected a case study of the pathology of interest for the largest number of sites to apply the advanced harmonized qMRI protocol using both clinical (WP2) and preclinical scanners (WP3).

WP2, delegated to experienced physicists and engineers, was organized into the following project tasks: (i) survey of MRI scanner and related equipment (e.g., vendor, gradient system, transmitter/receiver coils, software release, year of installation); (ii) selection of QC phantoms and definition of SOP to test scanner's performance (i.e., intra- and inter-scanner repeatability and reproducibility); (iii) set up and development of a harmonized qMRI protocol, described in an SOP and tested on healthy participants of similar age and gender; (iv) delineation of reproducibility standards for each qMRI metric as a possible biomarker across sites.

WP3, delegated to experienced physicists and engineers, was divided into the following project tasks: (i) use of standard Bruker and n-Tridecane phantoms (35) for the evaluation of preclinical scanner's performance; (ii) identification of a harmonized qMRI protocol on mice; (iii) delineation of reproducibility standards for each qMRI metric across sites.

WP4, carried out by experienced bioinformaticians and engineers, dealt with the (i) design and implementation of the IT infrastructures for data management and communication, according to the Findable, Accessible, Interoperable, and Reusable (FAIR) principles (36); (ii) implementation of analysis pipelines and data analytics using artificial intelligence and machine learning.

Data Analyses

WP1: Clinical Protocols

Twelve neurological diagnostic classes of interest were identified: dementia (Alzheimer's disease, frontotemporal dementia, vascular dementia), epilepsy, demyelinating diseases, motor neuron diseases, spinal cord pathologies, Parkinson's disease and parkinsonism, brain tumors, disorders of consciousness, rare diseases, pediatric diseases, headaches, and cerebrovascular diseases.

For each of these neurological diagnostic classes, neuroradiologists have established guidelines for the acquisition of a minimum standard clinical MRI protocol that includes the sequences required for radiological evaluation, as well as relevant geometry imaging parameters (i.e., slice orientation, phase encoding, the range for the field of view, voxel size, slice thickness). These guidelines were shared with the Italian Association of Neuroradiology for a wider application and diffusion to various centers at the national level. Neuroradiologists and researchers delineated MRI sequences for an advanced qMRI protocol, which includes techniques for both clinical, structural, and functional assessment of the brain according to the state of the art in the neuroimaging field. The working group identified its case study in dementia so that it can initiate the translation of all harmonization efforts into large-scale multimodal data collection for patients and animal models (Figure 3).

FIGURE 3

Figure 3. Schematic representation of the main results, roadmap, and integration between the 4 project WPs. WP, work packages; SOP, standard operating procedure; QSM, quantitative susceptibility mapping; DWI, diffusion-weighted imaging; rs-fMRI, resting-state functional magnetic resonance imaging; FAIR-ASL, flow-sensitive alternating inversion recovery arterial spin labeling; Clu-CEST, glutamate chemical exchange saturation transfer.

WP2: Clinical Scanners

The 3T scanner survey showed that within the Network there was a prevalence of Philips vendor (8 scanners), compared to General Electric (GE) (3 scanners) and Siemens (7 scanners) vendors. Moreover, heterogeneity in scanner configurations was found, particularly in the gradient system and the transmitter/receiver head coils. A head coil with 32 or fewer channels was available in 15 sites, while the latest technology with a 64 channels head coil was available in 3 centers. The size of the head coils affected the choice of phantoms for the QCs. QC phantom scans were conducted approximately monthly for 2 to 4 years. Through QC, we assessed the reproducibility over time among scanners, using published reference performance indices even when significant hardware and software upgrades were performed intra-site in response to the progress of technology.

For the evaluation of geometry and contrast parameters in QC, the American College of Radiology (ACR) phantom was identified and purchased in the large and/or small versions (37): the ACR large phantom fits head coils with 32 channels or less, while the ACR small phantom can fit head coils with 64 channels. For both phantoms, the MRI protocol, including T1- and T2-weighted sequences, was set up starting from the ACR recommendations (37). Additionally, specifications for the in-plane resolution, post-processing filters, and receiver bandwidth were introduced because these parameters are known to affect distortions of the images (38). Thus, an SOP for QC with ACR phantom (Figure 3) was established including care of the phantom (e.g., refill), its placement within the head coil, implementation of the MRI protocol, and image acquisition. To implement an automatic QC pipeline, Matlab script available at http://jidisun.wixsite.com/osaqa-project/resources/ was improved (39). This script includes the evaluation of the following quantitative parameters defined in ACR recommendation: geometry distortion, slice thickness accuracy, intensity uniformity, ghosting artifact, and high contrast spatial resolution. A further measure considering the geometric accuracy along the feet-to-head axis was introduced: the ratio between the elliptical areas of the first and the third/fifth slices was still obtained for small/large ACR. Each value derived from QC MRI images was compared with tolerance ranges defined by standard ACR recommendations (37) to define outlier measures.

For the evaluation of the stability of gradient echo-planar imaging (GE-EPI) sequences over time, the FUNSTAR (https://www.goldstandardphantoms.com/products/funstar/) phantom, compatible with all scanner head coils, was chosen. The MRI protocol was set up to use echo-planar sequences with similar parameters affecting gradients' stability and performance. Similar to the ACR phantom, a SOP for the QC with FUNSTAR phantom (Figure 3), including placement within the head coil, implementation of the MRI protocol, and image acquisition, was written. Data analysis was performed through a script implemented in Python (https://github.com/mri-group-opbg/stabilitycalc). This analysis estimated the number of parameters that are of interest for neuroimaging functional studies. It computed Signal to Noise Ratio (SNR), Signal to Fluctuation Noise Ratio (SFNR), Percentage Signal Change (PSC), Signal drift, temporal SNR (tSNR) (16, 40). It performed even-odd analysis producing an output to visualize structured noise and spike detection to identify anomalous volumes. Moreover, an additional analysis was implemented in our QC assessment on the FUNSTAR phantom. The Weiskoff analysis that we proposed covered not only a single plane as in the original paper (40), but innovatively all three orthogonal planes as well as the 3D volume in order to estimate the radius of DeCorrelation across all planes. This approach enables us to monitor the effect of noise across the slices and estimate poor slice selection and signal leakage. Values derived from FUNSTAR data were extracted from a central ROI as well as from multiple peripherical ROIs and they were compared with tolerance ranges defined as two standard deviations from the mean value among all the first nine acquisitions from each site, after excluding outliers.

SOPs for QC are available on request at the following link (https://zenodo.org/record/6320896).

After a period of trial and error, an advanced harmonized qMRI protocol in a clinical setting was then finalized. The final protocol now includes the following sequences: 3D T1 weighted images (3D T1w), 3D T2 weighted Fluid Attenuated Inversion Recovery (3D T2-FLAIR) images, quantitative susceptibility mapping imaging (QSM), 2-shell diffusion-weighted imaging (DWI), and resting-state functional magnetic resonance imaging (rs-fMRI). 3D T1w and 3D T2-FLAIR images, sagittally oriented, were acquired with the same geometry parameters (e.g., voxel = 1 x 1 x 1 mm³, number of slices = 175–180, depending on scanner). For the QSM data, a multi-echo gradient echo sequence (voxel size = 1 × 1 × 1 mm³) was defined. Given that different sequence implementations were available across vendors, standardizing the echo-train and echo time (TE) values, without using the scanners in research mode, was quite difficult. So, across all sites, we aimed to achieve a uniform average TE, according to literature standards at the time of implementation (41). For the DWI data, a 2-shell standard single-shot echo-planar imaging sequence (EPI) sequence (voxel size = 2.5 × 2.5 × 2.5 mm³, two shells with 30/32-depending on the scanner- isotropically distributed diffusion-weighted directions, diffusion weightings of 1,000 and 2,000 s/mm², 4–7 non-diffusion weighted b = 0 s/mm² images equally distributed among diffusion-weighted images) was implemented. In addition, 3 non-diffusion weighted images with the reversed phase-encoding acquisition were acquired for distortion correction. For the rs-fMRI data, a GE-EPI sequence (voxel size = 3 × 3 × 3 mm³, repetition time = 2,400 ms, echo time = 30 ms) was set. GE-EPI inverted blip acquisition was acquired as well. No advanced acceleration parameters were used in any of the protocols, as hardly any of the IRCCS scanners were equipped with options such as simultaneous excitation or compressed sensing as part of their software versions.

The total acquisition time for the qMRI protocol was approximately 40 min.

Thereafter, three test sites were identified: site CB for Philips, site DG for Siemens, site MA for GE. For each sequence of interest, the technical working group defined fundamental geometry and contrast parameters so that each scanner in the Network could implement them. The sequences were implemented and optimized in these 3 test sites. The guidelines on the correct implementation and acquisition of the harmonized qMRI protocol were summarized in an SOP document.

To verify the repeatability and reproducibility of qMRI measures extracted from each sequence, the protocol was acquired in 4 “traveling brains” at each test site. Not only the presence of motion artifacts in both structural and functional sequences but also excessive distortions along the antero-posterior phase encoding direction in DWI and rs-fMRI sequences were evaluated, together with contrast to noise ratio between gray and white matter, signal to noise ratio in different brain tissues and the coefficient of joint variation for the assessment of intensity non-uniformity. After this initial quality check, specific analysis pipelines were implemented based on Freesurfer (https://surfer.nmr.mgh.harvard.edu/) for cortical and subcortical thickness/volume assessment, on STI suite (https://people.eecs.berkeley.edu/~chunlei.liu/software.html) for QSM, on MRTrix/FSL (https://www.mrtrix.org/, https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FSL) for DWI, and CONN (https://web.conn-toolbox.org/) for rs-fMRI. Several qMRI measures were obtained: cortical and/or subcortical volumes and thickness in regions of interest (ROIs) and corpus callosum shape for 3D T1w; quantitative susceptibility values in subcortical ROIs for QSM; fractional anisotropy, mean diffusivity, mean kurtosis in white matter and gray matter masks for DWI; tSNR in white matter, gray matter, and cerebrospinal fluid masks for rs-fMRI. Similarity within each qMRI measure extracted from the “traveling brains” data was assessed by evaluating intra- and inter-scanner coefficients of variation. In particular, the extracted qMRI measures were considered robust if the coefficients of variation were <10% and no outliers were identified. Therefore, a tolerance range for each qMRI measure was defined as two standard deviations from the mean value calculated across all “traveling brains” data.

After the protocol optimization step in the three test sites, the remaining sites implemented the qMRI protocol following the SOP and/or through direct import of the file/s generated by the test scanners (e.g., examcard, DICOM files). To verify the protocol implementation at each site, the technical working group examined both the quality of the images acquired on a control subject-as outlined for the “traveling brain” -as well as the parameters set through checking automatically specific DICOM tags. Once the protocol was approved, each site proceeded with the acquisition of data using the qMRI protocol in 5 healthy participants (mean age: 29.7 ± 5.0; 32 male/ 45 female). To assess the reliability and robustness of the harmonized protocol across sites, the previously developed analysis pipelines were applied and the qMRI measurements obtained from each subject and each imaging modality were compared with the tolerance range derived from “traveling brains.”

WP3: Preclinical Scanners

WP3 of the RIN-Neuroimaging Network defined the first harmonized MRI study-both at the setup and analysis pipeline levels-carried out on a mouse model. The WP3 produced a SOP for advanced harmonized qMRI protocol in the preclinical setting. The survey showed that the 7T preclinical scanners were only from Bruker. Thus, all the QC (SNR, ghosting, intensity stability) were conducted using Bruker standard phantoms. The only exception was for the QC of diffusion measurements for which a homemade phantom filled with n-Tridecane was used to test the stability and reproducibility of diffusion maps (35). The harmonized preclinical protocol includes the following sequences: 3D T2-weighted images (3D T2w), DWI, Flow-sensitive Alternating Inversion Recovey Arterial Spin Labeling (FAIR-ASL), and Glutamate Chemical Exchange Saturation Transfer (Glu-CEST).

3D T2w provide a high resolution 0.1 x 0.1 x 0.1 mm³ isotropic anatomical scan. In preclinical imaging, it is imperative to pay attention to the placement of the surface head coil concerning the animal's head to avoid subsequent inaccurate brain extraction and segmentation. The diffusion protocol was set up as a single shell EPI sequence (voxel size = 0.115 × 0.115 × 0.66 mm³, with 30 isotropically distributed diffusion-weighted directions, diffusion weightings of 900 s/mm², 5 non-diffusion weighted b = 0 s/mm² images). Very precise positioning of the slice package is requested to prevent any misalignment along the head-tail axis that might introduce strong partial volume effects. FAIR-ASL is characterized by two T1 inversion recovery maps under both global and selective inversion regimes (voxel size = 0.156 × 0.208 × 1 mm³, T1 map with 15 inversion times). Measures of endogenous Glutamate levels were performed using a Glu-CEST acquisition with voxel size = 0.2 × 0.2 × 1 mm³, with the Z-spectrum generated with 22 off-resonance saturation pulses with lengths = 1 s and intensity = 5 mT.

The harmonized protocol and analysis pipelines allow obtaining the following qMRI measures: the whole brain volumes for 3D T2w, the white matter integrity through the fractional anisotropy, radial diffusion value, and axial diffusion values for DWI, the cerebral blood flow for FAIR-ASL, and the endogenous glutamate levels for Glu-CEST.

According to the outcome of discussions within WP1 regarding the first pathology to tackle, it was decided to study longitudinally (from 4 to 19 months) the transgenic (TG) mouse model APP/PS1. This model is a well-established model of Alzheimer's disease (42, 43).

WP4: e-Infrastructures

RIN–Neuroimaging Network identified three different web-based platforms developed under previous European and National calls, such as NeuGRID (44), ARIANNAFORIRCCS (45), and Medical Informatic Platform (MIP) (46) to provide complementary tools.

This project echoes the FAIR guiding principles of open science developed by the contemporary neuroscience community researchers for enhancing the reusability of research data (36, 47).

NeuGRID (https://neugrid2.eu) is a High-Performance Computing e-infrastructure aiming to collect a large amount of image data paired with computationally intensive data analyses. NeuGRID has been identified by the Re3data initiative as an official research data repository compatible with the FAIR principle. NeuGRID provides the following services to the RIN–Neuroimaging Network: (i) persistent storage and monitoring of harmonized acquisitions of phantoms and murine models with the settings of each scanner checked over time; (ii) imaging analysis using publicly available analysis software and artificial intelligence tools (46). Once data transfer from IRCCS hospitals to the central platform is complete, a QC procedure is automatically triggered for the recently uploaded data (i.e., ACR and FUNSTAR phantoms). QC results are exported to a spreadsheet and PNG snapshots, which are archived along with DICOM images for long-term performance reporting and automatically sent to the user as a final report.

MIP (https://ebrains.eu/service/medical-informatics-platform/), which is cross-linked with NeuGRID, is part of a large-scale European initiative, the Human Brain Project and aims at the integration of multiparametric neuroscientific data using advanced machine learning and deep learning algorithms for the investigation of the human brain in pathologies in the future development of this Network (46).

ARIANNAFORIRCCS platform (https://ariannaforirccs.pi.infn.it/), stemmed from the Arianna project (45) is a web-based research environment for the evaluation of advanced harmonized qMRI protocols on healthy participants. ARIANNAFORIRCCS provides the following services: (i) upload of demographics and brain imaging data; (ii) imaging analysis using public available analysis software.

Each of these platforms responds to a specific need of the RIN–Neuroimaging Network and the FAIR principles (36, 47). According to them, data are characterized by the following standards:

Findability-Persistent Identification and Description With Appropriate Metadata

Datasets within the web-based platforms are matched with metadata, including metadata directly extracted from DICOM image datasets, along with metadata provided by users at the time of upload. This persistent identifier is assigned to both the data and metadata.

Accessibility-Sufficient Storage for Human and Machine Access

Data and metadata can be retrieved using several access methods via standard certified protocols (https). Authentication is required to access data.

Interoperability-Structuring in a Way That Allows Plain Collaboration With Other Datasets

A structure made up of codified folder and subfolder, enclosing data and metadata, is used to ensure accessible interoperation of the data. Specifically, users specify the type of acquisition and upload a compressed folder containing DICOM files of the MRI sequence and image analyses are triggered in the RIN–Neuroimaging Network platforms.

Reuse-Licensed or Accompanied by Terms and Conditions of Use

The data are released with a clear data use agreement within the Network. Community standards and protocols are used to collect, process, and store data and metadata.

Discussion

In recent years, the considerable technological progress in the field of neuroimaging has increased the reliability, precision, and sensitivity of the MRI acquisitions, contributing to our understanding of the mechanisms of brain aging and pathological alterations, allowing for more accurate diagnosis and prognosis of neurological diseases, as well as the assessment of effects induced by pharmacological and non-pharmacological interventions through the identification of in vivo imaging biomarkers. The RIN–Neuroimaging network aims to increase the diagnostic and prognostic power of multimodal MRI data through the generation of “big data” of specific imaging biomarkers for neurological diseases collected at different sites through an integrated multidisciplinary and translational approach. Such an overarching aim can only be achieved through sharing clinical, radiological, and scientific knowledge and ongoing support in this collective effort.

Several factors are known to contribute to scanner variability in multisite studies, including-but not limited to-hardware and software scanner characteristics, data acquisition, and analysis (24, 48). Such sources of variability are often of the same order of magnitude as the disease-related variations, thus they could severely affect the diagnostic and prognostic potential of qMRI parameters (20). Inter-site harmonization strategies are needed to evaluate, manage and limit these sources of variability (20–24). Indeed, the definition of a unique gold standard for MRI protocol harmonization within multisite studies remains challenging. Strategies for sharing procedures and knowledge are key pillars to help overcome this challenge. We have presented here a Network of research hospitals whose first project, supported by Ministerial funds, aimed at converging all sorts of different clinical/preclinical and scientific expertise available between sites to promote sharing know-how of advanced imaging techniques and implement rigorous procedures for “big data” collection. The initiative, supported by the Ministry of Health, aims to reshape research in Italy toward a joint goal, integrating the unique clinical and research experience of IRCCSs, through a network of laboratories offering a global and multilevel approach to neuroscience research.

To date, we mainly focused on three key aspects.

The first one concerns feasibility. The trade-off between MRI procedures that are easily implementable in clinical routine and the completeness of the data collected is a driver to successful clinical research (6). This aspect was carefully considered during the definition of the guidelines for the acquisition of a minimum standard clinical MRI protocol as well as in the implementation of QC procedures and the advanced qMRI protocol. In particular, according to the state of the art in the field of neuroimaging and considering the characteristics of the subjects who have to undergo MRIs (children and non-cooperative patients and/or with difficulties in maintaining posture), the implementation of the qMRI protocol should aim to have an acquisition time of <1 h.

Furthermore, we consider a widely documented procedure that uses a traveling brain to assess the harmonized protocol in vivo in the human, but adapted to the widespread geographical area of our Network. The traveling brain approach is an effective method for controlling for site differences [e.g., (20, 22, 49–53)]. Using only a traveling brain approach for harmonization would require imaging the same participants at all participating sites, but it would also require significant efforts from participants and sites, particularly in a wide and distributed network (50). Additionally, if new sites are added to the Network, these traveling brains should be available to pursue further acquisitions and the effect of increasing age would need to be considered. Compared to previous traveling brain research, to address these needs, we tried to define a range of acceptability for each of the qMRI parameters for each vendor, based on traveling brain acquisitions, as a benchmark to assess consistency across IRCCSs.

A second aspect is the cross-cutting nature of these advanced imaging methodologies. Preclinical qMRI protocols have experienced less standardization compared to their clinical counterparts (6, 31, 32). By harmonizing QC approaches and assessing the reliability and reproducibility of quantitative data across preclinical scanners, on the same animal types and models of disease, it is possible to ensure a more accurate and precise transposition of results in the clinical setting.

Finally, a third and crucial aspect regards empowerment of knowledge. The multidisciplinary nature of the working teams, which involve professionals with substantially different yet complementary backgrounds, is the key ingredient to enrich and facilitate the standardization, harmonization, and successful sharing of the qMRI protocols. We have brought together clinical, infrastructure, and scientific competencies, such as knowledge of a wide range of physics principles required for advanced sequence implementations as well as image post-processing abilities and machine learning competencies.

Such skills have no boundaries as the working groups created a real synergy between members, through in-person and online regular events; this, in turn, ensured that all collaborators had easy access to all relevant information (17, 25). The work of the RIN-Neuroimaging Network is not ended with these SOPs and protocols, but will continue to integrate methods innovation (e.g., acceleration strategies, acquisition of other imaging biomarkers, novel post-processing algorithms).

The impact that the RIN–Neuroimaging Network actions provide to the scientific community can be summarized in four key points:

• Rationalization of the human and technological resources through constructive synergies;

• Standardization of protocols and analysis methods;

• Optimization of acquisition systems using shared QC;

• Sharing of procedures and data.

By grounding in these concepts, the RIN–Neuroimaging Network is now able to guarantee the effective harmonization and sharing of neuroimaging data to maximize the impact of public investment in scientific research and clinical practice.

Ethics and Dissemination

The Protocol Study was performed under the Declaration of Helsinki (59th General Assembly of the World Medical Association, Seoul, October 2008) and the Medical Research Involving Human Subjects Act (WMO). The procedures involving human participants were reviewed and approved by the Scientific Committee as part of the clinical and research criteria followed by the Neuroradiological Division. All the procedures described were performed in compliance with security, integrity, and privacy. Data protection is relevant due to the nature of the data, the individuals involved, and the purpose of the RIN–Neuroimaging Network, whose goal is to analyze and share information among research centers. Therefore, the data are treated following the General Data Protection Regulation (GDPR - 2016/679). No potentially identifiable human images or data is stored. A Publication Policy has been agreed upon by all participating partners. The Network shares the results of individual WPs through dedicated publications in open access peer-reviewed journals and participation in congresses, reaching a large audience of neuroimaging experts.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethics Committee Lombardy Region IRCCS Institute of Neurology Carlo Besta Foundation Section. The patients/participants provided their written informed consent to participate in this study. The animal study was reviewed and approved by Ufficio Animal Care Unit of IRCCS - Istituto di Ricerche Farmacologiche Mario Negri - Milano, Fondazione IRCCS Istituto Neurologico Carlo Besta - Milano, Ospedale San Raffaele S.r.l. - Milano, I.R.C.C.S. Neuromed-Pozzilli, and Ceinge Biotecnologie Avanzate S.c.a.r.l. - Napoli.

The RIN–Neuroimaging Network

Francesco Padelli (Fondazione IRCCS Istituto Neurologico Carlo Besta), Francesco Ghielmetti (Fondazione IRCCS Istituto Neurologico Carlo Besta), Jean Paul Medina (Fondazione IRCCS Istituto Neurologico Carlo Besta), Sara Palermo (Fondazione IRCCS Istituto Neurologico Carlo Besta), Mattia Colnaghi (Istituto Auxologico Italiano–IRCCS), Claudia Morelli (Istituto Auxologico Italiano–IRCCS), Maria Camilla Rossi-Espagnet (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Lorenzo Figà Talamanca (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Daniela Longo (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Chiara Carducci (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Giulia Lucignani (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Martina Lucignani (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Francesca Bottino (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Chiara Parrillo (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Emanuela Tagliente (IRCCS Istituto Ospedale Pediatrico Bambino Gesù), Claudia Ruvolo (IRCCS Centro Neurolesi Bonino Pulejo), Lilla Bonanno (IRCCS Centro Neurolesi Bonino Pulejo), Domenico Tortora (Ospedale Pediatrico Istituto Giannina Gaslini), Marco Grimaldi (IRCCS Istituto Clinico Humanitas), Maria Luisa Malosio (IRCCS Istituto Clinico Humanitas), Cira Fundaro' (IRCCS Istituti Clinici Scientifici Maugeri), Alessio Moscato (IRCCS Istituti Clinici Scientifici Maugeri), Denis Peruzo (IRCCS E. Medea dell'associazione “La Nostra Famiglia”), Stefano Bastianello (Fondazione IRCSS Istituto Neurologico Naz.le Mondino), Nikolaos Petsas (IRCCS Neuromed), Lorenzo Carnevale (IRCCS Neuromed), Patrizia Pantano (RCCS Neuromed), Fabio Maria Triulzi (Fondazione IRCCS Ca' Granda Osp. Maggiore Policlinico), Giorgio Conte (Fondazione IRCCS Ca' Granda Osp. Maggiore Policlinico), Antonella Iadanza (IRCCS Ospedale San Raffaele), Andrea Falini (IRCCS Ospedale San Raffaele), Giovanni Giulietti (Fondazione IRCCS Santa Lucia), Raffaele Agati (IRCCS Istituto delle Scienze Neurologiche di Bologna), Claudia Testa (IRCCS Istituto delle Scienze Neurologiche di Bologna), Marco Aiello (IRCCS SDN Istituto di Ricerca), Marco Salvatore (IRCCS SDN Istituto di Ricerca), Marta Lancione (Fondazione IRCCS Stella Maris), Mauro Costagli (University of Genova; Fondazione IRCCS Stella Maris), Fabrizio Levrero (IRCCS Ospedale Policlinico San Martino).

Author Contributions

ANi, SF, CG, MT, ARe, GFo, DA, RL, FT, and MGB contributed conception and design of the study. ANi, SF, CG, MT, ARe, DA, LB, PB, IC, GD, RG, ML, ANa, FP, AP, GS, VM, SM, FB, ARo, LSP, EA, FA, ER, LP, VC, GFe, AC, MB, CT, CC, LR, CA, SG, MGB, and RIN Neuroimaging Network development of clinical neuroimaging recommendations for harmonization. GFo, DA, EM, LP, AC, CC, MGB, and RIN Neuroimaging Network development of preclinical neuroimaging recommendations for harmonization. MT, ARe, LB, PB, SD, and RIN Neuroimaging Network organized the database and infrastructure. ANi, SF, CG, MT, and MGB wrote the first draft of the manuscript. ARe, GFo, DA, LB, PB, GD, EM, ANa, FP, and GS wrote sections of the manuscript. All authors and RIN Neuroimaging Network contributed to manuscript revision, read, and approved the submitted version.

Funding

This project was funded by the Italian Minister of Health (RRC-2016-2361095, RRC-2017-2364915, RRC-2018-2365796, and RCR-2019-23669119_001, and RCR 2020-23670067) and the Ministry of Economy and Finance (CCR-2017-23669078).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2022.855125/full#supplementary-material

References

1. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: Preferred definitions and conceptual framework. Clin Pharmacol Ther. (2001) 69:89–95. doi: 10.1067/mcp.2001.113989

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Shimizu S, Hirose D, Hatanaka H, Takenoshita N, Kaneko Y, Ogawa Y, et al. Role of neuroimaging as a biomarker for neurodegenerative diseases. Front Neurol. (2018) 9:1–6. doi: 10.3389/fneur.2018.00265

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Hiremath CS, Sagar KJV, Yamini BK, Girimaji AS, Kumar R, Sravanti SL, et al. Emerging behavioral and neuroimaging biomarkers for early and accurate characterization of autism spectrum disorders: a systematic review. Transl Psychiatry. (2021) 11:42. doi: 10.1038/s41398-020-01178-6

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Mitchell T, Lehéricy S, Chiu SY, Strafella AP, Stoessl AJ, Vaillancourt DE. Emerging neuroimaging biomarkers across disease stage in parkinson disease: a review. JAMA Neurol. (2021) 78:1262–72. doi: 10.1001/jamaneurol.2021.1312

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Young PNE, Estarellas M, Coomans E, Srikrishna M, Beaumont H, Maass A, et al. Imaging biomarkers in neurodegeneration: current and future practices. Alzheimer's Res Ther. (2020) 12:1–17. doi: 10.1186/s13195-020-00612-7

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Keenan KE, Biller JR, Delfino JG, Boss MA, Does MD, Evelhoch JL, et al. Recommendations towards standards for quantitative MRI (qMRI) and outstanding needs. J Magn Reson Imaging. (2019) 49:e26–39. doi: 10.1002/jmri.26598

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Van Horn JD, Toga AW. Human neuroimaging as a ‘big data' science. Brain Imaging Behav. (2014) 8:323–31. doi: 10.1007/s11682-013-9255-y

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Madan CR. Scan once, analyse many: using large open-access neuroimaging datasets to understand the brain. Neuroinformatics. (2021) 11:1–29. doi: 10.1007/s12021-021-09519-6

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Ioannidis JPA. Why most published research findings are false. Get to Good Res Integr Biomed Sci. (2018) 2:2–8. doi: 10.1371/journal.pmed.0020124

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Müller VI, Cieslik EC, Serbanescu I, Laird AR, Fox PT, Eickhoff SB. Altered brain activity in unipolar depression revisited: meta-analyses of neuroimaging studies. JAMA Psychiatry. (2017) 74:47–55. doi: 10.1001/jamapsychiatry.2016.2783

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Qiu S, Joshi PS, Miller MI, Xue C, Zhou X, Karjadi C, et al. Development and validation of an interpretable deep learning framework for Alzheimer's disease classification. Brain. (2020) 143:1920–33. doi: 10.1093/brain/awaa137

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Abrol A, Fu Z, Salman M, Silva R, Du Y, Plis S, et al. Deep learning encodes robust discriminative neuroimaging representations to outperform standard machine learning. Nat Commun. (2021) 12:1–17. doi: 10.1038/s41467-020-20655-6

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Woo CW, Wager TD. Neuroimaging-based biomarker discovery and validation. Pain. (2015) 156:1379–81. doi: 10.1097/j.pain.0000000000000223

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Sullivan DC, Obuchowski NA, Kessler LG, Raunig DL, Gatsonis C, Huang EP, et al. Metrology standards for quantitative imaging biomarkers1. Radiology. (2015) 277:813–25. doi: 10.1148/radiol.2015142202

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Helmer KG, Ambite JL, Ames J, Ananthakrishnan R, Burns G, Chervenak AL, et al. Enabling collaborative research using the biomedical informatics research network (BIRN). J Am Med Informatics Assoc. (2011) 18:416–22. doi: 10.1136/amiajnl-2010-000032

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Kayvanrad A, Arnott SR, Churchill N, Hassel S, Chemparathy A, Dong F, et al. Resting state fMRI scanner instabilities revealed by longitud inal phantom scans in a multi-center study. Neuroimage. (2021) 237:118197. doi: 10.1016/j.neuroimage.2021.118197

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Jovicich J, Barkhof F, Babiloni C, Herholz K, Mulert C, van Berckel BNM, et al. Harmonization of neuroimaging biomarkers for neurodegenerative diseases: a survey in the imaging community of perceived barriers and suggested actions. Alzheimers Dement. (2019) 11:69–73. doi: 10.1016/j.dadm.2018.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Keenan KE, Ainslie M, Barker AJ, Boss MA, Cecil KM, Charles C, et al. Quantitative magnetic resonance imaging phantoms: a review and the need for a system phantom. Magn Reson Med. (2018) 79:48–61. doi: 10.1002/mrm.26982

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Belli G, Busoni S, Ciccarone A, Coniglio A, Esposito M, Giannelli M, et al. Quality assurance multicenter comparison of different MR scanners for quantitative diffusion-weighted imaging. J Magn Reson Imaging. (2016) 43:213–9. doi: 10.1002/jmri.24956

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Pinto MS, Paolella R, Billiet T, Van Dyck P, Guns PJ, Jeurissen B, et al. Harmonization of brain diffusion MRI: concepts and methods. Front Neurosci. (2020) 14:396. doi: 10.3389/fnins.2020.00396

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Jovicich J, Minati L, Marizzoni M, Marchitelli R, Sala-Llonch R, Bartrés-Faz D, et al. Longitudinal reproducibility of default-mode network connectivity in healthy elderly participants: a multicentric resting-state fMRI study. Neuroimage. (2016) 124:442–54. doi: 10.1016/j.neuroimage.2015.07.010

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Badhwar AP, Collin-Verreault Y, Orban P, Urchs S, Chouinard I, Vogel J, et al. Multivariate consistency of resting-state fMRI connectivity maps acquired on a single individual over 25 years, 13 sites and 3 vendors. Neuroimage. (2020) 205:116210. doi: 10.1016/j.neuroimage.2019.116210

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Gracien RM, Maiworm M, Brüche N, Shrestha M, Nöth U, Hattingen E, et al. How stable is quantitative MRI? – assessment of intra- and inter-scanner-model reproducibility using identical acquisition sequences and data analysis programs. Neuroimage. (2020) 207:1–11. doi: 10.1016/j.neuroimage.2019.116364

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Lee H, Nakamura K, Narayanan S, Brown RA, Arnold DL. Estimating and accounting for the effect of MRI scanner changes on longitudinal whole-brain volume change measurements. Neuroimage. (2019) 184:555–65. doi: 10.1016/j.neuroimage.2018.09.062

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Keator DB, Helmer K, Steffener J, Turner JA, Van Erp TGM, Gadde S, et al. Towards structured sharing of raw and derived neuroimaging data across existing resources. Neuroimage. (2013) 82:647–61. doi: 10.1016/j.neuroimage.2013.05.094

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Das S, Glatard T, MacIntyre LC, Madjar C, Rogers C, Rousseau ME, et al. The MNI data-sharing and processing ecosystem. Neuroimage. (2016) 124:1188–95. doi: 10.1016/j.neuroimage.2015.08.076

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Mullard A, Mullard A. Preclinical cancer research suffers another reproducibility blow. Nat Rev Drug Discov. (2022) 2:89. doi: 10.1038/d41573-022-00012-6

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Immonen R, Smith G, Brady RD, Wright D, Johnston L, Harris NG, et al. Harmonization of pipeline for preclinical multicenter MRI biomarker discovery in a rat model of post-traumatic epileptogenesis. Epilepsy Res. (2019) 150:46–57. doi: 10.1016/j.eplepsyres.2019.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Deruelle T, Kober F, Perles-Barbacaru A, Delzescaux T, Noblet V, Barbier EL, et al. A multicenter preclinical MRI study: definition of rat brain relaxometry reference maps. Front Neuroinform. (2020) 14:1–9. doi: 10.3389/fninf.2020.00022

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Grandjean J, Canella C, Anckaerts C, Ayranci G, Bougacha S, Bienert T, et al. Common functional networks in the mouse brain revealed by multi-centre resting-state fMRI analysis. Neuroimage. (2020) 205:116278. doi: 10.1016/j.neuroimage.2019.116278

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Drude NI, Gamboa LM, Danziger M, Dirnagl U, Toelch U. Science forum improving preclinical studies through replications. Elife. (2021) 10:1–10. doi: 10.7554/eLife.62101

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Mannheim JG, Kara F, Doorduin J, Fuchs K, Reischl G, Liang S, et al. Standardization of small animal imaging—current status and future prospects. Mol Imaging Biol. (2018) 20:716–31. doi: 10.1007/s11307-017-1126-2

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Raggi A, Leonardi M. Burden and cost of neurological diseases: a European North-South comparison. Acta Neurol Scand. (2015) 132:16–22. doi: 10.1111/ane.12339

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Catalá-López F, Gènova-Maleras R, Vieta E, Tabarés-Seisdedos R. The increasing burden of mental and neurological disorders. Eur Neuropsychopharmacol. (2013) 23:1337–9. doi: 10.1016/j.euroneuro.2013.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Tofts PS, Lloyd D, Clark CA, Barker GJ, Parker GJM, McConville P, et al. Test liquids for quantitative MRI measurements of self-diffusion coefficient in vivo. Magn Reson Med. (2000) 43:368–74. doi: 10.1002/(sici)1522-2594(200003)43:3<368::aid-mrm8>3.0.co;2-b

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Wilkinson MD, Dumontier M. Aalbersberg IjJ, Appleton G, Axton M, Baak A, et al. Comment: the FAIR guiding principles for scientific data management and stewardship. Sci Data. (2016) 3:1–9. doi: 10.1038/sdata.2016.18

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Weinreb J, Wilcox P, Hayden J, Lewis R, Froelich J. ACR MRI accreditation: yesterday, today, and tomorrow. J Am Coll Radiol. (2005) 2:494–503. doi: 10.1016/j.jacr.2004.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Ihalainen TM, Lönnroth NT, Peltonen JI, Uusi-Simola JK, Timonen MH, Kuusela LJ, et al. MRI quality assurance using the ACR phantom in a multi-unit imaging center. Acta Oncol. (2011) 50:966–72. doi: 10.3109/0284186X.2011.582515

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Sun J, Barnes M, Dowling J, Menk F, Stanwell P, Greer PB. An open source automatic quality assurance (OSAQA) tool for the ACR MRI phantom. Australas Phys Eng Sci Med. (2015) 38:39–46. doi: 10.1007/s13246-014-0311-8

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Friedman L Glover GH The The FBIRN Consortium. Reducing interscanner variability of activation in a multicenter fMRI study: controlling for signal-to-fluctuation-noise-ratio (SFNR) differences. Neuroimage. (2006) 33:471–81. doi: 10.1016/j.neuroimage.2006.07.012

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Lancione M, Donatelli G, Cecchi P, Cosottini M, Tosetti M, Costagli M. Echo-time dependency of quantitative susceptibility mapping reproducibility at different magnetic field strengths. Neuroimage. (2019) 197:557–64. doi: 10.1016/j.neuroimage.2019.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Gengler S, Hamilton A, Hölscher C. Synaptic plasticity in the hippocampus of a APP/PS1 mouse model of alzheimer's disease is impaired in old but not young mice. PLoS ONE. (2010) 5:1–10. doi: 10.1371/journal.pone.0009764

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Jankowsky JL, Zheng H. Practical considerations for choosing a mouse model of Alzheimer's disease. Mol Neurodegener. (2017) 12:1–22. doi: 10.1186/s13024-017-0231-7

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Redolfi A, Manset D, Barkhof F, Wahlund LO, Glatard T, Mangin JF, et al. Head-to-head comparison of two popular cortical thickness extraction algorithms: a cross-sectional and longitudinal study. PLoS ONE. (2015) 10:1–22. doi: 10.1371/journal.pone.0117692

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Retico A, Arezzini S, Bosco P, Calderoni S, Ciampa A, Coscetti S, et al. ARIANNA: A research environment for neuroimaging studies in autism spectrum disorders. Comput Biol Med. (2017) 87:1–7. doi: 10.1016/j.compbiomed.2017.05.017

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Redolfi A, De Francesco S, Palesi F, Galluzzi S, Muscio C, Castellazzi G, et al. Medical informatics platform (MIP): a pilot study across clinical Italian cohorts. Front Neurol. (2020) 11:1021. doi: 10.3389/fneur.2020.01021

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Nichols TE, Das S, Eickhoff SB, Evans AC, Glatard T, Hanke M, et al. Best practices in data analysis and sharing in neuroimaging using MRI HHS public access author manuscript. Nat Neurosci. (2017) 20:299–303. doi: 10.1038/nn.4500

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Yan C-G, Craddock R, Zuo Z, Zang Y, Milham M. Standardizing the intrinsic brain : towards robust measurement of inter-individual variation in 1000 functional connectomes. Neuroimage. (2013) 80:246–62. doi: 10.1016/j.neuroimage.2013.04.081

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Voelker MN, Kraff O, Goerke S, Laun FB, Hanspach J, Pine KJ, et al. The traveling heads 20: Multicenter reproducibility of quantitative imaging methods at 7 Tesla. Neuroimage. (2021) 232:117910. doi: 10.1016/j.neuroimage.2021.117910

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Koike S, Tanaka SC, Okada T, Aso T, Yamashita A, Yamashita O, et al. Brain/MINDS beyond human brain MRI project: a protocol for multi-level harmonization across brain disorders throughout the lifespan. NeuroImage Clin. (2021) 30:102600. doi: 10.1016/j.nicl.2021.102600

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Maikusa N, Zhu Y, Uematsu A, Yamashita A, Saotome K, Okada N, et al. Comparison of traveling-subject and combat harmonization methods for assessing structural brain characteristics. Hum Brain Mapp. (2021) 42:5278–87. doi: 10.1002/hbm.25615

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Palacios EM, Martin AJ, Boss MA, Ezekiel F, Chang YS, Yuh EL, et al. Toward precision and reproducibility of diffusion tensor imaging: a multicenter diffusion phantom and traveling volunteer study. Am J Neuroradiol. (2017) 38:537–45. doi: 10.3174/ajnr.A5025

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Nencka AS, Meier TB, Wang Y, Muftuler LT, Wu YC, Saykin AJ, et al. Stability of MRI metrics in the advanced research core of the NCAA-DoD concussion assessment, research and education (CARE) consortium. Brain Imaging Behav. (2018) 12:1121–40. doi: 10.1007/s11682-017-9775-y

PubMed Abstract | CrossRef Full Text | Google Scholar