Functional MRI Preprocessing in Lesioned Brains: Manual Versus Automated Region of Interest Analysis

Functional magnetic resonance imaging (fMRI) has significant potential in the study and treatment of neurological disorders and stroke. Region of interest (ROI) analysis in such studies allows for testing of strong a priori clinical hypotheses with improved statistical power. A commonly used automated approach to ROI analysis is to spatially normalize each participant’s structural brain image to a template brain image and define ROIs using an atlas. However, in studies of individuals with structural brain lesions, such as stroke, the gold standard approach may be to manually hand-draw ROIs on each participant’s non-normalized structural brain image. Automated approaches to ROI analysis are faster and more standardized, yet are susceptible to preprocessing error (e.g., normalization error) that can be greater in lesioned brains. The manual approach to ROI analysis has high demand for time and expertise, but may provide a more accurate estimate of brain response. In this study, commonly used automated and manual approaches to ROI analysis were directly compared by reanalyzing data from a previously published hypothesis-driven cognitive fMRI study, involving individuals with stroke. The ROI evaluated is the pars opercularis of the inferior frontal gyrus. Significant differences were identified in task-related effect size and percent-activated voxels in this ROI between the automated and manual approaches to ROI analysis. Task interactions, however, were consistent across ROI analysis approaches. These findings support the use of automated approaches to ROI analysis in studies of lesioned brains, provided they employ a task interaction design.

Functional magnetic resonance imaging (fMRI) has significant potential in the study and treatment of neurological disorders and stroke. Region of interest (ROI) analysis in such studies allows for testing of strong a priori clinical hypotheses with improved statistical power. A commonly used automated approach to ROI analysis is to spatially normalize each participant's structural brain image to a template brain image and define ROIs using an atlas. However, in studies of individuals with structural brain lesions, such as stroke, the gold standard approach may be to manually hand-draw ROIs on each participant's non-normalized structural brain image. Automated approaches to ROI analysis are faster and more standardized, yet are susceptible to preprocessing error (e.g., normalization error) that can be greater in lesioned brains. The manual approach to ROI analysis has high demand for time and expertise, but may provide a more accurate estimate of brain response. In this study, commonly used automated and manual approaches to ROI analysis were directly compared by reanalyzing data from a previously published hypothesis-driven cognitive fMRI study, involving individuals with stroke. The ROI evaluated is the pars opercularis of the inferior frontal gyrus. Significant differences were identified in task-related effect size and percent-activated voxels in this ROI between the automated and manual approaches to ROI analysis. Task interactions, however, were consistent across ROI analysis approaches. These findings support the use of automated approaches to ROI analysis in studies of lesioned brains, provided they employ a task interaction design.
Keywords: stroke, lesion, spatial normalization, inferior frontal gyrus, region of interest analysis introduction Functional magnetic resonance imaging (fMRI) is used often in neuroscience and clinical practice (1) to study human brain function both in clinical and healthy populations. Studies in individuals with stroke, for example, use fMRI to evaluate changes in brain function and to relate these to changes in behavior, to estimate clinical outcomes and to attempt to reduce disability [e.g., Ref. (2,3)]. fMRI is also used to evaluate therapeutic efficacy, for example, of stroke neurorehabilitation approaches [e.g., Ref. (4)], or to provide proof-of-concept for novel clinical therapeutics (5,6). fMRI has the potential to provide important information about individualized treatments and outcomes (1).
A major challenge to the use of fMRI in clinical research and clinical practice is that it can be difficult to obtain robust enough signals in individuals to allow confident evaluations (1). In whole brain analyses, blood oxygen level-dependent (BOLD) signal is measured in each voxel and fit to a general linear model (GLM), and the test statistics are computed across thousands of voxels, leading to the well-known problem of multiple comparisons (7). One way to improve statistical power is to limit the number of tests; in neuroimaging, this can be accomplished by restricting analysis to an a priori ROI (8). In addition, corrections for multiple comparisons used to control for false positives resulting from correlation across voxels (e.g., Bonferroni correction) can be applied to an ROI or a small number of ROIs rather than to the thousands of voxels in the whole brain (8). Strong prior hypotheses are required in order to take advantage of the ROI approach. In clinical practice, ROI analyses can be useful when intervention decisions are linked to strong prior hypotheses about individual brain regions (9).
One approach for ROI analysis is to identify regions based on anatomical landmarks and define ROIs on each individual's structural brain image (10)(11)(12)(13)(14). This manual approach accommodates the inter-subject variability in neuroanatomy (8), including variance related to brain lesions, and may be considered the gold standard for studies in clinical populations. However, manual definition of ROIs is highly time consuming, labor intensive, and has to rely on expertise; therefore, it has the risk of interexperimenter variability. In addition, while some brain regions may be defined reliably, others lack clear anatomical landmarks and therefore may be more difficult to demarcate.
An alternative approach is to identify ROIs based on automatic demarcation using anatomical atlases or template brains. To do so, each individual's structural brain image must be first registered to the standard space defined by the template brain image; such registration is achieved by spatial normalization (15). In clinical populations, however, structural changes, such as brain lesions or loss of brain volume, may result in a lack of perfect correspondence between the individual's brain image and the template brain image, not allowing a perfect spatial normalization process (16). Such suboptimal normalization may lead to misalignment and therefore decreased sensitivity within a group of subjects, or worse, to false attribution of group differences to brain functional rather than structural differences (16,17). Normalization errors are always greater in damaged brains, introducing a confound in comparisons between patients and control groups. Spatial normalization algorithms attempt to reduce image mismatches, and the solutions have been improving. For example, a recent approach uses unified segmentation and normalization (18) to estimate the model parameters used to fit an individual's brain to standard space by alternating between image bias correction, tissue segmentation, and non-linear registration; an approach that has been optimized for lesioned brains (16). Nevertheless, differences between automated and manual ROI analysis have not been well quantified, especially in brains with structural lesions. This study examined the results of a new ROI analysis of a previously published hypothesis-driven cognitive fMRI study involving individuals with stroke (5), comparing commonly used approaches to automated and manual ROI analysis. The cognitive task employed in that study was action observation; therefore, the ROI examined in the current study was the inferior frontal gyrus pars opercularis (Brodmann area, BA 44, the posterior half of Broca's area), considered to be part of the putative human mirror neuron system that is activated during both action execution and action observation (19,20). This brain region is also implicated in a wide range of functions, including speech production (21) and perception (22,23), and working memory (24,25), among others. This ROI has potential clinical significance in utilizing action observation in stroke rehabilitation (26,27), measuring reactivation of language networks during stroke recovery (28,29), and as a focus of pre-surgical language network mapping (30). The current study compared activity in BA 44 using an automated and manual approach to ROI analysis. In the automated method, individual brain images were spatially normalized to a template brain image and ROIs were automatically defined in the standard space. In the manual method, ROIs were hand-drawn on each individual's non-normalized structural brain image using neuroanatomical landmarks. Differences in task-related effect size and percentactivated voxels between the automated and manual methods were characterized and compared between lesioned and control brains.

Participants
Twelve individuals with stroke (six females; mean age 66 years, range 40-86 years) and 12 control individuals (five females, mean age 66 years, range 40-82 years) participated in the prior study (5). All participants were right-handed (31) and had normal or corrected vision. All participants with stroke had chronic strokes imaging Procedure Images were acquired using a 3 T Siemens Trio MRI. T1-weighted anatomical images were acquired for participants with stroke (TR = 2350 ms, TE = 3.09 ms, 256 mm × 256 mm, 208 slices, slice thickness 1 mm, flip angle = 10°, 1 mm × 1 mm × 1 mm) and for healthy participants (TR = 1950 ms, TE = 2.26 ms, 256 mm × 256 mm, 176 slices, slice thickness 1 mm, flip angle = 9°, 1 mm × 1 mm × 1 mm). All participants took part in a hypothesis-driven fMRI study of action observation (5). Functional MRI included four 12 s blocked conditions: (a) right hand action observation, (b) left hand action observation, (c) static images of hands, and (d) rest. During action observation, participants watched videos of an actor grasping objects using either his right hand or his left hand. Participants were instructed to remain still and pay attention to the videos. Prior to actual scanning, the procedure was practiced in a mock scanner. During scanning, all participants were visually monitored for movement; no overt movement was detected in any participant. Each condition block was repeated 15 times followed by rest, randomized across three 6 min runs (gradient echo, TR = 2 s, TE = 30 ms, 64 mm × 64 mm, 37 slices, flip angle = 90°, 3.5 mm × 3.5 mm × 3.5 mm).

Image Preprocessing and Analysis
Image preprocessing was conducted using SPM12. 1 Functional images were realigned for motion correction and the resultant motion parameters were included as regressors of no interest in the fMRI model. In addition, Artifact detection tools (ART) 2 was used to identify mean global intensity and motion outliers in the fMRI time series using an outlier threshold of global signal >3 SDs and motion >1 mm, and the detected outliers were included as regressors of no interest in the fMRI model. For the manual ROI analysis, functional images were smoothed using a 6 mm full width half maximum (FWHM) Gaussian kernel, and no further preprocessing steps were taken. For the automated ROI analysis, the structural image was segmented and all images were normalized using SPM12's unified segmentation normalization (18), and smoothed using a 6 mm FWHM Gaussian kernel. Both approaches utilized smoothing in order to improve signal to noise (34). First level models were specified to estimate the betas for each participant for the conditions of right hand and left hand action observation separately. Rest was modeled as implicit baseline.

Region of Interest Definition
Region of interest (ROIs) included the left and right pars opercularis of the inferior frontal gyrus (BA 44). The methods to define ROIs using the manual approach and the automated approach are described below. Examples of ROIs defined using each of the approaches are displayed in Figure 1.

Manual region of interest definition
Region of interest were defined manually on each individual's non-normalized structural brain image based on neuroanatomical landmarks using the principles described by Allen et al. (35,36). The following landmarks were used to define the pars opercularis of the inferior frontal gyrus (BA 44): ascending branch of the Sylvian fissure (anterior boundary), inferior frontal sulcus (dorsal boundary), precentral sulcus (posterior boundary), Sylvian fissure (ventral boundary), depth of the ascending branch of the Sylvian fissure and depth of the inferior frontal sulcus (medial boundary).
ROIs were hand-drawn using MRIcron 3 (37) by a research assistant with no further involvement in the study, who had been trained to identify and trace neuroanatomical landmarks and lesion boundaries due to stroke, and who was blinded to the purpose of the current study, i.e., did not know that the manual drawings would be compared to an automated approach. ROI drawings were checked by an investigator (KAG); reviewed by a researcher with extensive neuroanatomy experience including related to BA 44 [CR; (22,23)]; overseen by an expert MRI neuroanatomist [HD; (38)]; and adjusted where necessary in discussion with the research team.

Automated region of interest definition
Automated ROI definition used the left and right BA 44 defined regions in SPM Anatomy toolbox (39) defined in Montreal Neurological Institute (MNI) space (40). In SPM Anatomy toolbox, anatomical regions have been defined based on maximum probability cytoarchitectonic maps.

Region of interest volume and overlap
Region of interest volume was calculated using fslstats from Fslutils (41). 4 ROI overlap was evaluated by first reslicing the ROIs into the same MNI space using SPM12. Spatial agreement between each manual ROI and the automated ROI for the same hemisphere was then calculated using Dice's coefficient (42), a measure of the volume of the overlap relative to the mean volume of the two ROIs, using a Matlab script (provided by Chris Rorden). 5 Dice's coefficient (d) ranges from 0 (no overlap) to 1 (complete overlap). ROI overlap was visualized using fslmaths and MRIcron.

Region of Interest Analysis
Region of interest were analyzed using MarsBar (43) to (1) extract all of the data within the ROI for each functional image to provide a voxel time course for each voxel in the ROI, (2) calculate a summary time course for each ROI as the mean of all voxel values in the ROI, (3) estimate the fMRI model with the ROI data according to SPM12's implementation of the GLM, (4) apply a contrast (e.g., "task minus rest") to the estimated model to derive an effect size for each contrast, and (5) extract the percent of activated voxels in the ROI (T = 1.7-5.0). Contrasts tested in this study included the main effects of right hand and left hand action observation, separately.

statistical analysis
One-sample t-tests were used to test whether the volume of ROIs defined manually differed from the volume of the automated ROI for each hemisphere, for each group. Repeated measures analyses of variance (ANOVA) were used to determine a difference in effect size or percent of activated voxels between automated and manual approaches to ROI analysis, with method (automated/ manual), condition (right hand/left hand action observation), and hemisphere (right/left) as within-subject factors, and group (stroke/control) as a between-subjects factor. Paired t-tests were used post hoc to determine differences between automated and manual approaches to ROI analysis for each group, hemisphere, and condition, for descriptive purposes.   ROI maps defined using the automated approach are displayed in blue. Spatial overlap between manual and automated maps is indicated in pink. ROIs are overlaid onto the MNI template brain image in neurological orientation.  Figure 3). Pairwise comparisons for each method, group, hemisphere, and condition are displayed in Figure 3. Additional within-subjects test results are provided in Table 1. Other main effects and interactions related to the methods comparison were not significant, including interactions between method and group (p = 0.416), method and hemisphere (p = 0.231), and method, group, and hemisphere (p = 0.418). However, as expected based on prior analysis of this dataset (5), a significant three-way interaction was found between cognitive task condition, group, and hemisphere [F(1,22) = 8.438, p = 0.008;  Figures 4-6 to illustrate the differences between automated and manual approaches to ROI definition, and are discussed in more detail below.

Percent activated Voxels Differs Between Manual and automated approaches to rOi analysis
Similarly, a four-way repeated measures ANOVA determined that percent activated voxels differed significantly between approaches [F(1,22) = 7.377, p = 0.013; Table 2]. A post hoc pairwise comparison using the Bonferroni correction revealed an overall significantly larger percent of activated voxels for      Table 2. Other main effects and interactions related to the methods comparison were not significant, including interactions between method and group (p = 0.546), method and hemisphere (p = 0.831), and method, group, and hemisphere (p = 0.839).

Discussion
These findings demonstrate significant differences between a manual approach and a commonly used automated approach to ROI analysis. The automated approach led to a larger estimated task-related effect size and percent activated voxels compared to the manual approach, in both lesioned and control brains, and in both right and left hemispheres (for participants with stroke, the left hemisphere was the lesioned hemisphere). These findings were consistent across two conditions of cognitive task employed in the fMRI study (5). As discussed, these differences in ROI measures between manual and automated approaches may be attributed to differences in ROI volume, spatial normalization error, and/or spatial smoothing. Region of interest volume for the left BA 44 was significantly smaller using manual as compared to automated ROI definition, in both groups. ROIs were defined automatically using SPM Anatomy toolbox in which BA 44 is derived from observerindependent analysis of cytoarchitectonic areas and generation of probabilistic maps of 10 post-mortem brains (44), spatially normalized to the T1-weighted MNI structural brain atlas. In Amunts et al., a left greater than right volume asymmetry for BA 44 was identified in all post-mortem brains used to map this brain region (44,45). Here, no such volume asymmetry was found between the left and right BA 44 using manual ROI definition in lesioned or control brains. The difference between the current findings and those of Amunts et al. is not likely to be age-related, because the mean age in that study was 66 years, and in the current study 65 years for control participants and 66 years for those with stroke. Moreover, in SPM Anatomy toolbox, the left greater than right volume asymmetry survived spatial normalization to the MNI single subject template brain, which was derived from a younger individual (46). The current data do not resolve this discrepancy, but may be more generally representative of inter-individual variability in neuroanatomy and brain volume. A smaller ROI volume may increase effect size by improving statistical power by reducing the number of computed tests, or decrease effect size by capturing fewer activated voxels. Likewise, a larger ROI volume may increase effect size by capturing a larger number of activated voxels, or decrease statistical power by averaging signal over more voxels and increasing the number of statistical tests. Moreover, there was low volume overlap between manual and automated ROIs in both lesioned and control brains, as indicated by low Dice's coefficients (d = 0.2). Several factors may contribute to low volume overlap, including volume differences between manual and automated ROIs, the effects of lesions involving the ROI, experimenter bias utilized in manually defining ROIs, and spatial normalization utilized in automatically defining ROIs. An earlier study reported 70-80% volume overlap (d = 0.7-0.8) between manual and automated definitions of the amygdala and hippocampus (47). The hippocampus has clearer anatomical boundaries than the cortical ROIs tested in the current study; that study also did not compare ROI approaches in lesioned brains. An example of the potential effect of ROI volume, in combination with other factors, is provided in Figure 4 from a control participant for whom the larger ROI defined by the automated method captured a larger number of activated voxels from the cluster of interest as compared to the smaller ROI defined by the manual method.
These findings demonstrate significant differences between manual and automated approaches to ROI analysis that are consistent in both lesioned and control brains, indicating that the findings cannot be exclusively attributed to error in spatial normalization of the lesioned brains. The automated method used SPM12's unified segmentation normalization algorithm, which has been optimized for lesioned brains (16). This approach combines bias correction, tissue segmentation, and spatial normalization in an iterative process to better fit an individual's brain image to the template brain image (18). Here, there is an overall good fit between individual participant's brain images and the template brain image (as indicated by a visual check of registration between the images). However, normalization error is always greater in lesioned brains due to intensity changes and/or tissue displacement, and this error is especially problematic when the lesion involves the ROI, as is the case for a number of participants with stroke involving left BA 44. In some cases, experimenter bias (i.e., neuroanatomical expertise) may be necessary to localize an ROI after sulcal changes due to brain injury. An example is provided in Figure 5 for a participant with stroke for whom the experimenter was able to manually define left BA 44, whereas the automated map does not contain the intact tissue from this brain region after tissue displacement due to stroke.
Spatial smoothing may also lead to differences in ROI measures in single subjects. Here, both the automated and manual methods of ROI analysis employed spatial smoothing, in order to improve signal to noise (34). Post hoc analysis indicated that the findings hold if the same methods are compared without spatially smoothing the data in either approach [F(1,22) = 21.111, p = 0.0001], or if smoothing is employed for the automated but not the manual method as is common practice [F(1,22) = 14.682, p = 0.001]. Spatial smoothing can impact ROI measures if activated voxels fall on the border of an ROI, or if smoothing decreases signal to noise. An example is provided in Figure 6 for a participant with stroke for whom peak task-related activation is localized to the left ventral premotor cortex according to the manual approach (with smoothing), and yet an increase in ROI volume combined with normalizing and smoothing in the automated approach leads to the peak activation being localized to left BA 44.
Despite the significant differences in ROI measures identified in this study between automated and manual approaches, there was a consistent three-way interaction of condition by group by hemisphere using either approach. This finding was reported previously using a similar automated method [(5); SPM8 was used in that study, and ART was not used] and was demonstrated in the current study using both the automated and manual approaches. This consistent group by task interaction across approaches is in line with prior recommendations to utilize task interaction designs in clinical neuroimaging studies, in particular in lesioned brains, rather than testing for main effects between groups (48). By testing for task interactions, any identified group differences cannot be attributed to pathology, such as changes in neurovasculature due to stroke, because those attributes should influence all tasks similarly.

limitations
Several limitations of the current study must be considered. First, although BA 44 roughly corresponds to the pars opercularis of the inferior frontal gyrus (44,45), the cytoarchitectonic areas may not consistently coincide with macroscopic landmarks (e.g., sulci) for this brain region (44), and it is therefore not entirely straightforward to compare them. Nevertheless, BA 44 is often used interchangeably with the designation of pars opercularis of the inferior frontal gyrus in fMRI studies and in those of action observation in particular (49). Thus the current methods comparison is in agreement with commonly used approaches to ROI analysis in such studies. Second, the manual ROIs in this study were drawn by one researcher and the reliability of the drawings was not tested. However, in similar studies, manual ROIs have been drawn by one researcher and compared to automated ROIs [e.g., Ref. (50)], or the accuracy of ROI drawings has been agreed upon by more than one researcher [e.g., Ref. (51)]. Future studies might directly test the inter-rater reliability of manually defined ROIs. Third, the sample size was small, reflecting the difficulty including individuals with chronic stroke and limited mobility in fMRI studies. A similar study used 20 control subjects (47). Statistical power for the methods comparison was also improved by testing two ROIs in each participant (left/right BA 44). Nevertheless, the small sample size limits the generalizability of the findings. The number of ROIs tested was also limited. This study reanalyzed data from an earlier cognitive fMRI study in which activation in BA 44 was hypothesized to be activated by the task and was involved in the lesion in some individuals (5). These attributes made this particular ROI appropriate to use in a comparison of manual and automated approaches to ROI analysis. The number of ROIs involved in the cognitive task and therefore potentially activated and measurable by ROI analysis was limited, as was the number of ROIs both involved in the task and involved in some lesions. Lastly, other preprocessing methods should be considered. Prior studies have used alternative approaches to spatial normalization, such as normalizing with lesion cost function masking (52), which has been shown to reduce errors in a direct comparison of unified segmentation with and without cost function masking (53). However, several automated methods of normalizing brains with lesions have been compared elsewhere, and none appeared to outperform the others (54). Alternative methods of ROI analysis could also be employed, such as using functionally defined ROIs [e.g., Ref. (55)] or alternative probabilistic brain atlases [e.g., Ref. (56)].
The generalizability of the current findings to comparisons between automated and manual approaches to ROI analysis in other ROIs is therefore limited by these and other factors.
conclusion In summary, this study identified significant differences in task-related effect size and percent activated voxels between a manual and automated approach to ROI analysis. These differences were found in lesioned and control brains, in lesioned and non-lesioned hemispheres, and across fMRI task conditions. Therefore it is possible that these findings are generalizable to other ROIs and hypotheses, although further direct testing is warranted. Despite these identified differences, condition by hemisphere by group interactions were consistent across the manual and automated approaches. Although other ROIs and groups should be tested, this finding suggests that a commonly used automated approach to ROI analysis is appropriate for fMRI studies in clinical populations, including individuals with structural brain lesions, although it is recommended that such studies employ a task interaction design. The automated approach is more easily implemented, with minimal requirements for intervention or expertise, thereby minimizing interexperimenter variability, and is fast and standardized. However, the automated approach is susceptible to spatial normalization error, and the accuracy of ROI definition is only as good as the spatial normalization achieved. Therefore, the automated approach may reduce the validity of ROI comparisons if spatial normalization error decreases sensitivity for the comparison, or if structural differences are incorrectly interpreted as functional differences. In studies with larger cohorts, the problems arising from automated ROI analysis might average out, however, most fMRI studies report statistics in small samples, especially in clinical populations in which recruitment and neuroimaging can prove challenging, such as in individuals with stroke. In these studies, the automated approach tested here may be appropriate if a task interaction design is employed, alternative automated approaches may be tested, or the manual approach should be used. In each, the results of preprocessing should be evaluated Manual versus automated ROI analysis in stroke Frontiers in Neurology | www.frontiersin.org at the single subject level and errors corrected or an alternative approach employed.
acknowledgments We thank our participants for their efforts. We thank Dr. Mara Mather and Matthew Konersman for their help with recruitment; Julie Werner, Alicia Johnson, and Sook Lei Liew for their assistance with scanning; and Ryan Cisneros for manual region of interest tracing. This study was supported by grants from the American Heart Association (14CRP18200010, 1310SDG3510062); National Institutes of Health, National Institute on Drug Abuse (K12DA00167); National Institute of Child Health and Human Development (R03HD067475-01A1l); and National Institute of Biomedical Imaging and Bioengineering (T32EB00438); and the Charles A. Dana Foundation.