Original Research ARTICLE
Sensitivity and specificity of interictal EEG-fMRI for detecting the ictal onset zone at different statistical thresholds
- 1Laboratory for Epilepsy Research, UZ Leuven and KU Leuven, Leuven, Belgium
- 2Medical Imaging Research Center, UZ Leuven and KU Leuven, Leuven, Belgium
- 3Laboratory for Cognitive Neurology, UZ Leuven and KU Leuven, Leuven, Belgium
- 4Department of Nuclear Medicine, UZ Leuven and KU Leuven, Leuven, Belgium
- 5Radiology Department, UZ Leuven and KU Leuven, Leuven, Belgium
There is currently a lack of knowledge about electroencephalography (EEG)-functional magnetic resonance imaging (fMRI) specificity. Our aim was to define sensitivity and specificity of blood oxygen level dependent (BOLD) responses to interictal epileptic spikes during EEG-fMRI for detecting the ictal onset zone (IOZ). We studied 21 refractory focal epilepsy patients who had a well-defined IOZ after a full presurgical evaluation and interictal spikes during EEG-fMRI. Areas of spike-related BOLD changes overlapping the IOZ in patients were considered as true positives; if no overlap was found, they were treated as false-negatives. Matched healthy case-controls had undergone similar EEG-fMRI in order to determine true-negative and false-positive fractions. The spike-related regressor of the patient was used in the design matrix of the healthy case-control. Suprathreshold BOLD changes in the brain of controls were considered as false positives, absence of these changes as true negatives. Sensitivity and specificity were calculated for different statistical thresholds at the voxel level combined with different cluster size thresholds and represented in receiver operating characteristic (ROC)-curves. Additionally, we calculated the ROC-curves based on the cluster containing the maximal significant activation. We achieved a combination of 100% specificity and 62% sensitivity, using a Z-threshold in the interval 3.4–3.5 and cluster size threshold of 350 voxels. We could obtain higher sensitivity at the expense of specificity. Similar performance was found when using the cluster containing the maximal significant activation. Our data provide a guideline for different EEG-fMRI settings with their respective sensitivity and specificity for detecting the IOZ. The unique cluster containing the maximal significant BOLD activation was a sensitive and specific marker of the IOZ.
The goal of the presurgical evaluation in refractory focal epilepsy is to define the epileptogenic zone, the area indispensable for the generation of epileptic seizures (1).
In the last decade, the value of simultaneous electroencephalography-functional magnetic resonance imaging (EEG-fMRI) as a localizing tool of the epileptogenic zone has been explored. In this technique, changes in blood oxygen level dependent (BOLD) contrast, related in a statistical way to interictal epileptic discharges or seizures, are displayed as spatial maps. Sensitivity of EEG-fMRI for localizing the epileptogenic zone has received a lot of attention, but specificity has largely been neglected (2), hampering the clinical implementation of EEG-fMRI in the presurgical evaluation of refractory focal epilepsy.
In order to address this issue, correlational studies of EEG-fMRI with a “gold standard” are crucial (3). Since the epileptogenic zone is a theoretical concept, the ictal onset zone (IOZ), the area from which seizures are generated, is a valuable alternative. The IOZ can be determined by ictal scalp/invasive EEG-registrations and/or ictal single photon emission computed tomography (SPECT) in concordance with other presurgical investigations (1).
Several validation studies assessed the sensitivity of spike-related EEG-fMRI using the results of ictal invasive EEG-registrations as indication of the IOZ (4–17). A major disadvantage of intracranial EEG is the limited spatial coverage and the necessity of an a priori hypothesis of the IOZ with possible non-localizing or misleading results (18, 19). The IOZ is then determined by arbitrary margins around contact points, active during seizure onset, but the technique is blind to uncovered areas, a particular problem when comparing with BOLD activity, as experienced by several authors (5, 8, 17, 20–23).
Ictal perfusion SPECT has the advantage of demonstrating dynamic seizure-related changes in cerebral perfusion on a whole brain scale, which offers ideal comparison with fMRI studies. Several interictal EEG-fMRI studies used the results of ictal SPECT in their validation (5, 6, 9, 11, 12, 24–26). Due to its low temporal resolution, both the IOZ and seizure propagation pathways can be found (27–30). Therefore, combinations of imaging modalities [structural MRI, interictal and ictal SPECT and subtraction ictal SPECT co-registered to MRI (SISCOM), and interictal 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET)], which integrate the additional information of each independent modality, may provide superior information compared to the information provided by ictal SPECT alone (29).
More recently, spike-related EEG-fMRI results have been compared with postsurgical resection zone and epilepsy outcome (5, 9–12, 14, 16, 17, 21, 24, 26, 31, 32). After successful epilepsy surgery, sufficient brain tissue has been resected. However, the resection zone depends on the surgical approach (conservative versus extended resection) (33) and the accessibility and can overestimate the actual IOZ.
A much larger problem than the exact definition of the IOZ to define sensitivity of EEG-fMRI is the lack of knowledge about specificity of EEG-fMRI (2). The extent and pattern of the BOLD changes are dependent on the statistical threshold levels that are used. Less stringent threshold levels will not only induce more widespread and multifocal patterns but also more false-positive results.
At conventional statistical threshold levels [family-wise error (FWE) corrected p < 0.05 and uncorrected p < 0.001] both focal and multifocal, widespread BOLD responses have been described. Widespread BOLD signal clusters have been interpreted as representing widespread epileptic abnormalities with poor surgical prognosis if not completely included in the resection (10, 16, 21, 31). However, in the absence of knowledge about specificity of EEG-fMRI, the clinical significance of these different patterns remains uncertain. As mentioned by Chaudhary and colleagues in 2012, EEG-fMRI studies demonstrate “often complex BOLD patterns, raising the issue of specificity of the findings and the unknown clinical relevance of individual BOLD clusters” (34). In a paper of van Houdt and colleagues, this was rephrased as “there are currently no standards for the statistical thresholds in EEG-fMRI analysis” (17).
In this study, we propose an innovative approach to quantitatively define the effect of different statistical thresholds on sensitivity and specificity of spike-related BOLD changes for detecting the IOZ. We determined true-positive and false-negative BOLD fluctuations in patients, and false-positive and true-negative BOLD fluctuations in age- and gender-matched healthy case-controls.
Materials and Methods
This study was approved by an independent ethical standards committee on human experimentation of the University Hospitals Leuven and written informed consent was obtained from all participants.
Inclusion criteria were (i) consecutive adults who underwent a full presurgical evaluation for refractory focal epilepsy between August 2010 and November 2013, including seizure history, neurological and physical examination, neuropsychological assessment, interictal and ictal scalp EEG-recordings, video-analysis of seizures, high-resolution MRI of the brain, and in most patients SISCOM and interictal 18F-FDG PET. In selected cases, intracranial EEG-recordings were performed; (ii) concordant data pointing to one epileptic focus using all available presurgical investigations, including a SISCOM or else successful outcome after epilepsy surgery [international league against epilepsy (ILAE) outcome classification 1–3 (1, completely seizure-free; 2, only auras; 3, one to three seizure days per year ± auras; 4, four seizure days per year to 50% reduction of baseline seizure days ± auras; 5, <50% reduction of baseline seizure days to 100% increase of baseline seizure days ± auras; 6, more than 100% increase of baseline seizure days ± auras)] (35); (iii) recording of interictal spikes during EEG-fMRI.
Operational Definition of the Ictal Onset Zone
The IOZ was defined as follows:
(i) In patients with successful outcome after epilepsy surgery, we considered the manually outlined resected brain area as the IOZ.
(ii) In patients, awaiting surgery, refusing surgery, or ineligible for surgery due to proximity of the epileptogenic zone to eloquent regions, we determined the IOZ as the hypothetical resection zone, based on multidisciplinary clinical consensus using all non-invasive and invasive data except EEG-fMRI results. As the patients were selected for concordant localizing data, we ensured not to rely on a single testing modality. The volume of this IOZ was further restricted to the region of ictal hyperperfusion on SISCOM within this hypothetical resection area. The hyperperfusion was thresholded with a Z-score = 1.5. This threshold was shown to be optimal for localizing the epileptogenic zone (36).
EEG-fMRI Acquisition and Processing
Functional images were acquired using a whole brain single-shot T2* gradient-echo Echo Planar Imaging sequence in one of two 3 T MR scanners (Achieva TX with a 32-channel head coil and Intera Achieva with an eight-channel head coil, Philips Medical Systems, Best, The Netherlands); TE = 33 ms, TR = 2.2 or 2.5 s, voxel size 2.6 mm × 3 mm × 2.6 mm.
A 64- or 32-channel MR compatible EEG cap was used for simultaneous EEG-fMRI recordings with a BrainAmp amplifier (Brain Products, Munich, Germany; sampling rate 5 kHz). In patients admitted to the hospital, we used a 24-channel MR compatible electrode set (Yves EEG solutions, Newburyport, MA, USA) both in the telemetry unit and in the scanner with the BrainAmp amplifier. Patients were asked to rest with closed eyes.
The EEG was filtered offline (bandpass 1–50 Hz) and gradient artifacts were removed using the Bergen plug-in (Bergen fMRI Group, Bergen, Norway)1 (37) for EEGLAB.2 Pulse artifacts were subtracted with Brain Vision Analyzer software (Brain Products, Munich, Germany) (38).
The fMRI data were analyzed with statistical parametric mapping (SPM), version 8 (Wellcome Department of Imaging Neuroscience, University College London, UK)3; running on MATLAB (MathWorks, Natick, MA, USA). Images were realigned, slice-time corrected, normalized into the Montreal Neurological Institute (MNI) space using the T1 MRI template available in SPM (voxel size: 2 mm × 2 mm × 2 mm), and spatially smoothed using an isotropic Gaussian kernel of 6 mm full width at half-maximum (FWHM).
Interictal spikes were visually marked by a neurologist (Simon Tousseyn) and discussed with a second neurologist (Wim Van Paesschen). Statistical analysis was performed using the general linear model approach. The regressor of the interictal spike was created using the timings of the event convolved with the canonical hemodynamic response function. Included as confounding covariates were (i) the six rigid-body motion correction parameters, (ii) the fMRI signal averaged over the lateral ventricles, and (iii) the fMRI signal averaged over a region centered in the white matter (39). When a sudden head movement (>1 mm translation) appeared, we added a dummy regressor, which was set to one for the corresponding scan as well as for the next three scans. The remainder of the regressor was set to 0. In case this sudden movement was present in different consecutive blocks, a dummy regressor was created for each block (40, 41).
A statistical Z-score map was obtained for the interictal spike event-related regressor. In case, a patient had more than one spike-type, only EEG-fMRI results corresponding to the most frequent spike-type, determined during video-telemetry, were used for the analysis.
Sensitivity and Specificity for Localizing the Ictal Onset Zone
Sensitivity and specificity were calculated as follows: true-positive cases were defined as those patients in whom we found a suprathreshold cluster of a suprathreshold size overlapping the IOZ. Patients, in whom this was not the case, were considered false-negative cases. Epilepsy can be regarded as a network disorder (42, 43). This network concept implies interregional interactions between the IOZ and other sites. Based on this theory, we believe it is not appropriate to classify activations outside the IOZ, related to spikes in patients, as false positives. To determine false-positive and true-negative cases, we introduced age- and gender-matched healthy controls assigned to each patient in order to obtain a statistical map using the spike event-related regressor of that patient (corresponding to nonsense events for the control subject). False-positive cases were those controls who showed a suprathreshold cluster of a suprathreshold size somewhere in the brain while true-negative cases were those controls for whom this was not the case (see Figure 1). In a way, we treated the controls as a surrogate for the patient group, assuming that the results would have been the same if we had been able to look at those parts of the brain, which were not linked to the epileptic network. Each control underwent EEG-fMRI using the same session length as the corresponding patient. The spatial normalization step ensured that the number of voxels, which were analyzed, as well as the cluster size was similar between all patients and controls.
Figure 1. Determination of test outcome. EEG-spikes = spike-time course based on manually indicated interictal spikes. Regressor = spike-time course of patient convolved with canonical hemodynamic response function. Areas of suprathreshold BOLD changes overlapping the ictal onset zone in patients were considered as true positives and if no overlap was found, they were treated as false negatives. Suprathreshold BOLD changes in any part of the brain in healthy controls were considered as false positives, absence of BOLD responses as true negatives.
At a certain statistical threshold, sensitivity was defined as the proportion of true-positive cases within the patient group and the “surrogate” specificity as the proportion of true-negative cases within the control group (see Figure 2). Sensitivity and specificity were calculated for different statistical thresholds at the voxel level (Z, 0–13; step-size, 0.1). This was combined with different thresholds for the minimal cluster size up to 600 voxels (step-size, 50 voxels, voxel size, 2 mm × 2 mm × 2 mm). Six hundred voxels correspond to a brain volume of 4800 mm3, comparable to the volumes of a hippocampus (44), or a focal cortical dysplasia (27). Based on these results, receiver operating characteristic (ROC)-curves were calculated. We performed the calculations for activations and deactivations, separately.
Figure 2. Illustration of a contingency table. (A) For a certain statistical threshold, true-positive (TP) or false-negative (FN) cases are determined in patients, false-positive (FP) or true-negative (TN) cases in control subjects. (B) Sensitivity (number of true positives over the number of true positives and false negatives) was defined solely in the patient group and specificity (number of true negatives over the number of true negatives and false positives) solely in the control group.
In an attempt to identify the most localizing cluster of BOLD changes, if widespread BOLD changes were present, several authors looked at the cluster containing the maximal significant activation or at the maximal significant activation voxel (45, 46). In an additional analysis, we calculated the ROC-curves based upon each of these selections.
Twenty-one consecutive patients (age: 36 ± 14 years, age at epilepsy onset: 12 ± 10 years, 16 women) met the inclusion criteria. Clinical data are presented in Table 1. Patients had a median seizure frequency of eight seizure days per month (range 0.5–30) and had failed 7 ± 4 antiepileptic drugs at the time of evaluation. Thirteen patients had temporal lobe epilepsy (TLE): six mesial TLE, seven lateral TLE, while seizure onset was extratemporal (ETLE) in the other eight patients. Structural imaging was normal in 3/21 patients. All ictal SPECT injections used for analysis in this study were performed during the ictal phase [median seizure duration, 84 s (range: 5–423); median time of initiation of ictal SPECT tracer injection, 17 s (range: 1–43)]. Intracranial EEG-recordings, available in five patients, were concordant with non-invasive investigations.
So far, eight patients underwent epilepsy surgery with a successful outcome [ILAE class 1 (completely seizure-free) in six cases, class 2 (only auras) in two cases] (35) (median follow-up time, 9 months; range, 2–27): temporal lobe resection (patients 3, 7, 17, and 18), frontal lesionectomy (patients 6 and 9), functional hemispherotomy (patient 12), and hippocampus/amygdala radiosurgery (patient 16).
Functional magnetic resonance imaging sessions lasted on average 49 ± 15 min. The median spike-rate during EEG-fMRI was 187 spikes/h (range: 8–2018). Twenty-one healthy case-controls (age, 36 ± 12 years; 16 women) underwent EEG-fMRI using the same session length.
Sensitivity and Specificity for Localizing the Ictal Onset Zone
Electroencephalography-functional magnetic resonance imaging BOLD activations corresponding to the statistical threshold of uncorrected p < 0.001 (Z = 3.1, no constraint on cluster size) resulted in 86% sensitivity (suprathreshold activations in the IOZ in 18 of the 21 patients) and 0% specificity (all controls had a suprathreshold detection in the brain). In contrast, when a significance level of FWE corrected p < 0.05 was used (corresponding to a Z between 4.9 and 5.1 in our study, no constraint on cluster size), sensitivity dropped to 62–57% but specificity increased to 95–100% (Figures 3 and 4).
Figure 3. Sensitivity and specificity. Receiver operating characteristic (ROC)-curves using different statistical Z-score thresholds at the voxel level (range, 0–13; step-size, 0.1) and (A) no constraint on cluster size, (B) a minimal cluster extent of 350 voxels. The results were based on all EEG-fMRI activations (blue) and upon the cluster containing the maximal significant activation only (green). The orange circle corresponds to the statistical threshold of uncorrected p < 0.001 (Z = 3.1), red circles to a significance level of FWE corrected p < 0.05 (Z = 4.9–5.1). The pink circle represents Z-score thresholds of 3.4 and 3.5. Note that not every change in Z-score threshold is associated with a change in sensitivity and specificity.
Figure 4. Electroencephalography-functional magnetic resonance imaging activations are overlaid on representative slices of the postsurgical structural image of patient 3 (A) after thresholding at an uncorrected p < 0.001 (Z = 3.1, no constraint on cluster size) and (B) at a FWE corrected p < 0.05 (Z = 5.1, no constraint on cluster size). (A) The EEG-fMRI cluster containing the maximal significant activation (green colored) is overlapping the IOZ (red colored). Widespread additional suprathreshold activation clusters (blue colored) are present. At these settings, it is unclear whether these activations outside the IOZ represent false- or true-positive activations. (B) Using a more stringent statistical threshold (corresponding to 100% specificity), fewer activation clusters survive. The left occipital activation (white circle) is not a false-positive but a true-positive detection and is considered part of a spike-related network; R, right; L, left.
We report the settings that give the highest sensitivity for maximal specificity. Z-score thresholds of 3.4 and 3.5 both resulted in 62% sensitivity and 100% specificity, using a minimal cluster size of 350 voxels (Figure 3). At these settings, 6 of the 13 patients (46%) with an overlap between a cluster of BOLD activation and the IOZ had at least one additional activation cluster in more remote areas, not overlapping the IOZ. Exclusion of these remote activations from resection did not preclude successful outcome in three of the eight operated patients (patients 3, 6, and 16). In patient 3, a dysembryoplastic neuroepithelial tumor (DNET) in the left temporal lobe was resected. Remote activations were mainly located in mesial occipital areas. Patient 6, with a focal cortical dysplasia in the right primary motor cortex, had a contralateral cerebellar activation (Figure 5). Successful radiosurgery involved the left hippocampus in patient 16 with left hippocampal sclerosis. Additional BOLD activations were present in the left temporal neocortex.
Figure 5. Electroencephalography-functional magnetic resonance imaging results after thresholding at Z = 3.5 at the voxel level combined with a minimal cluster extent of 350 voxels are overlaid on representative slices of the structural MRI in three patients. (A) Patient 16 had only 1 suprathreshold activation. This cluster containing the maximal significant activation (green colored) was localized within the IOZ (red colored). (B) In patient 6, the cluster containing the maximal significant activation (green colored) was overlapping the IOZ, while an additional activation cluster (blue colored) was situated within the contralateral cerebellum. (C) Patient 15 was the only patient with a BOLD activation cluster (blue colored) overlapping the IOZ, which did not contain the maximal significant activation (green colored); R, right; L, left.
In an additional analysis, we looked at the cluster containing the maximal significant BOLD activation. This resulted in similar sensitivity and specificity when using the same minimal cluster extent of 350 voxels (Figure 3). A sensitivity of 57% and a specificity of 100% were obtained for a broad range of Z-score thresholds from 3.2 to 3.5 (green areas in Figure 5). Interestingly, this combination of sensitivity and specificity could also be reached using other settings for minimal cluster size (from 250 voxels up to 600 voxels) but with a smaller range of corresponding Z-score thresholds. By using the cluster containing the maximal significant activation instead of all EEG-fMRI activations, we discarded the additional, non-localizing clusters (blue areas in Figure 4) distant from the IOZ with sacrificing minimal sensitivity (4.8%) at 100% specificity. Only 1 of 21 patients (4.8%) (patient 15) showed a BOLD activation cluster overlapping the IOZ, which did not contain the maximal significant activation (Figure 5).
Finally, when using the maximal significant activation voxel or when considering BOLD deactivations, an overall low performance was found.
Electroencephalography-functional magnetic resonance imaging has evolved from a research tool and is on the brink of becoming a clinical method to delineate the ictal onset in the presurgical evaluation of patients with refractory focal epilepsy. Before taking decisions based on EEG-fMRI, validation studies are a prerequisite. We felt that it was important to investigate sensitivity and specificity of interictal EEG-fMRI for localizing the IOZ, in those patients in whom spikes were visually detected during EEG-fMRI. A reliable test combines a high sensitivity with a high specificity.
The setting of an appropriate statistical threshold in functional imaging is a critical point (6, 32, 46, 47). EEG-fMRI validation studies focused on sensitivity for detecting the IOZ, using “standard” statistical thresholds, but the specificity of these results is still largely unknown. fMRI responses exceeding the epileptogenic zone are often reported (17, 48, 49). A possible explanation may be the choice of a low statistical threshold resulting in false-positive responses (25) and understanding how to minimize these false positives would be of great interest (50). On the other hand, it is not excluded that hemodynamic changes outside the IOZ are related to an epileptic network. Hence, it would not be appropriate to consider hemodynamic changes, associated to spikes in patients and localized partially within and partially outside the IOZ, as false positives. To tackle this issue, we chose to determine false-positive and true-negative cases in age- and gender-matched healthy case-controls. The interictal spike event-related regressor of the patient was used in the design matrix of the healthy case-control (corresponding to nonsense events for this control subject). False-positive cases were those controls who showed a suprathreshold BOLD change in the brain while true-negative cases were those controls for whom this was not the case. As argued before, we treated controls as a surrogate for the patient group with the assumption that results would have been the same if we were able to look at those parts of the brain that were not linked to the epileptic network. The surrogate specificity was subsequently determined as the proportion of true-negative cases within the control group. A disadvantage of our approach is the unknown contribution of differences in noise level between patient and control datasets.
Alternatively, false-positive and true-negative rates could be established in the patient group after random annotation of spike onsets (2). Specificity would then equal the proportion of patients who lack a suprathreshold BOLD fluctuation in the brain, related to these nonsense events. The calculation of true-positive and false-negative rates could remain unchanged, based on real spike onsets. This approach holds two potential risks. In patients with high spike rates, coincidental correlation between regressors related to nonsense events and to real spikes becomes more likely, causing an overestimation of false-positive rates. This problem does not apply in healthy volunteers. A second risk is related to the poorly understood occurrence in time of epileptic spikes. It is not excluded that this occurrence follows a rhythmic pattern, which exhibits (whether or not coincidental) temporal similarities with activity fluctuations of normal physiological brain processes. In that case, random assignment of spike onsets would break this rhythmicity and potentially cause an underestimation of false-positive rates. When we copy the regressor-of-interest to healthy volunteers, this rhythmicity remains unmodified. Notwithstanding the concerns of the alternative method described above, permutation of original spike onsets led to similar sensitivity and specificity (see alternative approach, included as Supplementary Material).
To the best of our knowledge, only three papers formally addressed the topic of specificity of interictal EEG-fMRI in a quantitative way. First, Waites and colleagues used a non-parametric permutation approach in two patients with childhood absence epilepsy and one healthy control to investigate if interictal discharges lead to a BOLD response that is significantly different from chance (2). It was shown that “activations” (at a corrected p < 0.05), related to randomly assigned events, survived more often than expected by chance (i.e., more than 1 in 20). Second, Flanagan and colleagues evaluated the effect of including non-epileptic sharp EEG transients in the EEG-fMRI analysis of clear epileptic spikes (51). These events can result in physiologically plausible BOLD changes that survive a statistical threshold (in both the patient and control group). Third, An and colleagues determined sensitivity and specificity of EEG-fMRI through a different approach, using surgical outcome as “ground truth” (32). True-positive (concordance with resection zone and good surgical outcome) and false-negative (discordance with resection zone and good surgical outcome) fractions were determined in the patients, as were true-negative (discordance with resection zone and poor surgical outcome) and false-positive (concordance with resection zone and poor surgical outcome) fractions. However, poor surgical outcome could have several reasons (incorrect location of surgery, correct location but intra- or post-operative complications, partial resection of the epileptogenic zone, and no resectable epileptogenic zone), leading to equivocal interpretation of the results (52). This is the reason why we established sensitivity only in successfully operated or well-defined patients, taking the (effective or hypothetical) resection zone as central point (patients with poor surgical outcome were not included). On the other hand, specificity was defined in healthy case-controls, taking absence of epileptic activity as “ground truth.”
Different statistical thresholds (uncorrected p < 0.001 and FWE corrected p < 0.05, for instance) can result in very divergent specificities and sensitivities. This information is crucial as these thresholds are commonly reported in EEG-fMRI validation studies. We argue that EEG-fMRI outcome studies should be reported with settings that have maximal specificity. However, when the purpose of EEG-fMRI is to guide the implantation of intracranial electrodes, a high sensitivity might be preferred (17).
The Cluster Containing the Maximal Significant Activation
The presence of multiple clusters of BOLD activation raises an important question: how can we identify the cluster overlapping the IOZ in a highly specific but often widespread interictal epileptic network without prior knowledge of the IOZ? The cluster, containing the maximal significant BOLD activation, with a minimal cluster size of 350 voxels, and with a broad range of Z-score thresholds from 3.2 to 3.5, had 57% sensitivity and 100% specificity for localizing the IOZ, similar to the accuracy of all EEG-fMRI activation clusters. The performance of this unique cluster was robust and did not critically depend on a single Z-score or cluster size threshold. Our findings confirm the observations that the cluster containing the maximal significant activation is important in the localization of the IOZ (45).
We considered two other aspects of the EEG-fMRI maps. First, the maximal significant activation voxel had a lower performance for localizing the IOZ compared with the cluster containing this voxel. In some patients, this voxel was localized at the border, but just outside the IOZ, while in others, it was found more remote. Hauf and colleagues (46) ascribed similar findings of distant fMRI peak activations to the effect of propagation. Second, deactivations were only infrequently found inside the IOZ, consistent with other reports (21, 53).
Involvement of Remote Regions: An Epileptic Network
There is a bulk of evidence that so called “focal” epilepsies are not strictly localized to well-circumscribed focal brain areas, but constitute larger epileptic networks (42, 43). When using a setting of high specificity (100%), almost half the patients with an activation overlapping the IOZ had at least one additional activation cluster in more distant areas. These remote findings can be considered as true positives. Activations at a distance have been interpreted as an extended or multifocal IOZ (10, 21). However, the presence of these remote activations did not preclude successful surgical outcome in three operated patients in our study. Therefore, spike-related BOLD clusters distant to the IOZ could also represent areas of propagated activity, as suggested by different authors (6, 25, 54, 55).
Validity of the Ictal Onset Zone Definition
Seizure freedom and good functional outcome are the ultimate goals of epilepsy surgery. So far, 8 of the 21 patients underwent successful surgery, and we considered the resection zone as IOZ. The extent of the resection zone depends on the surgical approach and can overestimate the actual IOZ. We have shown that only about one-quarter of resected brain tissue overlapped the structural lesion or SISCOM hyperperfusion cluster (27). However, after successful surgery, sufficient brain tissue has been resected.
Thirteen patients are awaiting surgery, refused surgery, or were ineligible for surgery due to proximity of the IOZ to eloquent regions. In this non-operated group, we chose to define the IOZ as the hypothetical resection zone, based on multidisciplinary clinical consensus and regardless of eloquent cortex. As already mentioned, a prerequisite for inclusion was concordance of all modalities, including electroclinical information, structural imaging, SISCOM, FDG PET, and intracranial EEG-recordings. Multimodal concordant seizure focus localizing data increase the likelihood of benefit from surgical treatment (56–58). To avoid a rater-dependent bias in the manual delineation, we restricted the volume of the IOZ to the region of ictal hyperperfusion within this hypothetical resection zone. In our center, ictal and interictal SPECT are part of the presurgical work-up. SISCOM has several advantages: (i) it samples the whole brain, which offers an ideal comparison with the results of EEG-fMRI, (ii) it displays relative changes in cerebral blood perfusion associated with neuronal metabolic activity, (iii) a SISCOM Z-threshold = 1.5 results in optimal localization of the IOZ (36), (iv) it is a non-invasive test, and (v) early ictal tracer injections, as achieved in most of our patients, are known to be related to correct localization of the IOZ (59).
We stress that sensitivity and specificity calculations only apply to patients in whom spikes were found during EEG-fMRI. In two patients, more than one spike-type (based on topography) was found during fMRI. In these cases, we decided to determine the results driven by the most frequent spike-type during video-telemetry only, similar to Elshoff and colleagues (26). High correlations between the localization of the lobe producing the most active spiking and that of the IOZ have been found for temporal lobe epilepsies (60). Moreover, it was shown that the lobe producing the most active spiking correlated highly with the ultimately resected lobe harboring cortical dysplasia (61).
The number of successfully operated patients and their follow-up period is limited. To increase the group size, a surrogate for the effective resection zone was adopted in those patients who could not undergo surgery. This allowed us to study a representative and larger sample of patients with a well-defined IOZ after a presurgical evaluation. Studies including larger number of patients and control subjects will be required to fine-tune EEG-fMRI settings. Furthermore, this could allow subpopulations (TLE versus ETLE) to be studied, as sensitivity and specificity are presumably also dependent on brain localization.
High sensitivity and specificity of spike-related EEG-fMRI for the detection of the IOZ are crucial for the clinical implementation of the technique in the presurgical planning of refractory focal epilepsy. Our data provide a guideline for different EEG-fMRI settings with their respective sensitivity and specificity for detecting the IOZ. Using optimal settings, we found that the unique cluster containing the maximal significant BOLD activation was a sensitive (57%) and specific (100%) marker of the IOZ.
Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work: Simon Tousseyn, Patrick Dupont, Karolien Goffin, Stefan Sunaert, and Wim Van Paesschen; drafting the work or revising it critically for important intellectual content: Simon Tousseyn, Patrick Dupont, Karolien Goffin, Stefan Sunaert, and Wim Van Paesschen; final approval of the version to be published: Simon Tousseyn, Patrick Dupont, Karolien Goffin, Stefan Sunaert, and Wim Van Paesschen; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved: Simon Tousseyn, Patrick Dupont, Karolien Goffin, Stefan Sunaert, Wim Van Paesschen.
Conflict of Interest Statement
Dr. Stefan Sunaert has received honoraria for speaking engagements from Philips Medical Systems. Dr. Simon Tousseyn reports industry-funded (UCB and GSK) travel, not included in the study funding, to attend AES meeting, ECE, and IEC. Dr. Wim Van Paesschen received honorary for speaking engagements (GSK), serves on a scientific advisory board (UCB and GSK), reports industry-funded travel (UCB and GSK), not included in the study funding, to attend AES meeting and ECE.
We acknowledge discussions with Maarten De Vos, Borbola Hunyadi, Ronald Peeters, David Robben, Laura Seynaeve and Katrien Vanderperren. We thank Bart De Dobbelaer for software support. The authors are grateful to Guido Van Driel and the epilepsy monitoring team for helping to collect the EEG data. This study was funded by a grant for Applied Biomedical Research (TBM) no. 080658 from the Flemish government agency for Innovation by Science and Technology (IWT); grant no. G.048010N from Fonds Wetenschappelijk Onderzoek-Vlaanderen; grant no. IUAP EMF-B6772-p6/29 from Inter-University-Attraction-Pole; and Belspo IAP grant P7/11 and grant no. FPV10/008 from Excellentie-Financiering.
The Supplementary Material for this article can be found online at http://www.frontiersin.org/Journal/10.3389/fneur.2014.00131/abstract
2. Waites AB, Shaw ME, Briellmann RS, Labate A, Abbott DF, Jackson GD. How reliable are fMRI-EEG studies of epilepsy? A nonparametric approach to analysis validation and optimization. Neuroimage (2005) 24:192–9. doi:10.1016/j.neuroimage.2004.09.005
4. Krakow K, Woermann FG, Symms MR, Allen PJ, Lemieux L, Barker GJ, et al. EEG-triggered functional MRI of interictal epileptiform activity in patients with partial seizures. Brain (1999) 122:1679–88. doi:10.1093/brain/122.9.1679
5. Lazeyras F, Blanke O, Perrig S, Zimine I, Golay X, Delavelle J, et al. EEG-triggered functional MRI in patients with pharmacoresistant epilepsy. J Magn Reson Imaging (2000) 12:177–85. doi:10.1002/1522-2586(200007)12:1<177::AID-JMRI20>3.0.CO;2-3
6. Al-Asmi A, Bénar CG, Gross DW, Khani YA, Andermann F, Pike B, et al. fMRI activation in continuous and spike-triggered EEG-fMRI studies of epileptic spikes. Epilepsia (2003) 44:1328–39. doi:10.1046/j.1528-1157.2003.01003.x
7. Bagshaw AP, Aghakhani Y, Bénar CG, Kobayashi E, Hawco C, Dubeau F, et al. EEG-fMRI of focal epileptic spikes: analysis with multiple haemodynamic functions and comparison with gadolinium-enhanced MR angiograms. Hum Brain Mapp (2004) 22:179–92. doi:10.1002/hbm.20024
8. Bénar CG, Grova C, Kobayashi E, Bagshaw AP, Aghakhani Y, Dubeau F, et al. EEG-fMRI of epileptic spikes: concordance with EEG source localization and intracranial EEG. Neuroimage (2006) 30:1161–70. doi:10.1016/j.neuroimage.2005.11.008
11. Tyvaert L, Hawco C, Kobayashi E, LeVan P, Dubeau F, Gotman J. Different structures involved during ictal and interictal epileptic activity in malformations of cortical development: an EEG-fMRI study. Brain (2008) 131:2042–60. doi:10.1093/brain/awn145
12. Moeller F, Tyvaert L, Nguyen DK, LeVan P, Bouthillier A, Kobayashi E, et al. EEG-fMRI: adding to standard evaluations of patients with nonlesional frontal lobe epilepsy. Neurology (2009) 73:2023–30. doi:10.1212/WNL.0b013e3181c55d17
13. Vulliemoz S, Rodionov R, Carmichael DW, Thornton R, Guye M, Lhatoo SD, et al. Continuous EEG source imaging enhances analysis of EEG-fMRI in focal epilepsy. Neuroimage (2010) 49:3219–29. doi:10.1016/j.neuroimage.2009.11.055
14. Grouiller F, Thornton RC, Groening K, Spinelli L, Duncan JS, Schaller K, et al. With or without spikes: localization of focal epileptic activity by simultaneous electroencephalography and functional magnetic resonance imaging. Brain (2011) 134:2867–86. doi:10.1093/brain/awr156
16. Donaire A, Capdevila A, Carreño M, Setoain X, Rumià J, Aparicio J, et al. Identifying the cortical substrates of interictal epileptiform activity in patients with extratemporal epilepsy: an EEG-fMRI sequential analysis and FDG-PET study. Epilepsia (2013) 54(4):678–90. doi:10.1111/epi.12091
17. van Houdt PJ, de Munck JC, Leijten FS, Huiskamp GJ, Colon AJ, Boon PA, et al. EEG-fMRI correlation patterns in the presurgical evaluation of focal epilepsy: a comparison with electrocorticographic data and surgical outcome measures. Neuroimage (2013) 75:238–48. doi:10.1016/j.neuroimage.2013.02.033
18. Wang ZI, Jin K, Kakisaka Y, Mosher JC, Bingaman WE, Kotagal P, et al. Imag(in)ing seizure propagation: MEG-guided interpretation of epileptic activity from a deep source. Hum Brain Mapp (2012) 33:2797–801. doi:10.1002/hbm.21401
19. Siegel AM, Roberts DW, Thadani VM, McInerney J, Jobst BC, Williamson PD. The role of intracranial electrode reevaluation in epilepsy patients after failed initial invasive monitoring. Epilepsia (2000) 41:571–80. doi:10.1111/j.1528-1157.2000.tb00211.x
20. Kobayashi E, Bagshaw AP, Grova C, Gotman J, Dubeau F. Grey matter heterotopia: what EEG-fMRI can tell us about epileptogenicity of neuronal migration disorders. Brain (2006) 129:366–74. doi:10.1093/brain/awh710
21. Thornton R, Vulliemoz S, Rodionov R, Carmichael DW, Chaudhary UJ, Diehl B, et al. Epileptic networks in focal cortical dysplasia revealed using electroencephalography-functional magnetic resonance imaging. Ann Neurol (2011) 70:822–37. doi:10.1002/ana.22535
23. van Houdt PJ, Ossenblok PP, Colon AJ, Boon PA, de Munck JC. A framework to integrate EEG-correlated fMRI and intracerebral recordings. Neuroimage (2012) 60:2042–53. doi:10.1016/j.neuroimage.2012.02.023
24. Jann K, Wiest R, Hauf M, Meyer K, Boesch C, Mathis J, et al. BOLD correlates of continuously fluctuating epileptic activity isolated by independent component analysis. Neuroimage (2008) 42:635–48. doi:10.1016/j.neuroimage.2008.05.001
25. Groening K, Brodbeck V, Moeller F, Wolff S, van Baalen A, Michel CM, et al. Combination of EEG-fMRI and EEG source analysis improves interpretation of spike-associated activation networks in paediatric pharmacoresistant focal epilepsies. Neuroimage (2009) 46:827–33. doi:10.1016/j.neuroimage.2009.02.026
26. Elshoff L, Groening K, Grouiller F, Wiegand G, Wolff S, Michel C, et al. The value of EEG-fMRI and EEG source analysis in the presurgical setup of children with refractory focal epilepsy. Epilepsia (2012) 53:1597–606. doi:10.1111/j.1528-1167.2012.03587.x
27. Dupont P, Van Paesschen W, Palmini A, Ambayi R, Van Loon J, Goffin J, et al. Ictal perfusion patterns associated with single MRI-visible focal dysplastic lesions: implications for the noninvasive delineation of the epileptogenic zone. Epilepsia (2006) 47:1550–7. doi:10.1111/j.1528-1167.2006.00628.x
28. Huberfeld G, Habert MO, Clemenceau S, Maksud P, Baulac M, Adam C. Ictal brain hyperperfusion contralateral to seizure onset: the SPECT mirror image. Epilepsia (2006) 47:123–33. doi:10.1111/j.1528-1167.2006.00378.x
29. Van Paesschen W, Dupont P, Sunaert S, Goffin K, Van Laere K. The use of SPECT and PET in routine clinical practice in epilepsy. Curr Opin Neurol (2007) 20:194–202. doi:10.1097/WCO.0b013e328042baf6
31. Thornton R, Laufs H, Rodionov R, Cannadathu S, Carmichael DW, Vulliemoz S, et al. EEG correlated functional MRI and postoperative outcome in focal epilepsy. J Neurol Neurosurg Psychiatry (2010) 81:922–7. doi:10.1136/jnnp.2009.196253
32. An D, Fahoum F, Hall J, Olivier A, Gotman J, Dubeau F. Electroencephalography/functional magnetic resonance imaging responses help predict surgical outcome in focal epilepsy. Epilepsia (2013) 54(12):2184–94. doi:10.1111/epi.12434
34. Chaudhary UJ, Rodionov R, Carmichael DW, Thornton RC, Duncan JS, Lemieux L. Improving the sensitivity of EEG-fMRI studies of epileptic activity by modelling eye blinks, swallowing and other video-EEG detected physiological confounds. Neuroimage (2012) 61:1383–93. doi:10.1016/j.neuroimage.2012.03.028
35. Wieser HG, Blume WT, Fish D, Goldensohn E, Hufnagel A, King D, et al. ILAE Commission Report. Proposal for a new classification of outcome with respect to epileptic seizures following epilepsy surgery. Epilepsia (2001) 42:282–6. doi:10.1046/j.1528-1157.2001.4220282.x
38. Allen PJ, Polizzi G, Krakow K, Fish DR, Lemieux L. Identification of EEG events in the MR scanner: the problem of pulse artifact and a method for its subtraction. Neuroimage (1998) 8:229–39. doi:10.1006/nimg.1998.0361
39. Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci U S A (2005) 102(27):9673–8. doi:10.1073/pnas.0504136102
40. Salek-Haddadi A, Dielh B, Hamandi K, Merschhemke M, Liston A, Friston K, et al. Hemodynamic correlates of epileptiform discharges: an EEG-fMRI study of 63 patients with focal epilepsy. Brain Res (2006) 1088:148–66. doi:10.1016/j.brainres.2006.02.098
41. Lemieux L, Salek-Haddadi A, Lund TE, Laufs H, Carmichael D. Modelling large motion events in fMRI studies of patients with epilepsy. Magn Reson Imaging (2007) 25:894–901. doi:10.1016/j.mri.2007.03.009
44. Van Paesschen W. Qualitative and quantitative imaging of the hippocampus in mesial temporal lobe epilepsy with hippocampal sclerosis. Neuroimaging Clin N Am (2004) 14:373–400. doi:10.1016/j.nic.2004.04.004
45. Liu Y, Yang T, Yang X, Liu I, Liao W, Lui S, et al. EEG-fMRI study of the interictal epileptic activity in patients with partial epilepsy. J Neurol Sci (2008) 268:117–23. doi:10.1016/j.jns.2007.11.019
46. Hauf M, Jann K, Schindler K, Scheidegger O, Meyer K, Rummel C, et al. Localizing seizure-onset zones in presurgical evaluation of drug-resistant epilepsy by electroencephalography/fMRI: effectiveness of alternative thresholding strategies. AJNR Am J Neuroradiol (2012) 33:1818–24. doi:10.3174/ajnr.A3052
48. Salek-Haddadi A, Diehl B, Hamandi K, Merschhemke M, Liston A, Friston K, et al. Hemodynamic correlates of epileptiform discharges: an EEG-fMRI study of 63 patients with focal epilepsy. Brain Res (2006) 1088:148–66. doi:10.1016/j.brainres.2006.02.098
49. Kobayashi E, Bagshaw AP, Bénar CG, Aghakhani Y, Andermann F, Dubeau F, et al. Temporal and extratemporal BOLD responses to temporal lobe interictal spikes. Epilepsia (2006) 47:343–54. doi:10.1111/j.1528-1167.2006.00427.x
52. Burch J, Marson A, Beyer F, Soares M, Hinde S, Wieshmann U, et al. Dilemmas in the interpretation of diagnostic accuracy studies on presurgical workup for epilepsy surgery. Epilepsia (2012) 53(8):1294–302. doi:10.1111/j.1528-1167.2012.03534.x
54. Hamandi K, Powell HW, Laufs H, Symms MR, Barker GJ, Parker GJ, et al. Combined EEG-fMRI and tractography to visualise propagation of epileptic activity. J Neurol Neurosurg Psychiatry (2008) 79:594–7. doi:10.1136/jnnp.2007.125401
55. Vulliemoz S, Thornton R, Rodionov R, Carmichael DW, Guye M, Lhatoo S, et al. The spatio-temporal mapping of epileptic networks: combination of EEG-fMRI and EEG source imaging. Neuroimage (2009) 46:834–43. doi:10.1016/j.neuroimage.2009.01.070
57. Labiner DM, Weinand ME, Brainerd CJ, Ahern GL, Herring AM, Melgar MA. Prognostic value of concordant seizure focus localizing data in the selection of temporal lobectomy candidates. Neurol Res (2002) 24:747–55. doi:10.1179/016164102101200843
58. Kurian M, Spinelli L, Delavelle J, Willi JP, Velazquez M, Chaves V, et al. Multimodality imaging for focus localization in pediatric pharmacoresistant epilepsy. Epileptic Disord (2007) 9:20–31. doi:10.1684/epd.2007.0070
59. Lee SK, Lee SY, Yun CH, Lee HY, Lee JS, Lee DS. Ictal SPECT in neocortical epilepsies: clinical usefulness and factors affecting the pattern of hyperperfusion. Neuroradiology (2006) 48:678–84. doi:10.1007/s00234-006-0106-z
Keywords: EEG-fMRI, refractory focal epilepsy, presurgical evaluation, sensitivity and specificity, interictal
Citation: Tousseyn S, Dupont P, Goffin K, Sunaert S and Van Paesschen W (2014) Sensitivity and specificity of interictal EEG-fMRI for detecting the ictal onset zone at different statistical thresholds. Front. Neurol. 5:131. doi: 10.3389/fneur.2014.00131
Received: 24 December 2013; Accepted: 03 July 2014;
Published online: 17 July 2014.
Edited by:David Vaughan, The Florey Institute of Neuroscience and Mental Health, Australia
Reviewed by:Milan Brazdil, St. Anne University Hospital Brno, Czech Republic
Louis Lemieux, University College London, UK
Copyright: © 2014 Tousseyn, Dupont, Goffin, Sunaert and Van Paesschen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Simon Tousseyn, Laboratory for Epilepsy Research, UZ Leuven and KU Leuven, Herestraat 49, Leuven 3000, Belgium e-mail: email@example.com