Comparison of two automated CT perfusion software packages in patients with ischemic stroke presenting within 24 h of onset

Background We compared the ischemic core and hypoperfused tissue volumes estimated by RAPID and JLK-CTP, a newly developed automated computed tomography perfusion (CTP) analysis package. We also assessed agreement between ischemic core volumes by two software packages against early follow-up infarct volumes on diffusion-weighted images (DWI). Methods This retrospective study analyzed 327 patients admitted to a single stroke center in Korea from January 2021 to May 2023, who underwent CTP scans within 24 h of onset. The concordance correlation coefficient (ρ) and Bland–Altman plots were utilized to compare the volumes of ischemic core and hypoperfused tissue volumes between the software packages. Agreement with early (within 3 h from CTP) follow-up infarct volumes on diffusion-weighted imaging (n = 217) was also evaluated. Results The mean age was 70.7 ± 13.0 and 137 (41.9%) were female. Ischemic core volumes by JLK-CTP and RAPID at the threshold of relative cerebral blood flow (rCBF) < 30% showed excellent agreement (ρ = 0.958 [95% CI, 0.949 to 0.966]). Excellent agreement was also observed for time to a maximum of the residue function (Tmax) > 6 s between JLK-CTP and RAPID (ρ = 0.835 [95% CI, 0.806 to 0.863]). Although early follow-up infarct volume showed substantial agreement in both packages (JLK-CTP, ρ = 0.751 and RAPID, ρ = 0.632), ischemic core volumes at the threshold of rCBF <30% tended to overestimate ischemic core volumes. Conclusion JLK-CTP and RAPID demonstrated remarkable concordance in estimating the volumes of the ischemic core and hypoperfused area based on CTP within 24 h from onset.


Introduction
In the rapidly evolving field of neuroimaging, the analysis of computed tomography perfusion (CTP) scans has become a cornerstone in the diagnosis and management of acute ischemic stroke (Abdalkader et al., 2023).CTP scans have played a major role in expanding the time window for endovascular treatment (EVT) in patients with ischemic stroke (Albers et al., 2018;Jovin et al., 2018).Using perfusion scans, clinical trials that compare EVT with medical treatment for patients with anterior circulation large vessel occlusion beyond 6 h have shown the clinical benefit of EVT in the extended time window (Albers et al., 2018;Jovin et al., 2018).Furthermore, our group has demonstrated that even beyond 24 h, selected individuals identified through perfusion imaging can benefit from EVT (Kim et al., 2020).In this context, it is of the utmost importance to precisely quantify the perfusion parameters to make an informed decision for acute ischemic stroke patients (Sarraj et al., 2020).
To streamline the analysis of perfusion and minimize variations among different observers, various commercial CTP software solutions have been introduced (Lim et al., 2023).These solutions automatically identify the ischemic core and penumbral regions.Nonetheless, there has been considerable inconsistency in the parameters of CTP and the quantitative benchmarks set for delineating the ischemic core and penumbra (Yedavalli et al., 2023).Gradually, relative CBF (rCBF) has emerged as the preferred metric for determining the ischemic core (Ballout et al., 2023).A threshold for the time to peak of the residue function (T max ) exceeding 6 s has been identified as a reliable predictor for tissues at risk of infarction if there is no reperfusion (Fainardi et al., 2022).However, the extent to which different software solutions can be used interchangeably, especially in terms of their clinical significance for planning treatment and estimating prognosis, remains uncertain.Previous research has indicated significant discrepancies in the calculated volumes of the ischemic core across different software, leading to inconsistent predictions of final infarct volume after EVT (Koopman et al., 2019;Yedavalli et al., 2023).
Previous research has shown that the epidemiology of stroke, encompassing its frequency, contributing factors, and death rates, differs markedly across ethnic groups (Venketasubramanian et al., 2017;Kim et al., 2018;Kang et al., 2024).For example, in comparison to North Americans and Europeans, the incidence of intracranial arterial disease is comparatively higher among Asians and Black individuals (Kim et al., 2018).One may postulate that ischemic core thresholds derived from studies on Caucasian populations may lead to an overestimation of ischemic core volumes in Asian patients due to the pre-stroke development of leptomeningeal collaterals after chronic intracranial stenosis (Sacco et al., 1995;Qiao et al., 2017).Additionally, disparities in ischemic core volume may be influenced by various comorbidities, such as hypertension and diabetes, as well as delays in both seeking and receiving timely stroke treatment.
In this study, we compared the volumes of ischemic core and hypoperfused tissue volumes estimated by RAPID (iSchemaView Inc., Menlo Park, CA) and JLK-CTP (JLK Inc., Seoul, Korea).In addition, we aimed to evaluate the agreement of patient selection eligible for EVT between the two packages.

Study design and study population
In this retrospective study using prospectively collected data, we included patients who were admitted to Seoul National University Bundang Hospital between January 2021 and May 2023 and underwent CTP scans within 24 h of symptom onset.We excluded patients with (1) severe motion artifacts on CTP or poor contrast bolus arrival and (2) failed automated perfusion calculation by RAPID.A subset of patients, who had available follow-up diffusionweighted images (DWI) taken within 3 h of the CTP, was used to compare the baseline ischemic core volumes predicted by the CTP with the early follow-up infarct volumes.The study protocol was approved by the institutional review board of Seoul National University Bundang hospital [IRB# B-1710-429-102], and patients or their legal representatives provided a written informed consent.

Clinical data collection
Using a standardized protocol (Kim et al., 2015), we prospectively collected demographic data, prior medication history, and the presence of vascular risk factors including hypertension, diabetes mellitus, hyperlipidemia, coronary artery disease, atrial fibrillation, and smoking history.Stroke subtypes were determined by an experienced vascular neurologist, using a validated MRI-based classification system built on the TOAST (Trial of ORG 10172 in Acute Stroke Treatment) criteria (Ko et al., 2014).

Imaging and image reconstruction
All CTP scans were performed using a 256-Slice CT scanner (Brilliance iCT 256, Philips Healthcare, Best, the Netherlands).The imaging parameters for CTP scans were as follows: 80 kVp, 150 mAs, Beam collimation 6 × 1.25 mm, rotation time 0.45 s.In the arterial phase scan, 30 cycles were captured every 2 s, resulting in a total duration of 60 s.Following a delay of 9 s, the delayed phase scan acquires 2 cycles at intervals of 10 s, amounting to a total duration of 20 s.A total of 50 mL of iodinated contrast agent (Iomeprol 400; Bracco Imaging, Milan, Italy) followed by 30 mL of saline flush was administered intravenously at a rate of 5 mL/s.The image matrix was 512 × 512 and the images were reconstructed with the slice thickness of 10 mm and the increment of 10 mm covering 8 cm in z-axis (Biesbroek et al., 2013).

Automated software analyzing CTP
JLK-CTP is a fully automated CTP software package to visualize and quantify ischemic core and hypoperfused tissue, which relies on a block-circulant singular value decomposition method (Kudo et al., 2010).We compared the volumes of ischemic core and hypoperfused tissue volumes estimated by RAPID and JLK-CTP, both of which are automated CTP analysis packages using a delay-insensitive algorithm.A default setting of rCBF <30% was used for defining the ischemic core.For assessing of hypoperfused tissue volume, we used the default Frontiers in Neuroscience 03 frontiersin.orgsetting of both packages (T max > 6 s).Both software packages carry out automated registration, segmentation, and motion correction, employing a delay-insensitive method, and conducting post-processing (Figure 1).The RAPID software package has been routinely used for CTP in our hospital setting.Restored CTP source images were analyzed with the research version of JLK-CTP within the hospital for this study.

Early follow-up infarct volume measurement
We collected DWI (3.0 Tesla, Ingenia CX, Phillips Medical Systems [n = 192] or MAGNETOM Vida, Siemens Healthcare system [n = 27]) taken within 3 h of the CTP scans.Areas of infarction were automatically segmented using a validated deep learning algorithm (JLK-DWI, JLK Inc., Seoul, Korea) (Ryu et al., 2023).Automatically segmented infarct areas underwent visual inspection by a stroke neurologist (W-SR) and were manually corrected as necessary.

Statistical analysis
Data were presented as mean ± standard deviation, median (interquartile range [IQR]), and number (percentage), as appropriate.
To compare the volumes of ischemic core and hypoperfused tissue using two software packages, we utilized the concordance correlation coefficient (ρ) with 95% confidence intervals (CIs) and further investigated the data using reduced major axis regression (Lin, 1989).The magnitude of agreement was classified as follows: values from 0.0 to 0.2 indicating poor agreement; 0.21 to 0.40 indicating fair agreement; 0.41 to 0.60 indicating moderate agreement; 0.61 to 0.80 indicating substantial agreement; and 0.81 to 1.0 indicating excellent agreement.Additionally, Bland-Altman plots were used to assess the agreement of ischemic core volumes and hypoperfused areas as determined by the two packages.For the comparison between follow-up infarct volumes and ischemic core volumes analyzed by the two packages, both concordance correlation coefficients and Bland-Altman plots were employed.In the subgroup analysis, concordance correlation coefficients and Bland-Altman plots were utilized after stratifying patients by EVT and early infarct volume, arbitrarily divided at 20 mL.The agreement between ischemic core volume and follow-up infarct volume, categorized according to the DAWN (DWI or CTP Assessment with Clinical Mismatch in the Triage of Wake-Up and Late Presenting Strokes Undergoing Neurointervention with Trevo) clinical trial's criteria (Jovin et al., 2018), was assessed using Cohen's kappa.A two-tailed p value of <0.05 was considered statistically significant.All statistical analyses were performed using STATA (version 16.0, StataCorp LP, College Station, TX).

Baseline characteristics of the study population
During the study period, a total of 2,544 patients were diagnosed with ischemic stroke, and 657 patients underwent a CTP scan.We excluded 315 patients who received CTP more than 24 h after their last known well and 15 patients who had severe motion artifacts or whose automated perfusion calculations failed using RAPID.Among the 327 patients included in the analysis, the mean age was 70.7 (SD 13.0), and 41.9% were women (Table 1).The median of the initial National Institute of Health stroke scale score was 9 (IQR, 4 to 17).The most common stroke etiology was identified as cardioembolism.Of the 205 (62.7%) patients who received revascularization therapy, 83 (25.3%) underwent intravenous thrombolysis alone, 59 (18.0%) underwent EVT alone, and 63 (19.3%) received combined therapy.The median interval between the time last known well to CTP scan was 192 min (IQR, 101 to 395 min).The median interval between CTP and DWI was 41 min (IQR, 27 to 100 min).

Concordance correlation analysis of the volumes of ischemic core and hypoperfused tissue by RAPID vs. JLK-CTP
The mean difference between ischemic core volumes calculated by RAPID and JLK-CTP was −0.54 mL (95% CI, −1.72 to 0.64 mL; p = 0.85).There was excellent agreement in between JLK-CTP and RAPID (ρ = 0.958 [95% CI, 0.949 to 0.966]; Figure 2A).In the Bland-Altman plot, the limits of agreement were −20.74 and 21.83 mL (Figure 2B).We observed excellent agreement between JLK-CTP and RAPID (ρ = 0.942 [95% CI, 0.916 to 0.960]) with the mean difference of −0.94 mL (95% CI, −4.90 to 3.00 mL), when we restricted the analysis to patients whose ischemic core volume was not zero by both JLK-CTP and RAPID (n = 90; Supplementary Figure S1).
The mean difference between hypoperfused tissue volumes calculated by JLK-CTP and RAPID was 13.31 mL (95% CI, 8.22 to 18.39; p = 0.06).There was excellent agreement between JLK-CTP and RAPID (ρ = 0.855 [95% CI, 0.828 to 0.877]; Figure 2C).In the Bland-Altman plot, the limits of agreement were −78.22 and 104.83 mL (Figure 2D).When the analysis was confined to patients receiving EVT (n = 122), there was excellent agreement in ischemic core volumes and substantial agreement in hypoperfused tissue volumes between JLK-CTP and RAPID (Supplementary Figure S2).

Comparison of ischemic core volumes calculated by software tools and follow-up infarct volumes on diffusion-weighted images
In patients with available early follow-up DWI (n = 218), there was substantial agreement between follow-up infarct volumes and ischemic core volumes as determined by JLK-CTP (ρ = 0.753 [95% CI, 0.699 to 0.800]; Figure 3A) and RAPID (ρ = 0.736 [95% CI, 0.685 to 0.780]; Figure 3C) at the default rCBF threshold of <30%.The limits of agreement for the volumes of the ischemic core and the follow-up infarct volume were comparable between the two packages (Figures 3B,D).Nevertheless, at the default rCBF threshold of <30%, both packages tended to overestimate infarct core volumes, as indicated by the slope of the reduced major axis being under one.Furthermore, in patients with visible infarcts on the follow-up DWI (n = 187), the ischemic core volume determined by RAPID was zero in 123 (65.8%) cases, whereas it was zero in 104 (55.6%; p = 0.04) cases as determined by JLK-CTP (see Table 2).

Mismatch volume analysis and application of clinical trial's criteria
The median mismatch volumes determined by JLK-CTP and RAPID were 23.26 (IQR 0 to 85.33) and 23 (0 to 101), respectively, and there was substantial agreement between the mismatch volumes determined by JLK-CTP and RAPID (ρ = 0.774; 95% CI 0.735 to 0.808; Figure 4A).The mean difference in mismatch volume between the two software tools was 13.85 mL (95% CI, 8.80 to 18.90 mL; p = 0.01; Figure 4B).When considering early infarct volume as a standard reference and categorizing ischemic core volumes by JLK-CTP and RAPID according to the DAWN trial's criteria, similar Cohen's kappa values were observed for JLK-CTP (0.55) and RAPID (0.51).However, in the medium-sized ischemic core volume groups (20-30 mL and 30-50 mL), the ischemic core volumes determined by both JLK-CTP and RAPID exhibited poor agreement with the categorization by early infarct volumes.

Discussion
We observed a comparable estimation of the ischemic core and hypoperfused area volume between RAPID and JLK-CTP, a newly developed CTP analysis package utilizing a delay-insensitive algorithm, based on 327 ischemic stroke patients who had CTP scans within 24 h from their last known well.Furthermore, both JLK-CTP and RAPID demonstrated a comparable correlation of ischemic core volumes with early DWI infarct volumes.Additionally, we observed that the default threshold of rCBF <30% tends to overestimate ischemic core volumes in comparison with early follow-up infarct volumes on DWI.
We observed an excellent agreement between ischemic core volumes calculated by RAPID and JLK-CTP.This correlation remained high even after excluding patients with ischemic core volumes of zero as determined by either JLK-CTP or RAPID.Additionally, hypoperfused tissue volumes identified using the default settings of both software tools also demonstrated substantial agreement.Consequently, mismatch volumes calculated by both software tools exhibited a substantial agreement.When the image criteria from the DAWN trial was applied (Jovin et al., 2018), ischemic core volumes calculated by the two packages showed overall good agreement with early follow-up infarct volumes.However, for medium-sized ischemic core volumes (20-30 mL and 30-50 mL), the estimated parameters differed substantially compared with early infarct volumes on DWI.These findings suggest that ischemic core volume in CTP scans, analyzed by automated software packages, should be interpreted with caution in patients with medium-sized ischemic core volumes.DWI is the method of choice when it comes to assessing ischemic core (van Everdingen et al., 1998).Our results showed that automated CTP analysis packages have a tendency to overestimate ischemic core volume, consistent with previous studies (Boned et al., 2017;Garcia-Tornel et al., 2021;Sarraj et al., 2022).Compared to JLK-CTP, RAPID was more likely to estimate the ischemic core as zero even when the ischemic core exceeded 20 mL.The discrepancy may result from differences in preprocessing images to correct patient motion and post-processing to reduce noise in both rCBF and T max maps.Moreover, our results contrast with a prior study demonstrating that CTP analysis packages tend to underestimate follow-up infarct volume on various imaging modalities such as DWI, fluid-attenuated inversion recovery image, and non-contrast CT (Pisani et al., 2023).This discrepancy may result from different study populations: consecutive patients presenting at the hospital within 24 h of onset versus patients undergoing mechanical thrombectomy.In addition, shorter time intervals from the last known well to CTP scans (median 168 vs. 402 min) and from CTP to follow-up images (median 0.7 vs. 18.7 h) in our study may have led to smaller follow-up infarct volumes on DWI, subsequently leading to the overestimation of ischemic core volume in CTP scans by software tools.Given that DWI is the gold standard for measuring ischemic core and CTP scans are screening tools to select patients who may benefit from intervention, a more stringent rCBF threshold may be required to accurately estimate infarct core, as shown in our analysis employing an rCBF threshold of 26%, which in turn ultimately provides opportunities for more patients with ischemic stroke to regain functional independence.The interpretation of our findings must be contextualized within the inherent limitations of this research.Conducted as a retrospective analysis at a single stroke center, our study's observational nature is susceptible to the influence of unmeasured confounding variables.The software utilized for analysis reflects the versions available at the time of the study, acknowledging that subsequent updates and modifications could affect the applicability of our results.Moreover, the management of patients included in this study was guided by the outcomes from the RAPID software, introducing a potential source of bias.Furthermore, we utilized follow-up infarct volumes that were automatically segmented by validated software.This approach may limit comparing our results with prior studies that utilized a threshold for apparent diffusion coefficients of 620 × 10 −6 mm 2 /s.

Conclusion
In conclusion, the perfusion parameters from the JLK-CTP and RAPID were documented to be substantially consistent.In addition, ischemic core volumes calculated by JLK-CTP and RAPID at the default rCBF threshold of <30% also showed excellent agreement with early follow-up infarct volumes on DWI.However, CTP parameters should be interpreted with caution when making recanalization treatment decisions.

FIGURE 1 A
FIGURE 1 A representative case employing JLK-CTP and RAPID.(A) Color map produced by RAPID, indicating an ischemic core volume (rCBF < 30%) of 10 mL and total hypoperfused tissue (T max > 6 s) of 168 mL.(B) Color map generated by JLK-CTP, showing an ischemic core volume (rCBF < 30%) of 8.8 mL and total hypoperfused tissue (T max > 6 s) of 159.1 mL.(C) Early follow-up DWI taken 110 min after CTP, following endovascular treatment with TICI 2a, reveals patchy areas of abnormally restricted diffusion in territories of the right middle cerebral artery (Top).RAPID DWI, based on the threshold of the apparent diffusion coefficient, estimated a total infarct volume of 3 mL (Middle).JLK-DWI, utilizing a deep learning algorithm, estimated a total infarct volume of 14.61 mL (Bottom).CTP, computed tomography perfusion; rCBF, relative cerebral blood flow; T max , time to maximum of the residue function; DWI, diffusion-weighted image.

FIGURE 2
FIGURE 2Comparison of ischemic core volumes and hypoperfused tissue volumes by JLK-CTP and RAPID.(A) Scatter plot illustrating ischemic core volumes as determined by JLK-CTP and RAPID, with the slope and intercept of the reduced major axis being 1.097 and −2.067, respectively.(B) Bland-Altman plot for the analysis of agreement in ischemic core volumes.(C) Scatter plot illustrating hypoperfused tissue volumes as determined by JLK-CTP and RAPID, with the slope and intercept of the reduced major axis being 1.337 and −8.350, respectively.(D) Bland-Altman plot for the analysis of agreement in hypoperfused tissue volumes.The green dotted line represents the line of perfect concordance, while the blue line denotes the reduced major axis.For (B,D), the blue solid line and the red dotted lines represent the mean difference and the limits of agreement between JLK-CTP and RAPID, respectively.

FIGURE 3
FIGURE 3Comparison of early follow-up infarct volume on diffusion-weighted image and ischemic core volumes by JLK-CTP and RAPID.(A) Scatter plot illustrating follow-up infarct volumes and ischemic core volumes as determined by JLK-CTP, with a concordance correlation coefficient of 0.753, and the slope and intercept of the reduced major axis being 0.706 and −0.598, respectively.(B) Bland-Altman plot for the analysis of agreement in between follow-up infarct volumes and ischemic core volumes by JLK-CTP.(C) Scatter plot illustrating follow-up infarct volumes and ischemic core volumes as determined by RAPID, with a concordance correlation coefficient of 0.736, and the slope and intercept of the reduced major axis being 0.633 and 0.804, respectively.(D) Bland-Altman plot for the analysis of agreement in between follow-up infarct volumes and ischemic core volumes by RAPID.The green dotted line represents the line of perfect concordance, while the blue line denotes the reduced major axis.For (B,D), the blue solid line and the red dotted lines represent the mean difference and the limits of agreement between follow-up infarct volume and ischemic core volumes calculated by JLK-CTP (B) and RAPID (D).