# REPERFUSION THERAPY FOR ACUTE ISCHEMIC STROKE

EDITED BY : Nishant K. Mishra and Bruce Campbell PUBLISHED IN : Frontiers in Neurology

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-243-5 DOI 10.3389/978-2-88963-243-5

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# REPERFUSION THERAPY FOR ACUTE ISCHEMIC STROKE

Topic Editors:

Nishant K. Mishra, Icahn School of Medicine at Mount Sinai, United States Bruce Campbell, The University of Melbourne, Australia

Citation: Mishra, N. K., Campbell, B., eds. (2019). Reperfusion Therapy for Acute Ischemic Stroke. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-243-5

# Table of Contents


Anke Wouters, Søren Christensen, Matus Straka, Michael Mlynash, John Liggins, Roland Bammer, Vincent Thijs, Robin Lemmens, Gregory W. Albers and Maarten G. Lansberg

*14 Functional Assessment for Acute Stroke Trials: Properties, Analysis, and Application*

Martin Taylor-Rowan, Alastair Wilson, Jesse Dawson and Terence J. Quinn

*24 Efficacy of Novel Carbon Nanoparticle Antioxidant Therapy in a Severe Model of Reversible Middle Cerebral Artery Stroke in Acutely Hyperglycemic Rats*

Roderic H. Fabian, Paul J. Derry, Harriett Charmaine Rea, William V. Dalmeida, Lizanne G. Nilewski, William K. A. Sikkema, Pitchaiah Mandava, Ah-Lim Tsai, Kimberly Mendoza, Vladimir Berka, James M. Tour and Thomas A. Kent


Mark R. Etherton, Andrew D. Barreto, Lee H. Schwamm and Ona Wu


Richard McKinley, Fan Hung, Roland Wiest, David S. Liebeskind and Fabien Scalzo

*90 Thinking About the Future: A Review of Prognostic Scales Used in Acute Stroke*

Bogna A. Drozdowska, Sarjit Singh and Terence J. Quinn

# Editorial: Reperfusion Therapy for Acute Ischemic Stroke

Nishant K. Mishra<sup>1</sup> \* and Bruce C. V. Campbell <sup>2</sup>

*<sup>1</sup> Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, NY, United States, <sup>2</sup> Department of Medicine and Neurology, Melbourne Brain Centre at the Royal Melbourne Hospital, University of Melbourne, Parkville, VIC, Australia*

Keywords: stroke, mismatch, endovascular, imaging, alteplase

**Editorial on the Research Topic**

#### **Reperfusion Therapy for Acute Ischaemic Stroke**

Endovascular thrombectomy to recanalize large vessel occlusions in ischemic stroke patients is now proven to improve clinical outcomes in randomized trials (1–6). This resulted in significant excitement in the field because, until these trials, intravenous (IV) alteplase was the only proven therapy available to treat these patients (7–9). Also a little prior to the positive endovascular trials, the field was subdued because the major promising endovascular trials, IMS III and MR RESCUE, had failed to show improved outcomes from endovascular therapy (10, 11). These trials were believed to have failed because of the long delay between symptom onset and treatment, inadequate patient selection, lower recanalization rates, and the use of older generation devices. Stroke centers across the globe are now incorporating lessons learnt from the recent positive trials (1–6) into restructuring the acute stroke pathway to expeditiously transfer eligible patients to the neurointerventionist (12). We invited manuscripts (Drozdowska et al.; Etherton et al.; Fabian et al.; Fandler et al.; McKinley et al.; Taylor-Rowanet al.; van de Graaf et al.; Wouters et al.; Zhou et al.) to evaluate the current state of knowledge about reperfusion therapy with the goal of educating our readership about the patient selection paradigms, challenges in clinical trial design, and identifying future directions.

We now know that the benefit of endovascular recanalization in ischemic stroke patients with a target perfusion mismatch profile is independent of the time since symptom onset (13–16). However, in 2010, a meta-analysis of trials that included patients using perfusion mismatch for delayed window reperfusion therapy using desmoteplase and alteplase failed to demonstrate utility of the mismatch paradigm (17). An important limitation noted was that the mismatch criteria used in these studies used visual assessment or lenient thresholds (time-to-maximum [Tmax] > 2 s) for the perfusion lesion which included mildly hypoperfused tissue not at risk of infarction (benign oligemia) (17). Subsequent positive studies have used a Tmax > 6 s delay threshold (13– 15, 17). Perfusion imaging data require complex analyses which can now be rapidly performed with automated post processing software e.g., RAPID (iSchemaView, Mountain View, CA) which was used in several pivotal trials to standardize recruitment criteria across different hardware platforms (18). Deconvolution algorithms rely on automated selection of an arterial input function which can lead to scan re-scan variation in Tmax and other perfusion parameters. There is therefore interest in using alternative measures like relative time-to-peak (rTTP) measures that normalize delay using a much larger area than the few voxels chosen for an arterial input function [Wouters et al.; (19)]. Wouters et al. contribute a manuscript to this collection reporting that a Tmax >6 s corresponds to rTTP >4.5 s and Tmax >10 s corresponds to rTTP >9.5 s. More importantly,

Edited and reviewed by: *Jean-Claude Baron, University of Cambridge, United Kingdom*

> \*Correspondence: *Nishant K. Mishra nishmishra@gmail.com*

#### Specialty section:

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

Received: *28 September 2019* Accepted: *11 October 2019* Published: *29 October 2019*

#### Citation:

*Mishra NK and Campbell BCV (2019) Editorial: Reperfusion Therapy for Acute Ischemic Stroke. Front. Neurol. 10:1139. doi: 10.3389/fneur.2019.01139* the authors highlight the stability of rTTP in a scan-rescan scenario (Wouters et al.). McKinley et al. also acknowledge that deconvolution techniques are highly susceptible to noise and artifacts; they list measures to overcome these issues e.g., by adoption of a smoother residue function; and they report machine learning methodologies to identify the tissue-at-risk. The authors suggest that the best algorithms are those which are based on neural networks and random forests (McKinley et al.). Finally, Etherton et al. summarize the neuroimaging paradigms and the clinical trials that tested reperfusion therapy in stroke patients with unknown time of symptom onset. Perfusion mismatch, DWI-FLAIR mismatch, and other mismatch paradigms are discussed, and prognostic value of some of these measures discussed (Etherton et al.). The authors propose that more patients can be offered reperfusion therapy by refining the approach to identify additional populations of stroke patients with unknown onset who may benefit (Etherton et al.). The target of therapy—the tissue-at-risk (penumbra) is better defined now and therefore there is greater opportunity to test preclinical research findings in the contemporary standard of care. To this end, we include Fabian et al. and Zhou et al. in this collection who report their data about methods to enhance ischemic preconditioning and to reduce oxidative stress respectively in acute stroke. Also, recanalization is not equivalent to reperfusion, and the goal should be to succeed with both (20). A large proportion of patients with successful recanalization do not show improved outcomes, possibly because of poor reperfusion, although the pre-treatment extent of injury is also a major contributor to poor outcome despite reperfusion. Periprocedural antithrombotic agents have a potential to improve reperfusion, but also have increased risk of intracranial bleeding. van de Graaf et al. conducted a systematic review to evaluate the use of periprocedural antithrombotic use in patients undergoing acute endovascular therapy. They found symptomatic ICH rate of 6–17% in patients with antiplatelet use and 5–12% in patients receiving heparin (van de Graaf et al.).

Endovascular stroke trials showed benefit in patients with proven large vessel occlusion and/or presence of mismatch. The investigators selected a population of patients with homogeneous clinical profile because the goal was to have a trial success (1, 5). In the real world, however, we often encounter patients who fall outside the randomized trial populations, and require

#### REFERENCES


individualized treatment decisions. Fandler et al. present one example as a case report highlighting excellent outcomes from endovascular therapy in a patient with ischemic stroke who had a stroke recurrence within 9 days of the first thrombectomy. The subsequent stroke occurred in exactly the same vascular territory, was associated with significant mismatch, and continued to show irregular shaped ulcerated plaque. Fandler et al. offered endovascular treatment with TICI 3 recanalization with excellent outcomes on follow up. She also received carotid thromboendarterectomy for the ulcerated plaque Fandler et al.. Patients treated off label are a highly heterogeneous group and therefore difficult to assess in randomized clinical trials. There is a need to collate these data in a registry setting.

Suboptimal trial design may well have been responsible for some of the neutral stroke trials reported in the last two decades. Drozdowska et al. and Taylor-Rowan et al. touch upon some of the relevant issues. Drozdowska et al. reviewed prognostic scales used in acute stroke and inform the readers about the validity of commonly used prognostic scores. Outcome prediction tools guide a stroke physician's discussion with patient's family in regard to possible outcomes. It also offers a clinical trialist a tool to stratify the patient population enrolled in studies to investigate a differential response to the treatment. Drozdowska et al., in their review, recommend the need for studies to investigate clinical usefulness in existing scales. Outcome measures used in complex trials like stroke trials should be valid, reliable, and responsive endpoint; and their analyses should be robust and acceptable to the regulatory bodies. In their review article, Taylor-Rowan et al. inform issues relevant to the selection of outcome measures and suggest robust analytics.

This Research Topic thus informs the current state of knowledge with respect to recanalization strategies and also stimulates readers to identify important research questions and tackle them through a robust research methodology.

# AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication. Opinions are personal and do not reflect that of the regulatory bodies or the authors' employers.


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Mishra and Campbell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Anke Wouters1,2,3\*, Søren Christensen4 , Matus Straka4 , Michael Mlynash4 , John Liggins4 , Roland Bammer <sup>4</sup> , Vincent Thijs5 , Robin Lemmens1,2,3, Gregory W. Albers4 and Maarten G. Lansberg4*

*1Department of Neurosciences, Experimental Neurology, Leuven Research Institute for Neuroscience and Disease (LIND), KU Leuven, Leuven, Belgium, 2 Laboratory of Neurobiology, Center for Brain and Disease Research, VIB, Leuven, Belgium, 3Department of Neurology, University Hospitals Leuven, Leuven, Belgium, 4Stanford Stroke Center, Stanford University Medical Center, Palo Alto, CA, United States, 5 Florey Institute of Neuroscience and Mental Health, Heidelberg, VIC, Australia*

Background and purpose: The perfusion-weighted imaging (PWI)/diffusion-weighted imaging (DWI) mismatch profile is used to select patients for endovascular treatment. A PWI map of Tmax is commonly used to identify tissue with critical hypoperfusion. A time to peak (TTP) map reflects similar hemodynamic properties with the added benefit that it does not require arterial input function (AIF) selection and deconvolution. We aimed to determine if TTP could substitute Tmax for mismatch categorization.

#### *Edited by:*

*Bruce Campbell, University of Melbourne, Australia*

#### *Reviewed by:*

*Nawaf Yassi, University of Melbourne, Australia Mark William Parsons, University of Newcastle, Australia*

*\*Correspondence: Anke Wouters anke.wouters@kuleuven.vib.be*

#### *Specialty section:*

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

*Received: 31 July 2017 Accepted: 26 September 2017 Published: 13 October 2017*

#### *Citation:*

*Wouters A, Christensen S, Straka M, Mlynash M, Liggins J, Bammer R, Thijs V, Lemmens R, Albers GW and Lansberg MG (2017) A Comparison of Relative Time to Peak and Tmax for Mismatch-Based Patient Selection. Front. Neurol. 8:539. doi: 10.3389/fneur.2017.00539*

Methods: Imaging data of the DEFUSE 2 trial were reprocessed to generate relative TTP (rTTP) maps. We identified the rTTP threshold that yielded lesion volumes comparable to Tmax > 6 s and assessed the effect of reperfusion according to mismatch status, determined based on Tmax and rTTP volumes.

results: Among 102 included cases, the Tmax > 6 s lesion volumes corresponded most closely with rTTP > 4.5 s lesion volumes: median absolute difference 6.9 mL (IQR: 2.3–13.0). There was 94% agreement in mismatch classification between Tmax and rTTP-based criteria. When mismatch was assessed by Tmax criteria, the odds ratio (OR) for favorable clinical response associated with reperfusion was 7.4 (95% CI 2.3–24.1) in patients with mismatch vs. 0.4 (95% CI 0.1–2.6) in patients without mismatch. When mismatch was assessed with rTTP criteria, these ORs were 7.2 (95% CI 2.3–22.2) and 0.3 (95% CI 0.1–2.2), respectively.

conclusion: rTTP yields lesion volumes that are comparable to Tmax and reliably identifies the PWI/DWI mismatch profile. Since rTTP is void of the problems associated with AIF selection, it is a suitable substitute for Tmax that could improve the robustness and reproducibility of mismatch classification in acute stroke.

Keywords: ischemic stroke, magnetic resonance imaging, perfusion imaging, thrombectomy, treatment

# INTRODUCTION

The combination of MRI diffusion-weighted imaging (DWI) and perfusion-weighted imaging (PWI) maps is used to assess PWI/DWI mismatch, which provides an estimate of the volume of penumbral tissue and has shown promise in identifying patients with a favorable response to reperfusion (1, 2). There is, however, variability between studies in the assessment of the PWI/DWI mismatch. One area of variability is the type of PWI map used to identify critically hypoperfused

**7**

tissue. The Tmax (time to the maximum of the residue function) map has gained popularity in recent endovascular stroke trials. Prior studies have shown that a Tmax delay of >6 s is a good predictor of critically hypoperfused tissue that is destined to infarction in the absence of timely reperfusion (2–6). The Tmax perfusion parameter primarily reflects the bolus delay between the site of the arterial input function (AIF) and the tissue (7). This delay sensitivity seems important, as Tmax has outperformed delay-corrected perfusion parameters such as cerebral blood flow (CBF) and mean transit time for identifying critically hypoperfused tissue (8–10). A drawback of Tmax is that calculation of this perfusion metric requires selection of an AIF (for deconvolution) and that the nature of the deconvolution algorithm renders the Tmax perfusion maps very sensitive to even minor changes in the shape of the AIF. Within patient variability in the AIF is unavoidable because the AIF is obtained by measuring the MRI signal in a few voxels in a main feeding artery (e.g., the middle cerebral artery) on the source perfusion images; a subjective process that results in profound variability in the shape of the AIF depending on which voxels are chosen. This in turn, causes variability in the Tmax perfusion maps and the Tmax lesion volumes. It also makes the Tmax map prone to errors resulting from imaging artifacts that perturb the AIF (11).

Time to peak (TTP) is a perfusion parameter that theoretically could be superior to Tmax for assessment of critically hypoperfused tissue because it does not require deconvolution and, therefore, is not dependent on an AIF. A potential drawback of TTP is that it is not only delay-sensitive (like Tmax) but also sensitive to arterial dispersion and tissue transit time. As a result, TTP reflects a sum of these three effects (12, 13). However, recently, it has been shown that summary parameters such as TTP display much less variability than AIF-based maps when properly normalized (11). Moreover, previous studies point to TTP in the range of 3–5 s as a sensitive and specific parameter to estimate penumbral tissue (8, 9, 14, 15). The most recent one, a combined MRI and PET study, showed that Tmax and TTP were the best predictors of penumbral tissue on PET, defined as CBF < 20 mL/100 g/min (9).

While these recent studies suggest that TTP and Tmax are both predictive of infarction, the impact on patient selection in clinical trials has not yet been compared (8, 9, 11). In this study, we used data from DEFUSE 2, a large prospective study, to compare relative TTP (rTTP) and Tmax in terms of image quality, lesion volumes, patient selection, and response to reperfusion among patients with a PWI/DWI mismatch (2).

#### MATERIALS AND METHODS

Imaging data were obtained from DEFUSE 2 (NCT01327989), a multicenter prospective cohort study. The local institutional review boards from all participating institutions approved the study. All subjects or legal representatives gave their informed consent. Study design and primary results are reported elsewhere (2). All patients eligible for the original study were included. Briefly, an MRI scan with PWI and DWI sequences was obtained on admission to classify patients according to target mismatch status. Definitions of target mismatch, malignant profile, reperfusion, and favorable clinical response at day 30 were adopted from the original DEFUSE 2 study (**Table 1**).

We generated Tmax—and DWI—maps using a research version of the RAPID software (v2.5) with a customized Matlab plug-in for our rTTP calculation (Mathworks, Natick, MA, USA) (16). TTP maps were created by smoothing the tissue concentration time curve of each voxel by a 3-point running average filter followed by a spline interpolation to 0.5 s time resolution. The TTP was then recorded as the time of the peak of this smoothed tissue concentration time curve. Using a manually positioned midline plane in 3D space, we extracted the median TTP of tissue contralateral to the stroke (contralateral to the DWI lesion or, when no DWI lesion was present, the TTP deficit). The TTP map was then normalized by subtraction of the contralateral median TTP from the absolute TTP in each voxel, yielding a map (rTTP). All preprocessing steps, including masking, source image smoothing, motion correction, and segmentation, were identical for rTTP and Tmax.

We compared differences between rTTP and Tmax, including (1) the number of cases that were uninterpretable due to excessive artifacts; (2) the number of cases that required any artifact removal; (3) the mean volume of artifact removal in these cases; (4) correlation and absolute difference between rTTP and Tmax lesion volumes; (5) agreement in target mismatch assessment; and (6) the response to reperfusion for patients with and without target mismatch. For this final analysis, we only included patients in whom rTTP, Tmax, and reperfusion status could be assessed.

To compare rTTP and Tmax, we first determined the rTTP thresholds that yielded lesion volumes, which corresponded most closely with the lesion volumes obtained with the Tmax thresholds used in the DEFUSE 2 trial (>6 s for critically hypoperfused tissue and >10 s for severely hypoperfused tissue). These rTTP thresholds were defined as the values at which the median difference between rTPP and Tmax lesion volumes was closest to 0. This optimization analysis was performed after imaging artifacts had been manually removed. All subsequent analyses were based on these optimal rTTP thresholds.


#### Statistical Analyses

Chi square test was used to compare categorical variables and Mann–Whitney *U* for continuous variables. Paired volumetric data was compared with the non-parametric Wilcoxon Signed Rank test. Cohen's Kappa (κ) was calculated to express the degree of agreement between rTTP and Tmax for target mismatch classification. The association between reperfusion and favorable clinical response in patients with or without the target mismatch profile was compared with a multivariate logistic regression analysis with favorable clinical response on day 30 as the dependent variable. Explanatory variables were the DWIvolume (log transformed), age, target mismatch, reperfusion status, and an interaction term between target mismatch and reperfusion. Results were considered statistically significant at a *p*-value < 0.05. Analyses were done using R software (R Development Core Team (2008). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria).

#### RESULTS

In this study, we reanalyzed MRI scans of 110 patients from DEFUSE 2 who received endovascular therapy. Excessive imaging artifacts rendered both Tmax and rTTP maps uninterpretable in six patients (5%). The reasons for these artifacts were severe patient motion in four and a failed contrast bolus injection in two patients. In an additional two patients, the Tmax map was uninterpretable due to poor AIF selection, while the rTTP map was of good quality.

Among the 102 patients with interpretable Tmax and rTTP perfusion maps, minor artifacts were manually removed from the Tmax maps in 24 patients (24%) and from the rTTP maps in 18 patients (18%; *p* for difference = 0.3) (**Figure 1**). Thirteen of these patients had artifacts on both Tmax and rTTP maps, 11 patients on Tmax alone, and five patients on rTTP alone. The mean clean-up volume was 14.2 mL for rTTP and 17.2 mL for Tmax (*p* = 0.7).

Following artifact removal, Tmax > 6 s corresponded best with rTTP > 4.5 s (median difference between Tmax and rTTP lesion volumes 0.1 mL, *p* = ns) and Tmax > 10 s corresponded best with rTTP > 9.5 s (median difference −0.3 mL, *p* = ns) (**Table 2**; **Figure 2**). Pearson correlation coefficients (R) between Tmax and rTTP volumes were 0.94 for both critical and severe hypoperfusion (**Figure 3**). An example of the close correspondence between the rTTP and Tmax maps is presented in **Figure 1**.

In 96 of the 102 patients (94%, κ = 0.82), there was agreement between target mismatch profiles assessed with Tmax and rTTP maps: 79 of these patients had the target mismatch profile and 17 did not. In six patients, the target mismatch profile classification differed between Tmax and rTTP. Two patients changed from target mismatch on Tmax to "no target mismatch" on rTTP and another four patients changed from "no target mismatch" on Tmax to target mismatch on rTTP. All six classification changes were the result of small differences in the lesion volumes between


Table 2 | Comparison of rTTP and Tmax lesion volumes for different rTTP thresholds.

*All represented data are median (IQR). Differences are calculated as rTTP* − *Tmax, absolute differences as |rTTP*−*Tmax|, and relative difference as |rTTP*−*Tmax|/Tmax. IQR, interquartile range; rTTP, relative time to peak; Tmax, time to the maximum of the residue function.*

Figure 3 | Comparison of Tmax vs. relative TTP (rTTP) lesion volumes. The scatterplots show the correlation between the volume of critical hypoperfusion assessed with a Tmax > 6 s vs. an rTTP > 4.5 s threshold (A) and the volume of severe hypoperfusion assessed with a Tmax > 10 s threshold vs. an rTTP > 9.5 s threshold (B). Identity lines are depicted for both graphs. R = Pearson's correlation coefficient.

Tmax and rTTP (median difference 9.5 mL, IQR −8.05–13.8; see **Table 3** for details).

We analyzed the effect of reperfusion on the 30 day favorable clinical response rate (the primary outcome for the DEFUSE 2 trial) in patients with and without target mismatch. When target mismatch status was assessed with Tmax, the odds ratio (OR) for favorable clinical response after reperfusion was 7.4 (95% CI 2.3–24.1) for patients with target mismatch and 0.4 (95% CI 0.1–2.6) for patients without. When target mismatch status was assessed with rTTP, these ORs were 7.2 (95% CI 2.3–22.2) and 0.3 (95% CI 0.1–2.2), respectively. The difference in ORs between patients with and without TMM was significant for both the Tmax and the rTTP-based analysis (*p* < 0.01).

#### DISCUSSION

In this study, rTTP thresholds of 4.5 and 9.5 s corresponded to the Tmax definitions for critical (Tmax > 6 s) and severe (Tmax > 10 s) hypoperfusion used in prior studies. The rTTP > 4.5 s threshold for critical hypoperfusion falls within the range of thresholds previously determined based on PET imaging (3–5 s), and is in very good concordance with the >4.8 s rTTP threshold determined in a recent study that used PET as the gold standard (9, 14, 15). The rTTP threshold of >9.5 s for severe hypoperfusion is novel since no previous studies have investigated rTTP thresholds that are comparable to Tmax > 10 s. Reanalysis of the primary DEFUSE 2 study results using these rTTP thresholds to identify patients with target mismatch yielded similar results as when Tmax thresholds were used.

In two patients, the Tmax maps were uninterpretable whereas the rTTP maps were of sufficient quality to determine target mismatch status. In these patients, the selection of the AIF failed, underscoring the advantages of rTTP since no AIF selection is required. Although AIF selection can be improved with engineering solutions, AIF selection, whether by humans or software, will remain a subjective choice that renders Tmax maps sensitive to small modifications in how this choice is made. Previous studies have shown that the location where the AIF is measured can highly influence Tmax perfusion volumes (17–19). Further, partial volume effects can lead to erroneous AIF measurements (20). Since calculation of rTTP does not require knowledge of the AIF, automated generation of rTTP maps may enable mismatch profiling when automated Tmax processing fails. Our study demonstrates in a large prospective cohort of stroke patients, this advantage of rTTP is observed in a small percentage (2%) of cases.

Both Tmax and rTTP are generated with respect to reference signals. Tmax is generated with the AIF as reference and rTTP is generated with the median TTP value in the contralesional hemisphere as reference. The rTTP reference is robust because it is the median of many observations. To illustrate this, consider a scan– rescan scenario with no change to the patient's hemodynamics or the injection. In this case, the reference TTP value would be virtually identical between scans given the high number of voxels in which noise will average out. In contrast, the AIF is not robust because it is derived from the signal intensity in just a few (4–5) voxels. In a scan–rescan scenario, even minor changes in head position would result in selection of different voxels for the AIF and standard image noise would result in different signal intensities even if identical voxels were selected (11). Consequently, the reference AIF will vary between scans and hence the Tmax maps and Tmax lesion volumes will vary as well, whereas the rTTP maps will not. An additional advantage of rTTP over Tmax is that the rTTP calculation is more straightforward and, therefore, easier for vendors to implement in an identical fashion across software solutions. Consequently, variability between software solutions in Tmax maps (and thus lesion volumes) will be greater than variability in rTTP maps.

The simplicity of the rTTP calculation comes at a potential cost. rTTP is a parameter, which is influenced by several aspects of the bolus passage, including arterial dispersion, tracer arrival delay, and tissue transit time. In contrast, Tmax is primarily sensitive to tracer arrival delays. It might be counterintuitive that a summary parameter such as rTTP is at least of similar quality compared to Tmax (8). However, studies have shown that inclusion of dispersion may be an advantage when identifying tissue at risk of infarction (8, 10, 11). Thus, the sensitivity of rTTP to multiple aspects of the bolus passage may in fact be a strength, making this parameter well-suited for estimating critically hypoperfused tissue.

Conditions that deserve special mention as they might influence perfusion measures include carotid stenosis and leukoariosis. In the presence of a chronic carotid stenosis, delay-sensitive parameters (such as TTP and Tmax) will overestimate the amount of critically hypoperfused tissue. However, since rTTP and Tmax (calculated using a global AIF) are both delay-sensitive, the effect of carotid stenosis on these perfusion parameters is likely similar. While we lack information about carotid status in our study, prior


*Tmax, time to the maximum of the residue function; rTTP, relative Time to Peak; DWI, diffusion-weighted imaging; PWI, perfusion-weighted imaging.*

studies have shown that the prolongation of bolus delay due to carotid stenosis is not clinically significant. One study in patients with acute MCA occlusions showed a median increase of only 1 s of Tmax delay between patients with and without ipsilateral carotid stenosis (21). Next, while it is well recognized that, in areas of leukoariosis, CBF is reduced (22, 23), there is likely little to no effect of leukoariosis on delay-sensitive parameters like TTP and Tmax.

It should be noted that although TTP maps have historically often been used to assess mismatch using a qualitative approach, the present analysis is a quantitative thresholding approach. We caution against patient selection using a qualitative review of the perfusion map, as this approach is prone to interrater variability and overestimation of tissue at risk.

Our study is limited to MR perfusion while many centers use CT perfusion to estimate core and penumbra in acute stroke patients. Future studies comparing Tmax to TTP would be required to support the use of TTP for assessment of critically hypoperfused tissue on CTP. Further, TTP is not a suitable parameter for estimating the ischemic core on CTP. The current standard for estimation of the ischemic core on CTP is CBF, and we do not expect that TTP can outperform CBF. Many software packages use an AIF for CBF calculation. Therefore, mismatch assessment on CTP would remain AIF dependent even if one were to substitute Tmax with TTP. It remains an open question whether non-AIF-dependent CBF techniques, such as the "maximal slope" method used by Siemens, can be as accurate as deconvolution-based CBF for estimating the core.

A limitation of the way we processed rTTP maps is that it required a manual step (to position the midline plane). Since this was a pilot study, no further automatization was pursued, but implementation of a fully automated analysis is easily feasible. Another limitation of this study is the lack of a gold standard to define the optimal rTTP threshold for critical hypoperfusion (tissue destined to go on to infarction in the absence of reperfusion). In this study, we used established Tmax thresholds to "calibrate" rTTP. Future research could use infarcts outlined on late follow-up MRI scans from patients without reperfusion as the gold standard for critical hypoperfusion. This was not possible in DEFUSE 2 due to the limited number of non-reperfused patients. It should also be noted

#### REFERENCES


that the use of late follow-up scans has its own limitations as there is no perfect time or method to accurately define the final infarct volume. For example, the presence of edema will lead to an overestimation of the infarct when the scan is obtained too early, whereas atrophy will lead to an underestimation when the scan is obtained late (24–26). Moreover, second strokes that occur during the follow-up period in the territory adjacent to the primary stroke can also complicate the accurate assessment of the final infarct volume.

In summary, this study, using a large prospective dataset, demonstrates that rTTP and Tmax provide comparable results in terms of lesion volumes and mismatch classification. Lesion volumes determined with rTTP thresholds of 4.5 and 9.5 s correspond closely with volumes obtained with previously identified Tmax thresholds for critical and severe hypoperfusion. Since the rTTP parameter is not AIF dependent and, therefore, not subject to the variability associated with AIF selection, it could serve as a substitute for Tmax that may improve the robustness and reproducibility of mismatch classification in acute stroke.

# ETHICS STATEMENT

The local institutional review boards from all participating institutions approved the study. All subjects or legal representatives gave their informed consent. Study design and primary results are reported elsewhere.

# AUTHOR CONTRIBUTIONS

AW and RL: study concept and design, analysis and interpretation of data, drafting the manuscript. SC, GA, and ML: study concept and design, analysis and interpretation of data, drafting the manuscript, acquisition of data. MS, MM, and RB: study concept and design, revising the manuscript, acquisition of data. JL and VT: study concept and design, analysis and interpretation of data, revising the manuscript.

# FUNDING

The study was funded by a grant from the National Institute for Neurological Disorders and Stroke (NINDS), R01 NS03932505 (GA).


optimal PWI maps and thresholds validated with PET. *J Cereb Blood Flow Metab* (2016) 37(9):3176–83. doi:10.1177/0271678X16685574


**Conflict of Interest Statement:** AW receives a grant from European Union. SC performs consulting work for iSchemaView. RL is senior clinical investigator of FWO Flanders. GA has received consulting fees and expenses from Lundbeck for Steering Committee work and consulting fees from Concentric for serving on a Data Safely and Monitory Board. GA and RB are equity shareholders in iSchemaView and perform consulting work for iSchemaView. The other authors report no conflicts.

The reviewer NY and handling editor declared their shared affiliation.

*Copyright © 2017 Wouters, Christensen, Straka, Mlynash, Liggins, Bammer, Thijs, Lemmens, Albers and Lansberg. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Functional Assessment for Acute Stroke Trials: Properties, Analysis, and Application

#### *Martin Taylor-Rowan\*, Alastair Wilson, Jesse Dawson and Terence J. Quinn*

*Institute of Cardiovascular and Medical Sciences, University of Glasgow, Glasgow, United Kingdom*

A measure of treatment effect is needed to assess the utility of any novel intervention in acute stroke. For a potentially disabling condition such as stroke, outcomes of interest should include some measure of functional recovery. There are many functional outcome assessments that can be used after stroke. In this narrative review, we discuss exemplars of assessments that describe impairment, activity, participation, and quality of life. We will consider the psychometric properties of assessment scales in the context of stroke trials, focusing on validity, reliability, responsiveness, and feasibility. We will consider approaches to the analysis of functional outcome measures, including novel statistical approaches. Finally, we will discuss how advances in audiovisual and information technology could further improve outcome assessment in trials.

Keywords: stroke, outcome, disability, modified Rankin scale, Barthel index, NIHSS

#### *Edited by:*

INTRODUCTION

*Nishant K. Mishra, Tulane University, United States*

#### *Reviewed by:*

*Katharina Stibrant Sunnerhagen, University of Gothenburg, Sweden Pitchaiah Mandava, Michael E. DeBakey VA Medical Center (VHA), United States*

#### *\*Correspondence:*

*Martin Taylor-Rowan m.taylor-rowan.1@research.gla.ac.uk*

#### *Specialty section:*

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

> *Received: 21 December 2017 Accepted: 12 March 2018 Published: 26 March 2018*

#### *Citation:*

*Taylor-Rowan M, Wilson A, Dawson J and Quinn TJ (2018) Functional Assessment for Acute Stroke Trials: Properties, Analysis, and Application. Front. Neurol. 9:191. doi: 10.3389/fneur.2018.00191*

Clinical trials are designed to assess the effect of a novel intervention versus a comparator. An archetypal stroke trial may, for example, assess the effect of endovascular treatment against a control of "usual care." The PICO framework can be used to describe any clinical trial in terms of Population, Intervention, Control, and Outcome. While it is typically the intervention that attracts attention and represents the exciting new chapter in stroke care, we should not forget about the other components of a trial. In particular, outcome assessment in stroke trials is critical and the approach to outcome assessment can be the difference between a positive and neutral trial.

The outcome of any trial should provide some quantifiable measure of the effect of the treatment. Historically, endpoints such as mortality or event recurrence have been used in stroke trials. While useful, particularly for trials of primary and secondary prevention, these "hard clinical endpoints" do not capture the full extent of outcomes for a disabling condition such as stroke. Therefore, assessment of patients' functional ability has been adopted and is now mandated by regulatory authorities for certain stroke trials. Multiple measures of post-stroke functional ability have been developed and many have been used in stroke trials.

In this review, commissioned as part of the themed series on hyper-acute stroke trials, we discuss commonly used functional outcome measures in these trials. We briefly describe their historical purpose before evaluating each in relation to core psychometric properties (see **Table 1**). We then discuss analytical approaches that can be used to assess stroke functional outcomes. Finally, we consider how training, structured assessment, and advances in technology may enhance stroke outcome assessment.

# A FRAMEWORK FOR CONSIDERING OUTCOME ASSESSMENT

There are numerous potential outcome assessments for stroke research (1). For example, even in a relatively niche area such as post-stroke psychological assessment, recent reviews have found

**14**


incorporate measures of acceptability to rater and patient

performed

more outcomes than trials (2). With such a range of potential functional assessment tools, it is useful to have a framework for considering the application of these tests. Our goal is functional outcome assessment, but "functional outcome" is a broad term that encompasses many constructs. A potentially useful way to categorize functional outcomes is to consider the World Health Organization International Classification of Function (WHO-ICF). The WHO-ICF describes function in terms of impairment, activity (formerly disability), participation (formerly handicap) (3), and we could add a fourth level of quality of life (QOL). Stroke assessment scales are available to describe functional outcome at each level of the WHO-ICF (**Figure 1**).

Even within each level of WHO-ICF, there can be many potential assessments to choose from. The science of psychometrics (sometimes called clinimetrics in the applied clinical context) describes properties of assessment scales. The classical properties that are important for a clinical assessment tool are validity, reliability, responsiveness to change, and feasibility/acceptability (4). Depending on the clinical context and population to be studied, some psychometric properties may be more important than others (**Figure 2**).

We will consider three outcome assessment scales that have been frequently used in acute stroke trials, each one has been chosen as an exemplar of a certain level of the original WHO-ICF framework. For each assessment scale, we will describe all four of the classical psychometric properties, using each scale to major on a particular aspect relevant to that scale.

Research into the properties of stroke scales is an evolving field. In this review, we highlight many of the seminal papers that have influenced our understanding of stroke clinimetrics. We recognize that in some instances authors may have used statistical approaches that are not reflective of current best practice. For example, statistical analyses based on parametric assumptions have often been applied to stroke scales that are ordinal or nominal in structure. While some argue that parametric statistics are inappropriate for evaluating stroke scales, it would be wrong to ignore all the available research that has used this approach. We also note that variations in language and translations can potentially affect scale properties. Our discussion will predominantly focus on the original (English language) versions of these tools.

#### IMPAIRMENT: NATIONAL INSTITUTES OF HEALTH STROKE SCALE

#### History and Purpose

The NIHSS was specifically designed for assessment of interventions in clinical trials. Of key intent was that the tool should be employed easily and quickly at the patient bedside to enable practicality of use (5). Rather than measuring function specifically, the NIHSS operates a 15-item ordinal, non-linear, neurological impairment scale covering consciousness, ocular movement, vision, coordination, speech and language, sensory function, upper and lower limb strength, facial muscle function, and hemi-neglect (6). Initial piloting took place in a controlled acute stroke trial assessing the effects of naloxone; it is now commonly used in acute-clinical stroke practice (see Table S1 in Supplementary Material).

#### Validity

The NIHSS' attention to specific neurological deficits engenders a high-concurrent validity (0.4–0.8) based upon association with infarct size (5, 7). Information on construct validity is lacking. The tool is well suited to early stroke severity assessment and baseline scores have strong predictive validity with outcome at 7 days and 3 months (8). Specifically, patients with a baseline score of <5 are almost always (80%) discharged home; scores of 6–13 often need inpatient rehabilitation; and scores of 14+ are strongly associated with need for longer-term care.

There are however questions as to how well the tool determines "real world" functional impact. For example, a lesion that

results in a hemianopia and a score of "1" on the NIHSS would typically be categorized as a "good" outcome (9). Yet, if such impairment precludes driving, consequences upon employment, independence and mood could be substantial. Additionally, the focus of the tool is weighted toward limb and speech impairments with reduced attention to cranial nerve-related lesions (10) and appears to have reduced validity when lesions present in the nondominant hemisphere (11).

#### Reliability

The scale has exhibited excellent inter (ICC = 0.95) and intraobserver reliability (ICC = 0.93). The high inter-rater reliability is observed in both neurologically trained and non-neurologically trained raters alike (11).

#### Responsiveness

The responsiveness of the tool has compared favorably to both the BI and mRS in detecting a treatment effect (5, 9).

#### Feasibility

The NIHSS is optimally generated using a formal observational patient assessment. Recognizing that unwell patients may be unable to participate in all aspects of testing, there is scoring guidance for incomplete test items. On average, NIHSS assessment takes around 5 min to complete. For retrospective assessment in audit or research, the NIHSS can be derived using medical records (12); this is not true for more complex assessments such as mRS (13).

#### Summary

The NIHSS has favorable properties, although as an impairmentbased scale it is not a good measure of the broader disability that can result from a stroke. The NIHSS is perhaps best used as case-mix adjuster or early outcome assessment measure for hyper-acute trials.

# ACTIVITY: BARTHEL INDEX

#### History and Purpose

Designed to measure independence, the Barthel index (BI) was originally used to assist patient discharge and long-term care planning in non-stroke settings (11). The BI operates according to a 10-item scale in which patients are judged upon degree of assistance required when carrying out a range of basic activities of daily living (ADL) (see Table S1 in Supplementary Material). The assessment is delivered through an established and validated questionnaire comprising a total score of 100 for the 10 items of the scale. The patient's answers on each item are scored based upon actual ability (preferably observed by the assessor). The usual scoring for each item is 0 points for "no ability" to do the item independently, 5 points for "moderate help" with the item, and 10 points for being able to manage the item independently. The BI has emerged as the second most popular tool for assessment of post-stroke outcome in clinical stroke trials (1).

# Validity

Concurrent validity, appropriated *via* correlation with infarct size, extent of motor loss and nursing-time requirements, appears to be moderate (*r*<sup>2</sup> = 0.3–0.5) (7, 14, 15). Construct validity is favorable when compared with other measures of activity (16), while predictive validity of the BI has been established on basis of low BI scores correlating with future disability, longer time to recovery, and heightened care needs (17). It is important to note, however, that the predictive validity of the BI can be suboptimal if it is conducted too early (within 5 days post-stroke) (18), and validity of the tool may be compromised by self-report measurement, particularly when cognitive impairment is present (19). Validity of the tool when used in the hyper-acute stroke period is also questionable as monitoring equipment, physical illness, and restrictions on mobility may all compromise the true score.

# Reliability

The inter and intra-reliability of the tool is judged to be moderate (*k*= 0.41–0.6) to high (*k*= 0.81–1.00) (20, 21). However, the studies from which this evidence pertains are limited in sample size and heterogeneous in both methodology and assessment quality (18). Of additional note, reliability seems to vary across specific items of the scale and is greatest at higher BI scores (22).

#### Responsiveness

Responsiveness to change has been described as a strength of the BI over other stroke functional assessment tools (23–25). The overall responsiveness of the tool is reasonable within a certain range of disability; however, it also appears vulnerable to floor and ceiling effects, which are largely attributable to the scales' assessment of basic ADLs only (11). Specifically, the tool is often insensitive to changes in patients whose general mobility and physical function is impaired, but who improve in other aspects—for example, cognitively (floor effect); or where there are limitations in extended ADL's—for example, due to cognitive impairment (ceiling effect).

# Feasibility

The simple structure of the BI allows for direct assessment, proxy-based assessment, telephone assessment, and postal questionnaire. Where possible the information should be based on direct observation of the tasks. BI is relatively quick to perform, but for large-scale audit and research shorter versions have been developed. Recent efforts to enhance feasibility include a short-form version of the BI which includes three items: bladder control, mobility, and transfers (26). This version of the tool has been validated *via* systematic review of short-form BIs, and while validity is reduced by comparison to the full scale, it is no worse than longer versions containing four and five items (27).

# Summary

Although still a popular outcome measure, the BI has properties that limit its utility as primary endpoint in an acute stroke trial. In particular, for those trials where moderate-to-severe disability is not expected, the usefulness of BI is limited by an emphasis on basic ADLs and physical constructs. The BI is perhaps best used to assess case-mix and early outcomes in stroke rehabilitation settings.

# PARTICIPATION: MODIFIED RANKIN SCALE

# History and Purpose

Adapted from the original 1957 Rankin scale (28), which was designed to assess patient outcomes in one of the first stroke units, the modified Rankin scale (mRs) was the first functional outcome assessment used in a stroke trial. The mRS is the most commonly used functional assessment measure and is recommend by professional societies and regulatory bodies1 for outcomes assessment in stroke trials. The mRS adopts a 7-point hierarchical, ordinal scale to measure functional independence (see Table S1 in Supplementary Material).

There has been some debate as to the nature of mRS scoring. We have classified as a measure of participation for the purpose of this review, as the scale offers a broad focus potentially going beyond the basic and extended ADL measures of an activity scale. Other scales are available that are more clearly aligned with the concept of participation but these tools are rarely used in stroke trials (29) whereas mRS is a common outcome assessment that at least serves as a proxy of participation.

# Validity

Analysis of clinical properties suggests concurrent validity based on correlation coefficients with infarct volume is of 0.4–0.5 (30)—comparable to the BI (14, 15, 31). Assessment of construct validity suggests that the mRS has excellent agreement with other stroke functional scales (32), while predictive validity is demonstrated by the association of short to medium term mRS with longer-term post-stroke care needs (17). The validity of the scale can however be affected when a proxy is used to generate a score, or when applied in the acute setting during when the patient has not yet had the chance to resume normal activities. When used in a retrospective fashion to determine the pre-stroke functional state, the mRS validity can be diminished, demonstrating moderate-concurrent validity (ρ > 0.4) when compared with other variables associated with function (33).

# Reliability

Reliability, particularly inter-observer variability, has been identified as the main drawback of the scale. This is a consequence of the simplicity of the tool and its use of a 7-point scale, which is both shorter than many other assessments and less categorical in descriptors at each point, thus requiring greater interpretation from assessors (34). Meta-analysis suggests an inter-rater reliability of κ = 0.62; however, in multicenter trials this maybe be as low as κ = 0.25 (35). This can be further compromised when telephone assessments are utilized to conduct the assessment (36). Statistical noise generated by the poor inter-rater reliability of the mRS increases vulnerability to type-2 errors, meaning that clinically significant treatment effects can be missed. Some of these issues can however be potentially alleviated *via* structured interview, training and central adjudication, all of which we discuss below (37–39).

# Responsiveness

The mRS responsiveness to change has received comparatively less attention than the scales' other properties. With a limited number of possible scores, the mRS may have inferior responsiveness to change compared with other measures of post-stroke function, although any change seen in the mRS is likely to be clinically meaningful. In a non-random sample of stroke rehab patients, the BI has demonstrated favorable responsiveness to change over the mRS (*p* = 0.002) (23). Further issues with regard to the responsiveness of the mRS over particularly short time-frames (i.e., admission to discharge) have also been highlighted (23).

<sup>1</sup>https://www.commondataelements.ninds.nih.gov/Doc/NOC/Modified\_Rankin\_ Scale\_NOC\_Public\_Domain.pdf (Accessed: November 30, 2017).

# Feasibility

The traditional method for mRS assessment is an unstructured direct to patient interview. These interviews are usually short, but the open nature of mRS questions can lead to longer interviews while issues are explored to the satisfaction of the assessor. Where a patient is unable to fully participate in an interview, a proxy can be used (40). The use of structured interviews may improve reliability, although this has not been consistently proven, and short structured mRS assessments have been used in some trials (41).

#### Summary

Overall, the mRS offers a brief yet broad ranging assessment of function. This comes at the price of reliability issues and potentially reduced responsiveness to subtle improvement or deterioration. As a global measure of functional recovery that captures clinically meaningful change, mRS is perhaps best suited as endpoint in large trials of potential stroke treatments.

# QUALITY OF LIFE

Moving beyond the WHO-ICF construct of participation, one can consider a further level of potential outcome assessment as QOL. Again, there are various QOL tools available; for clinical purposes we usually consider health-related assessment scales (HR-QOL) and these can be generic (e.g., the various iterations of the Euro-QOL) or stroke specific (e.g., the Stroke Impact Scale). QOL assessments have particular utility as they can be used to inform health economic analyses (42). The use of HR-QOL assessments is increasing, in part driven by the recognition of the value of patient reported outcome measures. At time of writing no positive stroke trial has used an HR-QOL as primary outcome but this may soon change. In the longer-term post-stroke, QOL will be a product of many factors many of which may be unrelated to the stroke. There is a tension between having a tool that allows comprehensive assessment and having a tool that does not require a lengthy and burdensome interview. In this regard, recent attempts to create shorter HR-QOL forms that retain the most discriminating questions are welcome (43).

# STATISTICAL ANALYSIS OF FUNCTIONAL ENDPOINTS

The statistical approach to analysis of functional outcomes can have implications for sample size, validity and ultimately the success of the trial. In this section, we will mostly discuss the mRS but many of the themes regarding analysis will equally apply to other assessment scales.

# Composite Endpoints

So far we have considered functional assessment scales in isolation. However, as the scales assess differing constructs there could be advantage in combining endpoints. Indeed in the seminal NINDS trial of tPA, scores on NIHSS, BI, mRS, and Glasgow Outcome Scale were assessed in aggregate. The use of composites may have particular utility where outcomes individually may be uncommon. Using a modeling approach, the utility of a composite outcomes to improve power in a trial in minor stroke and acute ischemic stroke have been described (44). The main limitation of composites is in the interpretation and there can be problems, if, for example, a patient has a favorable outcome on one component of the composite and an unfavorable outcome on another. Also, if measures are not independent of one another, error measurement can be exacerbated and there can be a temptation to adopt this approach *post hoc* because individual measures are non-significant (45).

## Cut Points and Shift

The mRS offers ordinal, hierarchical data, and historically the most commonly applied approach to analyses was to dichotomize scales at a set cutoff point, thus distinguishing those who achieve a "good outcome" from those who do not. Although there have been attempts to define an optimal cut point for the differing outcome assessments (9, 46), this approach is slightly misleading as the optimal cut point will vary with the population studied and the anticipation of functional recovery. So, for example, in studies of decompressive hemicraniectomy, a "good" outcome could be defined as less than or equal to mRS 3, while in a trial of tPA for minor stroke one would define a good outcome at a much narrower range, for example, mRS 0–1. What is clear from all the studies of dichotomized cut points is that the choice of scale and cut point will dictate the required sample size to demonstrate a treatment effect.

Dichotomization offers relatively simple comparative analyses, but this reductionist good versus bad outcome approach can miss important treatment effects and will be insensitive to partial, but meaningful improvements in functioning, such as an increase from a score of 5 to 3 in the mRS (i.e., an improvement from bed-ridden to independent mobility, a change which most would accept as clinically important). Indeed, adoption of a dichotomization approach has been implicated in false-neutral findings of stroke trials with examples of trials where dichotomized outcomes potentially missed a treatment effect that was observed using other approaches and examples of where dichotomized outcomes may have provided missed potentially harmful interventions (47).

It is possible to apply a prognosis adjusted endpoint method to analyses, whereby "good" outcomes are defined by achieving a standard dichotomized "good outcome" or by extent of improvement across the scale (e.g., an improvement in score of *n* points on NIHSS). This moderates some of the statistical limitations inherent to dichotomization as it allows more patients' data to contribute to the results (9). Again, however, there is some uncertainty as to how such "good outcomes" should be defined regarding the extent of change in score. Research into the NIHSS suggests the most discriminating prognosis adjusted endpoint appears to be a score of 1 or less overall, or a change in score of 11 points (9).

The alternative approach to segregating data and analyzing on the basis of dichotomized "good outcomes" is to evaluate more of the scale. Trichotomized endpoint analyses have been described but have been superseded by techniques that allow assessment of the entire ordinal scale range *via* a shift analysis. Such approaches that exploit the full distribution of outcomes include the proportional odds model and the Cochran–Mantel– Haenszel test. Shift analysis has been suggested to improve the overall power that can be generated compared with a dichotomized mRS (47). This seems to be particularly true when treatment effects are small, but uniform over all respective ranges of stroke severity (although dichotomization can be mildly more powerful than shift analysis when treatment effects are substantial in certain circumstances) (47). The potential utility of the shift analysis over dichotomization has been demonstrated empirically in recent trials. For example, the INTERACT-2 study of blood pressure reduction in intracerebral hemorrhage was neutral on a primary endpoint of dichotomized mRS, but demonstrated a treatment effect on prespecified secondary shift analyses (48).

A shift approach to mRS assessment is gaining traction in stroke research but we must be mindful of potential limitations in this approach. The main issue with this method is that there are implicit assumptions inherent to shift analysis that may not hold when applied to ordinal scales such as the mRS. For example, the Cochran–Mantel–Haenszel method of analysis assumes that treatment effects are uniform over the full range of the mRS scale; that is, that the treatment effects will be the same for those scoring mRS 0–1 as it is for those scoring mRS 2–3 (49, 50). Moreover, shift analysis is typically considered superior to dichotomized analysis when the error is uniform across the scale. However, inter-rater reliability and misclassification errors are often most problematic in the mid-range (mRS 2–4) of the mRS scale, meaning that errors are typically not evenly distributed (51). When error rates are high and non-uniform, shift analysis may reduce power by comparison to dichotomization (52). Due to these issues, some authors (45, 52, 53) advise against employing shift analysis to ordinal scales such as the mRS, particularly in early-phase trials with small patient samples.

# Utility Weighting

A novel approach to assessment that has been used in contemporary endovascular studies is to apply weighting to outcomes. In utility weighting, it is recognized that certain health outcomes and transitions between outcomes will be more desirable than others. A utility weighted mRS has been developed that incorporates patient and societal valuations of each potential mRS outcome (54). The weighting can be performed by mapping EQ-5D population data onto mRS or using disability weighting. Potential advantages of utility weighted mRS have been demonstrated using secondary analysis of existing trial data (54) but the real proof of the value of the utility weighted approach comes from the recent DAWN trial (55). As the greatest utility values are assigned to transitions from high disability states to lower, then the utility-based approach may have particular value in treatments with the potential to prevent disability such as large artery thrombectomy. It is however important to note that in adopting this approach the higher ends of the mRS scale are filtered out, which may produce a non-Gaussian distribution. Hence, this method can result in the inappropriate application of standard statistical tests.

# FUTURE DIRECTIONS IN STROKE OUTCOME ASSESSMENT

Even with an appropriate outcome measure and statistical analysis plan, demonstrating a treatment effect of a stroke intervention is not easy. Developments in audiovisual and information technology and best practice guidance in outcome assessment is helping to raise standards and improve the application of stroke outcome assessments.

#### Training

Although the assessment scales discussed are theoretically objective, there is always a degree of subjective interpretation. To ensure standardization of assessment, scoring rules and training materials have been developed. Direct training from an experienced assessor is not possible at scale across the many international sites that may participate in a stroke RCT. Training manuals and use of audiovisual materials is one potential solution. For example, mass training in NIHSS using video-recorded patient assessments has proven feasible and popular (56). The format has evolved with changes in available technology from videotape recordings (57), to DVD and now interactive online materials (58). Completion of NIHSS training has been shown to improve scoring and a certificate of completion of NIHSS training is now mandatory for many studies where NIHSS is an outcome measure. Similar resources are available for mRS (59) and BI and also seem to improve application of these scales (60). The mRS training is similar to NIHSS with teaching cases, tutorials, and a certification exam.2 BI training is a descriptive tutorial rather than video-based patient assessment (see text footnote 2). Although the use of these mRS and BI training materials seems intuitively attractive, there have been no suitably large trials that have demonstrated improvements in scoring with training. Nonetheless, it seems unlikely that training would worsen performance in assessment and so we would advocate continued use of such resources.

# Structured Assessments

The mRS and to a lesser extent the BI are based on an interview with the patient. To ensure interviews are focused and have consistency of content, a series of structured mRS' have been proposed. These can be structured, anchoring questions with guidance on interpretation or more formal questionnaires with a series of yes/no responses. Advocates of the structured approach report less time spent on interview and improved reliability. However, proponents of a less-structured interview note the benefits of a flexible approach. A structured interview can result in the discarding of essential information when contemplating a patient's functional ability, particularly concerning usual activities such as work or hobbies. Moreover, if a patient's answers do not "fit nicely" with a given item in the questionnaire, the rigid structured nature of the interview can be a hindrance rather than a benefit. A systematic review and meta-analysis that pooled all available data did not find benefits of structured interview over

<sup>2</sup>https://secure.trainingcampus.net/uas/modules/trees/windex.aspx?rx=rankinenglish.trainingcampus.net (Accessed: November 30, 2017).

standard face-to-face interview, albeit some of the structured interviews used in contemporary trials were not available at the time of the review (61).

# Centralized Adjudication

Expert group adjudication of outcome measures such as neuroimaging or electrocardiographs (ECG) has been routinely used in multicenter clinical trials as a method of reducing inter-observer variability and maintaining quality control. In contrast, traditionally functional outcomes were only assessed at participating sites, but the landscape is changing.

As mRS and to a lesser extent BI can be scored based on an interview, both have the potential for telephone administration. The properties of telephone mRS and BI are less well described than direct assessment and there may be some systematic differences in scoring. However, telephone assessment is attractive for a large multisite study, as it saves time, reduces patient/assessor travel and reduces test burden. In terms of centralized assessment, if telephone interviews are coordinated from a single center there can be more consistency of assessment and easier quality control. Telephone assessments can be audio recorded for off-line assessment by an adjudication panel. These processes were used for a subset of assessments in a recent thrombectomy trial (62).

Audio recording only gives a partial assessment and with the increasing availability of affordable portal video-recording equipment and high-speed data transfer there is increasing potential for audiovisual recording of stroke assessment. Such video assessment allows for remote centralized adjudication of any functional outcome assessment.

Centralized adjudication of the mRS has been employed in international trials with recruitment from a diverse range of countries from Vietnam to Kazakhstan and both North and South America (37). In this particular video-based platform, typically, the centralized adjudication of Rankin scoring employs a panel of 2 or more raters from a pool of expert assessors to score the mRS of the patient. A final score is assigned by a committee based on consensus agreement.

While video-based centralized adjudication necessitates an additional initial cost to the trialists, the availability of low-cost video-recording equipment and high-speed data transfer will mean that any initial outlay will be modest. The benefits gained from source data validation for the patients' existence and consent as well as more stringent blinding to treatment and quality control of the assessment all add value and likely become cost effective in medium- to large-scale trials.

Furthermore, although the approach is still evolving, the use of centralized adjudication begets improvements in inter-rater

# REFERENCES


reliability. Evidence to date suggests that centralized adjudication of the mRS can improve the inter-rater variability in multicenter trials from κ = 0.25 to 0.59 with an ICC = 0.87 for one rater and predicted to be 0.92 with four raters (37). This improvement in reliability can have a modest effect in the reduction of sample size required to see treatment effect. With the high per-patient cost in clinical trials, any potential reduction in patient numbers without sacrificing trial power is of benefit to trialists.

# CONCLUSION

There are many functional assessment scales available for use in stroke trials. It is possible that previous inappropriate choice of functional outcome assessment may have caused us to miss potential treatment effects in stroke trials. With some thought on the aspect of function of greatest interest (impairment, activity, participation), the preferred psychometric properties, and the proposed analytical technique, the researcher can make an informed choice as to the optimal outcome assessment for their study. The use of novel statistical techniques, rater training, and central adjudication have all been proven to improve the utility of outcomes assessments.

The stroke community has made substantial progress in outcome assessment methodology, but there is still more to do. The outcomes described are poor measures of cognitive and psychological outcomes and yet these are the outcomes of most importance to patients. As we make greater use of "big data," for example, national registers, we need methods to incorporate feasible but valid outcome assessment into routine data collection.

# AUTHOR CONTRIBUTIONS

MT-R and TQ drafted the paper. AW and JD contributed to writing, editing, and provided intellectual input.

# FUNDING

MT-R is part funded by Chest Heart and Stroke Scotland; this work is part of the APPLE program of work funded by the Stroke Association and Chief Scientist Office Scotland. TQ is supported by a Stroke Association, Chief Scientist Office Senior Lectureship.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at https://www.frontiersin.org/articles/10.3389/fneur.2018.00191/ full#supplementary-material.


stroke systematic review and external validation. *Stroke* (2017) 48:00–00. doi:10.1161/STROKEAHA.116.014789


**Conflict of Interest Statement:** TQ and JD assisted in design and validation of training materials for mRS and BI. TQ, JD, and AW have assisted in development, validation, and implementation of video-based mRS assessment.

*Copyright © 2018 Taylor-Rowan, Wilson, Dawson and Quinn. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Efficacy of Novel Carbon Nanoparticle Antioxidant Therapy in a Severe Model of Reversible Middle Cerebral Artery Stroke in Acutely Hyperglycemic Rats

*Roderic H. Fabian1‡, Paul J. Derry2‡, Harriett Charmaine Rea1 , William V. Dalmeida1 , Lizanne G. Nilewski3 , William K. A. Sikkema3 , Pitchaiah Mandava1 , Ah-Lim Tsai4 , Kimberly Mendoza3,5, Vladimir Berka4 , James M. Tour6 and Thomas A. Kent2 \*†*

*1Department of Neurology, Baylor College of Medicine, Michael E. DeBakey VA Medical Center, Houston, TX, United States, 2Department of Neurology and Center for Translational Research on Inflammatory Diseases, Baylor College of Medicine, Michael E. DeBakey VA Medical Center, Houston, TX, United States, 3Department of Chemistry, Rice University, Houston, TX, United States, 4Division of Hematology, Department of Internal Medicine, The University of Texas Health Science Center at Houston, McGovern Medical School, Houston, TX, United States, 5Department of Neurology, Baylor College of Medicine, Houston, TX, United States, 6Departments of Chemistry, Computer Science, Materials Science and NanoEngineering, Smalley-Curl Institute and the NanoCarbon Center, Rice University, Houston, TX, United States*

Introduction: While oxidative stress can be measured during transient cerebral ischemia, antioxidant therapies for ischemic stroke have been clinically unsuccessful. Many antioxidants are limited in their range and/or capacity for quenching radicals and can generate toxic intermediates overwhelming depleted endogenous protection. We developed a new antioxidant class, 40 nm × 2 nm carbon nanoparticles, hydrophilic carbon clusters, conjugated to poly(ethylene glycol) termed PEG-HCCs. These particles are high-capacity superoxide dismutase mimics, are effective against hydroxyl radical, and restore the balance between nitric oxide and superoxide in the vasculature. Here, we report the effects of PEG-HCCs administered during reperfusion after transient middle cerebral artery occlusion (tMCAO) by suture in the rat under hyperglycemic conditions. Hyperglycemia occurs in one-third of stroke patients and worsens clinical outcome. In animal models, this worsening occurs largely by accelerating elaboration of reactive oxygen species (ROS) during reperfusion.

Methods: PEG-HCCs were studied for their protective ability against hydrogen peroxide in b.End3 brain endothelial cell line and E17 primary cortical neuron cultures. *In vivo*, hyperglycemia was induced by streptozotocin injection 2 days before tMCAO. 58 Male Sprague-Dawley rats were analyzed. They were injected IV with PBS or PEG-HCCs (4 mg/kg 2×) at the time of recanalization after either 90- or 120-min occlusion. Rats were survived for up to 3 days, and infarct volume characteristics and neurological functional outcome (modified Bederson Score) were assessed.

Results: PEG-HCCs were protective against hydrogen peroxide in both culture models. *In vivo* improvement was found after PEG-HCCs with 90-min ischemia with reduction in infarct size (42%), hemisphere swelling (46%), hemorrhage score (53%), and improvement in Bederson score (70%) (*p* = 0.068–0.001). Early high mortality in the 2-h in the PBS control group precluded detailed analysis, but a trend was found in improvement in

#### *Edited by:*

*Nishant K. Mishra, Tulane University, United States*

#### *Reviewed by:*

*Maxime Gauberti, INSERM, France Yuchuan Ding, Wayne State University School of Medicine, United States*

#### *\*Correspondence:*

*Thomas A. Kent tkent@medicine.tamhsc.edu, thomaskent@prodigy.net*

#### *† Present address:*

*Thomas A. Kent, Engineering Medical School (EnMed), Texas A&M Health Science Center College of Medicine-Houston Campus and Texas A&M College of Engineering, Houston, TX, United States*

> *‡ These authors have contributed equally to this work.*

#### *Specialty section:*

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

*Received: 22 December 2017 Accepted: 14 March 2018 Published: 09 April 2018*

#### *Citation:*

*Fabian RH, Derry PJ, Rea HC, Dalmeida WV, Nilewski LG, Sikkema WKA, Mandava P, Tsai A-L, Mendoza K, Berka V, Tour JM and Kent TA (2018) Efficacy of Novel Carbon Nanoparticle Antioxidant Therapy in a Severe Model of Reversible Middle Cerebral Artery Stroke in Acutely Hyperglycemic Rats. Front. Neurol. 9:199. doi: 10.3389/fneur.2018.00199*

**24**

all factors, e.g., reduction in infarct volume (48%; *p* = 0.034) and a 56% improvement in Bederson score (*p* = 0.055) with PEG-HCCs.

Conclusion: This nano-antioxidant showed some improvement in several outcome measures in a severe model of tMCAO when administered at a clinically relevant time point. Long-term studies and additional models are required to assess potential for clinical use, especially for patients hyperglycemic at the time of their stroke, as these patients have the worst outcomes.

Keywords: diabetes mellitus, stroke, rat model, hyperglycemia, antioxidants, nanomedicine, transient middle cerebral artery occlusion

# INTRODUCTION

Based on many lines of evidence, oxidative stress is a major pathophysiological factor in ischemia and reperfusion injury. This evidence is exemplified by robust neuroprotection in multiple transgenic antioxidant overexpression models of ischemia/ reperfusion (1, 2). However, no clinical trial of antioxidant therapy in any form of brain injury has shown benefit (3, 4). We believe this failure is due to two major factors: (1) There are severe limitations in currently available antioxidants that hinder their effectiveness when employed *following* ischemia as opposed to pretreatment (5) and (2) oxidative stress injury is quantitatively more important under specific clinical circumstances, so a benefit might be missed if it is not tested under the most relevant conditions. In stroke, those conditions are typically those that have the worst outcomes such as *hyperglycemia* at the time of stroke when treated with recanalization therapy (6).

Several defense mechanisms exist to cope with oxidative radicals generated during normal physiology (2, 7, 8). These mechanisms consist of enzymes and other proteins that modify the radical species in a series of steps ultimately leading to water. For example, the fate of superoxide radical ( ) O S <sup>2</sup> •− ; O when dismutation catalyzed by superoxide dismutase (SOD) is to generate the intermediate unstable molecules (e.g., hydrogen peroxide; H2O2) or new radicals (hydroxyl; • OH) that can be generated by this process as H2O2 encounters iron as a catalyst through the Fenton reaction (9). Under normal conditions, there are sufficient levels of protective proteins for detoxification. Under pathological circumstances, however, these protective factors are depleted. After acute injury, they cannot upregulate fast enough. As a result, unstable intermediates are formed that become part of a radical cascade leading to damage and disruption of a wide variety of vital functions.

Given these considerations, once a radical cascade begins, we previously summarized the limitations of many current antioxidants (5) including the following: (A) mechanism of action: many antioxidants "transfer" the radical to another unstable species. SOD generates H2O2 that can subsequently generate • OH. Under normal circumstances, catalase, and glutathione are in sufficient quantities to quench the resultant radicals. This may not be the case under pathological conditions; SOD may actually generate more damaging species, (B) need for regeneration: many antioxidants, such as vitamin E and vitamin C, require regeneration (10) and require factors (glutathione) that are themselves consumed in the oxidative milieu, (C) limited capacity: most current antioxidants have limited capacity and are unlikely to be able to cope with a burst of radicals and their subsequent unstable products if administered after the burst is initiated. High dose albumin, recently failing to show benefit as an antioxidant in stroke (11), has a restricted number of thiol moieties that quench radicals (12) and (D) selectivity: high selectivity is a disadvantage if the agent's mechanism involves radical transfer and depends on downstream enzymes to cope with newly formed radicals. Nearly, every currently available antioxidant shares one or more of these limitations (5).

For this study, we have selected a condition that predicts a poor outcome in stroke: transient cerebral ischemia in the face of hyperglycemia at the time of the stroke. These circumstances are associated with increased expression of oxidative radicals (13–15). The kinetics of SO production is highly relevant to clinical outcomes in stroke. Our laboratory has previously studied this time course in a normoglycemic rat model of transient middle cerebral artery occlusion (tMCAO), using a cytochrome c-coated electrode on the cortical surface which detects SO release. In the case of normoglycemia, the SO radical is only released upon the onset of recanalization after occlusion time >90 min (13). Importantly, 90 min is considered an early time point that could be used widely to start catheter-based recanalization therapy. Longer time to recanalization is associated with declining benefit (16) and higher mortality after unselected endovascular procedures (17). Notably, hyperglycemia accelerates and magnifies oxidative burst in tMCAO (14) and worsens outcome in acute stroke models and stroke patients, especially those who receive recanalization therapy (18–20) by increasing mortality and hemorrhagic transformation (6). Hyperglycemic animal models demonstrate poor reflow, enhanced edema, higher mortality, and hemorrhagic conversion (14, 21–23), particularly with longer or more severe ischemia before recanalization (15, 6, 22).

In our reanalysis of the NINDS rt-PA dataset, patients with large artery stroke appear to be most susceptible to hyperglycemia when undergoing thrombolytic therapy (24, 25). In an earlier review, we concluded that poorer outcome is likely due to generation of a pro-thrombotic, pro-inflammatory, and vasospastic vascular phenotype (6). Variability of glucose may be a major contributing factor following clinical stroke (26). Notably, treatment of hyperglycemia after onset of stroke does not appear to improve outcomes (27, 28), although a definitive trial has not yet been performed specifically in the acute stroke setting. With a lack of a proven neuroprotectant therapy for hyperglycemic stroke, new approaches are needed for this especially vulnerable subgroup of patients especially in the context of newer recanalization therapies in which diabetes, glucose variability or hyperglycemia *per se* predicts a lower percentage of patients with a good outcome and/ or higher likelihood of hemorrhagic transformation (29–32).

In this study, we employ a novel class of antioxidants to address the oxidative imbalances seen following tMCAO under conditions in which oxidative stress is quantitatively more important. Soon after the discovery of carbon-based buckministerfullerenes (C60) (33), these nanomaterials were shown to have antioxidant characteristics (34). Subsequent modifications and applications to models of injury identified neuroprotective properties (35) but also a low threshold for further modification lest their antioxidant capacity be reduced (36). Subsequent generation of a series of different carbon nano-formulations by our research group (37) identified a formulation of carbon nanoparticles (38) that addresses the limitations of current antioxidants described earlier. Through a series of experiments, first in cell-free systems, then in tissue culture and finally *in vivo*, we identified a class of carbon nanoparticles that we term hydrophilic carbon clusters (HCCs) as highly effective antioxidants (39) with unique potential as *in vivo* therapeutics (40–43). We specifically demonstrated their superior efficacy to two clinical failed antioxidants, poly(ethylene glycol) (PEG)-SOD and the precursor to NXY-059 (4), phenyl butyl nitrone, PBN (41) in culture by their ability to reduce the damaging effect of the mitochondrial toxin, antimycin A, when given AFTER the toxin, while pretreatment was needed for the other agents. These particles are small (40 nm in length, 1–2 nm in diameter, comparable to a hydrated protein), highly functionalized to generate hydrophilic moieties with the addition of PEG to provide solubility in biological fluids, stable at room temperature and without apparent toxicity after systemic injection seen thus far (41).

Here, we report *in vitro* and *in vivo* evidence that hydrophilic carbon clusters, conjugated to poly(ethylene glycol) (PEG-HCCs) can mitigate major detrimental effects of oxidative stress. In tissue culture, we demonstrate that PEG-HCCs are able to mitigate the detrimental effects of H2O2 even though administered after the addition of the H2O2. *In vivo*, we show that PEG-HCCs administered intravenously at a clinical relevant time (onset of recanalization) can mitigate detrimental effects of the hyperglycemia.

# MATERIALS AND METHODS

The protective effects of PEG-HCCs in cell culture were tested using the murine brain endothelial cell line, bEnd.3 (44). This cell line was selected because of the delayed effects of transient ischemia at the neurovascular unit that impair reperfusion and promote edema (6). Experiments with neuronal cells were also performed with E17 murine cortical neurons. Oxidative injury rescue experiments were performed with 100 µM H2O2 because it achieved approximately 50% cell death after 24 h in bEnd.3 cells.

# Culture of bEnd.3 Cells

bEnd.3 cells (ATCC) were grown in T-75 (75 cm2 ) flasks containing Dulbecco's modified Eagle's medium (4 mM l-glutamine adjusted to contain 1.5 g/L sodium bicarbonate and 4.5 g/L glucose, 90%; fetal bovine serum, 10%) (Atlanta Biological) in an incubator. Aliquots of 30,000 cells in 0.1 mL were added directly onto sterile 24-well plates. The cells were allowed to attach for 15 min after which an additional 0.9 mL of media is added before the cells are placed in an incubator and allowed to grow for 48 h.

# Hydrogen Peroxide Protection by PEG-HCCs in bEnd.3 Cells

Cultured bEnd.3 cells were then treated with either PBS as a control or 100 µM hydrogen peroxide both with and without PEG-HCC (8 mg/L) added after 15 min. After all the additions, the cultures were incubated at 37°C in 5% CO2 overnight. The Live/Dead assay (calcein AM/ethidium homodimer-1) (Cat #L3224, ThermoFisher) was performed per the manufacturer's instructions and the number of live cells are counted using a Nikon eclipse 80i microscope set to the FITC channel.

# Culture of E17 Murine Cortical Neurons

E17 primary murine cortical neurons (A15586, ThermoFisher) were seeded on to a poly-d-lysine coated 48-well plate at a density of 50,000 cells/well in 500 µL of neurobasal (Cat #21103, ThermoFisher) media containing 1× B-27 supplement and 100 µM GlutaMAX (Cat #35050, ThermoFisher). The neurons were incubated overnight at 37°C in 5% CO2. The following morning, 250 µL of media in each well was exchanged with fresh complete media. Afterward, 250 µL of media was replaced twice on days 4 and 7.

# CellROX ROS Formation Assay

A 10 mM solution of H2O2 was prepared by diluting 51 µL of 9.8 M H2O2 in sterile water. Two wells containing 50,000 neurons were left as untreated controls, two wells were treated with 47 µL of 85 mg/L PEG-HCCs, and 250 µL of media was removed from four wells and replaced with 250 µL of 100 µM H2O2 in complete media. After 15 min, 47 µL of 85 mg/L PEG-HCCs was added by pipette, gently mixed, and incubated for 30 min. Simultaneously, 6 mL of a 10 µM solution of CellROX Deep Red (C10422, ThermoFisher) was prepared in complete Neurobasal media by the addition of 24 µL of 2.5 mM CellROX Deep Red dye. After incubating the neurons for 30 min, 250 µL of media was removed from each well and was replaced with 250 µL of 10 µM CellROX Deep Red solution and incubated for an additional 30 min at 37°C with 5% CO2. The neurons were rinsed twice by first removing 400 µL of media from each well and gently adding an additional 400 µL of warmed PBS. Finally, 400 µL of the media was removed and replaced with 4% formaldehyde in PBS and fixed for 30 min at 4°C in a refrigerator.

The fixed neurons were imaged at 20× magnification using a Nikon Eclipse Ti equipped with a Photometrics CoolSNAP HQ2 sensor and a 670 nm emission filter (Cy5). Phase contrast images and 670 nm fluorescence images were collected of each well. The average fluorescence signal from each cell was calculated by including only the fluorescence originating from the area of the cell soma. Average cellular fluorescence was normalized to the untreated control cells.

# Cytotoxicity Assay

Due to greater sensitivity to hydrogen peroxide, we tested both 50 and 100 µM H2O2 in plated neurons. PEG-HCCs were added right after the H2O2 and cells incubated overnight. Live/Dead assay was performed as above and live cells counted.

# *In Vivo* Testing in Hyperglycemia tMCAO Model

We utilized tMCAO and the filament model (45) in the context of acute hyperglycemia following streptozotocin injection (22, 46). We selected this method of generating hyperglycemia because acute hyperglycemia as a stress reaction in non-diabetics is associated with particularly poor outcomes (47) and less elevation in glucose is needed to increase poor outcomes in non-diabetics compared with diabetics (48).

# Synthesis and Characterization of PEG-HCCs

The carbon core of the PEG-HCCs is prepared by subjecting purified (removing exogenous carbon black and gross metal contaminants) single-walled carbon nanotubes (SWCNTs) to a harsh oxidation procedure which uses fuming sulfuric acid (excess SO3, oleum) and nitric acid (38, 39). Nitric acid initiates the oxidation and cutting process which both shortens the SWCNTs to ~35–40 nm and splits them to remove any tubular residues, thus generating shortened oxidized HCCs. Harsh acidic conditions dissolve and remove even trace metal contaminants as determined by inductively coupled plasma mass spectrometry. The surface of the HCCs is functionalized with various oxygencontaining moieties such as alcohols, ketones, and carboxylic acids, rendering the HCCs water soluble in spite of their many remaining hydrophobic domains. Characterization details including infrared spectroscopy (FTIR), Raman spectroscopy, X-ray photoelectron spectroscopy, atomic force microscopy, thermogravimetric analysis, UV–vis spectroscopy, dynamic light scattering, and zeta potential can be found in Berlin et al. (42).

# Induction of Hyperglycemia and tMCAO

All procedures were approved by the Baylor College of Medicine IACUC and the Michael E. DeBakey VA Medical Center R&D Committee. Outcome measurements were performed by coauthors blinded to expected outcomes (William V. Dalmeida and Harriett Charmaine Rea). Rats were rejected from subsequent analysis based on the surgeon's assessment of peri-procedural errors or procedure related death, concomitant illness (e.g., respiratory compromise) or mechanical dysfunction, with the surgeon blind to their quantified outcomes.

Male Sprague-Dawley rats weighing 330–350 g were delivered to the vivarium 1 week before experiments to allow for acclimation. Hyperglycemia was induced by injecting sterile filtered streptozotocin 60 mg/kg IP. Control rats are injected with sterile filtered normal saline. Two days later rats were subjected to tMCAO using the filament model as published (22). They were fed with standard chow and water *ad lib* and exposed to a standard light–dark daily cycle. In preparation for the MCA occlusion, rats were deeply anesthetized in an induction chamber with 3% isoflurane followed by intubation and mechanically ventilated with 2.0–2.5% isoflurane in an oxygen:air mix of 30:70. The tail artery was cannulated using sterile technique with PE-50 polyethylene catheters for monitoring mean arterial blood pressure, blood pH, PCO2, PO2, glucose, as well as additional blood chemistries. The tail vein was then cannulated with a 24-gage 0.75″ angiocath to administer an infusion of intravenous fluids. A rectal temperature probe was used, and the temperature maintained at 37 ± 0.5°C with a heating pad. Vitals such as O2 saturation, heart rate, average CO2, and total CO2 were monitored throughout the surgery. Analgesics were administered sub Q during the procedure to alleviate postsurgical pain and were continued postsurgery. Ketaprofen, an NSAID, was injected subcutaneously at 5 mg/kg in addition to buprenorphine at a dose of 0.05–0.1 mg/kg. Atropine was injected subcutaneously if needed at 0.054 mg/kg.

Without pausing anesthesia, focal cerebral ischemia was induced by occluding the origin of the MCA using the intraluminal suture insertion method. Rats were inverted following induction of anesthesia and an area of skin over the right carotid artery was prepped by clipping hair and scrubbing with betadine, rinsing with alcohol, and painting with iodine. An incision was made over the carotid bifurcation, and the carotid bifurcation was exposed with blunt dissection. The internal carotid artery and pterygopalatine artery were ligated at the origin. A small incision is placed in the internal carotid through which a 0.25 mm nylon monofilament is introduced. Initially, the filament was threaded up the internal carotid exactly 1.7 cm from the bifurcation to occlude the middle cerebral artery. Anesthesia was maintained through the duration of MCA occlusion. The filament was then withdrawn, and the internal carotid artery was ligated distal to the arterial incision. During ischemia, a warming blanket was used to maintain body temperature through the duration of MCA occlusion while the animal's body temperature is sustained with the heating pad. Rats were weaned off the respirator, observed and kept warm until alert and recumbent, and then returned to their cages.

Postsurgical animals received a soft, purified maintenance diet of 31M Diet Gel along with a 98% sterile water polymer HydroGel. In addition, moistened rat chow was placed at the bottom of the cage. Following surgery, blood was collected from rats every morning for glucose levels. Hyperglycemic rats were given NPH Lente insulin to keep glucose levels under 250 mg/dL. Overall, 21% of rats in the 90-min group and 22% of rats in the 120-min group were given insulin on day 0 and day 1, respectively. Pain medications ketaprofen and buprenorphine were continued daily postsurgery once or twice a day, respectively. Rats were observed twice a day for signs of distress and/or pain and euthanized if they meet the criteria.

# Administration of PEG-HCCs or PBS Control

PEG-HCCs were injected at a dose of 4 mg/kg (volume <0.1 mL) in a tail vein just before suture removal. Injection was repeated after 2 h. This dosing protocol was derived initially from concentrations that were maximally protective against various toxins *in vitro* culture and transformed into *in vivo* dosing based on estimated blood volume of distribution in rats. The dosing was confirmed as beneficial and well tolerated from that successfully used in mild traumatic brain injury complicated by hemorrhagic hypotension and repeated in 2 h based on the approximate 2-h blood half-life observed in our normal mice experiments (40, 41).

# Neurological Function (Bederson's Score)

A behavioral assay, a modified Bederson test was used for acute disability assessment on post-op day 3 (49). Neurological function is assessed from 0 (normal) to 6 at the end of the 3-day period. Either spontaneous death or euthanasia due to undue stress was scored 7. The scoring was as follows:


# Infarct Volume, Hemisphere Swelling

At the end of 72 h or at the time of early euthanasia, the rats were euthanized with 150 mg/kg of Nembutal IP. Rats were perfused transcardially with 100 mL of 0.9% saline. The brains were removed and immediately frozen at −20°C for 20 min and then sectioned into 1 mm thickness using a rotary hand microtome. The ischemic damage was evaluated by immunohistochemical analysis using 2% tetrazolium chloride (TTC) staining. Brains sections were incubated in 2% TTC in phosphate-buffered saline for 10 min in the dark at 37°C. Sections were fixed in buffered 4% paraformaldehyde pH 7.4 and photographed. After 30 min, the brains were sectioned into 10, 1 mm-thick slices from anterior to posterior. The area of non-stained infarct in each slice was measured using NIS Elements AR software (Nikon). Non-stained infarct areas of ischemic and control hemisphere areas were calculated and then multiplied by slice thickness and summed. Hemispheric swelling was assessed by the ratio of ischemic and contralateral hemisphere volume. This ratio was used to adjust infarct volume for edema modified from McBride et al. (50) and also served as hemisphere swelling index.

# Hemorrhage Assessment

Several rats undergoing tMCAO suffered a hemorrhage. The hemorrhages were assessed by visual examination of TTCstained coronal sections for each animal. The hemorrhage was documented by notating which areas in the brain were specifically affected as striatum and/or cortex, the number of sliced sections that intracranial bleeding had occurred, and the intensity of the hemorrhage seen in the affected areas such as petechial or confluent. Further evaluation entails scoring the hemorrhage by quantifying the size of area that was affected as follows: 0—no hemorrhage, 1—single petechial hemorrhage, 2—multiple petechial hemorrhages, 3—single confluent subcortical hemorrhage, and 4—hemorrhages including cortical region.

# Statistical Analysis

Cytotoxicity and cell fluorescence was tested by comparing means and SDs employing Student's *t*-test. For *in vivo* studies, baseline conditions and ordinal outcome measures were performed by Student's *t*-test. Proportions were tested using chi-square adjusted for small *n*. Outcome measures were those prospectively defined (infarct volume, hemisphere swelling index, hemorrhage score, and modified Bederson score) and tested by Mann–Whitney *U* non-parametric test due to the potential for non-normally distributed outcomes with the small *n*. Mortality was recorded.

# RESULTS

# *In Vitro* Protection Against H2O2

The protection of PEG-HCCs against hydrogen peroxide was measured in both cultured murine cortical endothelial bEnd.3 cells and in cultured primary murine cortical E17 neurons. We observed that 100 µM H2O2 reduced cell viability in bEnd.3 cells at 24 h by approximately 50% as indicated by a Live/Dead assay (**Figure 1**). The addition of PEG-HCCs after 15 min restored cell

Figure 1 | Cell viability following addition of hydrogen peroxide to cultured brain endothelial cells (b.End3). Live cell counts (Live/Dead cell viability assay) per well is presented on *y*-axis as mean and SD of replicates. 100 µM H2O2 was added and 15 min later either media or hydrophilic carbon clusters, conjugated to poly(ethylene glycol) (PEG-HCCs) (8 mg/mL) was added and live cell/well assessed the following day. H2O2 reduced cell viability by 50%, which was completely restored by PEG-HCCs.

number to baseline (*p*< 0.001 vs H2O2). In E17 neurons, we found that 100 µM H2O2 was more lethal in neurons than b.End3 cells, nevertheless, partial restoration was achieved with posttreatment with PEG-HCCs (**Figure 2**).

#### CellROX ROS Assay in E17 Neurons

ROS formation was measured using a CellROX assay in cultured murine neurons (**Figure 3**). E17 cells treated with PEG-HCCs

Figure 2 | Hydrophilic carbon clusters, conjugated to poly(ethylene glycol) (PEG-HCCs) reduce cytotoxicity of H2O2 on treated MCNs. PEG-HCCs given at a concentration of 8 mg/L treated immediately following exposure and overnight incubation reduce cell death restored cell number to baseline following 50 µM H2O2 and doubled cell count following the much more toxic 100 μM H2O2.

showed no increase in CellROX fluorescence compared with the untreated control (100.1 ± 8.8%). Cells treated with 50 µM H2O2 for 15 min showed a significant increase in CellROX fluorescence (200 ± 26.5%). Treatment of MCNs with 8 mg/L PEG-HCCs following 15 min of H2O2 exposure for 30 min showed an increase in CellROX fluorescence of 129 ± 3.4% but was smaller than with H2O2 by itself. Cell viability was reduced at 50 µM H2O2 by 20% and was fully restored by PEG-HCCs treatment.

#### *In Vivo* tMCAO

Seventy-two rats underwent the procedure. Fifty-eight met criteria for outcome analysis. In the 90-min occlusion, four PBS- and one PEG-HCC-treated rats were excluded, and in the 120-min occlusion group, seven PBS- and two PEG-HCC-treated rats were excluded, primarily for early illness/mortality or procedural problems identified by the operator before assessment of outcomes.

The target of 300 mg/dL preoperative glucose was achieved in the 90-min group. PBS-treated rats showed complete MCA territory infarction (**Figure 4A**) while PEG-HCCs treated rats showed mostly subcortical infarctions (**Figure 4B**). Quantification of outcome measures demonstrated that PEG-HCC treatment improved infarct volume, hemorrhagic conversion, hemisphere swelling and Bederson score, with a trend toward reduced mortality (**Table 1**).

Survival was markedly diminished at the 120-min time point in the PBS-treated controls, such that no rats survived the day of procedure at the original target glucose (300 mg/dL). We subsequently reduced the streptozotocin dosing until we achieved a target of 200 mg/dL glucose at the onset of the tMCAO procedure. Survival without apparent discomfort to at least 24 h marginally improved in the PBS-treated controls. However, this limited the information that we could obtain from the control group and we did not pursue this time point to full completion. Rats that required sacrifice before 12 h

Table 1 | Results of hydrophilic carbon cluster, conjugated to poly(ethylene glycol) (PEG-HCC) treatment compared with controls in hyperglycemia after 90 min occlusion and assessment at the end of experimental period.


*The mean overall survival was 2.8 days. Groups did not differ with respect to baseline glucose just before tMCAO or in blood gas parameters taken from a sample of each group. All outcomes were in the direction of improvement with PEG-HCC treatment compared with controls. \*P < 0.05.*

Table 2 | Results of hydrophilic carbon cluster, conjugated to poly(ethylene glycol) (PEG-HCC) treatment compared with controls in hyperglycemia after 120 min occlusion and assessment at the end of experimental period.


*The mean overall survival was 2.1 days. Glucose targets were lowered to improve survivability of the procedure. Groups did not differ with respect to baseline glucose just before tMCAO or in blood gas parameters from a representative sample except for trend toward lower pH in the PBS group. All outcomes were in the direction of improvement with PEG-HCC treatment compared with controls with significance achieved with modified Bederson Score. ND: not done because of premature termination of the experiment (see text). \*P < 0.05.*

postprocedure were not assessed for infarct characteristics as we felt this would be unreliable. In this time point, we observed positive trends in all measures, with significance achieved in the infarct volume (**Table 2**).

# DISCUSSION

In this report, we demonstrated that PEG-HCCs could improve cell survival in both tissue culture models of oxidative injury from H2O2, particularly in a brain endothelial cell line, an important target of hyperglycemia in stroke. From our CellROX assay on cultured neurons we can conclude that PEG-HCCs prevent the formation of oxidative radicals which would otherwise react with the non-fluorescent CellROX dye to produce a fluorescent derivative.

We also found that treatment with PEG-HCCs at a clinically relevant time point could improve several important features related to stroke outcome in a rat model of tMCAO complicated by acute hyperglycemia. Given that hyperglycemia has major influences on outcome in tMCAO through a dysfunctional vasculature (6), we speculate that the *in vitro* effects are indeed relevant to this *in vivo* protection, which is supported by benefit on two vascular measures: hemisphere swelling and hemorrhagic transformation. The dramatic worsening of outcome with hyperglycemia especially at 2 h in our hands was mitigated to some extent even in this severe condition by administration of PEG-HCCs.

There are several limitations of our study. There are different methods of inducing hyperglycemia that each encapsulate some aspect of both the acute and chronic effects of diabetes and/or hyperglycemia. We selected this acute model because analyses suggest that hyperglycemia in patients without prior diabetes have the worst outcomes (20). Here, we employed short survival periods, which was necessitated by the poor outcomes in the control group. An alternative strategy will be implemented in the future to look at the limits of occlusion time possible with PEG-HCCs without a concomitant delay in recanalization for a comparison control group given the severity of the injury (51). We selected only male rats for this proof of principle study and will need to address sex and age differences before expectation that these results can be clinically translated. In a different carotid occlusion model and hyperglycemia, female sex was associated with less severe outcomes (52). Clinically there are reported differences in both stroke risk and outcomes in diabetics related to gender (53, 54), which is complicated by different risk factors, stroke etiologies, and treatment responses but remains an important issue to address in preclinical models.

The occlusion method has some limitations as well. Endovascular therapy for ischemic stroke has been now shown to be overall beneficial even at longer time intervals in patients who maintain good collateral circulation and when using a new generation of removable stent retrievers (55). While not certain, these improved outcomes could be due to improved recanalization rates as well as less endothelial injury in the process. The suture model approximates some of the features of removable stent-retriever mechanical thrombectomy, but the principle is quite different including application of a removable stent. Use of analgesics and anti-inflammatory agents postprocedure was needed because of the severity of the insult; however, it is not clear what affect these may have had if they interacted with the PEG-HCCs. Also, the severity of the insult likely resulted in a relatively high percentage of subjects excluded (19%), although how this compares with other similar studies is not known since this number is not universally reported. We acknowledge that testing of our materials in larger animal models and more clinically realistic methods of inducing occlusion would be necessary before clinical translation. Nevertheless, the profound worsening by hyperglycemia in this model may model a worst-case scenario of endothelial occlusion/injury that suggest PEG-HCCs may be promising when used in combination with endovascular therapy.

The mechanism of worsened vascular outcomes in these models can be potentially explained at least in part by "uncoupling" of nitric oxide synthase (NOS) (46, 56–58), a phenomenon in which NOS dysfunction, often after oxidation of the cofactor, tetrahydrobiopterin, prevents proper coupling between the oxidase and reductase domains of NOS leading to generation of a SO radical in lieu of reducing l-arginine to NO and citrulline. *In vivo*, there are many potential sources of SO to initiate this effect [e.g., NADPH oxidase (14, 59)]. We have termed an overall increase in SO relative to NO as "functional uncoupling" since the net result, including the toxic product peroxynitrite, is similar regardless of the sources (46, 60). Consistent with this concept, we recently showed that, while both acute hyperglycemia and tMCAO individually cause functional uncoupling in the vasculature in the peri-infarct region, tMCAO with hyperglycemia had a 10-fold synergistic increase in SO relative to NO, still evident at 24 h (46). Persistent oxidative imbalance provides a potential late target for intervention, since the peri-infarct region is critical in mediating many of the vascular complications of stroke (61) such as edema and hemorrhage. The role of oxidative stress is partially supported by recent studies on post-recanalization beneficial effects of uric acid in a hyperglycemic mouse model as well (62).

Hydrophilic carbon clusters, conjugated to poly(ethylene glycol) are a unique antioxidant (39, 63, 64). While the antioxidant potency per carbon atom of PEG-HCCs is within an order of magnitude of prototype antioxidants such as Trolox and vitamin C, the capacity per particle is remarkable. Quantitative electron paramagnetic resonance (EPR) indicates the quenching effect of PEG-HCCs is equivalent to the total SOD activity in human spinal cord (39). Using EPR spectroscopy, and oxyhemoglobin, cytochrome *c*, and pyrogallol red decomposition assays we found that PEG-HCCs convert SO to O2, making them ideal for treating ischemia/reperfusion (39). Turnover numbers (moles of consumed SO/moles of PEG-HCCs) were 1 million at physiological pH. Nanomolar concentrations of PEG-HCCs showed typical Michaelis–Menten kinetics with turnovers in the same range as that obtained from the EPR. The catalytic turnover number is about an order of magnitude higher than most efficient single active site enzymes and suggests that a PEG-HCC could possess multiple catalytically active sites. Furthermore, 2.4 nM of PEG-HCCs are able to scavenge 2.8 and 53.7 µM of SO and of • OH, respectively. PEG-HCCs do not quench NO radicals and had no direct effect on peroxinitrate anion (ONOO<sup>−</sup>). Given that NO is constantly produced *in vivo*, is freely diffusible and PEG-HCCs efficiently scavenge SO, this upstream scavenging effect will likely also decrease the amount of ONOO<sup>−</sup> produced *in vivo*. Taken together, these studies demonstrate that PEG-HCCs address each of the hypothesized limitations of current antioxidants (5): capacity, selectivity, and quenching toxic intermediates. Prior work indicates rapid endothelial cellular uptake, although the mechanism is not yet identified (40).

While modern endovascular procedures show improved outcomes, many patients still do not fully recover. The precise role of reperfusion injury is controversial in those patients who do not recover. Several analyses of the most recent generation of endovascular therapies for the most part find an association of either diabetes *per se*, hyperglycemia or glucose dysregulation associated with poorer outcomes and/or hemorrhagic transformation (29–32). Indeed, even in a stent-retriever study, reanalysis that found that hyperglycemic patients did derive benefit, hyperglycemic patients were 36% less likely to have a good functional outcome, had nearly double the mortality and a fourfold increase in hemorrhagic transformation compared with non-hyperglycemic patients who received the catheter-based therapy (65).

While the presence of "reperfusion injury" remains controversial, we have suggested that it is most likely to be found in those patients that have a concomitant factor such as hyperglycemia or other sources of inflammation (66), and that an important factor to consider in the patients who do not recover are baseline factors that worsen outcome, of which hyperglycemia is a major consideration (6, 51, 67, 68). The abundant preclinical evidence supports that reperfusion oxidative stress is most prominent in this population, so we would expect strategies such as ours to be most effective in this group, if at all. Additional studies are warranted to in clinically relevant animal models encompassing a range of pathologies to address their suitability as an adjunct to recanalization therapies.

# ETHICS STATEMENT

All procedures were approved by the Baylor College of Medicine Institutional Animal Care and Use Committee and the Michael E. DeBakey VA Medical Center R&D Committee.

# AUTHOR CONTRIBUTIONS

Animal handling—RF and HR. *In vitro* cell experiments—WD, PD, and TK. Nanomaterials synthesis—WS, LN, JT, and KM. Nanomaterial characterization—A-LT and VB. Principal investigators—TK and JT.

# REFERENCES


# FUNDING

PJD was supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number T32HL007747. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Supported by NIH grants R21NS084290 and R01NS094535 (MPIs: TK, JT, and A-LT). The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government. A preliminary version of this manuscript was presented at the 2017 International Stroke Conference, Houston, TX, USA.


**Conflict of Interest Statement:** TK and JT are founders and shareholders in Acelerox, LLC. WD, LN, WS, PD, A-LT, JT, and TK are inventors on various approved and pending patents related to carbon nanomaterial synthesis and therapeutic use.

*Copyright © 2018 Fabian, Derry, Rea, Dalmeida, Nilewski, Sikkema, Mandava, Tsai, Mendoza, Berka, Tour and Kent. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Periprocedural Antithrombotic Treatment During Acute Mechanical Thrombectomy for Ischemic Stroke: A Systematic Review

*Rob A. van de Graaf1,2\*, Vicky Chalos1,2,3, Gregory J. del Zoppo4,5, Aad van der Lugt2 , Diederik W. J. Dippel1 and Bob Roozenbeek1,2*

*1Department of Neurology, Erasmus MC University Medical Center, Rotterdam, Netherlands, 2Department of Radiology, Erasmus MC University Medical Center, Rotterdam, Netherlands, 3Department of Public Health, Erasmus MC University Medical Center, Rotterdam, Netherlands, 4Division of Hematology, Department of Medicine, University of Washington School of Medicine, Seattle, WA, United States, 5Department of Neurology, University of Washington School of Medicine, Seattle, WA, United States*

Background: More than one-third of the patients with ischemic stroke caused by an intracranial large vessel occlusion do not recover to functional independence despite fast and successful recanalization by acute mechanical thrombectomy (MT). This may partially be explained by incomplete microvascular reperfusion. Some antithrombotics, e.g., antiplatelet agents and heparin, may be able to restore microvascular reperfusion. However, antithrombotics may also increase the risk of symptomatic intracranial hemorrhage (sICH). The aim of this review was to assess the potential safety and functional outcome of periprocedural antiplatelet or heparin use during acute MT for ischemic stroke.

#### *Edited by:*

*Bruce Campbell, University of Melbourne, Australia*

#### *Reviewed by:*

*Jelle Demeestere, University Hospitals Leuven, Belgium Joshua Z. Willey, Columbia University, United States*

*\*Correspondence:*

*Rob A. van de Graaf r.a.vandegraaf@erasmusmc.nl*

#### *Specialty section:*

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

*Received: 29 January 2018 Accepted: 26 March 2018 Published: 16 April 2018*

#### *Citation:*

*van de Graaf RA, Chalos V, del Zoppo GJ, van der Lugt A, Dippel DWJ and Roozenbeek B (2018) Periprocedural Antithrombotic Treatment During Acute Mechanical Thrombectomy for Ischemic Stroke: A Systematic Review. Front. Neurol. 9:238. doi: 10.3389/fneur.2018.00238*

Methods: We systematically searched *PubMed*, *Embase*, *Medline*, *Web of Science*, and *Cochrane* for studies investigating the safety and functional outcome of periprocedural antiplatelet or heparin treatment during acute MT for ischemic stroke. The primary outcome was the risk for sICH. Secondary outcomes were functional independence after 3–6 months (modified Rankin Scale 0–2) and mortality within 6 months.

Results: 837 studies were identified through the search, of which 19 studies were included. The sICH risks of the periprocedural use of antiplatelets ranged from 6 to 17%, and for heparin from 5 to 12%. Two of four studies reporting relative effects of the use of antithrombotics are pointing toward an increased risk of sICH. Among patients treated with antiplatelet agents, functional independence varied from 23 to 60% and mortality from 18 to 33%. For heparin, this was, respectively, 19–54% and 19–33%. The three studies presenting relative effects of antiplatelets on functional independence showed neutral effects. Both studies reporting relative effects of heparin on functional independence found it to increase this chance.

Conclusion: Randomized controlled trials investigating the effect of periprocedural antithrombotic treatment in MT are lacking. Some observational studies report a slight increase in sICH risk, which may be acceptable because they also suggest a beneficial effect on functional outcome. Therefore, randomized controlled trials are warranted to address the question whether the potentially higher risk of sICH could be outweighed by improved functional outcome.

Keywords: ischemic stroke, periprocedural, heparin, antiplatelet agents, antithrombotic agents, mechanical thrombectomy, endovascular treatment

# BACKGROUND

The introduction of endovascular treatment by means of acute mechanical thrombectomy (MT) has been a major change in the emergent treatment of ischemic stroke caused by an intracranial large vessel occlusion. An individual patient data meta-analysis of randomized trials showed that this approach is highly effective (1). In that meta-analysis, MT significantly improved functional outcome at 90 days, with a number needed to treat of 2.6 to reduce disability by one level on the modified Rankin Scale (mRS). Still, approximately one-third of the patients do not recover to functional independence despite fast and complete recanalization by MT (2, 3). This could partially be attributable to microvascular dysfunction also known as incomplete microvascular reperfusion (IMR). The concept of IMR stems from observations in the non-human primate of focal "no-reflow" following focal ischemia—caused by adhesion of polymorphonuclear leukocytes (4–6), and/or platelet-fibrin occlusions (7) within the downstream microvasculature—that could be prevented by anti-leukocyte or antithrombotic strategies. More recently, this concept has been described again (8). Antiplatelet agents in experimental systems have shown to prevent the microvascular occlusive events in both non-human primate and mouse models and to improve outcome (9, 10). Also heparin may be of additional value to MT, by preventing microthrombus formation and microvascular obstruction and potentially restore microvascular reperfusion. It has been suggested that microvascular obstructions could arise from neutrophil extracellular trap (NET) formation (11). NET formation can be dissolved by heparin, but not by tissue plasminogen activator (tPA) (12, 13). As antiplatelet agents and heparin seem promising in their ability to restore microvascular function, these drugs might contribute to the recovery of patients with ischemic stroke undergoing acute MT. A direct test of this hypothesis in humans has not yet taken place. An important disadvantage of both antiplatelet and heparin use in the setting of focal cerebral ischemia is the increased risk of intracranial hemorrhage. Symptomatic intracranial hemorrhage (sICH) leads to severe handicap or death in almost all patients (14). A randomized trial—in which patients with an ischemic stroke were either assigned to intravenous (IV) antiplatelet agents within 90 min after starting treatment with IV recombinant tPA or to no antiplatelet agents—was stopped before the intended conclusion due to non-superior outcomes and a higher risk of sICH in the group that received antiplatelet agents (15). Although the absolute sICH risk associated with acute antiplatelet administration was low (4.3%), concerns remain about this detrimental side effect. These concerns are also present with regard to the use of heparin in ischemic stroke. This may be due to the results of the International Stroke Trial, in which 19,435 patients were randomized to receive antiplatelet agents, heparin, both or neither within 48 h after symptom onset (16). In this study, the beneficial results (i.e., reduced risk of recurrent stroke and improved functional outcome) were offset by a higher sICH risk. Again, the absolute sICH risk was low in this trial, even in the high-dose group receiving 12,500 IU twice daily (2.0%). Yet, the balance between risk and benefit of these antithrombotic drugs for patients with ischemic stroke is uncertain in the setting of acute MT. Therefore, the aim of this review was to assess the potential safety and functional outcome of periprocedural antiplatelet or heparin use during acute MT for ischemic stroke.

# METHODS

# Search Strategy

A search strategy was developed in collaboration with a biomedical information specialist to systematically search *PubMed*, *Embase*, *Medline*, *Web of Science*, and *Cochrane*. The search was conducted in November 2017 and updated in March 2018. Two independent reviewers (RG and VC) screened all identified articles on titles and abstracts for eligibility. Articles identified as potentially eligible underwent a full text review. Disagreements between reviewers were resolved by a consensus meeting with a third reviewer (BR). The complete search strategy is listed as supplemental material (Data Sheet S1 in Supplementary Material).

# Inclusion and Exclusion Criteria

Studies were eligible for inclusion when:


Studies were excluded when:


In addition, studies reporting on patients with "tandem lesions" (i.e., an intracranial large vessel occlusion with simultaneous ipsilateral extracranial carotid occlusion) treated with intracranial MT with or without emergency carotid artery stenting were included through bibliographic review of the included studies. In these studies, antithrombotics were used as a part of protocol-based care to prevent stent occlusion. Finally, large randomized controlled trials (RCTs) investigating the effectiveness of MT, and in which periprocedural antithrombotics were used, were included through bibliographic reviewing.

# Data Extraction and Synthesis

We developed a data extraction form based on elements of the Cochrane Consumers and Communication Review Group's data extraction template (17). Two reviewers extracted the data independently: one reviewer extracted all the data (RG) and the other reviewer extracted 25% of the data (VC). Extracted data were checked during consensus meetings with three reviewers (RG, VC, and BR). For each included study, we aimed to specifically extract the available data for the patients treated with MT or the most representative group. The following information was extracted: study design, study population characteristics [sample size, age, National Institutes of Health Stroke Scale (NIHSS) at baseline, and occlusion location], recanalization therapy [administration of IV plasminogen activators, administration of intraarterial (IA) plasminogen activators, treatment with MT, and time from symptom onset to recanalization therapy]; study treatment and contrast [type of antithrombotic treatment, indication for antithrombotic administration, time from symptom onset to antithrombotic treatment, number of patients treated with antithrombotic treatment, and information about the control group (when available)]; safety (sICH and all-cause mortality within 6 months); and functional outcome (functional independence after 3–6 months, expressed as a mRS score of 0–2) (18, 19). Special note was made of the definition of sICH in each study.

When available, study characteristics were reported by mean (SDs) or median (interquartile ranges). Outcomes were reported as numbers of cases and percentages. When a comparison was performed or a contingency table could be prepared, odds ratios for both safety (sICH and all-cause mortality) and functional (mRS 0–2) outcomes were reported, with 95% confidence intervals (CIs). If present, adjusted odds ratios (aORs) were also reported. When data were unclear or missing, we extracted data from the related original study (when available) or approached the corresponding author for clarification. Data were reported according to the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) Statement (20). The checklist can be found in the supplementary material (Data Sheet S2 in Supplementary Material).

# RESULTS

#### Study Selection

The systematic literature search yielded a total of 1,270 studies (**Figure 1**). After removing duplicates, 837 articles remained, of which all titles and abstracts were screened. Full text of 17 articles was retrieved and assessed for eligibility. In addition, eight eligible studies were identified through bibliographic review of the included studies. Seven studies were identified in which tandem lesions were investigated, and one RCT investigating the effectiveness of MT was identified, in which periprocedural antithrombotics were used. A total of 19 articles met the selection criteria and were included in the review (21–39).

# Thrombectomy and *Antiplatelet Use*

We identified six studies investigating the periprocedural use of antiplatelet agents (22, 24, 31, 32, 36, 39). These studies include five cohort studies with sample sizes between 35 and 231 patients (22, 24, 31, 36, 39), and one *post hoc* analysis on a phase III RCT of 233 patients (**Table 1**) (32). The occlusion location varied between anterior circulation only (one study) (32), posterior circulation only (one study) (24), and both anterior and posterior circulation (three studies) (22, 36, 39). The occlusion location could not be retrieved in one study (31). In the cohort studies, 57–100% of the study population underwent MT, and in the *post hoc* analysis on phase III RCT data, all patients (in whom the effect of antiplatelet agents was investigated) underwent MT. The indication for antiplatelet use was mainly based on comorbidity (prior use) and prevention of re-occlusion of the vessel after recanalization. The sICH risk for periprocedural antiplatelet use ranged from 6 to 17%. Among the patients using antiplatelet agents, mortality varied from 18 to 33%, and functional independence from 23% to 60% (**Table 2**).

Four studies reported unadjusted relative effects of antiplatelet agents on the risk of sICH (22, 32, 36, 39). Antiplatelet use was associated with a higher relative effect on sICH in two studies in which patients were on prior antiplatelet treatment (OR, 4.80; 95% CI, 1.77–13.02, and OR, 5.43; 95% CI, 1.46–20.13) (32, 36), and a neutral effect in the other studies in which patients received acute antiplatelet treatment in one and were on prior antiplatelet treatment in the other (OR, 0.92; 95% CI, 0.24–3.46, and OR, 0.81; 95% CI, 0.25–2.68) (22, 39). Only one study adjusted the estimate of the relative sICH risk, attributable to antiplatelet use, for prognostic factors (glucose level and baseline NIHSS), but not for prior comorbidity or reperfusion (36). The population of this study was heterogeneous, concerning patients who received IA plasminogen activator and/or MT. The absolute sICH risk was 13% among patients receiving prior antiplatelet treatment and 3% among patients who did not. Prior use of an antiplatelet agent was an independent risk factor for sICH (aOR, 8.03; 95% CI, 1.83–41.70).

Three studies reported unadjusted effect estimates of antiplatelet use on mortality and functional independence (22, 32, 39). The relative effect on mortality was neutral in two studies (OR, 0.75; 95% CI, 0.34–1.67, and OR, 0.97; 95% CI, 0.50–1.85) (22, 39) and higher in the other (OR, 2.46; 95% CI, 1.27–4.76) (32), when antiplatelet agents were used. In all studies, the effect on functional independency was neutral (OR, 0.61; 95% CI, 0.32–1.17, and OR, 0.54; 95% CI, 0.28–1.05, and OR, 1.11; 95% CI, 0.63–1.97).

The *post hoc* analysis of the Multicenter Randomized CLinical trial of Endovascular treatment for Acute ischemic stroke in the Netherlands (MR CLEAN) was the only study in which prior antiplatelet use was directly compared to no prior antiplatelet use in patients who underwent acute MT (32). Prior antiplatelet use was associated with a higher risk of sICH (OR, 4.80; 95% CI, 1.77–13.02) and mortality (OR, 2.46; 95% CI, 1.27–4.76). However, prior antiplatelet use did not interact with MT treatment effect and safety parameters like sICH. Moreover, among patients with successful recanalization, patients on prior antiplatelet use were twice as likely to have a favorable functional outcome (39 vs. 18%, *P*interaction = 0.025). One other study that investigated the recanalization rate found that patients on prior antiplatelet treatment have higher odds for successful recanalization (39).

# Antiplatelet Use in Patients With Tandem Lesions

We identified eight cohort studies in which patients with tandem lesions—that required intracranial MT with or without

Figure 1 | Flowchart of the systematic literature search.

combined emergency carotid artery stenting—received antiplatelet agents as mandatory protocol-based care to prevent stent occlusion (**Table 3**) (21, 23, 25, 28–30, 34, 35). Antithrombotic agents in these studies included eptifibatide, tirofiban, abciximab, acetylsalicylic acid, clopidogrel, and heparin, alone or in combination. The observed sICH risk in the included studies ranged from 0 to 17% (**Table 4**). Mortality ranged from 0 to 39% and functional independence from 29 to 70%. No relative effects on sICH, mortality, or functional independence were reported.

#### Thrombectomy and Heparin Use

Four studies investigated the periprocedural use of heparin (**Table 5**) (27, 33, 37, 38). Two studies were *post hoc* analyses of RCT data (33, 37), one was a cohort study (38), and one was an RCT investigating the efficacy of acute endovascular treatment, which could include periprocedural heparin use (27). All studies investigated the use of unfractionated heparin (UFH). Both anterior and posterior circulation occlusions were included in all studies. The administered heparin dose was reported in all studies and varied between 2,000 and 5,000 IU. Heparin administration

Periprocedural Antithrombotics During MT

Table 1 | Characteristics of included studies investigating periprocedural antiplatelet use in patients with ischemic stroke who underwent acute MT.


*Characteristics of the included studies are presented by sample size (percentage), means (SDs), medians (interquartile ranges), or by remarks.*

*ASA, acetylsalicylic acid; IA, intraarterial; IV, intravenous; MT, mechanical thrombectomy (by means of stent retriever or aspiration); NIHSS, National Institutes of Health Stroke Scale; NR, not reported; RCT, randomized controlled trial; tPA, tissue plasminogen activator.*

*aTime from symptom onset to antithrombotic treatment was divided into acute treatment administration (*<*6 h) and early administration (6–24 h).*

*bNot solely MT.*

*cExtracted data from the subgroup that received antiplatelet agents.*


*Stroke Scale. aAdjusted for glucose level and NIHSS at baseline. bPROACT-II definition (40). cIntracranial hemorrhage resulting in NIHSS increase of* >*4 or non-definable neurologic status with PH and severe mass effect or subarachnoid hemorrhage with hydrocephalus.dIntracranial hemorrhage resulting in NIHSS increase of* ≥*4. eSITS-MOST definition (41).* discretion of the interventionalist in three studies (27, 33, 37). The observed risk of sICH varied between 5 and 12%, mortality between 19 and 33%, and functional independence between 19 and 54% (**Table 6**). Two studies reported an unadjusted effect estimate of heparin on the risk of sICH (33, 37). Both studies suggest that the effect of heparin use on sICH was neutral (8 vs. 11%; OR, 0.73; 95%

CI, 0.11–4.77, and 12 vs. 4%; OR, 3.02; 95% CI, 0.91–9.97) (33, 37). Both studies also reported unadjusted effect estimates for mortality and functional independence. For the latter, also adjusted effects were provided. Both studies suggested that the effect on mortality is neutral (OR, 0.73; 95% CI, 0.23–2.28, and OR, 1.08; 95% CI, 0.54–2.16). After adjustment for prognostic factors [age and final revascularization success in one study (33), and intubation during procedure, postdevice TICI 2b–3, diabetes mellitus, baseline NIHSS score, study device (Trevo vs. Merci), time from symptom onset to arterial puncture (hours), and congestive heart failure in the other (37)], periprocedural heparin use was positively associated with functional independence in both studies (aOR, 5.89; 95% CI, 1.34–25.92, and aOR, 5.30; 95% CI, 1.70–16.48).

was a part of standard care in one study (38) and left to the

One study—which identified predictors for sICH—used a periprocedural loading dose of 3,000 to 5,000 IU UFH, followed by a continuous infusion of 1,000 IU per hour according to standard care (referred to as systemic heparinization) (38). The absolute risk of sICH was 5% in patients who underwent MT and received systemic heparinization. No relative effect of heparin on sICH was reported in this study, neither were mortality nor functional independence.

In the one RCT investigating the effectiveness of MT, periprocedural heparin use was left to the discretion of the treating interventionalist (27). When used, an IV dose of 2,000 IU UFH followed by a subsequent continuous infusion of 500 IU per hour until the end of the procedure was recommended for patients undergoing MT. The risk of sICH in the MT group was 5%. No relative effect on sICH was reported. Both mortality and functional independence occurred in 19% of the patients in this study, but relative effects were not provided.

# Thrombectomy and Antithrombotic Combination Use

One study investigated different antithrombotic combination treatments in the early phase (<24 h) after ischemic stroke (26). Patients had relatively mild anterior or posterior circulation occlusions with a median baseline NIHSS of 11. The early antithrombotic treatment consisted of antiplatelet, anticoagulant, and combined antiplatelet with anticoagulant treatments (**Table 7**). The sICH rate in this study was 3%, mortality 8%, and functional independence 56% (**Table 8**). In this heterogeneous treatment group, in which patients received IV plasminogen activator, IA plasminogen activator, and/or MT, early antithrombotic treatment was not associated with sICH compared to standard antithrombotic treatment after multivariable adjustment (OR 0.56, 95% CI: 0.35 to 2.10) (26). However, both the small group that actually received the combination therapy and the lack of

*fECASS-II definition (42).*

van de Graaf et al.


(*Continued*) Periprocedural Antithrombotics During MT

#### TABLE 3 | Continued


*Characteristics of the included studies are presented by sample size (percentage), mean (SD), median (interquartile ranges), or by remarks.*

*ASA, acetylsalicylic acid; IA, intraarterial; IV, intravenous; MT, mechanical thrombectomy (by means of stent retriever or aspiration); NR, not reported; tPA, tissue plasminogen activator; UFH, unfractionated heparin. aTime from symptom onset to antithrombotic treatment was divided into acute administration (*<*6 h) and early administration (6–24 h). bNot solely MT.*

Table 4 | Outcomes of included studies investigating patients with ischemic stroke caused by a "tandem lesions" who underwent acute mechanical thrombectomy with or without emergency extracranial carotid stenting, who received periprocedural antithrombotic drugs as protocol-based care.


*mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; sICH, symptomatic intracranial hemorrhage.*

*a ECASS-II definition (42).*

*bIntracranial hemorrhage resulting in NIHSS increase of* ≥*4 within 36 h.*

*c SITS-MOST definition (41).*

*dIntracranial hemorrhage resulting in NIHSS increase of* >*4.*

*e No specific definition of sICH given.*

*f Intracranial hemorrhage resulting in NIHSS increase of* ≥*4 or death.*

subanalyses limit the ability to draw conclusions on combination antithrombotic treatments used during MT. This study is mentioned separately, because it did not report outcomes by separate antithombotic regimens.

# DISCUSSION

Based on the available literature, an increased sICH risk for both periprocedural administration of antiplatelet agents and heparin may be expected. Notwithstanding this higher risk of sICH, we found promising results of early antithrombotics regarding functional outcome in ischemic stroke patients undergoing MT. Future studies, especially RCTs, need to determine if the potentially higher sICH risk can be outweighed by improved functional outcome.

#### Antiplatelet Agents

Most studies reported a small but noteworthy higher risk of sICH. Only one study performed multivariable adjustment, in which an aOR of 8.03 was found (36). However, the CI was wide (95% CI, 1.83–41.70), and there may have been residual confounding. Promising results on functional outcome were seen when patients were on prior antiplatelet treatment and a complete recanalization was established, as patients were twice as likely to have a favorable functional outcome (32). This analysis has not been done by the other included studies. Furthermore, the effect of adding antiplatelet agents may have a different result in patients who were treated with IV rtPA (15). However, none of the included studies performed this additional analysis. No further inference was possible.

Previous large randomized trials have investigated the isolated use of antiplatelet agents in general populations of patients with ischemic stroke (i.e., no endovascular treatment) (16, 43). In these studies, the absolute sICH risk associated with antiplatelet administration was approximately 1% when the treatment was initiated within 48 h from symptom onset. MT with or without prior IV tPA bears a sICH risk of 4.4%, ranging from 0 to 7.7% in the large trials (1). The MR CLEAN *post hoc* analysis had not found an interaction between antiplatelet agents and the effect of MT on functional outcome. Taken together, the risk of sICH in patients who undergo MT for ischemic stroke within 6 h and the risk of sICH contributable to antiplatelet agents, this expected risk of sICH is in line with the range from 6 to 17% presented in our review (1, 16).

On the whole, periprocedural use of antiplatelet agents may be a useful adjunct, albeit with a higher sICH risk.

#### Heparin

Although at least one of the reported studies suggested that periprocedural heparin increased the risk of sICH (37), both studies that reported a relative effect of heparin on functional independence showed favorable results (33, 37). However, the true impact of adjunct heparin use remains difficult to determine in these observational studies. Substantial between-center variability in the use of periprocedural heparin exists. Indications varied from no heparin use, to use at the discretion of the interventionalist, and to standard care.

A large RCT has previously investigated the isolated effect of heparin treatment within 48 h in a general population of patients with ischemic stroke (i.e., no endovascular treatment), which resulted in an absolute sICH risk of 1.2% (16). Taken together with the sICH risk of MT, this is in line with the sICH range of 5–12% presented in our review (1, 16). This frequency of sICH is also comparable to the sICH risk in patients treated with acute systemic recombinant tPA in the NINDS and ECASS-III trials (44, 45). In the PROlyse (recombinant prourokinase) in Acute Cerebral Thromboembolism (PROACT) trial—the only randomized double-blind placebo-controlled trial of IA treatment—the use of heparin, at the outset (acute phase) of IA delivery of placebo or recombinant prourokinase (pro-UK), was a significant predictor of both recanalization efficacy and sICH frequency (46). That study set the heparin protocol for the PROACT-II study, in which heparin was administered in combination with recombinant pro-UK. In PROACT-II, both patients in the IA treatment arm and the control arm received heparin; 4,000 IU in total. A non-significant increase of 8% in sICH risk in the endovascular treatment arm compared to the control arm of the study was observed in the univariable analysis, but also an improvement in functional outcome just significant after stratification for stroke severity (40). Based on the available literature, the overall higher risk of sICH may be offset by the improved odds for a functional independence when heparin is used periprocedurally.

#### Strengths and Limitations

In light of two other reviews describing periprocedural antithrombotic use in ischemic stroke management, the strength of this study is the specific focus on MT, the emphasis on safety, the performance of a thorough systematic literature search and the identification of studies not included in both other reviews (47, 48). Another strength is the structured reporting of data according to the PRISMA Statement.

Table 5 | Characteristics of included studies investigating periprocedural heparin use in patients with ischemic stroke who underwent acute MT.


*Characteristics of the included studies are presented by sample size (percentage), mean (SDs), median (interquartile ranges), or by remarks.*

*IA, intraarterial; IU, international unit; IV, intravenous; MT, mechanical thrombectomy (by means of stent retrievers or aspiration); NIHSS, National Institutes of Health Stroke Scale; NR, not reported; RCT, randomized controlled trial; tPA, tissue plasminogen activator; UFH, unfractionated heparin.*

*aTime from symptom onset to antithrombotic treatment was divided in acute administration (*<*6 h) and early administration (6–24 h).*

*bNot solely MT.*

*cAntithrombotic treatment was used in all patients in the MT group.*

*dExtracted data from the subgroup that received heparin.*

*eExtracted data from the population that received penumbral embolectomy.* Table 6 | Outcomes of included studies investigating periprocedural heparin use in patients with ischemic stroke who underwent acute mechanical thrombectomy.


*C, control; CI, confidence interval; aOR, (adjusted) odds ratio; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; NR, not reported; OR, odds ratio; sICH, symptomatic intracranial hemorrhage; T, treated with heparin.*

*a Adjusted for age and final revascularization success in one study.*

*bAdjusted for intubation during procedure, postdevice TICI 2b–3, diabetes mellitus, baseline NIHSS score, study device (Trevo vs. Merci), time from symptom onset to arterial puncture (hours), and congestive heart failure in the other.*

*a SITS-MOST definition (41).*

*bECASS-II definition (42).*

*c No specific definition of sICH given.*

A limitation of this study is that some studies investigating periprocedural antithrombotic use in patients with tandem lesions were not identified by the initial search. This was because these studies did not provide keywords related to antithrombotic treatment use. When we became aware of this finding, we managed this problem through an extensive bibliographic review of the included studies related to this topic. We discussed this issue with our biomedical information specialist, and due to heterogeneity among keywords used in these studies, an additional search was not feasible. It is possible that selection bias has occurred regarding this distinct pathology. On the whole, the risk of sICH seems acceptable in patients with tandem lesions, but the results of this subpopulation should be interpreted with caution, as the causal effects of previous ischemia, misery perfusion, and sudden reperfusion alongside that of antiplatelet treatment cannot be untwined. Because patients with tandem lesions constitute a distinct subpopulation with ischemic stroke, results may also be less generalizable to results in patients with intracranial occlusions only, despite a similar treatment effect of acute MT in these patients (21). However, since limited evidence is available on the safety of periprocedural antiplatelet use in ischemic stroke patients undergoing acute MT, these studies provide valuable information and could therefore not be omitted.

Other limitations were the wide heterogeneity of inclusion criteria, treatment characteristics, and outcome definitions among studies. We found that some studies included patients with posterior circulation occlusion. These patients have a very poor prognosis with high mortality rates (49). Inclusion of these patients could have interfered with the reported outcomes. Furthermore, recanalization therapy varied among included studies from solely MT to more heterogeneous groups that received IV plasminogen activators, IA plasminogen activators, and/or MT. As no distinction was made in some studies, data specifically concerning patients who underwent MT could not always be extracted. This could have blurred the actual effect of periprocedural antithrombotic use in MT. Also, the use of IV plasminogen activator and IA plasminogen activator could have masked the actual sICH risk attributable to the antithrombotics. We also observed that the indication for antithrombotic administration depended on standard care, the discretion of the interventionist, and comorbidities, which dictated prior use. Patients with prior antiplatelet and heparin use were *a priori* more likely to have higher odds for worse outcome than the control group (due to comorbidity or occurrence of re-occlusion)—implying confounding by indication—which may hamper the interpretation of the outcomes and effect estimates. Even though few studies performed a multivariable analysis to adjust for confounding factors, this does not exclude the possibility that residual confounding has influenced our findings. Interpretation of the results of our review has been hampered by missing data in most studies regarding example collateral status, infarct size, and underlying medical conditions for which antithrombotics were administered. Besides, we cannot rule out the possibility of publication bias. Moreover, we could not take dosing into account because of the limited number of studies reporting this. As we focused on periprocedural antiplatelet and heparin use, the effect of other antithrombotic drugs such as direct oral anticoagulants (DOACs) and coumarin derivatives remains unanswered. We chose not to include DOACs and coumarin derivatives as these drugs are not readily available to be administered in the acute phase. Besides, there is no evidence that DOACs and coumarin derivatives can restore microvascular obstruction. Furthermore, the time interval between symptom onset and start of antithrombotic treatment ranged from naught (prior use), through 0–6 h (studies administering the antithrombotic drugs in the *acute* phase during MT), to 6–24 h (studies postponing the antithrombotic treatment to the *early* postprocedural phase). Based on the experimental work, it seems likely that the *acute* use of specific antithrombotic agents could (I) decrease the incidence of sICH by avoiding the later stages of injury evolution and (II) potentially add to improvement of outcomes by preventing or limiting microvascular occlusion within the regions of ischemic injury (6, 7, 9). As the exact underlying pathway by which antithrombotics act—direct link between IMR and antithrombotics—was not in the scope of this review, this should be explored in future research. An example supporting the statement that especially the *acute* phase is of clinical relevance


Table 8 | Outcomes of included studies investigating combined antithrombotic treatments use in patients with ischemic stroke who underwent acute mechanical thrombectomy.


*mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; sICH, symptomatic intracranial hemorrhage; T, treated with antithrombotics. a Intracranial hemorrhage resulting in NIHSS increase of* ≥*4.*

is the use of IV plasminogen activator in current practice. IV plasminogen activator seems safe when used within 4.5 h after stroke onset and improves functional outcome. However, extending this time window increases the risk of sICH significantly offsetting the beneficial effect (50). Possibly, as no clear distinction in time windows (i.e., *acute* or *early*) for antithrombotic treatment was made in most studies, the antithrombotic treatment effect may have been underestimated. Finally, sICH was defined according to various classifications, which makes it difficult to compare sICH risk among studies. Most studies elaborated on the exact sICH definition used (21–26, 28, 29, 31, 32, 34, 35, 38, 39). Most commonly, sICH was defined as neurologic deterioration with a 4 or more point increase in NIHSS score in combination with intracranial hemorrhage on imaging. Not all studies elaborated on the exact definition used. Therefore, heterogeneity among studies could have led to overestimation or underestimation of the actual risk. Due to the large variety in sample sizes and the heterogeneity between studies, a more in-depth exploration will not be helpful.

#### CONCLUSION AND FUTURE DIRECTIONS

Current evidence on periprocedural antiplatelet and heparin use in ischemic stroke patients undergoing acute MT relies on a limited number of *post hoc* analyses and cohort studies. Methodological limitations of these studies warrant cautious interpretation of the results. RCTs investigating the effect of periprocedural antithrombotic treatment in MT are lacking. Some observational studies report a slight increase in sICH risk, which may be acceptable because they also suggest a beneficial effect on functional outcome. Well-conducted phase III RCTs focusing on the acute use of antithrombotic agents alone and in combinations during MT are therefore required. MR CLEAN-MED ("Multicenter Randomized CLinical trial of Endovascular treatment for Acute ischemic stroke in the Netherlands; the effect of periprocedural MEDication: heparin, antiplatelet agents, both or neither") is an ongoing phase III trial that investigates the effect of periprocedural intravenous use of aspirin and/or UFH on functional outcome of ischemic stroke patients undergoing MT (ISRCTN 76741621). We expect that this trial will provide better insights in the balance between potential risks and benefits of the use of these periprocedural antithrombotics for these patients.

#### AUTHOR CONTRIBUTIONS

RG, VC, and BR wrote the first draft of the manuscript and contributed to conception and design of the study; BR, DD, and AL supervised the study; all authors contributed to manuscript revision and read and approved the submitted version.

# ACKNOWLEDGMENTS

We gratefully appreciate the support of our biomedical information specialist, Gerdien B. de Jonge, Erasmus MC, who contributed to the literature search.

# REFERENCES


# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at https://www.frontiersin.org/articles/10.3389/fneur.2018.00238/ full#supplementary-material.


MERCI trial. *J Stroke Cerebrovasc Dis* (2012) 21(8):790–3. doi:10.1016/j. jstrokecerebrovasdis.2011.04.009


**Conflict of Interest Statement:** RvdG has nothing to disclose. VC has nothing to disclose. GdZ has nothing to disclose. Erasmus MC received compensation from Stryker and Bracco Imaging Ltd for activities of DD and AL as a consultant. In addition, DD and AL are the recipients of unrestricted grants from Dutch Heart Foundation, Dutch Brain Foundation, Stryker and Penumbra for the conduct of trials on acute treatment for stroke. BR has nothing to disclose. RvdG, VC, DD, AL, and BR are investigators for the MR CLEAN-MED (ISRCTN 76741621).

*Copyright © 2018 van de Graaf, Chalos, del Zoppo, van der Lugt, Dippel and Roozenbeek. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Repeated Endovascular Treatment of Early Recurrent Proximal Middle Cerebral Artery Occlusion: Case Report and Brief Review of the Literature

#### *Simon Fandler1 , Hannes Deutschmann2 , Franz Fazekas <sup>1</sup> and Thomas Gattringer <sup>1</sup> \**

*1Department of Neurology, Medical University of Graz, Graz, Austria, 2Division of Neuroradiology, Vascular and Interventional Radiology, Department of Radiology, Medical University of Graz, Graz, Austria*

Mechanical thrombectomy (MT) is the gold standard treatment for large vessel occlusion (LVO) stroke of the anterior circulation. Whether MT can also be effectively and safely performed in early recurrent LVO is largely unclear. We present the case of a middle-aged patient who was successfully treated by MT for right proximal middle cerebral artery (MCA) occlusion with excellent outcome. One day after discharge (9 days after the first MT), the patient was readmitted with wake-up stroke. MRI again revealed right proximal MCA occlusion with severe diffusion–perfusion mismatch. Repeat MT was performed and once more led to almost full recovery. The recurrent strokes were attributed to ulcerated non-stenosing plaques in the ipsilateral internal carotid artery, which prompted thromboendarterectomy. In an 18-months follow-up period, no further vascular events occurred. In conclusion, repeated MT for early recurrent LVO appears feasible in carefully selected patients. The collection of similar cases *via* registries would be desirable.

#### *Edited by:*

*Nishant K. Mishra, Tulane University, United States*

#### *Reviewed by:*

*Nawaf Yassi, University of Melbourne, Australia Henry Ma, Monash University, Australia*

#### *\*Correspondence:*

*Thomas Gattringer thomas.gattringer@medunigraz.at*

#### *Specialty section:*

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

> *Received: 24 February 2018 Accepted: 16 April 2018 Published: 03 May 2018*

#### *Citation:*

*Fandler S, Deutschmann H, Fazekas F and Gattringer T (2018) Repeated Endovascular Treatment of Early Recurrent Proximal Middle Cerebral Artery Occlusion: Case Report and Brief Review of the Literature. Front. Neurol. 9:289. doi: 10.3389/fneur.2018.00289*

Keywords: thrombectomy, endovascular treatment, stroke, ischemic stroke, recurrent stroke, large vessel occlusion

#### BACKGROUND

Mechanical thrombectomy (MT) in combination with intravenous thrombolysis (IVT) has become the gold standard treatment for acute ischemic stroke due to proximal large vessel occlusion (LVO) of the anterior circulation (1). In special clinical situations where IVT is contraindicated, MT alone has also been found effective (2).

Intravenous thrombolysis is generally contraindicated in patients with a history of stroke within the last 3 months because of the assumed higher risk of intracranial hemorrhage. Especially the repeated use of IVT in early recurrent stroke might pose a risk, although a small case series has shown that repeated IVT can be safely and effectively administered in carefully selected patients (3).

In such a situation, MT might be an option. However, this has not yet been systematically evaluated and there are only few publications on repeated thrombectomies for LVO stroke (4–6). Data on repeated MT for early recurrent stroke caused by the occlusion of the same affected vessel are especially scarce.

Here, we present a patient with recurrent stroke due to occlusion of the same vessel who was successfully treated by MT twice within 9 days.

# CASE PRESENTATION

A 66-year-old retired female patient was admitted with left-sided hemiplegia, dysarthria, and hemineglect 30 min after symptom onset. The National Institutes of Health Stroke Scale (NIHSS) score was 13. Medical history included coronary heart disease, hypertension, diabetes, and smoking. Notably, she had a history of hemithyroidectomy 1 week prior because of goiter.

Acute non-contrast computed tomography (CT) was unremarkable. CT angiography revealed right-sided middle cerebral artery (MCA) M1 occlusion. IVT was contraindicated because of the recent surgery, and MT was initiated.

Mechanical thrombectomy was successfully performed using a Solitaire stent retriever (thrombolysis in cerebral infarction scale 3, **Figures 1A–B**). Symptom-to-recanalization time was 130 min.

After the procedure, the patient showed rapid major neurological improvement. Brain MRI on day 4 demonstrated small ischemic infarcts in the right posterior basal ganglia region and right temporal cortex (**Figure 1C**). While thorough cardiac work-up including stroke unit ECG monitoring, 24-h ECG and transesophageal echocardiography showed no cardioembolic source, duplex sonography revealed an irregular-shaped ulcerated plaque formation in the right internal carotid artery (ICA) origin but without relevant stenosis (peak systolic velocity of 66 cm/s on doppler sonography compared to 57 cm/s contralaterally; luminal stenosis of 40% on CT angiography). Therefore, only antiplatelet and statin therapy was initiated and the patient was discharged home with an excellent outcome on day 8 [NIHSS: 0, modified Rankin scale (mRS): 1].

On the next morning (day 9 after the index stroke), she was readmitted with wake-up-stroke (last seen well 10 h before) and had again a right total anterior circulation stroke syndrome (NIHSS: 16).

Multimodal MRI was performed. Aside from the past infarction, no new diffusion or FLAIR-positive lesions were found. However, a right-sided proximal MCA occlusion was present again and was associated with severe hypoperfusion (**Figures 2A–C**).

As thrombolysis was contraindicated (both because of recent surgery and recent stroke) and MRI showed an extensive diffusion–perfusion mismatch, MT was again successfully

Figure 2 | Initial MRI scan at second admission showing only a small DWI lesion corresponding to the preceding stroke (A), hypoperfusion of large parts of the middle cerebral artery (MCA) territory [(B), mean transit time], and right proximal MCA occlusion [(C), time-of-flight angiography]. Digital subtraction angiography pre- (D) and post-thrombectomy (E).

performed with a Solitaire stent retriever (door-to-recanalization time: 149 min, thrombolysis in cerebral infarction scale 3, **Figures 2D–E**).

The neurological exam on the following day showed once more remarkable recovery (NIHSS: 0). Postinterventional duplex sonography and contrast-enhanced MR-angiography did not show any signs of vasospasm or vessel dissection.

Detailed etiological re-evaluation including CT angiography of the aortic arch, repeated echocardiography, and 24-h ECG revealed no new findings. The already known ulcerated plaque at the right ICA origin (**Figure 3**) was, therefore, considered as the most likely cause of the two strokes, and uneventful carotid thrombendarteriectomy was performed. The patient was discharged home with only minimal clumsiness of the left hand (NIHSS: 0, mRS: 1).

Repeated neurological follow-up examinations were performed at 3, 6, and 18 months, each showing no further vascular events and stable neurosonographic findings.

# DISCUSSION

We here present the case of a patient with probable macroangiopathy-related recurrent severe stroke who underwent successful MT of the same occluded proximal MCA twice within a few days.

To our knowledge, there are only three publications on repeated MT for stroke within a short time frame. Those are two single case reports and one case series, the latter is composed of patients with a wide range of interthrombectomy intervals, ranging from less than 1 to 278 days (4–6). In total, eight cases treated by repeated MT in the MCA or intracranial ICA within <14 days have been reported to date. In half of these cases, re-thrombectomy was performed in the same vessel, in the other half, LVO had reoccurred contralaterally. While MT was technically successful in all these cases, patients' outcome varied substantially (mRS at 90 days or discharge, if former not available: 0–2: *n* = 5; mRS 3–5: *n* = 0; mRS 6/death: *n* = 3).

Contrary to our patient, cardioembolism was the predominant stroke etiology in prior reports. This is the first detailed description of a macroangiopathy-related early recurrent stroke treated by re-thrombectomy.

Because repeated and extensive stroke work-up showed no indication for other etiologies, the severe ulcerated plaque formation in the ipsilateral ICA was considered the most likely cause for the recurrent strokes (7). As the patient suffered from two LVO strokes within a short time period, we decided to perform carotid thromboendoarterectomy despite missing evidence for this approach. The absence of further cerebrovascular events during follow-up is in favor of this decision. Initiation of short-term (e.g., 3 months) dual antiplatelet therapy both after the first or second stroke event would have also been an alternative treatment approach (8).

The excellent outcome after two LVO strokes can be attributed to two main reasons. At the first stroke, symptom-to-recanalization time was rather short. For the second stroke, symptom onset was unclear due to the wake-up stroke scenario. However, MRI showed no new DWI lesion despite a large perfusion deficit. This fact also argues for a short interval between stroke onset and intervention alongside the presence of a good collateral circulation. Interestingly, final infarct size after both thrombectomies was minimal.

Repeated thrombectomy may lead to more severe disruption of the vascular endothelium, thereby increasing the risk of complications such as vasospasm, arterial dissection, as well as intracranial hemorrhage (9). In our case (with an interthrombectomy interval of 9 days), no such complications occurred. In line with this finding, a previous study using 3-T vessel wall MRI conducted within 1 week after MT reported no relevant residual vessel wall injuries (10). Furthermore, it has to be noted that our patient underwent MT immediately in both situations and did not receive IVT. IVT potentially promotes blood–brain barrier disruption and neurotoxicity, and thus might increase ICH risk (11). It is, therefore, unfortunate that the previously mentioned case series did not report on concomitant IVT treatment (4).

# CONCLUDING REMARKS

Repeated MT for early recurrent LVO stroke appears feasible in carefully selected patients. However, more experience in the management of such patients is important and the collection of similar cases *via* registries would be desirable.

# ETHICS STATEMENT

Written informed patient consent for publication of this case report has been obtained.

# AUTHOR CONTRIBUTIONS

SF and TG contributed conception and design of the study; SF wrote the first draft of the manuscript; HD, FF, and TG wrote sections of the manuscript. All authors contributed to data analysis and interpretation as well as to the manuscript revision, read, and approved the submitted version.

# REFERENCES


in patients with cryptogenic stroke. *J Am Heart Assoc* (2015) 4:e002012. doi:10.1161/JAHA.115.002012


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2018 Fandler, Deutschmann, Fazekas and Gattringer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Neuroimaging Paradigms to identify Patients for Reperfusion Therapy in Stroke of Unknown Onset

*Mark R. Etherton1 , Andrew D. Barreto2 , Lee H. Schwamm1 and Ona Wu1,3\**

*1Department of Neurology, JPK Stroke Research Center, Massachusetts General Hospital (MGH), Harvard Medical School, Boston, MA, United States, 2Stroke Division, Department of Neurology, McGovern Medical School at the University of Texas Health Science Center at Houston, Houston, TX, United States, 3Department of Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital (MGH), Charlestown, MA, United States*

Despite the proven efficacy of intravenous alteplase or endovascular thrombectomy for the treatment of patients with acute ischemic stroke, only a minority receive these treatments. This low treatment rate is due in large part to delay in hospital arrival or uncertainty as to the exact time of onset of ischemic stroke, which renders patients outside the current guideline-recommended window of eligibility for receiving these therapeutics. However, recent pivotal clinical trials of late-window thrombectomy now force us to rethink the value of a simplistic chronological formulation that "time is brain." We must recognize a more nuanced concept that the rate of tissue death as a function of time is not invariant, that still salvageable tissue at risk of infarction may be present up to 24 h after last-known well, and that those patients may strongly benefit from reperfusion. Multiple studies have sought to address this clinical dilemma using neuroimaging methods to identify a radiographic time-stamp of stroke onset or evidence of salvageable ischemic tissue and thereby increase the number of patients eligible for reperfusion therapies. In this review, we provide a critical analysis of the current state of neuroimaging techniques to select patients with unwitnessed stroke for revascularization therapies and speculate on the future direction of this clinically relevant area of stroke research.

Keywords: ischemic stroke, neuroimaging, reperfusion therapy, unwitnessed stroke, wake-up stroke

# INTRODUCTION

The treatment options for acute ischemic stroke are currently predicated on a confirmed last-known well (LKW) and the time-period from LKW to hospital evaluation. For those patients that present and start treatment within 4.5 h from LKW, administration of intravenous recombinant tissue plasminogen activator (IV tPA) reduces disability after acute ischemic stroke (1, 2). Likewise, those with large-vessel occlusions (LVO) of the anterior circulation who can start treatment within 6 h of LKW, endovascular thrombectomy (EVT) is a powerful therapy for improving long-term functional outcomes (3–6). Recently, two pivotal trials (7, 8) now extend that window up to 24 h in highly selected patients with imaging demonstrating small infarct core lesions and salvageable tissue by imaging or clinical measures. Unfortunately, these efficacious acute therapies are limited both by the relatively narrow treatment window for either IV tPA or EVT, and the relatively infrequent (5.7–12.8%) occurrence of LVO accompanied by a favorable tissue signature in the later time windows (9). Because of this, many ischemic stroke patients are not eligible for these therapies. While the rates of thrombolysis

#### *Edited by:*

*Nishant K. Mishra, Tulane University, United States*

#### *Reviewed by:*

*Richard Zweifler, Ochsner Health System, United States Vincent Thijs, Florey Institute of Neuroscience and Mental Health, Australia*

> *\*Correspondence: Ona Wu ona@nmr.mgh.harvard.edu*

#### *Specialty section:*

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

*Received: 01 March 2018 Accepted: 25 April 2018 Published: 15 May 2018*

#### *Citation:*

*Etherton MR, Barreto AD, Schwamm LH and Wu O (2018) Neuroimaging Paradigms to Identify Patients for Reperfusion Therapy in Stroke of Unknown Onset. Front. Neurol. 9:327. doi: 10.3389/fneur.2018.00327*

are increasing in the United States, conservative estimates in 2009 suggested that only 3–5% of all stroke patients receive treatment with IV tPA (10). One reason for its underuse may be the strict time restrictions from LKW (11). Exacerbating this issue, current estimates suggest that 31–36% of acute ischemic stroke patients have stroke of unknown symptom onset (SUSO) but do have an LKW time (12, 13), with a large proportion of these with deficits upon awakening, or "wake-up strokes" (WUS) (14–16). These patients with SUSO highlight the challenge of relying on a human witness of symptom onset, which greatly limits the opportunities for reperfusion therapy.

For these reasons, there is much interest in developing novel approaches to expand patient eligibility for revascularization therapies (e.g., IV tPA or EVT) to SUSO patients. Given the potentially large proportion of ischemic stroke patients that these populations represent, identifying approaches to discern which patients with SUSO may still safely benefit from reperfusion therapy holds enormous clinical and epidemiological ramifications. The interest in this question is exemplified by the increasing number of publications on this topic (**Figure 1**). Advanced neuroimaging has been applied to patients with SUSO based on two principles: (1) to substitute for the human witness of stroke onset by providing radiographic surrogates for stroke duration or (2) to identify patients with sufficient salvageable tissue at risk of dying to make the potential benefit of revascularization therapy worth the risk and considerable resource utilization of "late" intervention. The DAWN (DWI or CTP Assessment with Clinical Mismatch in the Triage of Wake-Up and Late Presenting Strokes Undergoing Neurointervention with Trevo) and DEFUSE 3 (Endovascular

Figure 1 | PUBMED search on January 11, 2018 (294 hits, 112 relevant) using the following terms: "stroke"[Title/Abstract] AND ("unwitnessed"[Title/ Abstract] OR "unclear onset"[Title/Abstract] OR "unclear-onset"[Title/ Abstract] OR "wake"[Title/Abstract] OR "wakeup"[Title/Abstract] OR awake\* [Title/Abstract] OR "unknown onset"[Title/Abstract] OR "unknownonset"[Title/Abstract]) AND ("trial"[Title/Abstract] OR therap\* [Title/Abstract] OR treat\*[Title/Abstract] OR thrombolysis [Title/Abstract]) NOT ("review"[Publication Type] OR "review literature as topic"[MeSH Terms]) NOT ("animals"[MeSH Terms:noexp] OR animal[All Fields]) demonstrate increasing interest in the treatment of patients with unknown symptom onset restricted up to December 31, 2017.

Therapy Following Imaging Evaluation for Ischemic Stroke) randomized clinical trials (RCTs) used different approaches to identify these potential candidates for EVT, and their success has much to teach us about the patients who can still benefit from late-window reperfusion. In this review, we will discuss the current evidence supporting the use of neuroimaging approaches for evaluation of patients with SUSO to identify populations that may benefit from delayed reperfusion interventions.

# WAKE-UP STROKE AS A DISTINCT CLASS OF SUSO

Wake-up strokes are hypothesized to represent a unique entity within SUSO as it is difficult to delineate the timing of stroke onset including the possibility that it may have occurred on awakening (17). Many have posited that there is a circadian variation in the frequency of ischemic stroke with most strokes occurring between 6:00 AM and 12:00 PM (15, 18–22). A metaanalysis of 8,250 patients with ischemic stroke demonstrated a 55% increased risk between 6 AM and noon (22). The diurnal variation in ischemic stroke is also thought to have contributions from morning increases in blood pressure, platelet aggregation, and prothrombotic factors (23–25). These observations have led many to speculate on circadian-related mechanisms underlying WUS, similar to those reported for myocardial infarction, and that WUS patients may have stroke onset contiguous with wakening. Multiple studies have in fact shown comparable presentation and outcomes in WUS patients vs. those with witnessed stroke onset (14, 19, 26, 27). One large study that investigated the cohort of WUS patients enrolled in the International Stroke Trial found that WUS patients, despite presenting with milder symptoms, had similar mortality rates and likelihood of poor outcome as patients with stroke onset while awake (28).

On the other hand, non-wake-up SUSO patients appear to represent a different clinical population than WUS SUSO patients. One study demonstrated that non-wake-up SUSO patients differ clinically from wake-up SUSO patients (more severe symptoms, faster arrival time from symptom discovery). Non-wake-up SUSO patients also appear to have worse prognosis than either WUS or witnessed stroke patients and the proportion of patients with non-wake-up SUSO may be increasing (29, 30). These findings suggest that both wake-up and non-wake-up SUSO patients represent a vulnerable subpopulation of AIS patients in need of developing new management paradigms for expanding reperfusion therapy eligibility. With the success of the late-window EVT trials (7, 8) and new guideline recommendations (11), the clinical focus should shift to expanding therapies for late-window patients without LVO or who lack rapid access to advanced neuroimaging and EVT.

# SUSO VS. STROKE OF KNOWN SYMPTOM ONSET (SKSO)

# Computed Tomography (CT)

There have been several CT-based approaches to characterizing SUSO patients as compared with their witnessed stroke counterparts. Multiple studies have demonstrated no significant difference in the extent of ischemic changes on the admission CT between WUS SUSO and SKSO patients (31–33) using the Alberta stroke program early CT score (ASPECTS) scale, which is a CT-based assessment of early ischemic changes in the middle cerebral artery (MCA) territory (34). Further supporting the argument that WUS SUSO patients may represent a distinct population, a study comparing cardioembolic SKSO (46 patients), non-WUS SUSO (18 patients), and WUS (17 patients) observed no significant difference between the SKSO and WUS groups in the number of patients presenting with a normal head CT (30 vs. 22%, *P* = 0.76) or hypodense area (0 vs. 11%, *P* = 0.069) (26). However, no patients in the non-WUS SUSO group had a normal head CT and 56% had a visualized hypodense area (*P* < 0.001) (26). Another study used CT perfusion (CTP) to characterize 420 stroke patients with known symptom onset, 131 patients with WUS, and 125 with non-wake-up SUSO (35). The non-wake-up SUSO group had larger lesion volumes on CT-angiogram source images compared with the other two groups (46.6-cm3 SUSO vs. 14.3-cm3 SKSO vs. 14.4-cm3 WUS, *P* = 0.04) but no difference in the frequency of CTP mismatch or presence of LVO (35).

#### Magnetic Resonance Imaging (MRI)

Magnetic resonance imaging-based approaches to identifying WUS patients who may benefit from reperfusion have also been performed. A retrospective study of 364 stroke patients, which included 100 patients with WUS, showed no differences in median stroke severity, as assessed by National Institutes of Health Stroke Scale (NIHSS) score (SKSO 7 vs. WUS 5; *P* = 0.06), age, or gender between the WUS and known stroke onset groups (14). Notably, while time from stroke onset was shorter in the known stroke onset group (6.0 vs. 13.3 h, *P* < 0.001), there was no significant difference in time from symptom detection (6.0 vs. 5.9 h, *P* = 0.83) (14). Of those patients imaged within 3 h of symptom discovery (*N* = 69), there was no difference in either the diffusion-weighted imaging (DWI: 26.8-cm3 SKSO vs. 19.6 cm3 WUS) or perfusion-weighted imaging (PWI) lesion volumes (107.7-cm3 SKSO vs. 82.7-cm3 WUS) (14).

Imaging findings of non-WUS SUSO patients have also been characterized with MRI, though not to the same extent as the WUS cohort. One retrospective study found that non-WUS SUSO patients (*N* = 104) were more likely to have DWI and FLAIR (fluid-attenuated inversion recovery) mismatch (non-WUS SUSO: 35.1 vs. WUS: 21.9%; *P* = 0.02), and DWI-PWI mismatch (*P* = 0.001) than WUS (*N* = 172) (13). However, a prospective study of SUSO patients found that the frequency of DWI-FLAIR mismatch (DFM), defined as a visible acute lesion on DWI but no obvious parenchymal hyperintensity in the corresponding region of the FLAIR sequence, was similar in both the WUS and non-WUS SUSO groups (43.7 vs. 48.7%; *P* = 0.3) (30).

#### NEUROIMAGING TIME-STAMP OF STROKE DURATION

The aforementioned studies suggest that SUSO patients present with similar clinical and imaging findings as their SKSO counterparts, as long as they are evaluated within a comparable time frame from stroke onset. This observation has prompted the utilization of imaging as a potential surrogate witness of onset when no human witness is available. Before using an imagingbased witness, it is critical to determine the key imaging features that can discriminate between patients within the therapeutic time window and those who are outside the window. Imagingsurrogates for stroke duration have been primarily based on MRI, specifically, the DWI (or apparent diffusion coefficient, ADC) and FLAIR sequences. Several clinical studies of acute ischemic stroke patients have strengthened the argument that patients with abnormal ADC and normal FLAIR are likely within 3–4 h of stroke onset and that assessing for DFM may be associated with stroke duration (**Figure 2**) (36–39).

In one study of AIS patients with brain MRI obtained within 12 h of stroke onset, the median time from known symptom onset was significantly longer in FLAIR-positive vs. FLAIRnegative patients (189 min, interquartile range 110–369 vs. 103 min, interquartile range 75–183 min; *P* = 0.011) (39). Moreover, in patients with infarct volume exceeding 0.5-cm3 on DWI, FLAIR-negative MRI showed 80% specificity and 51% sensitivity for imaging within 3 h of stroke onset (39). However, the authors observed no significant correlation between the signal intensity ratio and time from stroke onset. In contrast, another study showed a strong positive correlation between the time from stroke onset and the intensity of the FLAIR signal change relative to its contralateral homologous region (40). These findings support the idea that with longer stroke duration, the likelihood of visible FLAIR abnormalities increases. This allows for the hypothesis that patients with visible changes on DWI (or ADC) but normal FLAIR will likely have relatively recent stroke onset.

Several studies have strengthened the idea that DFM can inform on stroke duration. One retrospective investigation of 120 patients with AIS within 6 h of known symptom onset showed that presence of DFM identified patients with stroke onsets within 3 h or less with 93% specificity and 48% sensitivity (36). Importantly, 98.3% of the study population had confirmed arterial occlusions. Those patients that were FLAIR-positive were imaged significantly later than the FLAIR-negative group (180 vs. 120 min, *P* < 0.001) (36). In a retrospective, multicenter followup to this study involving 543 patients with AIS, DFM identified with 78% specificity and 62% sensitivity patients within 4.5 h and 87% specificity and 56% sensitivity patients within 6 h from stroke onset (41). Two additional studies demonstrated that scans with DFM were highly specific (71–80%) for identifying patients within 3 h of stroke onset (37, 38). Lastly, another study demonstrated that the presence of DFM on 3T MRI also has a high positive predictive value (88%) for the stroke occurring within 4.5 h; however, 44.5% of this population had positive FLAIR within 4.5 h of stroke onset and would not considered DFM (42). These results suggest that on 3T MRI, the presence of DFM can identify patients with stroke onset <4.5 h with high specificity but that a significant percentage of patients in the <4.5 h window can have positive FLAIR signals. Taken together, these findings demonstrate that MRI can, with high specificity, identify patients in the hyperacute (i.e., <3–4.5 h) phases of AIS based on DFM.

Using neuroimaging to serve as a radiographic biomarker of stroke onset holds much promise for potentially expanding eligibility for thrombolytic therapy. One analysis of WUS patients with DFM suggested that an additional 30% would be eligible for treatment with IV tPA (43). As such, several clinical trials have asked the question of whether DFM can be safely and efficaciously used for the treatment of SUSO patients with thrombolytics.

# MOVING BEYOND THE CLOCK: SHIFTING THE PARADIGM FROM "TIME IS BRAIN" TO "IMAGING IS BRAIN"

Complementary to the notion that imaging can serve as a surrogate for stroke duration is that imaging can directly measure the degree of injury the brain has already experienced from the ischemic event. While the duration of time since symptom onset is highly correlated with progression of brain tissue injury, there is tremendous between-subject variability as to the rate of tissue death in the face of a heterogeneous degree of ischemia. While one can calculate an average rate of neuronal death in AIS due to LVO (1.9 million neurons/min) (44), recent data confirm that many patients still have viable tissue well beyond 6 h after symptom onset. Early animal models of ischemic stroke have supported the hypothesis that mismatch between the DWI and T2-weighted signals reflect histologically salvageable tissue, which happen to also be associated with short stroke durations (45). This has prompted different neuroimaging approaches to quantify or characterize salvageable tissue as a radiographic surrogate of patients likely to benefit from reperfusion therapies. The importance of using neuroimaging to identify patients likely to benefit from reperfusion therapies is exemplified, in part, by the results of the International Management of Stroke III RCT (46), which enrolled 53.4% of subjects with no baseline CTA to confirm LVO and 45% of subjects with ASPECTS <8, and failed to show benefit of EVT, as compared with the positive EVT trials of 2015, which used stricter criteria for identifying LVO patients with small ischemic cores (4, 5, 47, 48). However, this trial used mostly first- and second-generation devices and it is unknown what the impact would have been if stent retrievers had been used. In the next sections, we will review the different imagingbased approaches to quantify viable tissue in patients with AIS independent from LKW.

# Infarct Core–Perfusion Mismatch

One approach has been to apply MRI or CT-based imaging techniques to quantify infarct core–perfusion mismatch as an indicator of salvageable tissue. While these two modalities measure core in very different ways, they both seek to differentiate irreparably injured tissue from tissue that is potentially recoverable.

#### MRI-Based Perfusion–Diffusion Mismatch

Magnetic resonance imaging-based techniques are one method that has been utilized to quantify salvageable tissue. Tissue that is abnormal on DWI due to restricted diffusion typically represents tissue that has the highest probability of infarction, with tissue salvage rare even with reperfusion (49), and therefore typically referred to as the infarcted "core" (50). PWIs, in particular gadolinium-arrival time measures such as *T*max [time to peak value of deconvolved residue function (51)], have been used to identify tissue that is at risk for ischemic infarction but has not yet irreversibly committed to cell death (45, 51–56). These observations represent the foundations of utilizing PWI–DWI mismatch to identify salvageable tissue as an alternative triage approach for SUSO patients (**Figure 3**).

Investigating this hypothesis, the Diffusion and Perfusion Imaging Evaluation for Understanding Stroke Evolution (DEFUSE) study was an observational study of IV tPA-treated patients for which target perfusion–diffusion mismatch was defined as mismatch volume (PWI–DWI) > 10 cm3 or mismatch ratio (PWI/DWI) > 1.2. PWI lesion was defined as tissue exhibiting *T*max ≥ 2 s. MRI was obtained before and 3–6 h after IV tPA treatment. A total of 68% of the study population had a confirmed partial or complete arterial occlusion of the internal carotid artery (ICA), MCA, or posterior cerebral artery (PCA). DEFUSE enrolled 74 patients and found that 56% (*N* = 18) with perfusion–diffusion mismatch and early reperfusion had a favorable outcome (defined as improvement of NIHSS between baseline and 30 days of 8 points or more or score of 0–1 at day 30) (57).

In a subsequent prospective cohort study, DEFUSE 2 (58), the same approach was applied to patients with LVO of the anterior circulation (defined as intracranial ICA or first segment of the MCA) treated with EVT within 12 h of LKW. The target mismatch profile was notably modified from that used in DEFUSE; mismatch ratio > 1.8 (*T*max > 6-s volume/DWI volume) and an absolute difference ≥15 cm3 , DWI lesion volume <70 cm3 , and *T*max > 10-s volume <100 cm3 . In the 78 patients with target mismatch, the adjusted odds ratio for favorable outcome (same definition as DEFUSE) with reperfusion was 8.8 (95%

CI 2.7–29) compared with 0.2 (95% CI 0–1.6) in the no target mismatch group (*P* = 0.003). Moreover, in the target mismatch group, reperfusion was associated with decreased infarct growth at 5 days (30 vs. 73 cm3 , *P* = 0.01). It should be noted that both DEFUSE and DEFUSE 2 were observational studies with treatment decisions made independent of imaging findings, which introduce selection bias; once enrolled, all patients received IV tPA or endovascular intervention.

#### CT-Based Infarct Core–Perfusion Mismatch

Due to the relative insensitivity of non-contrast CT for detecting early ischemic changes (59), alternative methods of defining core lesion volume are needed for CT-based screening methods. Thresholded relative cerebral blood flow (rCBF) maps have been used to approximate core lesion volumes, though, unlike DWI, they do not measure tissue infarction (**Figure 4**). Rather they are based on the probabilistic association that areas with substantive hypoperfusion are highly likely to progress to infarction despite reperfusion. These estimates can be highly variable at two extreme conditions: (1) patients with LVO stroke imaged early at stroke onset exhibiting large rCBF lesion volumes that grossly overestimate final infarct volume in settings of early reperfusion, and (2) patients with many hours of occlusion imaged after late

Figure 4 | CT perfusion (CTP) cerebral blood flow (CBF) maps do not correspond with infarct core. Eighty-four-year-old male with left middle cerebral artery (MCA) stroke with dense distal M2 occlusion presenting with initial National Institutes of Health Stroke Scale (NIHSS) of 9. By the time of the admission, NIHSS was 2 and patient did not receive IV tPA or endovascular treatment. The CTA/ CT perfusion (CTP) was acquired at 4.9 h from last-known well (LKW), magnetic resonance imaging was performed 19 min after the CTP. (A) CBF, (B) diffusion-weighted imaging (DWI) performed 19 min after CTP, (C) CTA shows occlusion of MCA superior division segment, (D) DWI with acute infarct mapped in red, (E) CT head at 24 h from LKW, and (F) follow-up FLAIR image at 34 days post-stroke depicting final infarct. Note the CTP CBF hypoperfused region identified as 30% of mean contralateral hemisphere is much larger than that of the DWI scan, and corresponded better with tissue infarction on follow-up. *Courtesy of William A. Copen, MD, Department of Radiology, Massachusetts General Hospital. Data analysis for figure was created under approval of local ethics committee.*

reperfusion demonstrating small or no CBF volumes (due to normal or increased CBF values in previously hypoperfused tissue) that would grossly underestimate final infarct volume. However, these conditions are infrequent, and many trials and centers have adopted a CT-based approach to identify subjects with tissue at risk, using very low values in CTP-derived CBF values to define "core," with CTP-derived tracer arrival time metrics used to represent tissue at risk (5) at centers which do not perform acute stroke MRI. Although the accuracy of using a perfusion metric to define infarction "core" is still debated (60–62), this approach has successfully identified a group of patients who respond to reperfusion therapy beyond 3- and 4.5-h time windows (see below). Some have suggested using thresholded relative cerebral blood volume (rCBV) maps to identify core (63) or absolute CBV values <2 cm3 /100 g (64, 65), but many studies have shown that CBV is not a robust surrogate for infarct core (66–71). However, investigators have shown that very low CBV might be an indicator of risk for future hemorrhagic transformation (72–75) or poor outcome after EVT (76, 77).

#### Clinical Trials of Infarct Core–Perfusion Mismatch Involving Non-SUSO Patients

There have been several trials applying the principles of infarct core–perfusion mismatch in the administration of IV thrombolytics to late-window AIS patients with varying degrees of success (**Table 1**). The Desmoteplase in Acute Ischemic Stroke Trials (DIAS and DIAS-2) used an alternative thrombolytic, desmoteplase (more specific for fibrin than alteplase) and core– perfusion mismatch (>20%) for the treatment of late-window AIS patients 3–9 h from LKW (78, 79). DIAS was a dose escalation study of desmoteplase. Phase 1 of DIAS was halted because of high rates of sICH with desmoteplase doses of 25–50 mg (26.7%) (78). Phase 2 of DIAS, however, showed that with desmoteplase doses of 62.5–125 µg/kg the rates of sICH were 2.2% and reperfusion rates were 71.4 vs. 19.2% with placebo. Of note, reperfusion in this trial was defined as a ≥30% reduction in mean transit time or ≥2 points improvement on the Thrombolysis in Myocardial


*a Combined analysis defined at 90 days as* ≥*8 point improvement or scoring 0 to 1 on NIHSS, score of 0 to 2 on mRS, and a BI score of 75 to 100.*

*bEighty-three patients treated using PWI/DWI mismatch. Fifteen patients treated based small-core defined as ASPECTS* ≥*6 on CT or MRI.*

*c Trial completed or terminated but not yet published. Trial in progress.*

*dEarly neurologic improvement defined as reduction of eight points or more on NIHSS or score of 0 or 1 at 72 h.*

*ACA, anterior cerebral artery; CTA, CT angiogram; CTP, CT perfusion; DWI, diffusion weighted imaging; EVT, endovascular thrombectomy; MCA, middle cerebral artery; mRS, modified Rankin scale; NIHSS, National Institutes of Health Stroke Scale; PCA, posterior cerebral artery; PWI, perfusion weighted imaging.*

*sICH criteria: ECASS II:* ≥*4pt* ↑ *NIHSS and any ICH; NINDS: any neurologic worsening due to any ICH; PROACT II:* ≥*4pt* ↑ *NIHSS at 36 h and any ICH; SITS-MOST:* ≥*4pt* ↑ *NIHSS at 24 h and PH2 HT.*

infarction grading scale (78). Although the trial was not powered to detect efficacy, at 90 days there was a dose-dependent rate of favorable outcome (defined as Barthel index > 75, modified Rankin scale (mRS) ≤ 2, and NIHSS 0–1 or improvement of 8 points) of 60% with 125 µg/kg vs. 18.2% placebo (78). The Dose Escalation of Desmoteplase for Acute ischemic Stroke (DEDAS) trial was a placebo-controlled, dose-escalation study of 90 and 125-µg/kg desmoteplase in 37 patients 3–9 h from LKW (80). No sICH occurred in any group and there appeared to be a dosedependent effect of desmoteplase on reperfusion rates (53.3% 125-µg/kg desmoteplase vs. 18.2% 90-µg/kg desmoteplase vs. 37.5% placebo) (80). In DIAS-2, 186 patients were randomized to either 90 or 125-µg/kg desmoteplase or placebo 3–9 h from LKW utilizing the same infarct core–perfusion mismatch criteria. Notably, in addition to MRI, CTP was also used in DIAS-2 for assessing infarct core–perfusion mismatch (64 patients); however, mismatch was determined based on a visual, qualitative assessment. DIAS-2 had a favorable safety profile but there was no difference in the rates of favorable outcome at 90 days, median change in infarct volume, or rates of sICH (79).

Further analysis of DIAS, DIAS 2, and DEDAS was pursued given the disparate results of Phase-2 trials (DIAS and DEDAS), suggesting efficacy and the negative efficacy results of Phase-3 trial (DIAS 2). In comparing the patient populations of the three trials, it was noted that there was a substantial difference between DIAS 2 and DIAS/DEDAS in the number of patients with intracranial vascular occlusion or high-grade stenosis (DIAS 2 30% vs. DIAS/ DEDAS 57%; *P* ≤ 0.0001) (81). Moreover, in the pooled analysis of DIAS, DIAS 2, and DEDAS, desmoteplase treatment showed a favorable effect at 90 days in patients with either an intracranial vascular occlusion or high-grade stenosis (OR 4.14; 95% CI 1.40–12.23; *P* = 0.01) (81). Of note, favorable clinical response was defined as the composite of ≥8 point improvement in NIHSS (or 0–1), mRS < 3, and a Barthel Index Score ≥75 at 90 days. The subsequent randomized control trials DIAS-3 (82) and DIAS-4 (83) notably did not require infarct core–perfusion mismatch for enrollment, but only occlusion or stenosis of proximal segments of the middle, posterior, or anterior cerebral arteries and acute infarct lesion (on DWI or non-contrast CT) involving less than 1/3 MCA territory or 1/2 the anterior cerebral artery (ACA) or PCA territory. Both studies showed no safety concerns, but also no benefit 90-day functional outcomes (mRS < 3). Taken together, the results of the DIAS and DEDAS trials are mixed with regard to utilizing neuroimaging to select late-window stroke patients for treatment with thrombolytic therapy. On one hand, desmoteplase 3–9 h from LKW did not improve functional outcomes. However, a positive aspect of these studies was their demonstration that infarct core–perfusion mismatch can be effectively used in the emergent setting to efficiently triage acute stroke patients for potential treatment with thrombolytic therapy.

Around the same time that the DIAS 1–2 and DEDAS trials were underway to investigate desmoteplase with neuroimagingbased patient selection, several studies were simultaneously studying selection approaches for IV tPA using perfusion–diffusion mismatch. The Echoplanar Imaging Thrombolytic Evaluation Trial (EPITHET) was a Phase 2, observational trial of IV tPA in AIS patients 3–6 h from symptom onset (84). Out of 101 patients, 86% had perfusion–diffusion mismatch, using the same definition as DEFUSE. Of those patients that received IV tPA, there was decreased infarct growth (growth > 0%: 54% IV tPA vs. 77% placebo, *P* = 0.032) and increased reperfusion > 90% (56% IV tPA vs. 26% placebo, *P* = 0.01). Overall, however, there was no difference in 90-day mRS between the IV tPA and placebo groups (mRS < 3: 45% IV tPA vs. 40% placebo, *P* = 0.66). *Post hoc* analysis suggested that the previous failure of EPITHET was potentially due to too low a threshold for defining the PWI lesion (*T*max > 2 s) (85) and subsequent studies by these investigators have used a stricter threshold of *T*max > 6 s to define salvageable tissue.

An RCT used an alternative tissue plasminogen activator, tenecteplase, in AIS patients with infarct core–perfusion mismatch. This Phase-2B trial of tenecteplase for AIS, two doses of tenecteplase (0.1 or 0.25 mg/kg) administered within 6 h of stroke onset, was compared with IV tPA (86). Eligibility criteria included a CTP mismatch of greater than 20% and verified occlusion of an anterior, middle, or PCA. Twenty-five patients were randomized to each group. For the co-primary endpoints, there appeared to be a dose-dependent effect of tenecteplase on the proportion of the perfusion lesion reperfused (as assessed by PWI) and improvement in NIHSS at 24 h. In the pooled analysis, the tenecteplase group had higher rates of reperfusion at 24 h (79.3 vs. 55.4%; *P* = 0.004), improvement in 24-h NIHSS score (8.0 vs. 3.0; *P* < 0.001), reduced infarct growth at 90 days (2 vs. 12 cm3 ; *P*= 0.01), and increased rates of good functional outcome at 90 days (mRS < 2: 36 vs. 11%; *P* = 0.02) (86). These promising findings prompted Phase 3, randomized tenecteplase trial (NOR-TEST) of 1,100 adults with AIS in 13 centers in Norway (87). No difference between the 0.4-mg/kg tenecteplase and IV tPA groups was observed for the primary outcome of 90-day mRS of 0–1 (64 vs. 63%; *P* = 0.52). Importantly, in contrast to the prior Phase-2B trial, there were no imaging inclusion criteria of documented occlusion of an intracranial artery or any perfusion mismatch. As a result, 17% of enrolled patients were later confirmed as stroke mimics. Lastly, the randomized control trial of 0.25-mg/kg tenecteplase in patients with WUS, Tenecteplase in Wake-up Ischemic Stroke Trial (TWIST), is currently ongoing (88).

Similar studies have also been conducted using infarct core– perfusion mismatch criteria for patient selection for EVT. The Mechanical Retrieval and Recanalization of Stroke Clots Using Embolectomy (MR RESCUE) trial was the first trial initiated using the concept of core–perfusion mismatch for AIS patient triage (89). MR RESCUE was a Phase 2b, multicenter, randomized, open-label study of anterior circulation LVO patients, within 8 h of LKW, to EVT vs. usual medical care. Patients were stratified according to a favorable vs. non-favorable penumbral pattern that was defined as a predicted infarct core of <90 cm3 and proportion of predicted infarct tissue within region of interest as <70%. Unlike the definition of core–perfusion mismatch utilized in other trials, MR RESCUE employed a complex voxel-by-voxel algorithm requiring 4–7 variables on CTP or PWI (90). No difference was observed in mean 90-day mRS (3.9 vs. 3.9, *P* = 0.99); however, there were several important limitations important in considering the overall results of this trial. First, the trial used first-generation thrombectomy devices. Second, the trial had an exceedingly difficult time with enrollment, taking 7 years to enroll 118 patients across 22 high-volume stroke centers, likely due to a bias to randomize patients at enrolling sites. Next, in contrast to the subsequent positive EVT trials, subjects in MR RESCUE in the embolectomy, favorable penumbral arm had large estimated core volumes (median 36.2 cm3 ) and low rates (24%) of successful revascularization defined as a thrombolysis In Cerebral Infarction scale 2b/3. Lastly, the automated imaging program for penumbral stratification failed in 42% of cases.

EXTEND-IA used automated imaging analysis of CTP to select patients with occlusion of the intracranial ICA or first or second segment of the MCA and with salvageable tissue profile for EVT within 6 h from LKW. The mismatch profile was defined as follows: perfusion lesion *T*max > 6 s, "infarct core" low CTP rCBF <30% normal tissue, low rCBF volume <70 cm3 , mismatch ratio > 1.2, and absolute mismatch volume > 10 cm3 . EVT initiated within 6 h of stroke onset and combined with mismatch for patient selection (see, e.g., **Figure 5**) significantly increased the likelihood of a favorable outcome (90-day mRS: generalized odds ratio, 2.0; 95%CI 1.2–3.8; 90-day mRS <3: 71 vs. 40%; *P* = 0.01) (5). EXTEND-IA demonstrated that early EVT, in combination with perfusion mismatch, was feasible for acute decision-making of LVO patients with SKSO. Moreover, in this trial EVT within 6 h of stroke onset was efficacious for reducing long-term disability.

These trials of late-window intravenous thrombolytic therapy and endovascular treatment have confirmed that advanced neuroimaging techniques can be employed in the emergent setting to triage acute stroke patients for acute therapies. As we will discuss

Figure 5 | CT perfusion to identify salvageable tissue. 81-year-old female with right middle cerebral artery (MCA) syndrome and occlusion of the MCA on CT-angiogram. CT-perfusion maps: (A) cerebral blood flow (CBF), (B) cerebral blood volume (CBV), and (C) time to peak (TTP). Elevated time-to-peak contrast enhancement (TTP) colors orange to red represent >6 s delay (C). Severely low blood flow and volume territories are violet color (gold arrow). The patient was rapidly revascularized and the final infarction is demonstrated on MRI diffusion weighted imaging [(D)––yellow asterisk]. Data analysis for figure was created under approval of local ethics committee.

below, these studies have prompted the application of infarct core–perfusion mismatch to the selection process of late-window or SUSO patients for revascularization therapies.

#### Clinical–Core Mismatch

An alternative approach to treating patients with sufficient salvageable "penumbra" tissue to make the likely benefit of reperfusion therapy outweigh its risk is to treat patients with large clinical––"core" mismatches. In one study (91), the authors demonstrated that in 166 patients imaged within 12 h of onset with small "core" (DWI lesion ≤ 25 mL), but large clinical deficits (NIHSS ≥8) were more likely to experience early neurological deterioration (increase of NIHSS ≥4 points). Similar findings were found in 87 patients imaged within 24 h of LKW using DWI-ASPECTS ≥8 score to define core (92). Another study showed that such clinical–diffusion mismatches are also associated with perfusion–diffusion mismatch, with 93% specificity and 53% sensitivity in 54 patients imaged within 24 h of LKW (93). However, a separate study showed clinical–diffusion mismatch predicted perfusion–diffusion mismatch with only 65% sensitivity and 42% specificity in 68 patients (94). While another cohort study of 99 patients showed that clinical–diffusion mismatch was only 46% sensitive but 86% specific for perfusion–diffusion mismatch, benefits of IV tPA and reperfusion were similar in both patient groups with or without clinical–diffusion mismatch (95). On the other hand, in 43 EVT-eligible patients (M1 segment of the MCA-occlusion) with DWI lesions <25 mL, clinical–diffusion mismatch was found to be a better predictor of infarct growth than perfusion–diffusion mismatch (96). There have also been studies of non-contrast CT-based approaches for clinical–core mismatches. One investigation found no combination of CT ASPECTS and NIHSS predicted perfusion–diffusion mismatch (97) and another study found no relationship between "clinical–CT mismatch" and likelihood of responding to IV tPA (98). Researchers have also shown that EVT decisions were changed rarely (5.6%) when including CTP in addition to NIHSS, noncontrast CT and CTA (99).

SWIFT PRIME, a prospective, randomized open-label clinical trial, which showed benefit for EVT for anterior circulation LVO stroke patients within 6 h of LKW also employed a modified clinical––"core" mismatch for part of the study (4). Patients were originally excluded based on MRI- or CT-assessed infarct core >50 cm3 , ischemic penumbra <15 cm3 , and mismatch ratio <1.8. After enrollment of the first 71 patients in SWIFT PRIME using infarct core–perfusion mismatch as part of its inclusion criteria, the approach was modified to accommodate study sites without perfusion imaging capabilities based only on the extent of ischemic changes on CT (ASPECTS ≥6) (4). Thirtyseven patients were enrolled with this modified inclusion criteria based on core size. Patients treated with Stent Retriever plus IV tPA were significantly more likely to be functionally independent at 90 days (mRS < 3: 60 vs. 35%; risk ratio 1.70, 95%CI 1.23–2.33; *P* < 0.001).

The results of these studies provide evidence that clinical– –"core" mismatch can function as an indicator of potentially salvageable tissue in the decision-making processes of acute stroke. A major advantage of this approach for patient triage is the independence from relying on perfusion imaging, which is not universally available at all hospitals that treat AIS patients. The best evidence for benefit in the late-window (beyond 6 h for EVT) currently relies on core volume estimates either by DWI, or by CT-rCBF, which implies that stroke centers of all levels will eventually need to become facile with some form of advanced imaging in late-window patients. In later sections, we will discuss the recent clinical trials that used this approach to expand the window of eligibility for thrombolysis in SUSO patients.

# Collateral Grade

A third approach for selecting patients for late thrombolysis relies on the status of the pial collaterals. This is also a pragmatic approach for EVT candidates since all patients are screened with vessel imaging to identify LVO that obviates additional imaging. Studies have shown that patients with a malignant CTA profile, defined as the absence of collaterals in >50% of an MCA M2 branch, also have large DWI lesions (100). A retrospective analysis of the IMS III trial of 95 patients with both diagnostic-quality CTA and CTP showed that patients presenting with good collaterals tend to have smaller cores and greater perfusion mismatch (101). In addition, in 276 patients with CTA, robust collaterals were associated with good clinical outcomes (102). However, another study of 60 patients imaged within 12 h of LKW showed that patients with target perfusion–diffusion mismatch did well irrespective of collateral score (103). In ESCAPE, which was the Canadian multicenter randomized Phase-3 trial of EVT for LVO in 316 patients up to 12 h from LKW, a notable, distinct inclusion criterion was evidence of moderate-to-good collateral circulation of the MCA territory on multiphase CTA (48). Importantly, 6.3% of participants enrolled had evidence of poor collateral circulation on analysis by the core laboratory. Although patients could be enrolled up to 12 h from LKW, the median time from LKW to reperfusion was approximately 4 h, with only 49 subjects randomized after 6 h, and thus the ESCAPE study cannot be considered a comprehensive study of late revascularization intervention.

These observations suggest that collateral status can function as another imaging surrogate of salvageable tissue. Specifically, patients with good pial collaterals are more likely capable of sustaining salvageable tissue for relatively longer periods of time and thereby could be candidates for extended window therapeutic interventions. Collateral status is likely an important variable in determining the rate of tissue death over time in hypoperfused brain. While there is a correlation between collateral status and CTP (101, 104), it is unclear whether collateral status is superior to CTP in the triage of SUSO patients.

# RETROSPECTIVE STUDIES OF OFF-LABEL REVASCULARIZATION TREATMENT OF SUSO PATIENTS

Because of encouraging studies characterizing SUSO patients and suggesting a potential benefit of reperfusion therapy due to similarity in imaging presentation with early witnessed stroke patients; there have been several retrospective analyses of patients who were treated with off-label IV tPA or EVT (**Table 2**) based on imaging techniques described previously. We will discuss briefly the safety and efficacy findings in these retrospective studies that paved the way for the pivotal prospective trials of extended window intervention of AIS patients.

# Intravenous Thrombolysis

A retrospective analysis of 32 SUSO patients treated with thrombolytic therapy at 3 Korean medical centers using MRI specific eligibility criteria (perfusion–diffusion mismatch > 20% MTT to DWI, no FLAIR changes, and DWI volume <50% MCA territory) showed that an MRI-based algorithm for thrombolysis of SUSO patients was feasible. In comparing the SUSO to SKSO groups, baseline characteristics were similar, including age and admission NIHSS scale, and no difference was observed in rates of recanalization (immediate 81.3 vs. 63.1%; *P* = 0.06; delayed 80.6 vs. 69.1%; *P* = 0.28), 90-day mRS ≤ 2 (50 vs. 49.3%, *P* = 1.0), or sICH (6.3 vs. 5.8%; *P* = 1.0) (105).

In another retrospective single-center study of thrombolytic therapy for WUS, administered on a compassionate basis, criteria for thrombolytic therapy in the WUS cohort included the following: (1) patients were neurologically normal before going to sleep,


*CT, computed tomography; DWI, diffusion-weighted imaging; EVT, endovascular thrombectomy; FLAIR, fluid attenuated inversion recovery; IA, intra-arterial; ICH, intracranial hemorrhage; IV, intravenous; MCA, middle cerebral artery; MRI, magnetic resonance imaging; mRS, modified Rankin scale; PH, parenchymal hematoma; PWI, perfusion weighted imaging; sICH, symptomatic intracranial hemorrhage; SKSO, stroke of known symptom onset; SUSO, stroke of unknown symptom onset; WUS, wake-up stroke.*


(2) patients awakened with a disabling deficit, and (3) CT head showed no hypodensity exceeding 1/3 the MCA (106). Forty-six WUS patients that received thrombolytic therapy were identified, of which 61% were treated with IV tPA and 30% with only EVT and the remaining receiving combination treatment (106). In the thrombolysed WUS group, two patients experienced sICH (4.3% thrombolysed WUS vs. 0% non-treated WUS). Despite higher mortality (15 vs. 0%) compared with non-treated WUS patients, thrombolysed WUS patients were more likely to experience a favorable outcome (90-day mRS 0–2: 28 vs. 13%, *P*= 0.006) (106). Compared with 174 standard-of-care 0 to 3-h IV tPA-treated patients, treated WUS patients had higher rates of sICH (4.3 vs. 2.9%; *P* = 0.64) and a lower, but statistically insignificant, likelihood of favorable outcome (28 vs. 48%, *P* = 0.64).

A retrospective analysis of 68 WUS patients presenting within 4.5–12 h from LKW and treated with thrombolytic therapy from another center showed no difference in the number of patients achieving a 90-day mRS of 0–2 (38 vs. 37%, *P* = 0.89) or rate of symptomatic ICH (sICH: 3.4 vs. 2.9%, *P* = 1.0) compared with 326 patients receiving IV tPA within 4.5 h of symptom onset (107). Notable inclusion criteria for the WUS group included: NIHSS ≥5 and no or early ischemic changes <1/3 MCA territory as assessed by ASPECTS (107).

#### Endovascular Treatment

With regard to EVT, a retrospective, single-center study of EVT without any advanced neuroimaging in WUS patients has also been reported. In 213 LVO anterior circulation ischemic stroke patients that underwent EVT after being deemed ineligible for IV tPA, including 21 WUS patients and 33 patients treated beyond 8 h from LKW, an increased odds of sICH (14.3%; odds ratio = 4.9, 95%CI 1.03–23.6; *P* = 0.047) in WUS patients compared with the group treated within 8 h of symptom onset (6.7%; odds ratio 3.8, 95%CI 1.07–13.7; *P* = 0.04) (108). Despite this observation, the authors reported no difference in the 90-day mRS between the WUS and group treated within 8 h of stroke onset (108). Another retrospective, single-center review of EVT comparing outcomes between 78 WUS patients and 128 late-window (beyond 8 h from LKW) SKSO patients who presented with small core and large perfusion defect found similar results (109). No significant difference was observed in baseline NIHSS, rates of successful recanalization, 90-day mRS ≤ 2 (43 vs. 50%, *P* = 0.3), parenchymal hematoma (9 vs. 5.5%; *P* = 0.3), or final infarct volume (75.2 vs. 61.4 cm3 ; *P* = 0.6).

A multicenter, retrospective analysis of patients with LVO (EVT initiated beyond 8 h from LKW) and perfusion imaging used for selection criteria suggested feasibility and potential efficacy of late-window EVT (110). In 237 patients meeting those inclusion criteria the mean treatment time was 15 h from LKW. Forty-five percent of the patients achieved a good functional outcome at 90-days or time of hospital discharge (mRS < 3). Parenchymal hematoma occurred in 8.9% of the patients and the 90-day mortality rate was 21.5%.

In addition to perfusion imaging, the status of the pial collateral circulation has also been evaluated as a potential metric for extending the window for EVT eligibility. A retrospective, singlecenter study of 61 anterior circulation LVO patients showed that in contrast to patients with poor collateral status, patients with good pial collaterals had no temporal cutoff point for total time of ischemia and predicting clinical improvement (111). In comparing good vs. poor collateral status, clinical improvement (4-point decline in NIHSS from baseline to discharge) beyond 300 min was significantly higher in the group with good pial collaterals (90.1 vs. 23.1%; *P* = 0.010). In agreement with these findings, another retrospective analysis of 237 patients with anterior circulation LVOs undergoing EVT also demonstrated that in patients with good collateral grades the probability of favorable outcome is not significantly influenced by onset-to-reperfusion time (112).

The interpretation of these retrospective studies is limited by the retrospective nature and inconsistency in neuroimaging selection criteria. Nonetheless, these findings demonstrate that neuroimaging-based triage for EVT is feasible and safe beyond 8 h from LKW and prompted the development of several prospective studies to further assess for efficacy.

# PROSPECTIVE CLINICAL TRIALS OF REVASCULARIZATION THERAPIES FOR SUSO PATIENTS

#### Intravenous Thrombolysis

Based on the promising findings of retrospective studies of revascularization interventions for SUSO patients, several prospective studies have been launched (**Table 3**). In 2003, one of the earliest studies involved abciximab, which had a prespecified cohort of WUS patients, although the primary cohort involved patients that could be treated within 5 h of stroke onset. Phase-3 RCT of abciximab (AbESTT-II), which is a platelet glycoprotein IIb/ IIIa inhibitor, was terminated early in 2005 due to a significantly increased rate of symptomatic and fatal ICH (5.5% of abciximabtreated vs. 0.5% placebo, *P* = 0.002) (113). Of the WUS cohort (43 patients, 22 treated with abciximab, 21 treated with placebo), there was no improvement in 90-day mRS and an increased rate of symptomatic and fatal ICH at 5 days (13.6 vs. 5% placebo, *P* = 0.347) and 3 months (18.2 vs. 5%, *P* = 0.193) Secondary analysis showed that the WUS cohort who received abciximab tended to have greater rates of new strokes on baseline CT and bleeding but otherwise were comparable to other patients in the study (114).

In 2013, another WUS investigation, Wake-up Stroke, completed involving a single-arm prospective open-label, multicenter safety trial of 40 WUS patients treated with IV tPA within 3 h of symptom discovery (115). The median NIHSS of this cohort was 6.5 and IV tPA was administered at a mean time of 10.3 ± 2.6 h from LKW. No sICH occurred in this population and 52.6% had an excellent functional outcome at 90 days (mRS 0–1) (115). While this trial is limited by its relatively small sample size, lack of control group, and open-label design, the strength of this trial is its pragmatic triage requirement of only a non-contrast head CT. A similarly designed prospective open-label, multicenter safety trial of IV tPA treatment within 4.5 h of symptom discovery of 20 WUS patients, Safety of intravenous thrombolytics in stroke on awakening (SAIL-ON), also reported no sICH (116). Both these


Table 3 | Prospective trials of thrombolysis in WUS and non-WUS SUSO patients.

*a Terminated (808 enrolled).*

*bHalted for overwhelming efficacy.*

*c Halted for funding stoppage.*

*dTreatment group compared with registry of controls.*

*e Trial enrolled on wake-up stroke patients.*

*CT, computed tomography; DWI, diffusion-weighted imaging; EVT, endovascular thrombectomy; FLAIR, fluid attenuated inversion recovery; IA, intra-arterial; ICH, intracranial hemorrhage; IV, intravenous; MCA, middle cerebral artery; MRI, magnetic resonance imaging; mRS, modified Rankin scale; PH, parenchymal hematoma; PWI, perfusion weighted imaging; sICH, symptomatic intracranial hemorrhage.*

trials enrolled stroke mimics, which might have contributed to the high rates of good outcome.

In parallel with the CT trials, MRI-based trials were underway investigating imaging-selected revascularization of SUSO patients. Launched in 2006, RESTORE (Reperfusion therapy in unclear-onset stroke based on MRI evaluation) was a prospective, multicenter, single-arm trial of SUSO patients with thrombolytic therapy within 6 h of symptom discovery (117). Patients were included if presenting with perfusion–diffusion mismatch, but excluded if FLAIR hyperintensities were noted. Out of 430 SUSO patients, 83 patients were treated with thrombolytic therapy including 63 WUS patients (117). Of those treated SUSO patients, the median LKW to hospital presentation was 8.6 h (interquartile range 5.4–11.1 h). In total, 89.2% of patients had an LVO and 68.7% of patients received only intra-arterial therapy, which included intra-arterial urokinase mechanical clot disruption, or angioplasty. At 3 months, 44.6% of patients had an mRS < 2 and only 3.6% had sICH. Compared with the non-treated registrybased control group, the treated group had increased odds of good outcome (mRS 0–2: OR 2.25; 95% CI 1.14–4.49) suggesting a potential benefit of revascularization therapy in this population. However, the interpretation and generalizability of these results are limited due to the use of registry patients as the control group. The RESTORE trial can be considered more aptly an investigation of perfusion–diffusion mismatch enrollment criteria, with an additional restriction of FLAIR negativity.

There have been trials that investigated directly the concept of using DFM for patient selection. In 2009, there was a small prospective trial of IV tPA (0.6 mg/kg) of SUSO patients with ICA or MCA M1 or M2 occlusions based on DFM who could be treated within 3 h of symptom discovery [fluid attenuated inversion recovery imaging-based intravenous recombinant tissue plasminogen activator (rt-PA) therapy study] (118). Ten subjects were enrolled, of which four were WUS. Favorable outcome was defined as mRS 0–2. No sICH was observed in this group and seven of the patients had recanalization at 7 days after IV tPA administration. Favorable outcome was observed in 40% of subjects. Notably 30% of the subjects had prestroke mRS greater than 2. In 2011, the MR WITNESS trial (A Study of Intravenous Thrombolysis with Alteplase in MRI-Selected Patients), a Phase 2a, open-label multicenter trial of IV tPA (0.9 mg/kg) in SUSO patients with DFM 4.5–24 h from LKW, launched (119). This trial enrolled 80 subjects with a primary safety outcome of sICH in only 1 subject, and a rate of excellent functional outcome at 90 days (mRS 0–1) of 44% among the 69 subjects with a prestroke mRS of 0–1. In summary, these findings suggest that the administration of IV tPA to SUSO patients within 3 h of symptom discovery is safe and feasible.

Two large, prospective clinical trials assessing DFM in the triage of AIS patients are in progress or recently completed that address efficacy issues. The Thrombolysis for Acute Wake-up and unclear-onset Strokes (THAWS) is a multicenter, prospective, open-label trial currently enrolling in Japan that is investigating a lower dose of IV tPA (0.6 mg/kg, which is the approved dose for Japanese stroke patients) in patients with stroke onset 4.5–12 h from LKW and DFM on MRI (120). The anticipated enrollment is 300 patients and the primary outcome measures are 90-day mRS < 2 and sICH. WAKE-UP (Efficacy and Safety of MRI-Based Thrombolysis in Wake-Up Stroke: A Randomized, Double-blind, Placebo-controlled Trial) is a European multicenter, randomized placebo-controlled, Phase-3 trial using DFM as criterion for IV tPA treatment of AIS patients with > 4.5 h LKW (30). This study was stopped due to lack of funding and results are anticipated this year. Out of 1,362 patients enrolled, 503 were randomized (planned 800) and 859 participants were screen failures. In **Tables 2** and **3**, we summarize the major prospective and retrospective studies on thrombolytic therapy of SUSO including WUS.

In addition to DFM trials, there is also a large Phase-3 trial Extending the time for Thrombolysis in Emergency Neurological Deficits (EXTEND) trial which uses infarct core–perfusion mismatch in patients 3 or 4.5–9 h from LKW or WUS within 9 h from midpoint of sleep duration to determine eligibility for treatment with IV tPA (121). The neuroimaging inclusion criteria of this trial are (1) a small "infarct core" defined as DWI or CT-rCBF lesion volume <70 cm3 and (2) core–perfusion mismatch > 1.2 and absolute mismatch > 10 cm3 . The perfusion lesion is defined on PWI or CTP as *T*max > 6 s. The primary outcome is rate of 90-day mRS 0–1 outcomes in the IV tPA group compared with the placebo group. The expected enrollment is 400 patients with an anticipated completion date of 2019.

While the current evidence for pretreatment advanced neuroimaging to guide decision-making for IV thrombolytic therapies in SUSO patients is intriguing, at present, there is no positive Phase-3 trial to warrant the routine use in clinical practice. This statement is supported by the 2018 American Heart Association/ American Stroke Association Guidelines recommendation of no benefit to this approach (11). There is therefore a clinical opportunity to expand IV tPA to more patients outside the current approved treatment window of 4.5 h from LKW with such a trial.

#### Endovascular Treatment

The recently published DAWN trial represents the first randomized, multicenter, Phase-3 trial utilizing an automated neuroimaging approach to triage late-window LVO patients for EVT (8). Distinct from the previously discussed studies, DAWN employed the concept of clinical–ischemic core mismatch to identify LVO patients with occlusion of the intracranial ICA and/or first segment of the MCA that were hypothesized to benefit from EVT. The inclusion criteria were therefore a combination of NIHSS and age-dependent infarct volume assessed by DWI or rCBF volume. Specifically, LVO patients 6–24 h from LKW and age less than 80 years were eligible if infarct volume was <31 cm3 and NIHSS ≥10 or infarct volume was 31–51 cm3 and NIHSS ≥20. For LVO patients greater than 80 years of age, inclusion criteria were NIHSS ≥10 and infarct volume <21 cm3 (8). A total of 206 patients were enrolled in the trial with 107 randomized to EVT. The median time from LKW to randomization was 12.2 h in the EVT group. The trial was stopped early for overwhelming efficacy according to a prespecified interval assessment. At 90 days, 49% of the EVT group vs. 13% of the standard medical therapy group achieved functional independence (mRS < 3; 95% credible interval 24–44; posterior probability of superiority, > 99.9%). There was no difference in the rate of sICH (6 vs. 3%; *P* = 0.5) or 90-day mortality (19 vs. 18%; *P* = 1.0) in the EVT group compared with standard medical therapy (8).

The DEFUSE-3 trial also showed substantial benefit of EVT from an infarct core–perfusion mismatch strategy of LVO patient selection 6–16 h after onset (7). DEFUSE 3 used the same neuroimaging definition of core–perfusion mismatch as DEFUSE 2 using MRI or CT-rCBF (infarct core < 70 cm3 , mismatch ratio ≥1.8, mismatch volume ≥15 cm3 ) in patients with anterior circulation LVO (defined as ICA or M1 segment MCA). Importantly, as a result of DAWN, the trial was halted prematurely for an interim analysis, which exceeded the efficacy endpoint. Of the 92 patients that were randomized to EVT, 53% were WUS and 75% received CTP to assess for core–perfusion mismatch. Both the EVT and standard medical therapy groups had small ischemic cores (9.4 cm3 EVT vs. 10.1-cm3 medical therapy) and large hypoperfused areas (114.7-cm3 EVT vs. 116.1-cm3 medical therapy). EVT plus standard medical therapy significantly reduced disability assessed by 90-day mRS (unadjusted common odds ratio 2.77, 95%CI 1.63–4.70; mRS 0–2 45% EVT vs. 17% standard medical therapy, *P* < 0.001). There was no difference in sICH between groups (7 vs. 4%; *P* = 0.75).

There are several additional points of DEFUSE 3 that further inform on the approach of core–perfusion mismatch in the triage of late-window LVO patients. First, of the 296 patients originally consented, 107 patients did not fulfill imaging inclusion criteria (36.1%). Secondly, 70 patients included in DEFUSE 3 would have been ineligible for DAWN, largely based on ischemic core size; however, the DAWN-ineligible group showed a similar benefit for late-window EVT (odds ratio 2.96, 95%CI 1.26–6.97). This observation speaks to the potential for core–perfusion mismatch to expand eligibility for reperfusion therapies beyond clinical– core mismatch. Third, WUS patients also showed a similar benefit of EVT (odds ratio 3.44, 95% CI 1.60–7.38). Lastly, and somewhat surprisingly, at 24-h post-revascularization therapy there was no significant difference in median infarct volume between groups (35-cm3 EVT vs. 41-cm3 medical therapy; *P* = 0.19). The underlying explanation for this observation is unclear and certainly warrants further investigation given the dramatic benefit of EVT on functional outcomes.

The results of DAWN and DEFUSE 3 are highly impactful since they will significantly alter the management and triage of patients previously thought to be "out of the window" for reperfusion therapy. In fact, the updated 2018 American Heart Association/American Stroke Association guidelines for management of anterior circulation LVO patients 6–24 h from LKW now recommend (level IA) obtaining CTP, DWI sequences or PWI to assist in patient selection for EVT (11). The findings of DAWN and DEFUSE 3 confirm that a subpopulation of LVO patients beyond 6 h from LKW with salvageable tissue, as evidenced by a clinical–ischemic core mismatch, that still benefit from EVT. In addition, while DAWN used CTP to determine the infarct core in a subset of patients, there was no core–perfusion mismatch requirement. There are, however, several additional findings from DAWN that merit discussion with regard to the overall generalizability of these findings. First, it is unknown how many patients were screened to enroll the 206 subjects reported in DAWN as screening logs were not collected. The discrepancy between the median NIHSS of 17 (interquartile range 13–21) in the EVT group but a median infarct volume of only 7.6 cm3 despite an LVO suggests a prolonged phase of penumbral survival and an opportunity to intervene may be present in more subjects than previously thought. These findings reinforce the critical role of collaterals in sustaining salvageable tissue until thrombolytic therapy is possible and affirm the inclusion criteria of ESCAPE. The poor outcomes seen in the medical arm of DEFUSE 3 and DAWN suggest that delayed collateral failure is common. The second important issue is how to generalize the results of DAWN and DEFUSE 3 to routine clinical practice. Future research should examine what additional subgroups of late-window LVO subjects can benefit from EVT. TENSION (Efficacy and safety of ThrombEctomy IN Stroke with extended lesion and extended time window) is one example of such a trial that plans to evaluate whether patients with severe strokes and large core volume (estimated by ASPECTS) can still have a relative benefit from EVT (122). TENSION is a prospective, open label, blinded endpoint, European randomized trial comparing the effectiveness of EVT in LVO patients with large infarcts (ASPECTS 3–5) up to 12 h or unknown LKW using an mRS ordinal analysis.

# REFERENCES


# CONCLUSION

The benefits of reperfusion therapies for acute ischemic stroke are well established for appropriately selected patients based on the duration of stroke symptoms. Neuroimaging-based methods of patient selection have, however, demonstrated the ability to identify additional populations of stroke patients that could benefit from late-window reperfusion therapy. Advanced neuroimaging techniques are both feasible and efficacious in the treatment allocation of SUSO patients based on either the presence of salvageable tissue on clinical-imaging mismatch or *via* a radiographic time-stamp of stroke duration. Going forward, with the anticipated results of several large Phase 3 trials, the management of this unique population of stroke patients will likely change for the better. Future research should continue to refine the approach to identifying additional populations of SUSO patients that would benefit from reperfusion therapy.

# AUTHOR CONTRIBUTIONS

ME conceived the idea, drafted the manuscript, designed the figures, and provided critical review of the manuscript and figures. AB designed the figures, and provided critical review of the manuscript and figures. LS provided critical review of the manuscript and figures. OW conceived the idea, drafted the manuscript, designed the figures, and provided critical review of the manuscript and figures.

# FUNDING

This research was carried out in part with support from the National Institutes of Health and NINDS Specialized Program of Transitional Research in Acute Stroke (SPOTRIAS) (grant number P50-NS051343) and at the Athinoula A. Martinos Center for Biomedical Imaging at the Massachusetts General Hospital, using resources provided by the Center for Functional Neuroimaging Technologies (grant number P41EB015896), a P41 Biotechnology Resource Grant supported by the National Institute of Biomedical Imaging and Bioengineering (NIBIB), National Institutes of Health.


of acute hemispheric stroke. *Stroke* (2006) 37(4):979–85. doi:10.1161/01. STR.0000209238.61459.39


for treatment with desmoteplase. *Stroke* (2012) 43(6):1561–6. doi:10.1161/ STROKEAHA.111.642322


stroke therapy. *AJNR Am J Neuroradiol* (2012) 33(10):1893–900. doi:10.3174/ ajnr.A3102


**Conflict of Interest Statement:** ME reports no disclosures. OW is the co-inventor of a patent on Delay-compensated calculation of tissue blood flow, US Patent 7,512,435. 31 March 2009, and the patent has been licensed to General Electric, Siemens, Imaging Biometrics and Olea Medical, consultant to Penumbra on topics unrelated to the article. OW and LS report being the principal investigator of an investigator-initiated study of extended-window intravenous thrombolysis funded by the National Institutes of Neurological Disorders and Stroke (clinicaltrials.gov/ show/NCT01282242) for which Genentech provides alteplase free of charge to Massachusetts General Hospital as well as supplemental per-patient payments to participating sites. LS reports serving as chair of the AHA/ASA GWTG stroke clinical work group and hospital accreditation Science Committee; serving as a stroke systems consultant to the Massachusetts Department of Public Health; and serving as a scientific consultant to LifeImage regarding user interface design and usability, and regarding trial design and conduct to Lundbeck (international steering committee, DIAS3, 4 trial), Penumbra (data and safety monitoring committee, Separator 3D and MIND trials) and Medtronic (Victory AF and Stroke AF trials). AB reports that the Wake-up Stroke study was partially supported by Genentech Inc., which provided study medication without charge as well as an investigator initiated grant to another investigator for monetary support of enrollment and pharmacy fees.

*Copyright © 2018 Etherton, Barreto, Schwamm and Wu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Remote ischemic Conditioning in Cerebral Diseases and Neurointerventional Procedures: Recent Research Progress

*Geng Zhou1,2, Ming Hua Li1 , Gabriel Tudor2 , Hai Tao Lu1 , Ramanathan Kadirvel2 \* and David Kallmes2*

*1Shanghai Jiao Tong University Affiliated Sixth People's Hospital, Shanghai, China, 2Mayo Clinic, Rochester, MN, United States*

#### *Edited by:*

*Bruce Campbell, University of Melbourne, Australia*

#### *Reviewed by:*

*David Charles Hess, Augusta University, United States Neil James Spratt, Hunter Medical Research Institute, University of Newcastle and Hunter New England Local Health District, Australia*

> *\*Correspondence: Ramanathan Kadirvel kadir@mayo.edu*

#### *Specialty section:*

*This article was submitted to Endovascular and Interventional Neurology, a section of the journal Frontiers in Neurology*

*Received: 12 February 2018 Accepted: 30 April 2018 Published: 16 May 2018*

#### *Citation:*

*Zhou G, Li MH, Tudor G, Lu HT, Kadirvel R and Kallmes D (2018) Remote Ischemic Conditioning in Cerebral Diseases and Neurointerventional Procedures: Recent Research Progress. Front. Neurol. 9:339. doi: 10.3389/fneur.2018.00339*

Cerebral ischemia and stroke are increasing in prevalence and are among the leading causes of morbidity and mortality in both developed and developing countries. Despite the progress in endovascular treatment, ischemia/reperfusion (IR) injury is an important contributor to post-surgical mortality and morbidity affecting a wide range of neurointerventional procedures. However, pharmacological recruitment of effective cerebral protective signaling has been largely disappointing to date. In remote ischemic conditioning (RIC), repetitive transient mechanical obstruction of vessels at a limb remote from the IR injury site protects vital organs from IR injury and confers infarction size reduction following prolonged arterial occlusion. Results of pharmacologic agents appear to be species specific, while RIC is based on the neuroprotective influences of phosphorylated protein kinase B, signaling proteins, nitric oxide, and transcriptional activators, the benefits of which have been confirmed in many species. Inducing RIC protection in patients undergoing cerebral vascular surgery or those who are at high risk of brain injury has been the subject of research and has been enacted in clinical settings. Its simplicity and non-invasive nature, as well as the flexibility of the timing of RIC stimulus, also makes it feasible to apply alongside neurointerventional procedures. Furthermore, despite nonuniform RIC protocols, emerging literature demonstrates improved clinical outcomes. The aims of this article are to summarize the potential mechanisms underlying different forms of conditioning, to explore the current translation of this paradigm from laboratory to neurovascular diseases, and to outline applications for patient care.

Keywords: remote ischemic conditioning, acute ischemic stroke, ischemia/reperfusion injury, neuroprotection, neurointerventional procedures

# INTRODUCTION

Recent studies show that ischemia/reperfusion (IR) injury is an important contributor to postsurgical mortality and morbidity affecting those undergoing a wide range of neurointerventional procedures (1, 2). Effective protection attenuating IR injury is therefore an important factor in improving patient prognosis. However, pharmacological strategy to protect the brain against IR injury has been largely disappointing to date.

Ischemic conditioning, a powerful non-pharmacological strategy for reducing IR injury, was recognized in animal models in 1986 (3), though this innate cytoprotective mechanism in the brain was noted as early as the 1940s (4). By 1996, its use extended to organs remote from the heart in the form of remote ischemic conditioning (RIC) (5). Today, RIC is a remarkably simple and low-cost intervention that employs repetitive inflation and deflation of a standard arm or leg blood pressure cuff and constitutes a highly effective therapy for protecting vital organs from IR injury. Base on its simplicity, accessibility, and non-invasive nature, RIC has the potential for treatment in a wide variety of conditions including acute, subacute, and chronic neurological diseases with an ischemic basis, such as acute ischemic stroke (AIS) (6).

The aims of this article are to summarize the potential mechanisms underlying different forms of conditioning, to explore the current translation of this paradigm from laboratory to neurovascular diseases, and to outline applications for patient care.

# RIC PROTOCOL

The most effective RIC protocol has yet to be fully defined. Currently, the most commonly employed technique across clinical settings is three to four repetitions of 5-min inflation/deflation using a standard blood pressure cuff. Tourniquet pressure should be above the systolic pressure to ensure arterial occlusion. Its localization (arm versus thigh) does not affect cytoprotection (7). However, more than eight ischemic cycles or cycles >10 min did not lead to better results and possibly even increased injury in mice (8). If RIC were considered in the manner one would analyze a therapeutic drug, its exact dosage, pharmacokinetics, and pharmacodynamics would remain largely unclear.

Experimental and clinical evidence suggests that RIC, as well as other preconditioning stimuli, activates at least two distinct time frames of protection against IR injury of brain and heart. The time window of brain protection by preconditioning has also been demonstrated *in vitro* model (9). The initial time window of brain protection is short lasting as a result of changes in ion channel permeabilities, protein phosphorylation, and release of several mediators [including adenosine and bradykinin (BK)]. It occurs immediately after the RIC stimulus and lasts 2 h (10). The delayed form of protection, referred to as the second window of protection (SWOP), follows 12–24 h later, and lasts 48–72 h (as shown across multiple species) (11). SWOP may be triggered by the reactive oxygen species (ROS) and mediated by modulated inflammatory response, improved endothelial function, and activation of gene expression (such as HIF, toll-like receptor caspases, and heat shock proteins) (**Figure 1**) (12, 13). Various clinical studies have demonstrated the SWOP in RIC, although all the studies are in cardiac surgery settings (14).

The concept of RIC has now expanded into three temporal variants after its initial application: remote ischemic preconditioning (RIPreC), perconditioning (RIPerC), and postconditioning (RIPostC) (15–17). Brain mechanisms are independent of the timing of conditioning strategies (pre-, per-, postconditioning), and their effects have a great deal of overlap.

## RIC MECHANISMS

The mechanisms underlying RIC include neurovascular protection, anti-inflammatory action, reduced excitotoxicity, and metabolic protection, which are associated with influences on mitochondria, circulating inflammatory cells, or transcriptional upregulation of protective pathways (**Figure 2**) (18, 19). There is a consensus that the infarct-sparing effect of all forms of ischemic conditioning involves the upregulation of several signal transduction cascades, which serve to stabilize the mitochondria (20).

Although neurons are assumed to be the cellular target of cerebral conditioning, ischemic tolerance occurring at the level of endothelial and smooth muscle cells contributes to neuronal protection (21). RIPreC was first shown to protect against endothelial injury during IR in humans in 2002 (22), and vasodilation was shown to be better preserved in a preconditioned brain (23). Trans-cranial Doppler measurements of patients undergoing RIC indicated transient cerebral vasodilation over the duration of conditioning (24). All temporal variants of RIC have been proven to prolong protein kinase B (Akt) activity in the endothelium, which increases nitric oxide (NO) production through improved endothelial nitric oxide synthase (eNOS) activity and helps to maintain vascular homeostasis (25–27).

#### Cell-Level Mechanisms Underlying RIPreC

The mechanism of brain preconditioning involves a shift in the neuronal excitotoxic/inhibitory balance and a reduction in inflammatory sequelae. Several intracellular signaling pathways and various intercellular mediators and kinases have been identified in tissue protection by RIC. The protective reperfusion injury salvage kinase pathway (RISK) including the phosphoinositide-3 kinase/Akt signaling cascade and the pro-survival survivor activating factor enhancement (SAFE) pathway including the Janus kinase 2 (JAK2)/signal transducer and activator of transcription (STAT)3 signaling cascade are the most important pathways involved in ischemia cytoprotection and eNOS activation (28, 29). And the SAFE pathway was shown to lead to tissue protection independently of the RISK pathway (28). Phosphorylation of JAK2, STAT3, STAT5, Akt, and other signaling complexes may ultimately reduce apoptosis, ROS production, and inflammation (30, 31). In addition, STAT3 located in the matrix of subsarcolemmal and interfibrillar mitochondria also serves to improve mitochondrial respiration and attenuate mPTP opening, and ROS formation (32, 33). And Akt activation, in interaction with STAT3 activation, was mandatory for ischemic preconditioning (34). The activation of the STATs also results in transcriptional upregulation of inducible nitric oxide synthase (iNOS) and cyclooxygenase-2, known distal mediators/effectors of protection (35, 36). There are direct evidences for STATs involvement in patients with RIC (37, 38). A recent study demonstrated that RIPreC could enhance the phosphorylated Akt, STAT3, STAT5, and eNOS expression levels and activating the pro-survival signaling pathway in humans (39). In addition, previous reports showed that NO, hypoxiainducible factor (HIFs), erythropoietin, free radicals, BK,

adenosine, opioids, activation of the ATP-sensitive potassium (KATP) channel, and norepinephrine all have roles in RIPreC (40–42). One of the key regulators of the genomic response after RIPreC is the transcriptional activator HIF. HIF-1 activation is neuroprotective, and a neuron-specific HIF-1α deletion demonstrated exacerbation of brain injury in an experimental model of stroke (43). The growth of new vessels stimulated by the VEGF and erythropoietin cytokines are also regulated by HIF-1 (43). Some researchers believe that expression of HIF-1α—but not phosphorylation of extracellular signal regulated kinase 1/2 (ERK1/2), Akt, or STAT5—is required for RIPreC (44). Inflammatory mediators, such as interleukin-6, tumor necrosis factor (TNF), intracellular adhesion molecule, matrix metalloproteinase 9, and C-reactive protein are downregulated through RIPreC (45).

Microarrays indicate that preconditioning stimulates a genomic reprogramming of cells that confers cytoprotection, recovery, neurogenesis, and angiogenesis (46). In particular, genes regulating cell metabolism, signal transport, growth factors, ion channels, metallothionins, or cell cycle/apoptosis are selectively upregulated (46, 47). The microRNA for glutamate receptor, ionotropic delta 2, was reported to be downregulated in the mouse brain after RIPreC (46).

Using a global model of ischemia preconditioning in gerbils, short stimuli were shown to induce an increase in dendritic spine density of vulnerable hippocampal CA1 neurons 3 days after reperfusion, comparable to the SWOP of the neuroprotective effect induced by preconditioning (48). Preconditioning in immature brains also increases the concentration of astrocytic glycogen, which is neuroprotective, and delays energy depletion caused by ischemia (49). Moncada found that preconditioning increases expression of cyclooxygenase 1 and prostacyclin synthase; these enzymes act successively to produce prostacyclin, which inhibits platelet aggregation and vasoconstriction (50). Røpcke et al. also demonstrated that RIPreC reduces arterial thrombus formation and embolization in rats (51). Several clinical trials are underway to test the safety and efficacy of RIPreC for protecting the brain against anticipated damage (52, 53),

transcription.

Figure 2 | Overview of the proposed signaling cascades recruited in the setting of remote ischemic conditioning based on available data. Abbreviations: Akt, protein kinase B; AR, aldose reductase; AP-1, activator protein 1; cGMP, cyclic guanosine monophosphate; COX-2, cyclooxygenase-2; Cx 43, connexin 43; DAG, diacylglycerol; eNOS, endothelial nitric oxide synthase; ERK, extracellular signal regulated kinase; Gs/Gi/q, stimulatory/inhibitory G protein; GPCR, G protein-coupled receptor; gp130, glycoprotein 130; GSK3β, glycogen synthase kinase 3 β; HIF-1α, hypoxia-inducible factor 1α; HSP, heat shock protein; IR, ischemia/reperfusion; iNOS, inducible nitric oxide synthase; JAK, Janus kinase; KATP, ATP-sensitive potassium channel; mPTP, mitochondrial permeability transition pore; Mito, mitochondria; NFκB, nuclear factor κB; NO, nitric oxide; P70S6K, p70 ribosomal S6 protein kinase; P90RSK, 90 ribosomal S6 kinase; PI3k, phosphoinositide-3 kinase; PKC, protein kinase C; PKG, protein kinase G; PLC, phospholipase C; RISK, reperfusion injury salvage kinase pathway; ROS, reactive oxygen species; sarcKATP, sarcolemmal potassium channels; sGC, soluble guanylate cyclase; SAFE, survivor activating factor enhancement; STAT, signal transducer and activator of transcription; TK, tyrosine kinase; TNFR, tumor necrosis factor receptor.

and its procedural simplicity makes it an excellent candidate for study in future clinical trials.

#### Corroborating Evidence Based on Transient Ischemic Attack (TIA) Neuroprotection

Patients who suffer a TIA show better clinical outcomes in subsequent strokes compared to those who suffer similar strokes without first having suffered a TIA, which may be due to activation of the same neuroprotective pathways as RIPreC (54). Schaller found that stroke patients showed more favorable neurological outcomes when the preceding TIAs occurred 1–7 days prior to stroke (55). Similarly, in a German study comprised of 7,611 patients, TIA was associated with reduced stroke severity (56). Recent data also suggests that peripheral vascular disease with chronic limb hypoperfusion was associated with less disability and lower mortality in AIS (57). In contrast to the findings, Kim et al. reported that a low ankle-brachial blood pressure index (ABI) (<0.9) was associated with an increased risk of poor functional outcome in patients with acute cerebral infarction (odds ratio 3.452, *P* < 0.001) than patients without low ABI (58). However, in this study, the patients with a low ABI were more likely to have a high NIHSS score at baseline. Besides, the patients with a low ABI more often had diabetes mellitus (44.9 versus 29.5%, *P*= 0.007). Diabetes mellitus itself may attenuate the effectiveness of RIC (59). In future trials, subgroup analysis of patients with comorbidities such as diabetes is needed.

# Alternative Method: RIPerC

Remote ischemic preconditioning may be not practical in acute clinical settings because it must be initiated before the ischemic event. The neuroprotective efficacy of RIPerC has been proven in a number of animal models (10, 14, 25, 60). Furthermore, mild to moderate hemorrhage after tissue plasminogen activator (tPA) was attenuated when RIPerC therapy was performed 2 h before tPA infusion, making it an excellent candidate for combination therapy with tPA (61). Clinical MRI evidence suggests RIPerC treatment induces an immediate neuroprotective effect by reducing cytotoxic cerebral edema when perfusion is restored (62). RIPerC also upregulates mRNA expression of eNOS about 10-fold in the blood vessels, from the site of conditioning, and increases the concentration of NO in plasma (63).

#### The Reasoning Behind RIPostC

Remote ischemic postconditioning can be used in both elective and acute settings. Evidence from experimental and trial studies supports an additive protective effect of combined RIPreC and postconditioning, as reperfusion itself is associated with cell injury and cell death in its very early moments (64–66). Postconditioning likely mitigates damage from sudden reperfusion, plausibly blocking production of ROS and reactive nitrogen species and thus attenuating reperfusion-induced brain injury (67), or possibly by attenuating endoplasmic reticulum stress response-induced apoptosis (68). The pro-survival protein kinases extracellular signal-regulated kinases (ERK), p38 mitogen-activated protein kinase (MAPK), and Akt showed prolonged phosphorylation in the cortex of postconditioned rats (69). Protection from RIPostC is blocked in animal models by removing the influence of STAT3 and mitochondrial KATP channels, as well as TNF α (33, 70).

# MITOCHONDRIA AND RIC

Mitochondria play critical roles in all pathways triggered by RIC. RIC causes recruitment of ligands such as adenosine and opioids to Gprotein-coupled receptors. This action leads to the activation of signaling protein kinases and the opening of mitochondrial KATP channels, which subsequently prevents the opening of the mitochondrial permeability transition pore (mPTP) after the first minutes of reperfusion whereby tissue protection is activated (71–73).

The role of signal transduction pathways during RIC has predominately been demonstrated in the heart. However, the presence of STATs in the mitochondria was confirmed in a number of organs including heart, kidney, and brain (74). A few reports in the literature have suggested the involvement of MAPKs, Akt, HIF-1α, and STATs in mitochondrial neuroprotection following preconditioning (30, 75–77). STATs have been shown to regulate mitochondrial function by preserving efficiency of electron transport chain complexes (35, 78).

# TRANSFER OF THE CEREBRAL PROTECTIVE STIMULUS

In RIC, transient, reversible episodes of ischemia with reperfusion in the stimulus location render remote tissues and target organs resistant to IR injury. At present, transfer of the cerebral protective stimulus is not well understood, though studies have shown it to act through multiple pathways (15).

# Humoral Pathways

The humoral pathway has been most extensively studied. Some studies have identified specific factors, such as stromal cellderived factor-1 α, interleukin, nitrite, cysteine-rich secretory protein 3, and microRNA-144 as possible candidate transfer factors (51, 79, 80). Ueno et al. suggest that RIPreC transiently increases plasma VEGF levels by downregulating miR-762 and miR-3072-5p in CD34-positive bone marrow cells, leading to protection against organ ischemia (81). In a recent human study, only STAT5 signaling was identified to be associated with RIPreC humoral transfer (38). Endothelial cells were suggested as the target for RIPreC-released mediators (82). Finally, Dong et al. suggest that humoral factors, rather than the neural pathway, play an important role in the formation of the tolerance against spinal cord ischemia by limb RIPreC (83).

#### Nerve Pathway

Occlusion with a tourniquet on the arm can stimulate the release of autacoids that activate an afferent neural pathway and/or cause the release of NO from blood vessels (80, 84, 85). Transection of the femoral nerve or spinal cord can abrogate the effect of RIC in rabbits (86). The dependence of remote conditioning on intact neural pathways also may explain why its effects seem to be attenuated in patients with neuropathy (87).

Mastitskaya et al.'s study used viral gene transfer and optogenetics to show that the dorsal motor neurons of the vagus in the brainstem were required for RIPreC to have a cardioprotective effect, and that stimulation of these neurons mimicked the effect of RIPreC (88). Interestingly, femoral nerve or sciatic nerve resection alone only partially abolished the infarct-limiting effect of RIPreC in mice, suggesting the influence of both neural and humoral pathways (89).

# Inflammatory Pathway

Remote ischemic preconditioning has been shown to have a systemic anti-inflammatory influence through upregulation of cytoprotective genes and suppression of proinflammatory genes in immune cells (90). Circulating monocytes and neutrophil infiltration play a key role in IR injury. RIPreC downregulated the expression of a broad spectrum of proinflammatory genes in circulating monocytes. For circulating neutrophil, RIPreC activated signal pathways in neutrophils modulating the release of proinflammatory cytokines and the expression of adhesion markers. Consequently, RIPreC negatively affected their function (18). Microarray analysis showed that reduction of inflammatory gene expression takes place within 15 min of RIC and at 24 h after conditioning in humans (18). Humoral, neural, and antiinflammatory pathways probably interact with each other and are not necessarily mutually exclusive (91).

# CLINICAL APPLICATIONS

Larger trials of RIC, especially for cardioprotection but also for kidney and neuroprotection, have largely supported the consensus of RIC's lack of harmful influence and reduction of IR injury when established protocols are used and in the absence of propofol (6, 92). Several clinical studies are also underway to expand the literature on neuroprotection specifically (52, 53).

#### RIC in AIS

Over 10 million people worldwide suffer an AIS each year (93), yet few neuroprotective treatments against IR injury have been proven effective: clinical trials of more than 50 compounds for treatment of IR injury secondary to AIS all showed negative results. Mechanical thrombectomy has been widely accepted as an effective treatment for AIS. Despite the sharp increase in recanalization rate with current thrombectomy devices compared with tPA, cerebral reperfusion after endovascular embolectomy and/or tPA may cause deterioration of penumbra, disruption of the blood–brain barrier, cerebral edema, and intracerebral hemorrhage (94). Thus, there is an urgent need for effective forms of secondary prevention after the acute phase of AIS intervention, for which RIC is an excellent candidate.

In a model of autologous thromboembolic clots, RIPerC has been effective in mice models when applied 2 h after stroke onset with or without late (4 h after stroke onset) intravenous (IV) tPA (25). Hahn et al. show that infarct size in a rat AIS model was reduced by RIPreC but even further by RIPerC (17). In an analogous study, RIPerC therapy also improved the cerebral blood flow (CBF) and the hemorrhage, edema, and neurobehavioral outcomes significantly on top of the reduction in infarction size compared to IV-tPA alone at 4 h post-stroke (95). Hess et al. show optimal results occurred when RIPerC was started as soon as possible after stroke onset and RIPostC was administered two to three times during first day and repeated daily during the following week (96).

#### Trials in AIS

Several trials studying the effect of RIC on AIS patient outcomes have shown benefits when RIC is administered during ischemia. Hougaard et al. (62) found an overall reduction in the risk of infarction for tissue subjected to pre-hospital RIPerC at 1 month but the study was not powered to show effect in clinical outcome at 3 months. The Remote Ischemic Conditioning After Stroke Trial study (64), a blinded placebo-controlled trial of RIC in AIS patients, showed improved clinical outcome in the RIC group. Compared with sham, 90-day NIHSS score was significantly lower in the RIC group (1 versus 3, *P* = 0.04). RIC also increased plasma heat shock protein 27 (HSP27, *P* < 0.05) level in the study, compared with control. The investigators suggested that the neuroprotective effects may be mediated through phosphorylated HSP27. A research group in Denmark administered RIPerC during transportation in the ambulance as a pretreatment to IV alteplase. Overall, the study showed RIPerC to be safe and feasible in the setting of AIS, with the likely benefit of greater tissue survival in the penumbra than the control (62). Another randomized trial also found that high prestroke physical activity is associated with reduced infarct size after IV tPA treatment only in patients receiving adjuvant RIPerC (97). While a French multicentric trial of RIC for ischemic stroke within 6 h of symptom onset is currently underway. Results of this trial have not yet been reported (98).

#### Other Clinical Applications for RIC Intracranial Atherosclerotic Stenosis

Endovascular treatment of ICAS carries a risk of intraoperative and postoperative ischemic events, allowing for non-urgent consideration of protection against IR injury. RIPreC alone was recently found to significantly decrease the incidence of stroke in patients with ICAS (26.7 versus 7.9%), increase CBF, and protect against ischemia-related neurological morbidity (99). Meng et al. (99) found that RIC could improve the cerebral circulation in patients with intracranial arterial stenosis. While RIC was also reported to be effective in cerebral small vessel disease (SVD) related cognitive impairment. Wang et al. (100) randomly assigned 30 patients with mild cognitive impairment caused by cerebral SVD to receive RIC (by the method used by Meng et al. twice daily for 12 months) or to receive a sham intervention; the patients who received RIC had a higher reduction of white matter hyperintensities volume (−2.632 versus −0.935 cm3 , *P* = 0.049), with a better visuospatial and executive ability at 1 year (0.639 versus 0.191, *P* = 0.048). Meanwhile, in a bilateral carotid artery stenosis mouse model with vascular cognitive impairment, RIC was effective in improving cognition and CBF, attenuating tissue damage (101).

#### Carotid Artery Stenting (CAS)

Carotid artery stenting is a selective procedure used to tread carotid artery stenosis, RIC has been evaluated in surgical brain injury paradigms such as hypothermic circulatory arrest and following carotid endarterectomy. Though a pilot study of 70 patients who received RIC showed no statistically significant improvement in neurological outcome (53), the first proof-of-concept trial of RIC before CAS found that RIC can ameliorate the complications of distal thromboembolization (102). This is the first study to show effect of RIC given before CAS on ischemic lesions size and number assessed by MRI. The authors reported that the incidence of new ischemic lesions were lower in patients who received RIC than in patients who did not (15.87 versus 36.51%, *P* < 0.01), with smaller infarct volume (0.06 versus 0.17 ml).

#### Subarachnoid Hemorrhage (SAH) From Intracranial Aneurysm

The leading cause of SAH is rupture of an intracranial aneurysm, accounting for roughly 80% of cases. Even if embolization of the ruptured intracranial aneurysm is successful, delayed cerebral ischemia may occur (103). Preconditioning before the induction of SAH in rats was shown to improve vasospasm, reduce cerebral inflammatory cytokines, attenuate tissue hypoxia, and prevent neurological deterioration (51). Some authors believe that SAH is a particularly feasible clinical setting to evaluate human response because RIPreC activates multiple pathways that have been invoked in SAH (104).

Laiwalla et al. reported a matched cohort analysis of RIPostC for patients with aSAH.

Remote ischemic conditioning was independently associated with good outcomes and lower incidence of delayed cerebral ischemia (105). A longitudinal human pilot study in aSAH patients undergoing RIC found coordinated expression and methylation of a small set of key genes in mitotic cell cycle, defense, and inflammatory responses after RIC (106). Other human studies have confirmed the safety and feasibility of lower limb RIC in individuals with aSAH in which no patient experienced delayed cerebral ischemia (51).

#### LIMITATIONS OF RIC

Remote ischemic conditioning can be initiated during prehospital transport, through which the patient would receive benefit during triage, imaging, and reperfusion therapy by IV or endovascular methods with low known risk of adverse effects. In the study by Botker et al. (107), the RIC stimulus was initiated in ambulance during transfer for angioplasty, resulting in increased myocardial salvage (36%). RIC intervention can also be delivered on immediate arrival at interventional center when ambulance transit times are short, and even at the onset of reperfusion (108). However, most of the current trials are studies mainly focusing on cardioprotective effects. These studies provided further opportunities to investigate the neuroprotective effect of limb RIC applied in an ambulance, helicopter, or emergency departments, in advance of interventional reperfusion. Moreover, preclinical trial in murine thromboembolic stroke model and pilot trials suggest that RIC can be combined with recombinant tissue plasminogen activator in the pre-hospital setting to increase the protective effect. In the Denmark trial, patients were randomly assigned to receive or not receive RIPerC treatment, and RIPerC was completed during transportation in the ambulance before a final diagnosis of ischemic stroke (62, 109). However, it has been reported that about 3% patients will not able to tolerate tourniquet inflation on their arm (94). Furthermore, RIC would also predetermine the arm to be used for arterial and venous access. Other considerations include the influence on obtaining endovascular access during vascular intervention (110). Finally, the time window and the primary RIC protocol in neuroprotection are still not fully determined.

In two large trials, the benefits from RIC were not confirmed in patients undergoing cardiovascular surgery. However, a point of critique in their studies is that the use of propofol anesthesia in most (111) or all patients (112). The second problem is the inclusion of many patients who also underwent valve surgery. RIC protects only from IR injury and not from traumatic injury at the target organ. Propofol is known to disrupt RIC (113–115). Neither

#### REFERENCES


RIC cardioprotection nor STAT5 activation were observed under propofol anesthesia (115). In clinical studies reporting protective effects of RIC, the RIC procedure was either completed without anesthetic intervention or completed during anesthesia induction with anesthetics other than propofol (116). The use of propofol has been suggested to be avoided in future studies on RIC (117). And the efficacy of RIC could also be influenced by many other variables including conditioning protocol, concomitant medications, and coexisting conditions (118–121).

Most animal studies have been performed in reductionist approaches which lack risk factors and comorbidities (122). Additional sources of variation should be considered in future studies, including the choice of anesthesia, patient's comorbidities and comedications, and the temporal aspects of the remote conditioning algorithm (122). Caution should be exercised when assessing outcomes because patient selection and trial design may affect outcomes.

# CONCLUSION

Remote ischemic conditioning is protective against reperfusion injury, and further research will expand our knowledge in the field of cerebral vascular diseases. Its simplicity and non-invasive nature, as well as the flexibility of the timing of RIC stimulus, make it feasible to apply alongside neurointerventional procedures. Precise knowledge of its optimal dosage and timing of administration is yet to be found. RIC has promising but understudied potential neuroprotective influences on patients undergoing endovascular treatments who have risks of IR injury. Further validation using well-designed randomized controlled trials is necessary to document the efficacy of differing RIC protocols across a range of cerebrovascular diseases.

# AUTHOR CONTRIBUTIONS

DK: concept, design, and development of the study; MHL: development of the study; GZ: acquisition and analysis of the data, writing of the article; GT: article writing; HTL: development of the study; RK: critical review of the article.

# FUNDING

This study was supported by grants from National Natural Science Foundation of China, grant no. 81370041, 81471760, 81671655 and National Institutes of Health, grant no. NS076491.


versus thigh: is one raise of the cuff enough? *Redox Biol* (2017) 12:491–8. doi:10.1016/j.redox.2017.03.010


anism. *Circulation* (2007) 116:1386–95. doi:10.1161/CIRCULATIONAHA. 106.653782


intravenous tissue-type". *Stroke* (2013) 44:e37. doi:10.1161/STROKEAHA. 111.000541


preconditioning protects human endothelial cells from hypoxia-induced cell damage. *Basic Res Cardiol* (2015) 110:17. doi:10.1007/s00395-015-0474-9


vessel disease. *Stroke* (2017) 48:3064–72. doi:10.1161/STROKEAHA. 117.017691


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2018 Zhou, Li, Tudor, Lu, Kadirvel and Kallmes. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# A Machine Learning Approach to Perfusion Imaging With Dynamic Susceptibility Contrast MR

Richard McKinley <sup>1</sup> \*, Fan Hung<sup>2</sup> , Roland Wiest <sup>1</sup> , David S. Liebeskind<sup>2</sup> and Fabien Scalzo<sup>2</sup>

*<sup>1</sup> Support Center for Advanced Neuroimaging, Inselspital, University of Bern, Bern, Switzerland, <sup>2</sup> Department of Neurology, University of California, Los Angeles, Los Angeles, CA, United States*

Background: Dynamic susceptibility contrast (DSC) MR perfusion is a frequently-used technique for neurovascular imaging. The progress of a bolus of contrast agent through the tissue of the brain is imaged via a series of T2\*-weighted MRI scans. Clinically relevant parameters such as blood flow and Tmax can be calculated by deconvolving the contrast-time curves with the bolus shape (arterial input function). In acute stroke, for instance, these parameters may help distinguish between the likely salvageable tissue and irreversibly damaged infarct core. Deconvolution typically relies on singular value decomposition (SVD): however, studies have shown that these algorithms are very sensitive to noise and artifacts present in the image and therefore may introduce distortions that influence the estimated output parameters.

#### Edited by:

*Nishant K. Mishra, Icahn School of Medicine at Mount Sinai, United States*

#### Reviewed by:

*Richard Leigh, National Institutes of Health (NIH), United States Vincent Thijs, Florey Institute of Neuroscience and Mental Health, Australia*

> \*Correspondence: *Richard McKinley richard.mckinley@insel.ch*

#### Specialty section:

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

Received: *10 May 2018* Accepted: *08 August 2018* Published: *04 September 2018*

#### Citation:

*McKinley R, Hung F, Wiest R, Liebeskind DS and Scalzo F (2018) A Machine Learning Approach to Perfusion Imaging With Dynamic Susceptibility Contrast MR. Front. Neurol. 9:717. doi: 10.3389/fneur.2018.00717* Methods: In this work, we present a machine learning approach to the estimation of perfusion parameters in DSC-MRI. Various machine learning models using as input the raw MR source data were trained to reproduce the output of an FDA approved commercial implementation of the SVD deconvolution algorithm. Experiments were conducted to determine the effect of training set size, optimal patch size, and the effect of using different machine-learning models for regression.

Results: Model performance increased with training set size, but after 5,000 samples (voxels) this effect was minimal. Models inferring perfusion maps from a 5 by 5 voxel patch outperformed models able to use the information in a single voxel, but larger patches led to worse performance. Random Forest models produced had the lowest root mean squared error, with neural networks performing second best: however, a phantom study revealed that the random forest was highly susceptible to noise levels, while the neural network was more robust.

Conclusion: The machine learning-based approach produces estimates of the perfusion parameters invariant to the noise and artifacts that commonly occur as part of MR acquisition. As a result, better robustness to noise is obtained, when evaluated against the FDA approved software on acute stroke patients and simulated phantom data.

Keywords: machine learning, stroke, perfusion, reperfusion, penumbra

# 1. INTRODUCTION

Perfusion imaging is a vital tool in clinical neuroimaging, and in particular in the imaging of acute stroke patients. In magnetic resonance imaging, Dynamic Susceptibility Contrast (DSC) MR Perfusion imaging is a modality in which a bolus of contrast agent that reduces the signal intensity of T2− and T2 <sup>∗</sup>− weighted images is allowed to perfuse through neural tissue while a series of consecutive MRIs is taken. The signal attenuation resulting from the contrast agent can be used to infer the concentration of contrast agent in the volume over time (1). These concentrationtime curves on their own cannot be directly interpreted, but clinically relevant measures such as cerebral blood flow (CBF), cerebral blood volume (CBV), mean transit time (MTT), timeto-peak (TTP), and time-to-maximum (Tmax) can be inferred by deconvolving the arterial input function to obtain the residue function: a curve characterizing blood flow through that volume element. These fluid measurements have been widely used in assessing brain damage, abnormalities, and recovery (2). In acute stroke, treatment selection is performed by comparing the volume of the ischemic core (the tissue undergoing cytotoxic edema) with that of the penumbra: the hypoperfused tissue which is at risk, but which may still be salvaged. Parameters extracted from perfusion imaging are vital for identifying the tissue at risk. Various studies have shown correlations between the perfusion parameters and clinical outcome in terms of Rankin score and Barthel Index (3). Perfusion imaging has also been used to assess collateral circulation and indirectly qualify clinical outcome (4). Transient ischemic attack and internal carotid artery blockage and stenosis are also identifiable with perfusion imaging (5, 6).

The inverse problem of inferring the residue function (and thus the perfusion maps) is ill-conditioned, and standard deconvolution techniques such as singular value decomposition (SVD) are highly susceptible to noise and artifacts in the DSC sequence, causing underestimates for some parameters and overestimates for others (7). Certain techniques have been developed to reduce this problem. A smoother residue function can be achieved through a Gaussian process for deconvolution (GPD) (8), Tikhonov Regularization (9), and a physiological model of microvasculature (10). Attempts to provide better estimates of perfusion parameters have also used Maximum Likelihood Estimation Maximization (ML-EM) (11), and Bayesian estimation (10, 12). These novel algorithms have provided encouraging results to improve the robustness of the deconvolution in the context of DSC. In some instances, however, some of these techniques may not be suitable in the setting of acute stroke due to the increased processing time.

In the clinical setting, perfusion maps are interpreted in two distinct ways: by visual inspection, and by thresholding at standard parameter values. These interpretations are complementary: visual interpretation can provide valuable insight into subtleties of the patient's condition, while thresholding can provide volumetric assessments of the extent of hypoperfusion. For example, a threshold of 6s is used as standard in clinical trials to define the ischemic penumbra (13–15). However, both kinds of interpretation are susceptible to noise. As an alternative to improving the quality of perfusion maps by altering post-processing, several attempts have been made to improve interpretation of standard perfusion maps, moving beyond thresholds to apply machine-learning to standard perfusion maps, identifying tissue-at-risk by learning a mapping from perfusion parameters to tissue risk, as learned from a databank of retrospective cases (16). Other data driven approaches have demonstrated significant improvements in predicting tissue fate based on advanced nonlinear regression (17) and deep learning (18), for example. More recently, Yu et al. presented a model predicting hemorrhagic transformation severity directly from source perfusion imaging [i.e. without first performing deconvolution on the concentration-time curves (19)].

While automated prediction and detection hold enormous promise, interpreting the output of a system derived from machine-learning is often difficult. In particular, clinicians base decisions on the appearance of standard perfusion maps, whose relationship to the outputs of an unfamiliar algorithm may be difficult to discern. To mitigate this, we propose to train a machine-learning models to reconstruct standard perfusion maps from source perfusion, without passing via SVD. The models are trained on a large number of voxels from perfusion imaging in ischemic stroke cases: the variability of these cases allows our models to disregard erroneous measurements and produce better estimates of perfusion parameters, as we demonstrate by synthetically adding noise to both perfusion cases and phantom data.

# 2. DSC MR PERFUSION IMAGING

From each voxel, DSC imaging gives rise to a signal intensity time curve. From this curve, a concentration time curve (CTC) of the contrast agent can be computed following the relation: (10):

$$CTC(t) = \frac{1}{TE} \* \log \frac{I(t\_0)}{I(t)},\tag{1}$$

where TE is the echo time of the MRI, and I(t) is the pixel intensity at a pixel as a function of time t, and t<sup>0</sup> is the first time of the series. The concentration time curve of a voxel of interest is modeled with the following relation:

$$\kappa \,\mathrm{CTC}(t) = \mathrm{CBF} \int\_0^t A \mathrm{IF}(\tau) \mathrm{R}(t - \tau) d\tau \tag{2}$$

Here, κ is a constant dependent on hematocrit levels in the arterioli and the density of brain tissue, CBF is the cerebral blood flow. CTC(t) is modeled as the signal response of the system of neural tissue and vasculature that the contrast agent moves through to reach the voxel of interest. AIF(t), an arterial input function, which is a CTC(t) at the chosen voxel representing the source of incoming contrast agent, is convolved with R(t), which is the impulse response of the system of neural tissue and vasculature. Using the fluid model, it is possible to compute estimates of the desired parameters from the CTC at all voxels. Each pixel's residue function, R(t), can be recovered using deconvolution. The CBF can be computed from Equation (2). CBV is calculated as a ratio between total volume of incoming contrast agent and total volume of contrast agent moving through the voxel of interest.

$$\text{CBV} = \frac{\int \text{CTC}(t)dt}{\int AIF(t)dt} \tag{3}$$

Mean Transit Time (MTT) is the average time that blood may spend in the voxel and is computed as:

$$MTT = \frac{CBV}{CBF} \tag{4}$$

TTP and Tmax are defined as the times at which the CTC(t) and R(t) respectively reach their maximum and are calculated accordingly.

#### 3. METHODS

#### 3.1. Study Design and Data Acquisition

The study is based on imaging and clinical data from the UCLA stroke registry, a database approved by the internal review board (IRB). All patients included in this study were treated for an acute ischemic stroke at the UCLA Ronald Reagan Hospital in Los Angeles between 2010 and 2016. Inclusion criteria for this study included: (1) Diagnosis of acute ischemic stroke in the middle cerebral artery (MCA) territory or border-zone areas, (2) last known well time within 24 h, (3) MRI of the brain performed before IV-tPA administration or endovascular clotretrieval therapy. A total of 344 patients (mean age, 61 years; range 13 − 97; average NIHSS, 14; range 0 − 38) satisfied the above criteria and underwent MRI using a 1.5 or 3.0 Tesla echo planar MR imaging scanner (Siemens Medical Systems).

#### 3.2. Perfusion Image Processing

Ground truth perfusion maps, used to train and evaluate the predictive models, were computed using Olea Sphere's Perfusion MRI oSVD algorithm (OLEA S.A., La Ciotat, France). To reduce artifacts, motion correction, spatial and temporal smoothing are applied. CTCs are computed from the image intensity curve I(t) from the DSC MRI's signal intensity at each voxel. An arterial input function (AIF) was also identified from the CTCs of pixels in major arteries, and manually validated by an expert. An oSVD-based deconvolution was used within the Olea software to compute rCBV, rCBF, MTT, TTP, and Tmax. This deconvolution is a cyclic convolution of the AIF and residue function (R) and is represented as a matrix multiplication of the form: CTC = A × R ∗ CBF, with × as matrix multiplication, and ∗ as scalar multiplication. Here, CTC and R are represented as column vectors where each component is the function value at a point in time. A is the cyclic convolution matrix constructed from the AIF so that matrix multiplication by A results in a discrete convolution. SVD is then run to invert A and compute R. The perfusion parameters are then computed from R based on its functional form.

#### 3.3. Data Preparation

To handle the different time resolution of the DSC acquisition across patients, CTCs and AIFs vectors were resized in the temporal domain to a set of 40 values using bicubic interpolation. AIFs used in training were those chosen by Olea Sphere's automatic AIF inference, to ensure that any difference between the output of our models and the OLEA sphere arises from differences in the model, and not on differences in AIF.

The data from each patient was resampled uniformly across the range of the perfusion parameter of interest. The rationale for this was that taking a random subset from the true frequency distribution of the perfusion parameters would bias our function to unevenly represent the full range of the parameters. Lower sections of the brain included many faulty parameter measurements: these slices were excluded from the training data.

#### 3.4. Regression of Parameters From Source Perfusion Imaging

We introduce here a regression-based formulation of the reconstruction of perfusion parameters from source MRI images as summarized in **Figure 1**. The regression model is trained to predict one of the perfusion parameters (i.e., rCBF, rCBV, MTT, Tmax, and TTP) from CTC data at the voxel level. The input to these algorithms takes the form of a one-dimensional vector, containing concentration time curve information combined with the arterial input function. The output of the model is set as the perfusion parameters previously computed used a FDA approved software. All models were trained using Matlab (The MathWorks, Inc., Natick, Massachusetts, United States.)

For each pixel, we define a feature vector, consisting of the concatenation of CTC and AIF data. For a window of size 2e + 1, the feature vector for a voxel located at the coordinate (x, y) is of the form:

$$\mathbf{x} = \left[ \mathbf{C}\_{\mathbf{x}-\varepsilon, \mathbf{y}-\varepsilon}, \mathbf{C}\_{\mathbf{x}-(\varepsilon-1), \mathbf{y}-\varepsilon}, \dots, \mathbf{C}\_{\mathbf{x}, \mathbf{y}}, \dots, \mathbf{C}\_{\mathbf{x}+\varepsilon, \mathbf{y}+\varepsilon}, A \right] \tag{5}$$

where Ci,<sup>j</sup> is a vector of values representing the CTC of the pixel at position (i, j):

$$\mathbf{C}\_{i,j} = \mathbf{C}\_{i,j,t\_0}, \mathbf{C}\_{i,j,t\_1}, \dots, \mathbf{C}\_{i,j,t\_n} \tag{6}$$

and A is the AIF:

$$A = A\_{t\_0}, A\_{t\_1} \dots A\_{t\_n} \tag{7}$$

Note that the explicit spatial/temporal relationships between features is recorded nowhere in the feature vector.

The computation of the value y<sup>i</sup> of a perfusion parameter at a given voxel i is posed as a regression one, such as y<sup>i</sup> = F(xi). The function F that maps the observed CTC and AIF (xi) to the output parameter is represented by a regression model. When a large number of labeled data points is available, numerous algorithms are available to solve the regression problem. We focus here on standard methods that have been successfully used on a wide variety of applications: support vector machines (SVM), neural network, ridge regression (both linear and kernel), and random forests. In addition, we also include a simple multilinear regression model as baseline method.

#### 3.4.1. Multiple Linear Regression

Multiple Linear Regression analysis (20) aims at fitting a model such that the sum-of-squares error (SSE) between the observed and the predicted values is minimized. Let β a matrix of s parameters,

$$Y = \beta X + \epsilon \tag{8}$$

retrospective stroke cases. Once trained, the performance of the model is assessed on new cases not present in the training data.

$$\Leftrightarrow y\_i = \beta\_1 \mathbf{x}\_i(1) + \beta\_2 \mathbf{x}\_i(2) + \dots + \beta\_s \mathbf{x}\_i(s) + \epsilon\_i \tag{9}$$

where i = 1 . . . n and ǫ<sup>i</sup> = N(0, σ 2 ) denotes the noise variables. Multiple Linear Regression analysis finds estimates coefficients βˆ such that they minimize the sum-of-squares error (SSE) which measures the total error between each prediction and the actual value of the output variable,

$$\hat{\beta} = \operatorname{argmin}\_{\beta} \sum\_{i=1}^{n} (\beta x\_i - \jmath\_i)^2 \tag{10}$$

The optimal βˆ can be expressed as βˆ = (XX<sup>T</sup> ) <sup>−</sup>1X <sup>T</sup>Y. We used a QR factorization to obtain estimated regression coefficients βˆ.

#### 3.4.2. Random Forests

Decision trees use a hierarchical structure that represents a series of recursive tests performed on the input features to produce an output class. To build the structure of the decision trees, we use the Classification and Regression Trees (CART) algorithm (21). CART is a standard learning algorithm for decision trees based on binary recursive partitioning.

The CART algorithm iterates through three steps to create new nodes in the tree, starting with a single root node:


The procedure expands the tree until the minimum number of samples in a leaf node is reached. After all the terminal nodes are found, the tree acquires its maximum size and can be pruned to produce the final tree. Bootstrap aggregating was then used to generate a forest of 100 decision trees. Predictions were obtained by averaging the output of the trees.

#### 3.4.3. Neural Network

A standard feedforward neural network was implemented as baseline technique. It consists of three types of layers: the input layer which is connected to the input features, the hidden layers that are connected in a cascade, and the output layer that is used to produce the output label. Each layer is associated with a transfer function that applies a weight and bias to its input; these parameters are optimized during the training phase of the model. In this study, a total of 7 layers was optimized using a scaled conjugate backpropagation gradient algorithm.

#### 3.4.4. Support Vector Machines (SVM)

SVM (22) aims at finding the optimal separating hyperplane that minimizes the misclassification rate, while maximizing the sum of distances of the samples from the hyperplane. Formally, this problem amounts at finding the parameter α,

$$\begin{aligned} \underset{\alpha}{\text{argmin}} & \quad \frac{1}{2} \boldsymbol{\alpha}^T \mathbf{Q} \boldsymbol{\alpha} - \boldsymbol{e}^T \\ \text{subject to} & \quad \boldsymbol{\gamma}^T \boldsymbol{\alpha} = \mathbf{0} \end{aligned} \tag{11}$$

$$\boldsymbol{0} \le \boldsymbol{\alpha}\_i \le \mathbf{C}, \ i = 1, \dots, n$$

where C is a constant that controls the amount of penalty on the error term during the minimization process, e is a vector of all ones, and matrix Q defined as:

$$Q\_{i\circ} = \mathcal{y}\_i \mathcal{y}\_j K(\boldsymbol{\omega}\_i, \boldsymbol{\omega}\_j) \tag{12}$$

$$K(\mathbf{x}\_i, \mathbf{x}\_j) = \exp - \|\mathbf{x}\_i - \mathbf{x}\_j\|^2 / 2\sigma^2 \tag{13}$$

K represents a Gaussian kernel that maps samples into another space and σ is the standard deviation of the kernel. After learning, SVM can be used to make predictions on new samples x by evaluating the weighted sum of the distances between the sample and each of the training vectors x<sup>t</sup> in the kernel space. Class membership probability estimates were obtained using Platt's scaling method (23) which uses logistic regression on the top of the SVM's scores.

#### 3.4.5. Ridge Regression

Ridge regression (24) is a standard technique that aims at minimizing the residual sum of square (RSS) to infer the projection vectors a:

$$\arg\min\_{a} \sum\_{i=1} (y\_i - a^T \bar{\mathbf{x}}\_i)^2 + \alpha \|a\| \tag{14}$$

where x¯<sup>i</sup> = x<sup>i</sup> − µ are the centered data points with respect to mean µ, y<sup>i</sup> is the response vector, and α is a regularization factor on the norm of a.

The problem can be formulated as

$$(K + \alpha I) \; a \; = \; y \tag{15}$$

where K equals to XX<sup>T</sup> in linear ridge regression, and a Gaussian kernel projection for kernel ridge regression, I is the identity matrix and α > 0 is a regularization parameter. Solving for a can be performed using the Cholesky factorization. Because no eigenvector computation is involved, there is a considerable reduction of computational cost while providing nonlinearity.

#### 3.5. Experiments

The purpose of the experiments is to examine if the computation of perfusion parameters from DSC imaging can be performed using a regression formulation (section 3.4). Here, we focus on the following parameters: CBF, CBV, MTT, TTP, and Tmax. For each patient, our dataset holds unprocessed, source perfusion MRI scans within 24 h and their corresponding perfusion parameters computed with a FDA-approved commercial software (Olea Sphere from Olea medical). As part of our experiments, we compare the predicted output of the models to the groundtruth. We report the normalized root-mean-square error (NRMSE) and the coefficient of repeatability (CR) which are two recommended techniques to evaluate regression models (section 3.5.1).

In addition to comparing the equivalence between the maps produced by Olea sphere software and the output of the ML framework, we evaluate the robustness of the ML models in the case of known parameter values using a virtual phantom model (section 3.5.4).

#### 3.5.1. Validation and Metrics of Accuracy

For the purposes of training and validation, a train-test split was used, in which four fifths of the available cases were randomly selected and used to generate training vectors (as described in section 3.4) and the remaining one fifth of cases were used to validate the models, using the accuracy metrics defined in the following paragraph.

The accuracy of each regression model is assessed using the normalized root-mean-square error (NRMSE) and the coeficient of repeatability (CR); two standard metrics of accuracy recommended in such setting (25) in such setting. The NRMSE is defined from the root-mean-square error (RMSE):

$$\text{RMSE} = \sqrt{\frac{\sum\_{i=1}^{n} (\hat{y}\_i - y\_i)^2}{n}} \tag{16}$$

where y<sup>i</sup> is the groundtruth value, yˆ<sup>i</sup> is the predicted output, and n is the number of data samples being tested.

$$\text{NRMSE} = \frac{\text{RMSE}}{\nu\_{\text{max}} - \nu\_{\text{min}}} \tag{17}$$

where ymax − ymin represents the range of the output values.

The coefficient of repeatability (CR) originates from the bland-altman (BA) plot which represents the differences between groundtruth (yi) and predictions (yˆi):

$$BA(\mathbf{x}, \boldsymbol{\mathcal{y}}) = \left(\frac{\hat{\boldsymbol{\mathcal{y}}}\_{i} - \boldsymbol{\mathcal{y}}\_{i}}{2}, \hat{\boldsymbol{\mathcal{y}}}\_{i} - \boldsymbol{\mathcal{y}}\_{i}\right) \tag{18}$$

BA captures the error with respect to the value in the output space. It is common to look at the standard deviation within that space; the smaller the standard deviation, the closer the groundtruth and predictions tend to be on average (26). The coefficient of repeatability (CR) precisely captures this notion:

$$\text{CR} = 1.96 \times \sqrt{\frac{\sum (\hat{y}\_i - y\_i)^2}{n}} \tag{19}$$

The CR means that the difference between any pairs of prediction, groundtruth is expected to be in the interval [-CR, CR] for 95% of samples.

#### 3.5.2. Training Sample

The first experiment evaluates the effect of the number of training samples on the two metrics of accuracy (i.e., NMSRE and CR). A different model is trained for each of the six regression techniques (linear, ridge, kernel ridge, SVM, neural network, random forests) by varying the number of training samples while keeping the test sample fixed. The number of samples using for training the models was generated using a logarithmic distribution L ranging from 100 to 16, 000 samples. The NMRSE and CR metrics are reported for each combination of number of samples and regression model, on each PWI modality; CBF, CBV, MTT, Tmax, and TPP.

#### 3.5.3. Patch Size

Regression models are evaluated with a patch size varying from 1 × 1, to 17 × 17. Here, we set the number of training samples to 15, 000 samples. Similarly to the evaluation of the number of training samples, we compute the NMRSE and CR error for each combination of regression models, PWI perfusion parameter, and patch size.

#### 3.5.4. CBF Phantom Model

The CBF phantom model is constructed by selecting 20 patients at random within our cohort. For each patient, the range of CBF values is discretized into 12 bins within the 5th and 95th percentile. The CTC curves of each pixel falling within each bin are used to compute a trimmed mean (by removing the top and lower 10% outliers). At the end of the process, each patient p is characterized by 12 average and idealized CTC curves F<sup>p</sup> = C1...<sup>12</sup> together with a manually validated, low-pass filtered AIF curve Ap.

To evaluate the robustness of each regression models to noise, the phantom is produced by altering the idealized CTC curves F<sup>p</sup> using additive white Gaussian noise (AWGN), ranging from a SNR of 50 to 1 dB. Similarly to our previous experiments, we report the NRMSE as metric of accuracy to correctly estimate the correct CBF. As baseline method, we use a deconvolution method of block-circulant singular value decomposition (cSVD) which is a delay-insensitive method typically used to compute CBF (27).

#### 3.5.5. Computational Cost

A crucial aspect of the processing of perfusion-weighted MRI is the time required to compute the maps. While some methods may be more accurate and robust to noise, they might also take a prohibitive time to compute new images. In this experiment, we report the average time to compute a CBF perfusion maps from a 128 × 128 × 40 source PWI. Our goal is to find the method that has the best trade-off between computational cost and accuracy.

# 4. RESULTS

The results of the sample size experiments indicate that both error metrics are decreasing significantly in the first 4, 000 training samples. When looking at the average of all regression models in **Figure 2**, the reduction goes from 0.225 ± 0.01 to

0.205 ± 0.02 for CR and from 0.12 ± 0.005 to 0.105 ± 0.005 for NRMSE; both are statistically significant reductions (p < 0.01).

**Figure 3** summarizes the results of the patch size effect on CR (left) and NRMSE (right) error metrics. In both cases, the average error over all regression models reaches a minimum (CR = 0.23, NRMSE = 0.13) at a size of 5 × 5 pixels. This confirms previous findings (17) where the use regional information using of local patches in the context of regression tend to provide more robust predictions.

Overall results are summarized in **Tables 1**, **2** for each parameters (TTP, MTT, Tmax, rCBV, rCBF) and each regression model (Linear, Ridge, Kernel ridge, SVM, neural network, random forests). Overall, Random forest was the best method with respect to both error metrics. SVM, Kernel Ridge, and Neural network perform equivalently and can be considered as a good alternative. Linear regression and Ridge regression on the other hand performed more poorly than the rest of the models.

The results of the phantom experiment are illustrated in **Figure 4** where the linear and random forests models do not perform as well as the other methods (including the cSVD deconvolution method). Other regression methods are more stable than cSVD. The differences between these models (SVM, Ridge, Neural Network, Kernel ridge) was not significant. Please note that this result measures the stability of the predictions of each model with respect to noise, but does not reflect the bias associated with the models. The ground truth used here is the output of each model on the idealized curves computed without noise. By adding noise to these idealized curves of known rCBF, we can test if the models produce an estimated output that is similar to what they predict without noise.

In terms of processing speed, linear and ridge regression models perform best with a similar execution time of 0.006 s. Neural network and kernel ridge regression follow with 0.61 and 2.83 s, respectively. Without optimization, SVM and random forests models were more costly with 6.73 7.23 s for predicting a slice of 128 × 128 voxels.

#### 5. DISCUSSION

The experimental results above demonstrate that machine learning models can produce perfusion maps from source perfusion imaging. The best-performing algorithms in our analysis were neural networks and Random Forests: these techniques produce perfusion maps with similar visual appearance and good numerical correspondence to the output of a standard, FDA-approved perfusion processing algorithm (see **Figure 5** for a visual comparison of the machine-learning algorithms and SVD). While Random Forests performed marginally better than neural networks on the original testing data, addition of white Gaussian noise caused this gap to widen significantly, with performance of the random forest model degrading substantially at low SNR, while the neural network model was more robust to noise than SVD. For this

TABLE 2 | 15,000 training samples–5 × 5 patch–Coefficient of Repeatability (CR) \* 100.


FIGURE 4 | NRMSE of regression-based estimation of rCBF using phantom model, showing the effect of adding noise to the phantom perfusion curves on reconstruction of rCBF. For each model considered, the error is relative to the output of same model trained on data without added noise. The cSVD deconvolution model is used a baseline method.


reason, overall we judge that the neural network model is the best-performing of the models tested, in terms of ability to reproduce perfusion maps in the presence of noise. Moreover, the neural network model reproduced perfusion maps within less than one second, per slice: given that five perfusion maps must be processed per acquisition, this leads to an unoptimized processing time approximately two minutes per case, which is comfortably within the timescales expected in the clinical setting of acute stroke. Timely, robust processing of perfusion data is vital for assessment and treatment selection in acute stroke: perfusion maps derived from learning algorithms may allow clinical decision making to be made faster, and to more robust to noise in sequence acquisition.

Perfusion-processing typically operates on a voxel-by-voxel level, calculating the perfusion parameters in a voxel from the concentration-time curve of that voxel, together with the arterial input function. Information from neighboring voxels is only incorporated by first applying a spatial smoothing before deconvolution. However, our experiments suggest that robustness of perfusion parameters is improved by incorporating data also from surrounding voxels. This effect is only apparent with small patch sizes. with larger patches leading to an increase in RMSE and CR. This may be a result of the flat data representation used: a model based on, for example, spatial convolutions might be better able to incorporate larger patches without overfitting. The fate of tissue in ischemic stroke is betterpredicted from spatial features derived from standard perfusion parameters than from just the voxel-by-voxel parameters (16), but improved spatial perfusion processing incorporating may reduce this effect. Although slice spacing in MR perfusion is rather large (3–6 mm), the use of 3D context (using, for example, three-dimensional convolutional neural networks) may provide further useful information.

The experiments here provide a starting point for the use of other machine learning models on raw CTC data in predicting perfusion parameters as well as us of raw CTC data on predictions in general. One limitation of our approach is that source of the training data, since this is limited to stroke cases from a single center. Further studies would benefit from external validation incorporating multicenter data and covering a number of different pathologies. Perfusion imaging is used, for example, in tumor typing and grading (28, 29). A further limitation of the study is our reliance on an external algorithm for the automatic inference of the arterial input function. This decision allowed us to be certain that, any difference in perfusion maps calculated was due to the machine-learning method, and not a difference in arterial input function. One final limitation of our study is that we do not assess the clinical impact or advantages of the method, concentrating instead on the quality of reconstruction with respect to established methods. A follow-up paper is in preparation which assesses the differences between penumbral volumes, ASPECTS scoring, and eligibility for treatment according to DEFUSE 3, between a machinelearning method and oSVD. Having demonstrated the robustness of machine-learning tools for perfusion analysis to noise, we can, in a further step, analyze the robustness of the generated maps to changes in arterial input function: more ambitiously, we can

#### REFERENCES

1. Essig M, Shiroishi MS, Nguyen TB, Saake M, Provenzale JM, Enterline D, et al. Perfusion MRI: the five most frequently asked technical questions. AJR Am J Roentgenol. (2013) 200:24–34. doi: 10.2214/AJR.12.9543

envisage machine-learning systems which also infer the arterial input function, or even systems which implicitly incorporate arterial blood flow as a latent variable inferred directly from imaging.

As well as reconstructing perfusion maps, there is also potential for these machine learning models to predict other Perfusion MRI related values. In particular, sequence-tosequence models could be devised to infer the residue function, rather than its related perfusion maps, from the concentrationtime curves. Finally, since the goal of perfusion imaging in stroke is to assess the extent of likely tissue damage, we are working currently on models to predict tissue fate directly from source perfusion imaging.

# 6. CONCLUSION

This paper represents a proof of concept that standard perfusion maps, as used in clinical routine, can be reproduced quickly and with low reconstruction error using simple supervised learning techniques. Nonlinear models such as Random Forests and feedforward neural networks outperformed simpler linear and regularized linear models, and well as kernel-based methods. While the mean squared error of the random forest models were lower than those of the neural network models, the neural network models were more robust to noise. This study paves the way for further advances in the processing of perfusion data by means of machine learning.

#### ETHICS STATEMENT

Study was performed using retrospective data, which was collected from acute ischemic stroke patients admitted at the University of California, Los Angeles Medical Center. The use of this dataset was approved by the local Institutional Review Board (UCLA IRB).

# AUTHOR CONTRIBUTIONS

RM concepted the study, analyzed the data, drafted and edited the article. FH contributed code, drafted and edited the article. RW concepted the study, drafted and edited the article. DL concepted the study, provided data, drafted and edited the article. FS concepted the study, contributed code, analyzed the data, drafted and edited the article.

# FUNDING

RM was supported by Swiss National Science Foundation grants 320030L\_170060 and 32003B\_160107, and by a grant from the Schweizerische Herzstiftung (FASTER-CT).


J Cereb Blood Flow Metab. (2010) 30:703–17. doi: 10.1038/jcbfm. 2010.5


on Bioinformatics and Biomedicine (BIBM). Washington, DC (2015). p. 1316–21.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 McKinley, Hung, Wiest, Liebeskind and Scalzo. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Thinking About the Future: A Review of Prognostic Scales Used in Acute Stroke

Bogna A. Drozdowska, Sarjit Singh and Terence J. Quinn\*

*Institute of Cardiovascular and Medical Sciences, College of Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow, United Kingdom*

Background: There are many prognostic scales that aim to predict functional outcome following acute stroke. Despite considerable research interest, these scales have had limited impact in routine clinical practice. This may be due to perceived problems with internal validity (quality of research), as well as external validity (generalizability of results). We set out to collate information on exemplar stroke prognosis scales, giving particular attention to the scale content, derivation, and validation.

#### Edited by:

*Bruce Campbell, The University of Melbourne, Australia*

#### Reviewed by:

*Daniel Strbian, Helsinki University Hospital, Finland Jens Fiehler, University Medical Center Hamburg-Eppendorf, Germany*

> \*Correspondence: *Terence J. Quinn terry.quinn@glasgow.ac.uk*

#### Specialty section:

*This article was submitted to Stroke, a section of the journal Frontiers in Neurology*

Received: *07 January 2019* Accepted: *01 March 2019* Published: *21 March 2019*

#### Citation:

*Drozdowska BA, Singh S and Quinn TJ (2019) Thinking About the Future: A Review of Prognostic Scales Used in Acute Stroke. Front. Neurol. 10:274. doi: 10.3389/fneur.2019.00274* Methods: We performed a focused literature search, designed to return high profile scales that use baseline clinical data to predict mortality or disability. We described prognostic utility and collated information on the content, development and validation of the tools. We critically appraised chosen scales based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies (CHARMS).

Results: We chose 10 primary scales that met our inclusion criteria, six of which had revised/modified versions. Most primary scales used 5 input variables (range: 4–13), with substantial overlap in the variables included. All scales included age, eight included a measure of stroke severity, while five scales incorporated pre-stroke level of function (often using modified Rankin Scale), comorbidities and classification of stroke type. Through our critical appraisal, we found issues relating to excluding patients with missing data from derivation studies, and basing the selection of model variable on significance in univariable analysis (in both cases noted for six studies). We identified separate external validation studies for all primary scales but one, with a total of 60 validation studies.

Conclusions: Most acute stroke prognosis scales use similar variables to predict long-term outcomes and most have reasonable prognostic accuracy. While not all published scales followed best practice in development, most have been subsequently validated. Lack of clinical uptake may relate more to practical application of scales rather than validity. Impact studies are now necessary to investigate clinical usefulness of existing scales.

Keywords: stroke, functional outcome, prognostic scale, risk score, validation

Outcomes following a stroke event can range from full recovery, through varying degrees of disability to death. Given the subsequent need for intervention planning, resource use, and lifestyle adjustments, predicting outcome following stroke is of key interest and importance to patients, their families, clinicians, and hospital administrators. Various tools exist to assist in estimating stroke-related prognosis. For example, the ABCD2 score uses clinical features to predict risk of stroke following transient ischemic attack (TIA) (1). Although there are criticisms of ABCD2, it is widely used and included in stroke guidelines (2).

Scales for predicting acute stroke outcomes from baseline features are also described in the scientific literature (3–5). Often prognosis scales report mortality; however, given the disabling nature of stroke, scales predicting death and/or longerterm disability may be more useful in the stroke setting (6). However, these prognostic scales have had limited clinical traction and have not been incorporated into routine clinical practice (3). There are many plausible reasons why these scales have not been adopted by the stroke community (6). In an acute setting, scales may be perceived as being too complex to use or may require information that is not routinely available (for example, sophisticated neuroimaging) (3). Clinicians may moreover be concerned that scales are inherently too generic, and may not provide insight over what the clinician can conclude based on individual patient factors and clinical gestalt (7).

For many scales, clinicians may simply not be convinced of their utility or the rigor of the underpinning science. These points can be addressed by describing the validity of the scales. Issues with validity could relate to the methodological quality of the initial derivation of the scale (internal validity) or the generalizability of a scale to a real-world population (external validity). Robust evidence of validity requires assessment of the scale in cohorts independent of the population used to derive that scale (8). However, In some areas of stroke practice, for example rehabilitation, it has been demonstrated that independent validation studies are lacking for many scales (5).

Collating evidence around the quality of the research that led to development of prognostic scales and also the results of subsequent validation work could be useful for various stakeholders. For clinicians it may convince of the utility, or lack of utility, of certain tools; for researchers it may point to common methodological limitations that need to be addressed in future work and for policy developers, if a certain tool has a more compelling evidence base than others, then this scale may be preferred in guidelines.

Previous reviews have reported that many stroke prognosis scales have similar properties such as discrimination and calibration. These reviews also highlight the limited evidence for external validity of many commonly used stroke scales (9, 10). Distinguishing an optimal prognostic tool may not be possible based on psychometric properties alone and factors such as feasibility and acceptability in the real world setting need to be considered.

We sought to collate and appraise a selection of exemplar published stroke scales, designed for use in acute care settings. We used these as a platform to discuss methodological quality of prognostic scale development, while also considering potential barriers or facilitators to implementation of the scales in clinical practice.

# METHODS

We performed a focused review of the literature to find scales predicting post-stroke mortality and/or function. Our approach followed that used in a recent comparative efficacy review of stroke scales (9). Rather than assess every tool that has ever been used to make outcome predictions in stroke, we were interested in examples of high profile prognostic scales. Although our intention was not a comprehensive search, we followed, where relevant, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance for designing and reporting our study (11). For consistency in use of terminology, we have referred to the prediction models as "scales," and the calculated outputs of models as "scores."

# Inclusion/Exclusion Criteria

We defined a scale as any tool that uses more than two determinants to estimate the probability of a certain outcome. We focused on scales with predominant clinical input variables that can be applied without specialist resources or tests and to this end we excluded scales that had more than two neuroimaging input variables. We limited to ischemic or all cause (undifferentiated) stroke scales, recognizing the differing natural progression of ischemic and hemorrhagic strokes.

# Search Strategy

Our focus was on scales that are well known in the stroke field and so we adapted our search using an approach that has been used in other focused stroke studies (12, 13). We limited our search to 11 high profile, international journals, chosen based on relevance to stroke and clinical impact, covering fields of stroke, neurology, internal medicine, and geriatric medicine (a full list of journals and the search strategy are included in **Supplementary Materials**).

Searches were from inception to May 2018. Once we had selected chosen scales we used PUBMED and Google Scholar electronic search engines to find the initial development paper and any potential validation papers. A single researcher (SS) performed the search and screened the results We assessed internal validity of the search results by screening title lists twice (October 2015 and May 2018).

# Data Extraction and Critical Appraisal

Two researchers (BD, SS) extracted data from selected studies, using a pre-specified proforma. This included information on: data source, study sample characteristics, predictor, and outcome variables, procedures involved in model derivation, methods of validation, measures of performance and presentation of results. Extracted data were comprehensively reviewed to inform critical appraisal, a process in which all authors (BD, SS, TQ) were involved.

The methodological assessment of prognostic scales is an evolving landscape. Although there is no consensus preferred approach to this, there are certain features common to most tools that purport to assess validity of prognosis research. We based our assessment on recommendations from the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) checklist (14). Discrepancies in assessment between researchers were discussed and resolved through consensus.

#### Data Used for Scale Development

We assessed the representativeness of the sample from which information was collected. Generalizability of a scale to a broader patient population may be compromised when recruitment takes place in a highly specific context or is limited to a relatively homogeneous group; when multiple inclusion and exclusion criteria are applied; and finally, when patients with missing data are excluded from the study (complete-case analysis). The latter presents itself as an issue, as it is uncommon for variable values to be missing completely at random. Often this is related to other predictors, the outcome, or even the value of that variable itself (15). Therefore, patients with missing data are likely to form a selective rather than random subsample of the initial baseline cohort, and may substantially differ from those included in the analysis (14, 16).

#### Scale Variables

For predictor and outcome variables, a particular concern was whether they are precisely defined and measured in a way that can be reproduced across different centers. It is recommended that continuous data (e.g., age) are not categorized when introduced to a model as a predictor (17, 18). Doing so is associated with loss of information and power, and increases the risk of generating inaccurate estimates and residual confounding. Finally, bias may arise from lack of blinding to predictors when assessing an outcome, or blinding to the outcome where predictors are assessed retrospectively.

#### Scale Development Process

In this context, we assessed study sample size against the number of candidate predictors being tested. For logistic regression procedures, we considered a minimum of ten events (number of patients with the less frequent outcome) per variable to be sufficient (19, 20). Evaluating the selection process of predictors for inclusion in scales presents a challenge, as there is no agreed approach (21). There are however certain practices that are consistently stated to increase risk of bias. One is selecting predictors for inclusion in multivariable analysis based on significance in univariable analyses (22). This approach may lead to exclusion of predictors that could be associated with the outcome after adjusting for the effects of other factors. A data-driven approach to variable selection may lead to model overfitting (23) and forward selection techniques should be avoided in multivariable modeling (24). Either a full model approach (all candidate variables included in the model) or backwards elimination (beginning with all candidate predictors, removing those that do not satisfy a pre-specified statistical criterion) is preferable (25).

#### Assessment of Scale Performance

We distinguished three levels of validation: apparent, internal and external (26–28). In apparent validation, predictive ability is assessed in the development set itself and may give overoptimistic performance estimates. With internal validation two approaches are described, split-sample and cross-validation. These involve randomly splitting the baseline sample into development and assessment sets. In the split-sample technique, the population is divided once, in cross-validation, the process of sample division is repeated for consecutive fractions of subjects, thus allowing for each participant to be included in the validation set once. Here, a larger part of the baseline sample can be used for model derivation, avoiding the considerable loss of power associated with split sample approaches (14). The most efficient method of internal validation is considered to be bootstrapping, where samples are drawn with replacement from the original dataset, replicating sampling from an underlying population (26). The generated sample is of the same size as the original dataset.

Importantly, even with use of internal validation techniques, assessing a scale's performance in the development cohort is considered insufficient to confirm its value and general applicability (27, 29), In view of this, we prioritized findings from external validation studies. External data can differ from derivation data in terms of when and where it was collected, as well as by research group. Typically, an external dataset is comparable to the original, however in some studies a model is intentionally tested in a population characterized by different clinical features.

Reviewing study results on predictive performance, we focused on measures of discrimination and calibration, as these properties are necessary (although not sufficient) to ensure clinical usefulness of a prognostic scale (14, 27). Discrimination relates to the ability of a model to accurately distinguish between those who develop a certain outcome and those who do not, and is commonly expressed as the area under the receiver operating characteristic curve (AUROC) (30). To aid interpretation of results, we applied to following AUROC cutoff values: 1.00—perfect discrimination; 0.90 to 0.99—excellent; 0.80 to 0.89—good; 0.70 to 0.79—fair; 0.51 to 0.69—poor; 0.50 of no value, equivalent to chance (31). Calibration refers to the level of agreement between observed and predicted outcome probabilities, with assessment preferably based on inspection of calibration plots/curves (32). Graphical evaluation can be accompanied by reporting results of the Hosmer-Lemeshow test, assessing whether there is a significant difference between observed and predicted outcomes. The test has however limited power for detection of poor calibration, is oversensitive in large samples, and does not allow to determine the direction of miscalibration (33).

# RESULTS

#### Overview of Scale Content and Quality

Our search returned 3817 results. We found 10 primary scales that met our inclusion criteria, six of which also had modified versions published (**Tables 1**, **2**). Scales used from four to thirteen input variables, with a mode of five. There was considerable overlap in the predictors used, including variables relating to demographics, past medical history and the acute stroke event (**Table 3**). The most commonly incorporated predictors were: age (all ten scales), a measure of stroke severity (eight); prestroke function, comorbidities and stroke subtype (each present in five scales). Individual scale content, with scoring, is presented in **Table 4**.

For seven scales the outcome of interest was a specified range of scores on the modified Rankin Scale (mRS) (34–36). The scale is a measure of functional outcome following stroke, ranging from no symptoms (a score of zero) through increasing levels of disability, to death (a score of six). Five scales focused on mortality, while one scale aimed to predict recurrent stroke and another length of hospital stay. Four scales predicted more than one outcome.

Our critical appraisal of scales' development process identified potential sources of bias in each study, as well as issues related to incompleteness of reporting for methods and results. Most common limitations were around handling missing data and model development. In relation to the former, in two studies it was not clearly stated how missing data was handled. Six studies used complete-case analysis, and the remaining two excluded participants from analyses involving the particular variables they had no data for. For model development, six studies selected variables for multivariable modeling based on the univariable significance (**Table 5**).

We present an overview of each scale, focusing on scale content, development, validation, and where applicable any modification to the scale. We summarize our critical appraisal of derivation studies and discuss potential issues around implementation of the scales in routine clinical practice.

# Acute Stroke Registry and Analysis of Lausanne (ASTRAL)

#### Scale Content and Development

The ASTRAL scale uses six input variables to predict unfavorable functional outcome at 3 months (mRS>2): age, stroke severity according to the National Institutes of Health Stroke Scale (NIHSS) (37), time from symptom onset to admission, range of visual fields, acute glucose, and level of consciousness (38). Based on these variables, an integer score is assigned, from zero with no upper limit. Higher scores are associated with a greater probability of an unfavorable outcome. Through a logistic regression procedure, the scale was developed in a sample of 1,633 ischemic stroke patients from the Acute Stroke Registry and Analysis of Lausanne (39).

#### Scale Validation and Updating

Using a 2-fold cross-validation technique for internal validation, the scale was found to have good discriminatory power, AUROC:0.85 for prediction of mRS>2 at 3 months. The derivation paper further described external validation of the scale in two independent cohorts from Athens and Vienna (40, 41), reporting AUROC values of 0.94 and 0.77, respectively. Calibration was assessed in all three cohorts based on Hosmer-Lemeshow test and inspection of calibration plots, indicating a good fit with the data.

The ASTRAL scale has been subsequently externally validated by seven studies, with six assessing predictive value based on AUROC estimates (42–48). Within these, ASTRAL was found to have fair to good discriminatory power, with the exception of one study, involving a Brazilian cohort (AUROC 0.67) (44). These external validation studies used differing time points for outcome assessment (up to 5 years post-stroke) and differing outcomes, including mortality and symptomatic intracerebral hemorrhage (sICH).

#### Critical Appraisal and Clinical Application

In the ASTRAL derivation study we found potential sources of bias relating to participant selection, namely excluding all patients with pre-stroke dependency and any missing data. In addition, treatment effects were not accounted for. We also noted that some issues relevant to scale development were unclear: whether any method of blinding was used, the number of candidate predictors (which allows to estimate whether the sample size was sufficient), and finally whether there were any significant baseline differences between the derivation and validation cohorts.

Despite these concerns, evidence from validation studies suggests that the predictive performance of ASTRAL is sufficient for the clinical setting. The scale was designed with the acute context in mind, and does not require sophisticated diagnostic tests. Nonetheless, in some cases, estimating onset to admission time may not be possible. Where all necessary information is accessible, the ASTRAL offers an easilycalculable score, with use aided by color-coded graphs to assign a percentage probability of unfavorable outcome based on clinical features. There is also a score calculator available online (49).

#### Dense Artery, mRS, Age, Glucose, Onset-to-Treatment, and NIHSS (DRAGON) Scale Content and Development

The DRAGON scale incorporates the six variables in its acronym, as well as early infarct signs on computed tomography (CT). It was developed to predict functional outcome at 3 months in stroke patients treated with intravenous tissue plasminogen activator (IV-tPA) (50). The outcome was trichotomized according to mRS scores, where mRS 0–2 was defined as "good outcome," mRS 3–4 as "poor outcome" and mRS 5–6 as "miserable outcome." Scale scores range from one to ten, with higher values associated with poorer outcomes. The scale was derived in a single-center Finish cohort of 1,319 ischemic stroke patients, using a logistic regression procedure.

#### TABLE 1 | Derivation study characteristics.



*IV-tPA, intravenous tissue plasminogen activator; mRS, modified Rankin Scale.*

\**Combined recruitment period for all trials.*

#### Scale Validation and Updating

On internal validation, using 1,000 bootstrap replications, DRAGON was found to have an AUROC of 0.84 (95%CI: 0.80–0.87) for prediction of miserable outcome. A comparable AUROC value was obtained through external validation, performed by the authors in a cohort of 330 Swiss patients: 0.80 (95%CI: 0.74–0.86). Calibration was not assessed. DRAGON has undergone subsequent external validation in ten studies (42, 43, 46, 51–57), all of which concluded the scale performs well, and (where assessed) had fair to good discriminatory power. In majority of cases, the scale was used in a similar context and for the same purpose as in the derivation study. However, one study assessed prediction of sICH (42).

Recognizing the increasing use of magnetic resonance imaging (MRI), the original DRAGON scale was adapted to include MRI based variables (58). Namely, with all clinical variables remaining unchanged, proximal middle cerebral artery occlusion on MR angiography replaced hyperdense artery sign, and the diffusion-weighted imaging Alberta Stroke Program Early Computed Tomography Score (DWI ASPECTS) replaced CT early infarct signs (59, 60). The scale was derived in a French cohort of 228 patients treated with IV-tPA. Internal validation was performed using a bootstrapping method. For prediction of 3-month mRS>2, MRI-DRAGON was found to have an AUROC of 0.83 (95%CI: 0.78–0.88). The scale was externally validated in one subsequent study, where reported AUROC values for prediction of poor and miserable outcome were 0.81 (95%CI: 0.75–0.87) and 0.89 (95%CI: 0.84–0.95), respectively (61).

#### Critical Appraisal and Clinical Application

We identified issues in the DRAGON derivation study. All continuous candidate predictors were categorized. A completecase analysis approach was employed and discriminatory power was only estimated for prediction of miserable outcome, while calibration was not assessed at all. Moreover, it seemed unclear whether any blinding method was applied for assigning mRS scores, and there were no description of the multivariable method for selection of final predictors.

In the context of clinical practice, DRAGON score should be easy to calculate [online score calculator available (62)]. Again, estimating symptom onset-to-treatment time may not be possible in some cases. There is potential for misinterpretation of early infarct and hyperdense cerebral artery signs (63–65). From this point of view, MRI-DRAGON appears a valuable alternative. MRI has been found to be a more sensitive method for ischemia detection than CT, and use of a semi-quantitative assessment of lesions is likely to ensure higher reproducibility (66). Importantly, based on results of validation studies, both versions of the scale seem to have satisfactory predictive ability, although evidence on performance of MRI-DRAGON is still limited.

# Five Simple Variables (FSV)

#### Scale Content and Development

The FSV scale incorporates two models for predicting functional outcome at 6 months post-stroke (67–69). One is used for good (mRS<3) or excellent (mRS<2) outcomes, and one for prediction of a devastating outcome (mRS>4; FSVDEV). The two models share four input variables: age, pre-stroke functional status (Oxford Handicap Score) (70), ability to lift both arms off the bed, and normal verbal response on the Glasgow Coma Scale (71). The first model additionally includes ability to walk unaided, while the FSVDEV incorporates stroke subtype. Prediction scores created based on the models range from−5 to 5 for the positive outcomes, and 0 to 15 for the devastating outcome. In both cases, a higher score is associated with a

#### TABLE 2 | Derivation study participants, outcomes and scale discriminatory power.


*mRS, modified Rankin Scale; LOS, length of hospital stay.*

\**Where reported in derivation study.*

\*\**AUROC for model, not derived risk score.*

TABLE 3 | Categories of input variables included in post-stroke prognostic scales.

*Green fill denotes inclusion of a variable from the given category in a prognostic scale.*

greater likelihood of having the outcome of interest. Both FSV models were derived in a single-center Canadian cohort of 538 stroke patients.

#### Scale Validation and Updating

Internal validation of the prediction scores, using 500 bootstrap replications, indicated good discriminatory power, with AUROC values of 0.88, 0.87, and 0.86 for good, excellent and devastating outcomes, respectively. Similar results were reported for initial external validation, conducted in a sample of patients from the Oxfordshire Community Stroke Project (OCSP), with AUROC values ranging from 0.86 to 0.89. Calibration was assessed only in the derivation sample for prediction of good outcome, and, based plotted calibration curves, concluded to be good (67).

FSV scores have been externally validated in one study (72), reporting good discriminatory power for prediction of good and devastating outcomes at 6 months in a Scottish stroke cohort. The use of five variables for predicting post-stroke functional outcome was also assessed in a cohort combining six European populations. However, here a similar scale was being


#### TABLE 4 | Continued


*(Continued)*

*(Continued)*

#### TABLE 4 | Continued


*ASPECTS, Alberta Stroke Program Early Computed Tomography Score; CNS, Canadian Neurological Scale; CT, computed tomography; GCS, Glasgow Coma Scale; IV-tPA, intravenous tissue plasminogen activator; LACS, lacunar stroke syndrome; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; OCSP, Oxfordshire Community Stroke Project; PACS, partial anterior circulation stroke syndrome; POCS, posterior circulation stroke syndrome; TACS, total anterior circulation stroke syndrome; TIA, transient ischemic attack; TICI, Thrombolysis In Cerebral Ischemia; TOAST, Trial of ORG 10172 in Acute Stroke Treatment.*

\**Risk score was not developed.*

\*\**Optional variable.*

independently developed rather than the FSV being externally validated (73). The described model included the same variables, although a different measure was used for estimating pre-stroke functional status [Barthel index (74)]. The authors reported good discriminatory power on both internal and external validation.

#### Critical Appraisal and Clinical Application

To assess FSV derivation, we reviewed three publications and identified potential sources of bias. The sample size was insufficient for the number of tested candidate predictors. Although a complete-case analysis method was not applied, with no data imputation, participants with missing data were excluded from particular analyses. Blinding was unclear. Input variables for multivariable analyses were selected based on univariate significance. In the paper where models for excellent and devastating outcomes were developed, calibration was not assessed, while in the remaining two, the procedure was mentioned but no calibration plots were presented. In relation to study results, differences in baseline characteristics between derivation and validation datasets were not assessed, and a datadriven approach was applied when selecting cut-off scores for outcome prediction (75).

In clinical practice, a significant advantage of FSV is the use of easily accessible and often routinely collected information. Moreover, for patients and their families, the differentiation between recovering to a level of functional independence with and without disability can be of particular value. It is unlikely however for this useful concept to be transferred into practice, as the same FSV cut-off score was chosen for both outcomes, the difference lying in prognostic accuracy for prediction of each. Finally, although reports on FSV performance are encouraging, further external validation studies are necessary before it can be considered for use in a clinical setting.

#### iScore

#### Scale Content and Development

The iScore was developed using a logistic regression procedure to predict death at two timepoints. The derivation study included 12,262 ischemic stroke patients from the Registry of the Canadian Stroke Network (76). For outcome prediction at 3 months, an integer score (from zero, with no defined upper limit) is calculated based on: age, sex, stroke severity assessed with the Canadian Neurological Scale (77), stroke subtype according to the Trial of ORG 10172 in Acute Stroke Treatment (TOAST) (78), acute glucose, history of atrial fibrillation, congestive heart failure, cancer, kidney disease, and preadmission dependency. For predicting one-year mortality, previous myocardial infarction and smoking status are added. Higher scores associated with greater mortality.

#### Scale Validation and Updating

In the derivation study, a split-sample validation method was chosen, with 8223 patients assigned to the development set (AUROC: 0.85 and 0.82 for 30-day and 1-year mortality, respectively) and 4039 in the internal validation (AUROC: 0.85 and 0.84). External validation used data from 3270 patients from the Ontario Stroke Audit. AUROC: 0.79 and 0.78, for 30-day and 1-year mortality, respectively.

The scale has been further externally validated in 15 studies (48, 54, 79–91). The iScore has been applied not only to predict mortality, but also poor functional outcome, institutionalization, clinical response, hemorrhagic transformations following thrombolytic therapy, and healthcare costs. All studies concluded that iScore is useful, predicting outcomes of interest with sufficient accuracy. Where AUROC values were estimated, they were fair to good, apart from one study where AUROC was 0.68 for 30-day mortality or disability at discharge (79).

Scale Data source Participants Outcome Predictors Sample size Missing data Model development Model performance Model evaluation Results

*Color code: green, low risk of bias; yellow, unclear or medium risk of bias; red, high risk of bias.*

Recognizing the difficulty of etiological classification (92), a revised iScore (iScore-r) was developed, replacing TOAST with OCSP (93). The revised scale was validated in a Taiwanese cohort of 3,504 ischemic stroke patients, for prediction of poor functional outcome (mRS>2) at discharge and at 3-months. Assessment of discriminatory power in an external cohort of iScore and iScore-r indicated comparable performance of the scales. AUROC of 0.78 and 0.77 for discharge outcome, and AUROC of 0.81 and 0.80 for 3-month outcome, with lower values reported for iScore-r.

#### Critical Appraisal and Clinical Application

We identified limitations in the iScore derivation. A completecase analysis approach was applied. Variables were selected based on univariable significance. Administration of treatments was not accounted for. A split-sample method was used for internal validation, while the external validation cohort was partially recruited from the same centers as the derivation cohort, which gives overoptimistic estimates of performance in independent populations. It was unclear whether blinding was applied; which inputs were included in the model as continuous and which were categorized; and how pre-stroke dementia (a candidate predictor) and dependency were operationalized.

The iScore scale has many external validation studies, which indicate sufficient prognostic ability for outcomes other than just mortality. Use of the scale can be aided by an online score calculator (94). Nonetheless, compared to most scales included in this review iScore require substantial baseline information. The revised scale may offer a solution to the issues of acute classification, yet the iScore-r derivation study reported high attrition rates, and with no further external validation studies, the generalizability of the scale remains uncertain.

# Preadmission Comorbidities, Level of Consciousness, Age, and Neurological Deficit (PLAN)

#### Scale Content and Development

The PLAN scale was developed to estimate probability of death and severe disability following ischemic stroke, specifically 30 day and 1-year mortality, and mRS>4 at discharge (95). A risk score ranging from 0-25 is calculated based on: pre-admission dependency, history of cancer, congestive heart failure, atrial fibrillation, consciousness, age, proximal weakness of the leg, weakness of the arm, aphasia and neglect. Higher scores are associated with greater likelihood of death or severe disability. The scale was derived through logistic regression using the same multicenter data source as in the case of iScore. The baseline sample comprised 9,847 patients. However, as a split-sample validation method was applied, only 4,943 of subjects were included in the development set.

#### Scale Validation and Updating

The derivation study reported results of both apparent and internal validation, with AUROC values ranging from 0.82 to 0.89 for all three outcomes. The scale's performance was not assessed in an independent dataset. External validation was however conducted in two subsequent studies (48, 73). The scale was applied for prediction of good functional outcome, poor outcome, and mortality. In all analyses, PLAN was found to have AUROC values above 0.80.

#### Critical Appraisal and Clinical Application

Our assessment of PLAN revealed issues predominantly related to three aspects of scale development: predictors, the model derivation procedure, and assessment of performance. In relation to predictors, all originally continuous variables were categorized. There was also a lack of reporting on how pre-stroke dementia and dependency were operationalized, as well as on blinding to outcome for assessment of input variables. In terms of creating the model, variables for multivariable analysis were chosen based on estimated associations in univariable analysis, while the method for selecting final predictors in multivariable analysis seemed unclear. Finally, the scale was only internally validated, using a split-sample method. Calibration was assessed alongside discrimination, however this was limited to performing the Hosmer-Lemeshow test and correlations between observed and expected outcomes. An additional concern is the lack of statement on the method of handling missing data.

Given the increasing use of IV-tPA as a treatment option in ischemic stroke, it is noteworthy that patients receiving this intervention were excluded from the PLAN derivation study. This does not necessarily entail limited applicability of the scale, particularly as the external validation studies, reporting good performance for PLAN, both included IV-tPA-treated patients. However, as the scale was only applied in two independent dataset, it seems that more evidence is necessary before reaching conclusions on PLAN's generalizability. If an acceptable level of performance is consistently indicated, another issue worth investigating will be whether the relative complexity in scoring impedes implementation of the scale in clinical practice.

# Symptomatic Hemorrhage, Baseline NIHSS, Age, Reperfusion, and Location of Clot (SNARL)

#### Scale Content and Development

The SNARL scale uses the three clinical and two imaging variables in its acronym to predict a good outcome (mRS<3) at 3 months following ischemic stroke treated with endovascular therapy (96). Scores can range from zero to eleven, with higher scores associated with a greater probability of a good outcome. The scale was derived through a logistic regression procedure, using data of 511 patients from a multicenter registry.

#### Scale Validation and Updating

Based on results of apparent validation, reported AUROC was 0.79 (95%CI: 0.75–0.83). The study also assessed the scale's performance in an independent cohort, comprising 223 patients from the North American Solitaire Acute Stroke registry. For this dataset, AUROC was 0.74 (95%CI: 0.68–0.81). In addition, the authors reported that compared to the THRIVE scale (described below), SNARL presented a 35% improvement in terms of accurately classifying patients' probability of a good outcome. We did not identify any further external validation studies assessing this scale.

#### Critical Appraisal and Clinical Application

Through our critical appraisal of the SNARL derivation study we identified two sources of bias, both common across the reviewed scales, use of a complete-case analysis approach and selection of predictors based on associations in a univariable statistical procedure. The applied input selection process in multivariable analysis, on the other hand, seemed unclear, as did the use of any blinding methods. Finally, although predictors were well-operationalized, interpretation of imaging findings may be subject to relatively high interobserver variability.

# Stroke Subtype, OCSP, Age, and Pre-stroke mRS (SOAR)

#### Scale Content and Development

SOAR was developed to predict early mortality (inpatient and 7-day) and length of hospital stay, based on the four clinical variables of the scale's acronym (97). Using a logistic regression model, a scoring system ranging from 0 to 8 was derived, with higher scores associated with a greater likelihood of death and extended length of stay. The derivation cohort included 12,355 acute stroke patients (91% ischemic) from a multicenter register, based in the United Kingdom.

#### Scale Validation and Updating

SOAR was internally validated using a bootstrapping resampling method, with reported AUROC values being the same for both 7-day and inpatient mortality: 0.79 (95%CI: 0.78–0.80). For predicting length of hospital stay, dichotomized at seven days, AUROC was 0.61 (95%CI: 0.60–0.62). Although external validation was not included as part of the derivation paper, SOAR has been subsequently assessed in independent datasets in five studies (98–103). Four studies assessed the scale's performance for predicting early mortality (inpatient, 7-day, discharge, and 90-day). Three found SOAR to have fair discriminatory power, and one, good. One study applied the scale for prediction of length of hospital stay. Discrimination was not formally assessed, however the authors reported that SOAR scores were significantly associated with the outcome (100).

Three external validation studies additionally aimed to improve the scales predictive performance by adding new variables. The modified SOAR (mSOAR) added stroke severity (NIHSS) (98). When compared to SOAR, mSOAR was found to have superior discriminatory power: AUROC of 0.83 (95%CI: 0.79–0.86) vs. 0.79 (95%CI: 0.75–0.84). Noteworthy, performance was assessed in the mSOAR derivation set. Nonetheless, the finding was confirmed in an independent Chinese patient sample: AUROC of 0.78 (95%CI: 0.76–0.81) and 0.79 (95%CI: 0.77–0.80) for discharge and 90-day mortality, respectively, compared to 0.72 (95%CI: 0.70–0.75) and 0.70 (95%CI: 0.69–0.72), respectively, with higher estimates reported for mSOAR. The remaining two updates of SOAR included adding admission blood glucose levels (SOAR-G) and admission sodium (SOAR-Na) (101, 103). Both concluded that the original and revised scales performed well, however without evidence of the latter offering a significant improvement in discriminatory power.

#### Critical Appraisal and Clinical Application

Reviewing the SOAR derivation study, we noted that the authors intended to select predictors for multivariable analysis based on univariable associations. However, as all candidate predictors were found to be significantly associated with the outcome, using this approach would not have influenced the results. In this case, what seems to be a greater issue, is that sex was not included in the final model, despite the significance of its association in both univariable and multivariable analyses. Risk of bias was increased by excluding all patient with missing data from the study, as well as by not accounting for effects of administered treatments.

For implementation in clinical practice, the simplicity of SOAR appears a major advantage, including easily accessible information on only four variables. Adding NIHSS is the only attempted modification that has significantly improved scale performance. In many centers, where the measure is not routinely used, this will introduce an additional challenge, yet it is worth considering that stroke severity has been consistently found to be associated with post-stroke outcomes. Calculation of mSOAR can be aided by use of an online tool (104).

# Stroke Prognosis Instrument (SPI)

#### Scale Content and Development

SPI was developed to predict risk of stroke or death within 2 years of TIA or minor stroke (105). A score ranging from 0 to 11 is calculated based on five variables: age, history of diabetes and coronary heart disease, acute hypertension, and presentation (TIA or minor stroke). This score assigns patients to one of three risk groups: low (0–2 points), medium (3–6 points), and high (7–11 points). The scale was developed based on survival analysis, specifically using a Cox proportional hazards model. The derivation cohort included 142 patients, who had undergone carotid ultrasonography in a United States tertiary care hospital. Based on data from this sample, an initial SPI score was developed, including only three variables: age, diabetes, and hypertension.

#### Scale Validation and Updating

In the derivation study, the SPI score was assessed based on its ability to accurately stratify patients according to risk of stroke or death, using data from the development sample, as well as in an independent Canadian cohort, including 330 patients. In the derivation set, the results showed that 3% of patients estimated as being at low risk had a subsequent stroke or died within 2 years of the initial neurovascular event, while the incidence for patients assigned to the medium risk group was 27%, and for those in the high-risk group 48%. For the validation cohort, the incidence of stroke and death were 10, 21, and 59%, for the 3 risk groups respectively. To ameliorate decreased performance estimates in the external set, two more variables were added to the scale, differentiation between a TIA and a minor stroke, and a history of coronary heart disease.

The authors of SPI subsequently externally validated the final scale in four independent cohorts, and used one of these cohorts to develop a modified version of the scale (106). SPI-II was derived based on data from 525 female patients, who participated in the Women's Estrogen for Stroke Trial (107). In addition to the original variables, SPI-II incorporates history of congestive heart failure and prior stroke, with total scores ranging from 0 to 15. Data from three cohorts, with a total of 9,220 patients, were used in a pooled analysis to estimate the AUROC values for both scales, concluding that SPI-II (0.63; 95%CI: 0.62–0.65) had superior discriminatory power to SPI-I (0.59; 95%CI: 0.57–0.60).

SPI-II has been subsequently externally validated in two studies (108, 109). The first found that for prediction of both stroke and death at 1 year, SPI-II had poor discriminatory power (0.62; 95%CI: 0.61–0.64), which further decreased when limiting the outcome measure to recurrent stroke (0.55; 95%CI: 0.51– 0.59). In the second study, groups identified as medium and high risk were combined, and the scale applied to predict 3-month recurrence of ischemic events. Here, the scale was found to have an AUROC of 0.55 (95%CI: 0.41–0.69).

#### Critical Appraisal and Clinical Application

The SPI derivation study had a high risk of bias. Exclusion criteria for study participation included previous stroke and any missing data on variables of interest. As a result, close to 60% of the baseline sample were not included in the analyses, leaving an insufficient number of participants relative to the number of predictors that were investigated. All of these predictors were categorized. Distinguishing between TIAs, minor strokes, and stroke has potential for interobserver variability. All candidate predictors were included in analysis, however forward selection method was used.

In relation to assessing scale performance, we noted that neither discrimination nor calibration were assessed in the SPI derivation study. The chosen validation cohort also differed from the derivation set in that some predictors were measured in alternative ways, and patients with previous strokes were included. The latter introduced an additional problem, as a history of cerebrovascular events was found to be significantly associated with the outcome. However, as this could not be investigated in the derivation set, the variable was not incorporated as a predictor. The he final SPI score seemed to be derived on the basis of a partially erroneous process of rounding up variable coefficient values.

SPI-II is also at high risk of bias, using data from a femaleonly patient sample. Although the revised scale was found to have significantly increased discriminatory power compared to the original, it was nonetheless poor, as confirmed in subsequent validation studies. The scale's predicted outcome is also problematic, creating a highly heterogenous risk group. On one hand, with a highly diverse range of possible scenarios, identifying a set of predictors both necessary and sufficient for accurate outcome prognosis seems extremely difficult. On the other hand, for clinicians, and particularly for patients, identifying that one belongs to a high-risk group seems of limited value, when this can indicate increased likelihood of anything from a minor stroke with no residual disability to death.

# Stroke Thrombolytic Predictive Instrument (S-TPI)

#### Scale Content and Development

S-TPI was developed to assist clinicians in predicting the outcome of ischemic stroke patients following intravenous IVtPA (110). Two logistic regression models were created: one for prediction of good outcome (mRS<2) and one for prediction of catastrophic outcome (mRS>4), at 3 months. In addition to IV-tPA treatment, the former model included the following variables: age, initial systolic blood pressure, diabetes, sex, baseline NIHSS score, prior stroke, and symptom onset to treatment time; as well as interaction terms: treatment with blood pressure, sex, prior stroke, and onset to treatment time, and age with NIHSS. For prediction of catastrophic outcome, the model consisted of considerably fewer inputs: age, NIHSS, serum glucose and ASPECTS score, the latter treated as an optional variable, with inclusion subject to availability. The models were derived using a combined dataset from five randomized clinical trials of IV-tPA, involving 1983, 1967, and 1883 patients (depending on the model), out of an initial cohort of 2184.

#### Scale Validation and Updating

The models were internally validated using a bootstrapping method, creating development and independent test datasets. In the latter, AUROC values were 0.77 [interquartile range (IQR): 0.76–0.78] and 0.76 (IQR: 0.75–0.78), for prediction of good outcome and catastrophic outcome without ASPECTS, respectively. Calibration was graphically assessed through plotting mean predicted vs. observed rates of patient outcomes across quintiles, and was concluded to be excellent.

S-TPI was subsequently externally validated in three studies (111–113). Two studies assessed discriminatory power based on AUROC, finding the scale to have fair to good performance for both good and catastrophic outcomes. Calibration curves were investigated in all three studies. In each case, S-TPI was found to overestimate the likelihood of a good outcome, particularly at higher levels of observed probabilities. In relation to a catastrophic outcome, findings were mixed, two studies reported the scale to underestimate the likelihood of this outcome, while the third concluded the opposite. One of the studies undertook recalibration of the scale and further added two variables for prediction of good outcome, signs of infarction on brain scan and serum glucose level. The authors reported this improved the scales discriminatory power (AUROC of 0.77 vs. 0.75) (112).

In contrast, the group that developed S-TPI sought to simplify the scale by reducing the number of predictors, with an aim to makes its implementation in routine clinical practice more feasible (114). The process involved removing interaction terms with limited external supporting evidence, removing the ASPECT score, and exploring the use of simpler stroke severity measures. A total of nine models were generated through logistic regression, for prediction of three outcome levels: mRS<2, mRS<3, and mRS>4. Results from apparent validation showed that AUROC values for all models ranged from 0.75 to 0.80. External validation was performed for models predicting mRS<2 and mRS<3, with findings indicating comparable discriminatory power as in the derivation set. The authors concluded that reducing model components did not lead to a substantial deterioration in performance. We have not identified any further publications externally validating the simplified S-TPI models.

#### Critical Appraisal and Clinical Application

Risk of bias in the derivation paper was increased by use of data from randomized control trials. Inclusion and exclusion criteria for such trials typically lead to recruiting a highly selective group of participants, thus decreasing the generalizability of scales developed based on their data. In addition, it seemed that trial investigators were not blinded to predictors (with the exception of use of IV-tPA vs. placebo) when assessing the outcome. Finally, although patients with missing data were not excluded from the derivation study outright, lack of data imputation would have led to participants being excluded from particular analyses, when they had no data for one or more of the variables used.

In view of scale implementation, it is important to note that individual patient outcome predictions were to be estimated automatically using a computer system, with an open-access version of the instrument also published online. The latter is however no longer available. With no presentation of an easily calculable risk score, estimating probabilities of patient outcomes would be a challenging task for clinicians, particularly taking into account the complexity of the S-TPI models. Despite the effort to simplify the scale, its use would nonetheless require applying the regression model formulae itself. There also seems to be no clear indication of which version of the multiple S-TPI models is the best candidate for implementation. Overall, it appears more evidence of predictive performance is needed before the simplified models can be considered for clinical use, as well as an easily-applicable scoring system.

# Totaled Health Risks in Vascular Events Stroke (THRIVE)

#### Scale Content and Development

The THRIVE scale was originally developed with an aim to support identification of patients who may benefit from endovascular stroke treatments, in terms of 3-month functional outcome and risk of death (115). The scale includes five clinical variables: age, stroke severity, and history of hypertension, diabetes mellitus and atrial fibrillation. On their basis, an integer score is calculated ranging from 0 to 9, with higher scores associated with a greater probability of a poor outcome. The scale was developed using logistic and ordinal regression models. The derivation cohort included participants of the MERCI and Multi MERCI trials of mechanical thrombectomy, with a total of 305 ischemic stroke patients (116, 117).

#### Scale Validation and Updating

The derivation paper reported results of apparent validation for prediction of good outcome (mRS<3), finding that the final prognostic model had an AUROC of 0.71. The THRIVE score, developed based on estimated odds ratios for each predictor, was assessed in terms of its association with the percentage of patients with a particular outcome. A good outcome was observed in 64.7% of patients with a score of 0–2 and in 10.6% of cases with a score of 6–9. Reported mortality rates were 5.9 and 56.4% for patients with low and high THRIVE scores, respectively.

There have been 16 subsequent studies externally validating THRIVE. These involved patient groups receiving intra-arterial therapy, intravenous thrombolysis, and no acute treatments, and focused on a number of different outcomes – good functional outcome, poor outcome, risk of hemorrhagic transformations, infarct size, and even pulmonary infection (44, 118–132). The majority of studies aimed to predict multiple outcomes, and 15 assessed the scales predictive performance in terms of AUROC values, typically alongside other estimates. Seven studies found the discriminatory power to be either poor or fair, depending on the outcome, four reported it to be poor for all used outcomes, three to be fair, and only one study found the performance to be good, specifically for prediction of mortality rates.

With an aim to improve the scale's predictive performance, a revised version was developed, the THRIVE-c Calculation (133). The modified tool includes the same variables as the original scale, with age and NIHSS score entered as continuous rather than categorized variables. The derivation study reported results of apparent, internal (split-sample) and external validation, with AUROC values ranging from 0.77 to 0.80 for prediction of poor outcome. In the overall study cohort, THRIVE-c was found to have significantly superior predictive performance compared to the original THRIVE score (0.79, 95%CI: 0.78–0.79 vs. 0.75, 95%CI: 0.74–0.76). THRIVE-c has been subsequently externally

validated in a Chinese population of patients receiving IV-tPA (134). The scale was used to predict symptomatic hemorrhage, poor functional outcome and mortality, with reported AUROC values of 0.70, 0.75, and 0.81, respectively.

#### Critical Appraisal and Clinical Application

Our critical appraisal of the THRIVE derivation study indicated issues with each of the assessed aspects, either due to methodological quality or incomplete reporting. The derivation set consisted of participants recruited to a clinical trial, thus leading to participation of a selective group of subjects. Moreover, the final sample size and method of handling missing data seemed unclear. In relation to input variables, the initial set of candidate predictors appeared limited, omitting a number of factors found to be associated with functional outcome in previous research. Inclusion of specific chronic diseases in multivariable analysis was based on significance of associations in univariable analysis. Three factors, age, stroke severity and success of vessel recanalization, were included in multivariable analyses outright, and all were found to be independently associated with the outcome. It is however unclear why vessel recanalization was not incorporated into the final THRIVE scale. There was also no report of how cut-offs were determined for the derived THRIVE score.

Assessment of scale performance in the derivation study was limited to apparent validation. Moreover, discriminatory power was tested only for the model predicting good outcome; it was not assessed for the model predicting mortality or for the derived THRIVE score. Although the scale has undergone extensive external validation since its development, findings from these studies do not seem to support a favorable judgement on the scale's prognostic performance. THRIVE-c appears to be a superior alternative, yet up-to-date we have found only one independent validation study assessing the scale's predictive ability. In view of use in routine clinical practice, inclusion of relatively few variables, based on information typically available in an acute setting, is a relevant advantage of THRIVE. Score calculation can additionally be aided be use of an online tool (135). However, existing evidence on predictive ability does not seem to merit implementation.

# DISCUSSION

There are many prognostic tools available for use in acute stroke settings. We have reviewed a selection of these and common themes emerge. Our primary interests were methodological quality of derivation, subsequent external validation and scale usability in routine clinical practice. Across 10 primary derivation studies of better-known scales, we identified potential sources of bias in each. However, it is the results of external validation studies that allow us to conclude on the scales prognostic value and applicability. We found that all scales, but one, were externally validated.

While there was a range of prognostic accuracies reported, most scales had properties that would be considered "acceptable." This is perhaps not surprising as the scales tended to measure the same concepts of demographics, comorbidity, initial stroke severity and pre-stroke functional status. Where scale developers have tried to add additional elements to these core predictors, the gain in predictive power has been limited. However, most scales have been developed with a biomedical focus and it is plausible that other less traditional factors could improve utility of the scales, for example incorporating measures of frailty, resilience, provision of rehabilitation services and social support, or the clinician's clinical gestalt.

Based on our literature search, we identified the highest number of external validation studies for THRIVE. Yet results indicated a level of predictive performance insufficient to merit the scale's use in a clinical setting. Our critical appraisal may partly explain this, identifying concerns relating to all aspects of THRIVE's derivation process. Four other scales, ASTRAL, DRAGON, iScore, and SOAR, have also been validated in multiple independent datasets. For all, findings suggested a level of predictive ability that would merit implementing the scales in clinical practice. Moreover, one study reported ASTRAL and DRAGON to predict patient outcomes more accurately than clinicians (46). Although evidence regarding the performance of other scales included in this review seemed insufficient to reach firm conclusions, a number of these tools were derived with relatively low risk of bias, and future research is likely to confirm their prognostic value. These include PLAN, SNARL, S-TPI, as well as updated scale versions: MRI-DRAGON, iScore-r and mSOAR.

For a number of reasons, it is challenging to directly compare the reviewed prognostic tools, and we have deliberately chosen to avoid naming a single preferred tool. Firstly, studies assessing more than one scale as part of an external validation process are relatively uncommon. When conducted, findings are often difficult to interpret, small differences in predictive ability between scales may arise from the superiority of one over another, but they could also be attributed to an incidentally greater similarity between the validation set and the derivation set of the scale found to perform best. Secondly, although satisfactory predictive ability is essential for a scale to be clinically useful, it is not sufficient. A number of other factors need to be considered, including feasibility in routine clinical practice, the relevance of the predicted outcome to the specific context, and whether applying the tool improves clinical decision-making, patient outcomes or cost-effectiveness of services (136). To help answer these questions, it is necessary to conduct impact studies, a stage in prognostic research that to our knowledge none of the described scales have yet reached.

Our focused literature review has strengths and limitations. We recognize that there have been many high quality systematic and narrative reviews of stroke prognosis scales (10, 137). We hope that our review offers a novel focus. We have appraised relevant stroke scales against each other; very few derivation papers have done this, despite its importance when choosing which scale to use. Additionally, we have followed the PRISMA systematic review guidelines (9) when designing and completing our study and based our appraisal on the CHARMS checklist (16). Our intention was not to offer a comprehensive review, rather we choose exemplar scales that featured in high impact journals and so by implication would be amongst the best known in the clinical community. In our assessment of feasibility we identified clinical and radiological features that may be challenging to assess in the acute setting (63, 138). Our focus was routine stroke practice (139) and our comments on feasibility may not apply to specialist stroke centers. It takes time for scales to become established and our review did not include recently published scales, for example those designed to inform thrombectomy decisions (140). However, the literature describing these scales is increasing rapidly and soon there may be sufficient validation studies.

We used data from our focused literature review to compare long-term stroke prognosis scales. We found many scales with similar content and properties. Although development of the scales did not always follow methodological best practice, most of these scales have been subsequently validated. Rather than developing new scales, prognostic research in stroke should now focus on implementation and comparative analyses.

# AUTHOR CONTRIBUTIONS

SS and TQ contributed to the conception and design of the study. TQ designed the literature search strategy, SS performed the

#### REFERENCES


search and screened the results. SS and BD extracted study data, all authors were involved in critical appraisal of included studies. BD and SS drafted the final manuscript. All authors critically revised the manuscript, approved its final version, and agreed to be accountable for its content.

# FUNDING

SS was supported by the Michael Harrison Summer Studentship, University of Glasgow; BD was supported by a Stroke Association Priority Program Award, grant reference: PPA 2015/01\_CSO; TQ was supported by a Chief Scientist Office and Stroke Association Senior Clinical Lectureship, grant reference: TSA LECT 2015/05.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur. 2019.00274/full#supplementary-material


comparison of computed tomography and magnetic resonance diffusionweighted imaging. J Neurol Neurosur Psychiatry. (2005) 76:1528–33. doi: 10.1136/jnnp.2004.059261


Age, and Prestroke Modified Rankin) to Predict Early Mortality in Acute Stroke. Stroke. (2016) 47:74–9. doi: 10.1161/STROKEAHA.115. 009898


Final results of the multi MERCI trial. Stroke. (2008) 39:1205–12. doi: 10.1161/STROKEAHA.107.497115


probability: the THRIVE-c Calculation. Int J Stroke. (2015) 10:815–21. doi: 10.1111/ijs.12529


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Drozdowska, Singh and Quinn. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.