Stroke is a leading cause of acquired adult disability worldwide (1). Recent substantial advances in acute stroke treatments (2, 3) and novel approaches to stroke rehabilitation (4, 5) have resulted in significant improvements in poststroke outcomes. To systematically evaluate the real-world benefit of such interventions, it is essential to reliably collect outcomes for patients after acute stroke discharge.

Current approaches to outcomes data collection face many logistical barriers. Follow-up care, during which outcomes are traditionally collected, requires patients to return to specialized stroke centers and can be time-intensive, cost-prohibitive, and burdensome, relying on interaction with trained healthcare providers (6, 7). Phone calls to stroke patients to assess outcomes are also time-consuming and require dedicated and trained staff.

In recent years, there has been rapid adoption of digital and telehealth approaches in clinical care (8–10). Since mobile phones are one of the most popular forms of digital interaction (11) and are ubiquitous even among diverse demographic groups (11, 12), there is the potential to utilize short message service (SMS) texting for gathering assessments after stroke. SMS-texting programs have been used in a range of health conditions (13–15) for varying utilities, including intervention (16, 17), adherence (13), and data collection (18). Although app-based collection of outcomes after stroke has been explored (19), the feasibility of using SMS texting has not yet been examined in stroke. This study aimed to examine the feasibility, reliability, cost, and acceptability of an SMS-texting approach to gather health outcomes in the first 90 days after acute stroke.

Our health system has articulated a goal of collecting functional outcomes after acute stroke discharge on all patients but has lacked the resources to accomplish this. As part of a clinical quality improvement initiative to assess barriers to success, we sought to increase the likelihood of data collection by leveraging an SMS-text messaging-based program on all discharged acute stroke patients for a several-month period. Using the services offered by a digital health technology company [Philips Patient Navigation Manager (formerly Medumo), Boston, MA, United States], we developed and launched an SMS-text follow-up program for patients discharged from the Massachusetts General Hospital (MGH) with a stroke ICD-10 code (I63, I60, I61, and G45) between June 8, 2020, and February 1, 2021. Patients were eligible to participate in the program if they had a valid mobile phone contact number in their medical chart. Patients who had not previously consented to receive SMS texts from their clinical care team at MGH received one consent SMS-text message at the time of acute hospital discharge, which remained active (i.e., giving the option to consent) for the duration of the program. If they did not consent, they did not receive any further messages. Patients had the option to decline participation in the program by responding “STOP” or simply not responding to the consent message.

Patients who consented to receiving SMS texts were enrolled in the program at the time of discharge regardless of their discharge destination (home or facility) and were provided instructions for unsubscribing (Supplementary Table S1). To familiarize patients with the SMS-texting method of communication and optimize patient engagement, patients also received weekly brain health educational tips developed by a multidisciplinary panel of clinicians, including neurologists, dietitians, and therapists (Supplementary Table S1).

Enrolled patients received an SMS text at 30, 60, and 90 days after hospital discharge, prompting them to complete the simplified modified Rankin scale (mRS) (20), a single-item, seven-level, ordinal measure of global disability, and then the Patient-Reported Outcomes Measurement Information System (PROMIS) Global-10, a 10-item measure of physical health, mental health, social health, pain, fatigue, and overall perceived quality of life. Individual items from each assessment were sent via SMS-text messages one at a time (i.e., each question of each assessment was one text message). Participants responded by directly texting the number corresponding to the answer of their selection. Participants had 1 week from receiving the prompt to complete the assessment at each time point. Responses were automatically saved in a secure database. Participants who completed all questions associated with the mRS at any given time point were considered responders, while those that did not were considered nonresponders. Participants who completed the mRS but did not complete all questions associated with the PROMIS Global-10 were still considered responders but were not scored on this assessment. The mRS can be dichotomized into good (score 0–2) and poor (score 3–6) outcomes (21). Global Physical Health (GPH) and Global Mental Health (GMH) z-scores (mean: 50; standard deviation (SD): 10) were derived from two 4-item summary scores extracted from the PROMIS Global-10 questions (22). At the conclusion of the program, enrolled patients received a satisfaction survey via an SMS text with six questions that had multiple-choice response options. Participants were counted as responders to the satisfaction survey if they responded to at least one question.

To evaluate the reliability of outcome measure scores obtained via SMS texting, mRS and PROMIS Global-10 scores closest in time to SMS-text responses were extracted from documented traditional follow-up encounters (clinic visit or follow-up phone call), when available. The clinically documented score was compared to the score from the closest SMS-text response in time.

To compare the yield and cost of the SMS texting approach to gather outcomes after acute stroke discharge with traditional methods of ascertaining outcomes, we added clinical staff and utilized a trained coordinator to call all consecutive patients discharged with stroke during a 3-month period to obtain their outcomes approximately 90 days after hospitalization discharge. The mRS and PROMIS Global-10 scores were also assessed via phone calls in the same order as SMS texts. Three attempts were made to reach each patient. Call attempts and time lengths were documented. The results of this intervention were then used to compare the cost between the two strategies (SMS texts vs. phone calls) for gathering poststroke outcomes.

Of the 530 patients discharged from MGH with a stroke ICD-10 code between June 8, 2020, and February 1, 2021, 350 patients (66.04%) were enrolled in the program. Patients enrolled on average 6.8 ± 8.2 (mean ± SD) days after acute stroke hospital admission. Patients were not enrolled if they did not have valid contact information in the medical chart (n = 151) or declined to receive messages (n = 29) (Figure 1). Enrolled patients were more likely to have English listed as their preferred language (χ² = 5.44, p = 0.02), have a lower NIH stroke scale upon admission (t = 5.0, p ≤ 0.001), and have been discharged directly home from the hospital (χ² = 4.20, p = 0.04) compared to those patients who did not enroll (Table 1).

Of those who enrolled, 40.5% (n = 142) responded to at least one SMS text to complete an assessment. The response rate at 30-day postdischarge was 28.6%, and the response rates at the 60- and 90-day time points were 24.3%. Of the responders, 30% (n = 42) responded at any two time points and 30% (n = 43) of responders responded at all three time points (Figure 2A). SMS-text response compliance is presented in Figure 2B. Responders to message prompts were more likely to have English as their preferred language (χ² = 9.33, p = 0.009), have shorter acute hospital length of stay (t = 2.5, p = 0.01), have a lower NIH stroke scale upon admission (t = 3.98, p < 0.001), and have been discharged directly home (χ² = 24.94, p < 0.001) (Table 1).

The median (IQR) modified Rankin scale score collected by SMS testing was 1 (0–3) at 30-, 60-, and 90-day postdischarge. The median (IQR) GPH scores were 44.9 (42.3–50.8), 47.7 (42.3–54.1), and 47.7 (41.1–54.1) and the median GMH scores were 45.8 (38.8–50.8), 43.5 (38.8–53.3), and 45.8 (36.9–50.8) at 30-, 60-, and 90-day postdischarge, respectively (Supplementary Table S2). The distributions of these outcomes gathered via SMS texting at 90 days are shown in Figure 3.

The mRS [median: 1 IQR: 1–3) from clinical follow-up encounters, within 13.0 ± 14.9 days of SMS-text responses across collection time points (Supplementary Figure S1), was available for 113 of the 142 patients who responded (Figure 4). Weighted Cohen’s kappa between mRS scores obtained from SMS texting compared to follow-up encounters revealed almost perfect agreement (κ = 0.8, p < 0.001) for dichotomized (good vs. poor) and ordinal (with quadratic weights, Supplementary Table S3) levels (κ = 0.8, p < 0.001) of the mRS. The PROMIS Global-10 score was not routinely collected and so was only available for 19 patients from clinical encounters. There were significant associations between traditional methods and SMS texting of ascertaining PROMIS subscores (ρ = 0.4, p < 0.01, GPH, and ρ = 0.4, p < 0.01, GMH) (Figure 4).

To compare the yield of SMS texts with that of phone calls, we attempted to complete a 90-day phone call for all patients discharged within a 3-month time period. Of the 169 stroke patients discharged, we reached 104 (61.4%) by phone. Of those reached by phone, 59 (56.7%) were contacted on the first call attempt, 28 (26.9%) were contacted on the second attempt, and 17 (10.1%) were contacted on the third attempt. For every successful phone call, there were 2.5 unanswered calls. As compared to those who did not answer, patients who answered the phone calls were more likely to have English listed as their preferred language (t = 3.9, p ≤ 0.001). Phone calls took 5.4 ± 2.9 minutes to complete. Unanswered calls took approximately 1.5 minutes.

We estimated the cost per successful assessment achieved via SMS texts vs. phone calls. The total annual cost of the SMS-text program was $12,500 to configure and run, with a cost per assessment defined as $12,500a (where a = # of assessments). For phone calls, we estimated a fixed cost of 20% of a coordinator’s time (for training and documentation) and added the average cost per phone call (both successful and unsuccessful) to each patient (n). In Boston, Massachusetts, coordinators make an average (avg.) of $60,000 and thus $0.48 per minute. Answered phone calls cost $2.50 (5.4min×$0.48/min), while unanswered phone calls cost $0.70(1.5min×$0.48) on average. This analysis yielded the following:[fixedcoordinatorcost+(avg.costofasuccessfulcall×n)+(avg.costofanunsuccessfulcall×avg.numberofunansweredcalls×n)]nor[$12,000+(2.5×n+$0.70×2.5×n)]numberofsubjects,suggesting that SMS texts become less costly than traditional phone calls if used for outcomes assessment in more than 181 stroke patients per year.

The results of the satisfaction survey revealed that the majority of patients found the educational material received via SMS texts to be helpful (53%), with only a small portion of patients finding information burdensome (6%). Most participants felt there was just the right number of tips (79%) and that the messages were clear and easy to understand (97%). Most felt that they were able to easily pick between the choices (91%) in the SMS texts that best described their recovery. Most participants (91%) reported feeling as comfortable answering questions by SMS texts than by answering in person or via phone calls (Table 2).

This is the first study to systematically examine the feasibility, reliability, cost, and acceptability of gathering outcomes after acute stroke via SMS texts. This novel method was found to be highly reliable for collecting the mRS scores and moderately reliable for collecting PROM scores. Even without specific program marketing or patient engagement campaigns, the SMS-text program yielded a 40.5% response rate. Compared to direct patient phone calls, SMS texting yielded fewer responses but is cost-saving for centers with annual stroke discharges exceeding 181 patients based on our costing equation. The experience of receiving text messages with assessments and brain health tips was overall very well received by stroke patients.

We found almost perfect agreement between mRS scores at both the dichotomized (good vs. poor outcome) and ordinal levels obtained via SMS texting compared to traditional methods. This high reliability provides the foundation for systematic evaluation of stroke survivors’ outcomes at a large scale, which could help evaluate the efficacy of stroke treatments. We found moderate agreement with traditional methods for patient-reported measures. This suggests that certain stroke outcomes (i.e., ordinal ratings of global disability) may be more suitable for text-message programs. Prior studies for smoking cessation (24) and depression (25) have also found mixed reliability (fair to substantial) for self-report assessments. Digital means for collecting stroke outcomes may be limited by stroke-related impairments (i.e., language or cognitive deficits). Further exploration is required to examine the reliability of collecting different types of outcomes via SMS texting, particularly in stroke. Furthermore, modality-specific outcome measures, such as motor or language assessments after stroke, may require other types of digital or sensor-based approaches (26–28).

SMS texting had an overall lower yield (40.5%) in this study compared to targeted phone call follow-up method (61%, phone calls). The first attempt at reaching participants via SMS texts also yielded lower responses (28.6%) than the first attempt at reaching participants via phone calls (56.7%). This differs from a prior study that received more SMS-text data than paper diary data after birth control insertion, although notably this patient population was substantially younger as compared to ours (29). Stroke survivors who tend to be older and often suffer stroke-related deficits may be limited in their ability to use cell phones or read and write SMS texts. Survivors might also have limited access to their mobile devices when discharged to a facility, as a majority (70.4%) of responders were discharged home. Engaging caregivers to help with outcomes data collection via digital technology may be helpful in these cases. Low SMS-text response rates could also be attributed to participants feeling more comfortable declining to participate via SMS texts rather than directly to a care team member via phone calls. Another reason could be that the number of required questions needed to complete the assessments by SMS texts was too burdensome, and future research should determine the optimal number of questions to response ratio. In addition, future programs with a dedicated patient and caregiver outreach and SMS-text reminders (30) will likely yield higher response rates.

While the majority of stroke patients consented to SMS-text communication, there were differences between those who consented and those who did not. Individuals who consented to receive SMS texts were more likely to have English listed as their preferred language than another language, have a lower NIH stroke scale at admission, and be discharged home rather than to a postacute care facility. Similar differences were found between those who responded to SMS-text assessments vs. those who did not. Furthermore, outcomes gathered via SMS texts revealed that responders had predominantly mild disability (median mRS of 1, with an interquartile range of 0-3). Outcomes gathered via SMS texts may not be representative of all stroke survivors. A nutrition education program for low-income parents that used SMS texts for program evaluation also found limitations in those who could be reached (31). In stroke, different approaches may be required to reach non-English speakers (32) and those with more severe disabilities (33). If the use of SMS-text programs can diminish the burden of manual collection, outcome collection systems can work in parallel to focus human resources on the more healthcare marginalized and disabled.

Due to the fixed cost of the SMS-text program for an unlimited number of participants, higher response rates would render a lower cost per participant. This contrasts with traditional methods such that outcomes collected during clinic visits or by phone calls require more resources for additional participants and thus cost increases per participant. Therefore, identifying the cost-to-response ratio that would favor SMS texting over traditional methods is essential for developing cost-saving programs. In our study, we show that gathering data via SMS texting would be cost-saving at scale for larger populations. Alternatively, it could also be cost-effective for smaller populations requiring outcome assessments at a greater frequency. For example, a previous study showed that SMS texting was more cost-effective for a weekly, two-question survey than the same paper-based survey (34) due to the frequency demand of the assessment. Future programs should consider the number of participants, data type, and sampling frequency and length in determining the cost-to-benefit ratio when using SMS texting.

Patients who participated in the program found it acceptable without additional burden. The majority found that the messages were clear, easy to understand, and easy to answer. The delivery of brain health tips was well received. These results are consistent with results from the acceptability of SMS texting in diverse clinical populations including individuals with depression (25), high blood pressure (35), and psychosis (36). Given the pervasive use of cell phones in modern society, cell phone-based outcome programs have significant promise for a wide range of patient populations including those with stroke. Although it is feasible to collect the mRS score via a mobile app (19), SMS texting leverages existing software without requiring a smartphone, additional download, or application knowledge.

Overall, our findings suggest that collecting functional outcomes via SMS texting during the first 3 months after stroke is feasible, acceptable, and reliable but that reach and cost-effectiveness should be further considered for broad clinical translation. Future programs should consider developing content in multiple languages and incorporating dedicated patient outreach materials. For example, educational material on how to view and respond to SMS-text communications delivered during the acute stroke inpatient stay could be helpful. Such content would help reach vulnerable populations. Increasing the yield of SMS-text outcome programs would decrease the cost per participant and make broad adoption across healthcare systems more feasible.

This study has several important limitations. The study was conducted at a single, urban, academic medical center in the northeastern United States with a predominantly White patient population. Findings may not be translatable to other hospitals in different locations with different patient populations. A limitation to communication via SMS texting is the chance that someone other than the intended recipient is receiving or interacting with the SMS texts. Moreover, the program could not gather information on the number of subjects who passed away during the study. In longitudinal data collection via SMS texting, subjects may see their responses from prior time points (in the SMS-text chain), which may lead to recall bias. Our sample of PROM data via both SMS texting and traditional methods was small (n = 19), and thus future studies with larger samples are needed to draw definitive conclusions. At last, the potential effects of our stroke and brain health educational program delivered via SMS texting were not systematically considered in our cost analysis.

Our study suggests that it is feasible, reliable, and acceptable to provide general stroke education and gather functional outcome measures via SMS-text messaging after acute stroke discharge. Replication of our results in an independent cohort and further validation of specific outcome types and assessment frequency and length are warranted. Our findings lay the foundation for using SMS texting to gather outcomes after stroke to better evaluate the real-world efficacy of stroke therapies.