Can We Improve the Monitoring of People With Multiple Sclerosis Using Simple Tools, Data Sharing, and Patient Engagement?

Technological innovation is transforming traditional clinical practice, enabling people with multiple sclerosis (pwMS) to contribute health care outcome data remotely between clinic visits. In both relapsing and progressive forms of multiple sclerosis (MS), patients may experience variable disability accrual and symptoms throughout their disease course. The potential impact on the quality of life (QoL) in pwMS and their families and carers is profound. The introduction of treatment targets, such as NEDA (no evidence of disease activity) and NEPAD (no evidence of progression or active disease), that guide clinical decision-making, highlight the importance of utilizing sensitive instruments to measure and track disease activity and progression. However, the gold standard neurological disability tool—expanded disability severity scale (EDSS)—has universally recognized limitations. With strides made in our understanding of MS pathophysiology and DMT responsiveness, maintaining the status quo of measuring disability progression is no longer the recommended option. Outside the clinical trial setting, a comprehensive monitoring system has not been robustly established for pwMS. A 21st-century approach is required to integrate clinical, paraclinical, and patient-reported outcome (PRO) data from electronic health records, local databases, and patient registries. Patient and public involvement (PPI) is critical in the design and implementation of this workflow. To take full advantage of the potential of digital technology in the monitoring and care and QoL of pwMS will require iterative feedback between pwMS, health care professionals (HCPs), scientists, and digital experts.

Introduction

Multiple sclerosis (MS) is a chronic inflammatory, demyelinating, and degenerative disease of the central nervous system (CNS). MS affects more than 130,000 people in the UK and over 2.5 million worldwide (1–3). While prediction of the disease trajectory in individual people with MS (pwMS) remains challenging, accrual of chronic disability is the norm (4, 5), particularly if pwMS are left without disease-modifying treatment (DMT) (6). Dependable outcome measures are highly desirable to assess the clinical course of MS and inform patient management. Given the heterogeneity of clinical presentation, systems involved, and speed of progression, assessing outcomes in pwMS requires systematic, multidimensional tools. Comprehensive follow-up of pwMS has been demonstrated in a number of clinical trials (7–10). However, systematic monitoring of pwMS in clinical practice is often incompatible with the limited time available for patient review (11), particularly when using the expanded disability status scale (EDSS) (12), which nevertheless remains key to determine DMT eligibility (13), and despite its well-rehearsed shortcomings (14).

PwMS with advanced disease, for example those having an EDSS ≥ 6.5, and elderly pwMS are at particular risk of being less carefully followed up (15). These patients are more likely not on a licensed DMT and are commonly considered “beyond” immunotherapy, despite mounting evidence that neurologic function can potentially be preserved, even at a later stage of the disease (16, 17).

Here, we provide a perspective on using a new approach of collecting data in pwMS that combines (i) clinical assessments with potential for self-monitoring and (ii) patient-reported outcomes (PROs) using a platform shared between a large data repository, the UK MS Register at Swansea University, and BartsMS in east London, UK. We describe how such point-of-care data collection may serve both research and the individual pwMS in clinic and highlight the role of patient and public involvement (PPI) in facilitating the “buy-in” of pwMS underpinned by some preliminary data on patient engagement with the UK MS Register portal and corresponding data sharing preferences.

Quantifying Neurologic Disability

The introduction of the EDSS (12) as the key outcome measure of disability in MS DMT trials cemented its role as the neurologist's “gold standard” rating scale of disability in pwMS. However, while clinical trials usually allocate sufficient time to complete and fully document an EDSS (which takes ~20–30 min), the time constraints of clinical practice regularly lead to either an “estimated” EDSS, or systematic clinical assessments remain patchy, or are not undertaken at all (11). To overcome this shortcoming, various versions of a patient-reported EDSS (PREDSS) have been proposed. These are either paper based, administered via telephone, or, more recently, via an online application, the “webEDSS” (18). Correlation has been observed between EDSS and all versions of PREDSS; however, limitations of agreement were identified, particularly at low EDSS levels (11). However, even if these limitations could be minimized, the non-parametric character of the EDSS, its ambulatory bias, and lack of sensitivity at high values remain problematic. Moreover, decline in cognitive function is not well covered, in spite of its key importance in pwMS, especially given the implications for employment opportunities (19, 20).

As a result, the National MS Society's Clinical Outcomes Assessment Task Force started more than 25 years ago developing a new set of outcome measures. Ultimately, a set of three tests was agreed, making up what was coined the Multiple Sclerosis Functional Composite (MSFC). The MSFC consists of the Paced Auditory Serial Addition Test (PASAT), Timed 25-foot walking (T25ftWT), and the Nine Hole Peg Test (9HPT) and has been implemented in a number of clinical trials (21). However, only this year, 2020, will a DMT licensing trial for the first time use one element of the MSFC, the 9HPT, as its primary outcome measure (22).

“BartsMS” is a clinic–academic partnership based at The Royal London Hospital (Barts Health NHS Trust) and The Blizard Institute/Queen Mary University of London providing clinical care to over 3000 pwMS. Faced with the same discrepancy between high expectations and the reality of limited resources (6), BartsMS introduced a modified version of the MSFC in their clinical practice in 2016. While T25ftWT and 9HPT were retained, PASAT was replaced with the Symbol Digit Modality Test (SDMT; oral version) following the recommendation by Drake and coworkers (23), among others (24). The SDMT has equal psychometric validity to the PASAT and is associated with lesser confounding by training and more congenial for both patient and assessor (23). It takes less time to complete, requires less expertise and experience of the assessor, and, unlike the PASAT, does not require special equipment for auditory presentation of stimuli (24). In practice, we summarize the three elements (T25ftWT, 9HPT, and SDMT) simply as “3TEST.” Given a clinical and research focus of BartsMS on advanced MS, i.e., people with an EDSS of ≥ 6.5 (25), the ABILHAND questionnaire is also regularly administered to capture perceived manual ability (26). Obtaining such “real world” outcome measures in routine clinical practice and trials has also been recognized by the European Medicines Agency (EMA) as an important component of disease management (27).

The Evolving Role of Remote Self-Monitoring

The relative simplicity of the MSFC or variations thereof, such as the 3TEST, combined with advances in technology and ever-increasing online resources and capabilities have led to the expansion and uptake of self-monitoring applications (28). Self-monitoring enables tracking the disease course in pwMS unable to travel to clinic, e.g., due to their disability or them living in remote locations. Given the often-extended intervals between follow-up in clinic (commonly 6–12 months), systematic self-monitoring may improve detection of changes not captured during visits, including relapses and disability accrual, thereby enabling earlier detection of disease progression and trajectories of long-term outcomes. Moreover, self-monitoring has inherent potential to empower pwMS to manage their condition pro-actively, with likely benefits for their care and self-management (29). Alongside other measures, such as written decision aids (30), self-monitoring may help remove hierarchical barriers and level the platform for shared decision-making between health care professionals (HCPs) and pwMS. It would be expected that such change will improve treatment satisfaction and adherence (31). Against this backdrop, numerous self-assessment tools have been developed (32, 33). As part of this effort, our group developed portable versions of the 9HPTs and the T25ftWTs (34, 35), while the UKMSR produced an online version of the SDMT (MSiDMT) (36).

In addition, wearable technologies, including motion detecting devices (MTDs) and smartphone applications may facilitate minimally intrusive assessment of outcomes such as step count, walking speed, and gait (37) and support neurorehabilitation (38).

A Model of Integrated Monitoring and Patient Engagement

Results from tests that (i) are relatively straightforward to implement in clinic and (ii) can be translated into self-monitoring tools can be combined with PRO questionnaires and fed into the patient record, which, in health care settings covering large numbers of pwMS, is usually an electronic health record (EHR). EHRs facilitate the timely recording of patient data and the simultaneous navigation by multiple HCPs from different specialities (39). Coding terminology, such as Systematized Nomenclature for Medicine (SNOMED), provides a powerful resource to collate individual patient data as well as to identify, stratify, and audit patient cohorts and outcomes.

We use the generic Barts Health NHS Trust-wide EHR Cerner Millennium Clinical Record System (CRS). This system enables extraction of coded information to populate our database of pwMS (the “BartsMS Database”) in Excel (40), thereby providing both an individual record and a point-of-care data collection, including 3TEST data, fed by the various HCPs at the Trust involved in the care of the pwMS. Our dataset is further enriched by the UK MS Register (UKMSR), an MS Society (UK)-sponsored resource that collects PRO data on pwMS throughout the UK (41). The UKMSR was conceived on the understanding that PRO data are important to capture the experience of pwMS and their families, friends, and carers (42–44). PROs are also commonly used as secondary endpoints in clinical trials to determine and compare the effect of DMTs. The core validated instruments collected by the UKMSR are EuroQol 5D (EQ-5D), Multiple Sclerosis Impact Scale 29v2 (MSIS-29), Hospital Anxiety and Depression (HADS) Scale, Fatigue Severity Scale (FSS), the Multiple Sclerosis Walking Scale (MSWS-12), and Patient Determined Disease Steps (PDDS) (45–51). The webEDSS is also available as an ad hoc questionnaire (52).

Since 2017, BartsMS and the UKMSR have been developing a hub/spoke monitoring system (Figure 1). The intention of the algorithm is to (i) facilitate research through high-quality data collection, (ii) support the clinical service provision with PRO data, and (iii) enable the latter via a patient portal. PwMS who consent to join the UKMSR will have their minimum dataset (demographics, MS history, risk factors, disease course, EDSS scores, relapses, DMT, and symptomatic information) collected and securely uploaded via a REDCap electronic clinical record form (53). In addition, pwMS are prompted via email, at regular (currently 6-monthly) intervals, to fill in PRO questionnaires. This information can then be linked to their unique study ID provided at the hospital site, and thereby merged with their clinical record.

FIGURE 1

Figure 1. The hub/spoke system between BartsMS and the UK MS Register. (A) The interaction between the BartsMS service & Database, and the UK MS Register. (B) My MS Hub page on the patient portal. (C) Graph of Multiple Sclerosis Impact Scale (MSIS29 V2) results viewed in the patient portal. Accompanying description for people with MS: “Used to measure the impact that your MS is having on you physically and psychologically at any given time. This questionnaire was designed specifically for MS. It means that researchers can measure how much your MS affects your quality of life. It is increasingly being used in MS research and clinical settings. What the graph means: the MSIS is composed of three scores, a total, a psychological sub-score and a physical sub-score. In this graph we show the psychological and physical sub-scores. In both Scales, higher values indicate that your MS is causing you more trouble and lower ones indicate less impact.” (D) Graph of EQ-5D 3L results viewed in the patient portal. Accompanying description for people with MS: “This is one of the most commonly used health status measurements. It is not MS specific and so it can be used to make comparisons with various other health conditions/chronic diseases. What the graph means: the EQ5D has five questions specifically related to aspects of your general health and one scale marked out of 100 in which you indicate your overall quality of life. High scores indicate a better overall quality of life”.

Patient Engagement

We learned that patient and public involvement (PPI) is pivotal to maintain and expand data collection through the UKMSR. Valuable insights and feedback were provided through a PPI meeting held at The Royal London Hospital (Barts Health NHS Trust) on 16 February 2018. Key outcomes of this engagement day were (i) a re-designed, visually more attractive website enabling easier navigation and providing better sectioning, including a “My MS” hub page. This hub contains easily identifiable and accessible open questionnaires, including estimates of the time required for completion. This feature also provides pwMS with a snapshot of the information they have contributed and highlights any data that they should still provide; (ii) radio boxes for questionnaires, rather than drop-down menus since less mouse movement is required, making it easier to navigate for pwMS with upper limb function impairment; (iii) reduced frequency of questionnaire responses requested (bi-annually instead of quarterly); (iv) more tangible benefits for UKMSR subscribers, who were keen to receive comprehensive feedback about their collected questionnaire data—we therefore decided that the facility of viewing personal response data should be provided as an option; (v) since September 2018, participants who join the UKMSR and opt in to feedback are being offered a downloadable version of their results. By December 2019, 67% of new subscribers (total n = 2712) had had opted into this facility. This is designed so that it can be taken along to clinic appointments. Information is displayed in easily accessible graphs, allowing pwMS to track their condition over time. Explanations in lay terms are included about what the instruments and graphs mean and their relevance to pwMS (Figures 1C,D).

Further insights from our PPI exercise included an understanding that pwMS wanted the UKMSR portal to enable them (i) to have better control over their health care including treatment options, (ii) access to clinical trials, and (iii) improved self-management. PwMS were also passionate about furthering research both for short-term benefit and for future generations, including their own children.

To estimate the effect of our response to the PPI input received on the rate of questionnaires, we extracted the number of completed questionnaires at three time points; Winter 2018 (before implementation of the above changes to the portal), Spring 2019, and Winter 2019. Data were extracted from the UKMSR production databases running Microsoft SQL Server 2014.

Figure 2 illustrates a significant increase in the number of completed questionnaires between the launch of the new website in Winter 2018 and the latest cutoff in Winter 2019. This increase suggests a significant impact of PPI on the new UKMSR portal design and functionality.

FIGURE 2

Figure 2. Core Questionnaire Response rates following PPI and redesign of the UK MS Register portal.

Discussion

Optimizing the landscape of individualized, effective, and compassionate care with and for pwMS remains a work in progress. Whereas clinical trials provide data on a cohort level, the evidence produced can only provide a backdrop for decisions that need to be tailored to the individual pwMS. Clinical monitoring is essential to detect treatment success and failure, in order to make individual decisions. While various digital tools for disease monitoring in pwMS have been developed, their value in clinical practice is not yet established, and their adoption limited (54). We found validated measures that are easily applicable and straightforward to interpret a useful way to quantify change in an era where pwMS expect their care to catch up with the efficacy of the latest DMTs. The administration of 3TEST does not require any special qualification—virtually any HCP can be trained to apply it in a short timeframe. Since all three parts of the 3TEST can be done remotely, the limit for self-monitoring is now mainly a question of frequency and logistics (how often to test, how to feedback results to the health care team, and how to embed the data in the daily routine of neurologists and MS specialists between appointments). The simplicity and compatibility for remote testing of 3TEST also highlight the potential for relatively straightforward multi-center adoption and inclusion in large datasets, such as the UKMSR or MSBase (55), and there is obvious potential for remote testing in exceptional situations, such as a pandemic (56). Furthermore, 3TEST is likely going to be of use when screening for trials where measures other than the EDSS are being used for inclusion as well as outcome (22). New systems intended to both serve individual monitoring of pwMS and contribute to large datasets, such as Floodlight (33, 57), will need to be validated using well-established tests such as those combined in 3TEST (32).

Our experience trying to combine clinical and PRO data collection via the UKMSR in order to facilitate databasing for research, service audit, and individual patient care highlights the important role of PPI throughout the design and implementation process. To truly deliver patient-centered care and at the same time enable high-quality data collection, any system for pwMS needs to be developed jointly with pwMS. In our example, PPI led to a significantly increased number of completed PRO questionnaires. We are currently optimizing and streamlining mutual data exchange between BartsMS and the UKMSR to provide an integrate model of point-of- care data collection. This system may provide a model of data collection and sharing that can be adopted by other centers across the UK and beyond.

Data Availability Statement

All datasets generated for this study are included in the article/supplementary material.

Author Contributions

KS initiated the BartsMS Database and conceptualized the setup between the UK MS Register and BartsMS clinical interface, which the team helped establish. KA-P, RM, and KS drafted the manuscript. CA, AS, and KT-D contributed toward the subsequent revisions. RM and EB performed the data extraction and analysis. All authors read and approved the final manuscript.

Funding

KA-P was supported by the National Institute for Health Research North Thames Clinical Research Network. The UKMSR was funded by the Multiple Sclerosis Society of Great Britain & Northern Ireland. The initiation and early maintenance of the BartsMS Database were supported by non-promotional research grants from Novartis Pharmaceuticals UK Ltd.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor declared a past co-authorship with one of the authors KS.

Acknowledgments

We would like to thank members of the BartsMS advisory group led by Alison Thomson for their valuable contributions toward development of the UK MS Register portal and collaboration with the BartsMS clinical service.

References

1. Multiple Sclerosis: Prevalence, Incidence And Smoking Status–Data Briefing [online]. Public Health England (2020). Available online at: https://www.gov.uk/government/publications/multiple-sclerosis-prevalence-incidence-and-smoking-status/multiple-sclerosis-prevalence-incidence-and-smoking-status-data-briefing#epidemiology-of-multiple-sclerosis (accessed February 6, 2020)

2. MS Incidence and Prevalence Report [online]. MS Society (2020). Available online at: https://www.mssociety.org.uk/what-we-do/our-work/our-evidence/ms-in-the-uk (accessed February 6, 2020)

3. Wallin MT, Culpepper WJ, Campbell JD, Nelson LM, Langer-Gould A, Marrie RA, et al. The prevalence of MS in the United States: A population-based estimate using health claims data. Neurology. (2019) 92:e1029–e40. doi: 10.1212/WNL.0000000000007035

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Leray E, Yaouanq J, Le Page E, Coustans M, Laplaud D, Oger J, et al. Evidence for a two-stage disability progression in multiple sclerosis. Brain. (2010) 133:1900–13. doi: 10.1093/brain/awq076

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Scalfari A, Neuhaus A, Degenhardt A, Rice GP, Muraro PA, Daumer M, et al. The natural history of multiple sclerosis: a geographically based study 10: relapses and long-term disability. Brain. (2010) 133:1914–29. doi: 10.1093/brain/awq118

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Hobart J, Bowen A, Pepper G, Crofts H, Eberhard L, Berger T, et al. International consensus on quality standards for brain health-focused care in multiple sclerosis. Mult Scler. (2019) 25:1809–18. doi: 10.1177/1352458518809326

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Hauser SL, Bar-Or A, Comi G, Giovannoni G, Hartung H-P, Hemmer B, et al. Ocrelizumab versus interferon beta-1a in relapsing multiple sclerosis. N Engl J Med. (2017) 376:221–34. doi: 10.1056/NEJMoa1601277

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Fox EJ, Markowitz C, Applebee A, Montalban X, Wolinsky JS, Belachew S, et al. Ocrelizumab reduces progression of upper extremity impairment in patients with primary progressive multiple sclerosis: Findings from the phase III randomized ORATORIO trial. Multiple Sclerosis J. (2018) 24:1862–70. doi: 10.1177/1352458518808189

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Kapoor R, Ho P-R, Campbell N, Chang I, Deykin A, Forrestal F, et al. Effect of natalizumab on disease progression in secondary progressive multiple sclerosis (ASCEND): a phase 3, randomised, double-blind, placebo-controlled trial with an open-label extension. Lancet Neurol. (2018) 17:405–15. doi: 10.1016/S1474-4422(18)30069-3

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Kappos L, Bar-Or A, Cree BAC, Fox RJ, Giovannoni G, Gold R, et al. Siponimod versus placebo in secondary progressive multiple sclerosis (EXPAND): a double-blind, randomised, phase 3 study. Lancet. (2018) 391:1263–73. doi: 10.1016/S0140-6736(18)30475-6

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Collins CDE, Ivry B, Bowen JD, Cheng EM, Dobson R, Goodin GS, et al. A comparative analysis of patient-reported expanded disability status scale tools. Multiple Sclerosis J. (2016) 22:1349–58. doi: 10.1177/1352458515616205

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. (1983) 33:1444–52. doi: 10.1212/WNL.33.11.1444

PubMed Abstract | CrossRef Full Text | Google Scholar

13. NHS England. Treatment Algorithm for Multiple Sclerosis Disease-Modifying Therapies [online]. (2018). Available online at: https://www.england.nhs.uk/commissioning/wp-content/uploads/sites/12/2019/03/Treatment-Algorithm-for-Multiple-Sclerosis-Disease-Modifying-Therapies-08-03-2019-1.pdf (accessed January 10, 2020)

14. Dubuisson N, Baker D, Thomson A, Marta M, Gnanapavan S, Turner B, et al. Disease modification in advanced MS: Focus on upper limb function. Mult Scler. (2017) 23:1956–7. doi: 10.1177/1352458517717811

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Roberts M, Bowen A. Improving Services for People With Advanced MS. Letchworth: MS Trust. (2016) Available online at: https://multiplesclerosisacademy.org/wp-content/uploads/sites/3/2018/11/Improving-services-for-people-with-Advanced-MS.pdf

Google Scholar

16. Alvarez-Gonzalez C, Adams A, Mathews J, Turner BP, Giovannoni G, Baker D, et al. Cladribine to treat disease exacerbation after fingolimod discontinuation in progressive multiple sclerosis. Ann Clin Trans Neurol. (2017) 4:506–11. doi: 10.1002/acn3.410

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Mao Z, Álvarez-González C, Allen-Philbey K, De Trane S, Yildiz O, Campion T, et al. Treating the ineligible: Disease modification in people with multiple sclerosis beyond NHS England commissioning policies. Mult Scler Relat Disord. (2019) 27:247–53. doi: 10.1016/j.msard.2018.11.001

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Leddy S, Hadavi S, McCarren A, Giovannoni G, Dobson R. Validating a novel web-based method to capture disease progression outcomes in multiple sclerosis. J Neurol. (2013) 260:2505–10. doi: 10.1007/s00415-013-7004-1

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Rao SM, Leo GJ, Bernardin L, Unverzagt F. Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology. (1991) 41:685–91. doi: 10.1212/WNL.41.5.685

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Clemens L, Langdon D. How does cognition relate to employment in multiple sclerosis? A systematic review. Mult Scler Relat Disord. (2018) 26:183–91. doi: 10.1016/j.msard.2018.09.018

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Fischer JS, Rudick RA, Cutter GR, Reingold SC. The Multiple Sclerosis Functional Composite measure (MSFC): an integrated approach to MS clinical outcome assessment. Mult Scler. (1999) 5:244–50. doi: 10.1191/135245899678846168

CrossRef Full Text | Google Scholar

22. A study to evaluate the efficacy and safety of ocrelizumab in adults with primary progressive multiple sclerosis (O'HAND) [online]. Clin Trials. (2019). Available online at: https://clinicaltrials.gov/ct2/show/NCT04035005 (accessed January 10, 2020).

23. Drake AS, Weinstock-Guttman B, Morrow SA, Hojnacki D, Munschauer FE, Benedict RHB. Psychometrics and normative data for the Multiple Sclerosis Functional Composite: replacing the PASAT with the Symbol Digit Modalities Test. Mult Scler. (2010) 16:228–37. doi: 10.1177/1352458509354552

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Langdon DW, Amato MP, Boringa J, Brochet B, Foley F, Fredrikson S, et al. Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS). Multiple Sclerosis J. (2012) 18:891–8. doi: 10.1177/1352458511431076

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Dubuisson N, Marta M, Gnanapavan S, Turner B, Baker D, Thomson A, et al. Inclusion criteria used in trials of people with progressive multiple sclerosis. Mult Scler. (2020) 26:279–83. doi: 10.1177/1352458518803769

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Cano S, Cleanthous S, Marquis P, Hobart J, Naoshy S, Mikol D, et al. Measuring upper limb function in multiple sclerosis: Enhancing the ABILHAND's performance. Value Health. (2015) 18:A24. doi: 10.1016/j.jval.2015.03.149

CrossRef Full Text | Google Scholar

27. EMA. Patient registries [online]. Eur Med Agency. Available online at: https://www.ema.europa.eu/en/documents/report/report-patient-registries-workshop_en.pdf (accessed December 5, 2019).

28. Lavorgna L, Brigo F, Moccia M, Leocani L, Lanzillo R, Clerico M, et al. e-Health and multiple sclerosis: An update. Mult Scler. (2018) 24:1657–64. doi: 10.1177/1352458518799629

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Stepleman L, Rutter M-C, Hibbard J, Johns L, Wright D, Hughes M. Validation of the patient activation measure in a multiple sclerosis clinic sample and implications for care. Disabil Rehabil. (2010) 32:1558–67. doi: 10.3109/09638280903567885

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Manzano A, Ford HL, Potrata B, Eskyte I, Meads D, Webb E, et al. Treatment Decision Making and Relapsing Remitting Multiple Sclerosis. The CRIMSON Project Decision Aid Booklet. Leeds: University of Leeds (2019).

Google Scholar

31. Oreja-Guevara C, Potra S, Bauer B, Centonze D, Giambastiani M-P, Giovannoni G, et al. Joint healthcare professional and patient development of communication tools to improve the standard of MS care. Adv Ther. (2019) 36:3238–52. doi: 10.1007/s12325-019-01071-9

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Baldassari LE, Nakamura K, Moss BP, Macaron G, Li H, Weber M, et al. Technology-enabled comprehensive characterization of multiple sclerosis in clinical practice. Mult Scler Relat Disord. (2019) 38:101525. doi: 10.1016/j.msard.2019.101525

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Midaglia L, Mulero P, Montalban X, Graves J, Hauser SL, Julian L, et al. Adherence and satisfaction of smartphone- and smartwatch-based remote active testing and passive monitoring in people with multiple sclerosis: nonrandomized interventional feasibility study. J Med Internet Res. (2019) 21:e14863. doi: 10.2196/14863

CrossRef Full Text | Google Scholar

34. Dubuisson N, Bauer A, Buckley M, Gilbert R, Paterson A, Marta M, et al. Validation of an environmentally-friendly and affordable cardboard 9-hole peg test. Mult Scler Relat Disord. (2017) 17:172–76. doi: 10.1016/j.msard.2017.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Allen-Philbey K, Yildiz O, Raciborska D, Stennett A, Mathews J, Turner B, et al. Systematic multidimensional clinical point-of-care monitoring of people with multiple sclerosis using 3TEST. Multiple Sclerosis J. (2018) 24:569. doi: 10.1177/1352458518798591

CrossRef Full Text | Google Scholar

36. Middleton R, Pearson O, Ingram G, Watson L, Craig E, Roberts C, et al. MSiDMT: Development of a Consistent Electronic Cognitive Scoring Method for the UKMS Register. ECTRIMS Online Library (2019) Available online at: https://onlinelibrary.ectrims-congress.eu/ectrims/2019/stockholm/279173/rod.middleton.msidmt.development.of.a.consistent.electronic.cognitive.scoring.html (accessed January 23, 2020).

37. Sparaco M, Lavorgna L, Conforti R, Tedeschi G, Bonavita S. The role of wearable devices in multiple sclerosis. Multiple Sclerosis Int. (2018) 2018:1–7. doi: 10.1155/2018/7627643

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Isernia S, Pagliari C, Jonsdottir J, Castiglioni C, Gindri P, Gramigna C, et al. Efficiency and patient-reported outcome measures from clinic to home: the human empowerment aging and disability program for digital-health rehabilitation. Front Neurol. (2019) 10:1206. doi: 10.3389/fneur.2019.01206

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Claire Simon K, Hentati A, Rubin S, Franada T, Maurer D, Hillman L, et al. Successful utilization of the EMR in a multiple sclerosis clinic to support quality improvement and research initiatives at the point of care. Mult Scler J Exp Transl Clin. (2018) 4:2055217318813736. doi: 10.1177/2055217318813736

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Albor C, Richards O, Ramagopalan S, Boomla K, Schmierer K. Using routine point-of-care data for research: the east London multiple sclerosis cohort. J Neurol Neurosurg Psychiatry. (2013) 84:e2–e2. doi: 10.1136/jnnp-2013-306573.170

CrossRef Full Text | Google Scholar

41. Middleton RM, Rodgers WJ, Chataway J, Schmierer K, Rog D, Galea I, et al. Validating the portal population of the United Kingdom Multiple Sclerosis Register. Mult Scler Relat Disord. (2018) 24:3–10. doi: 10.1016/j.msard.2018.05.015

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Jones KH, Ford DV, Jones PA, John A, Middleton RM, Lockhart-Jones H, et al. The physical and psychological impact of multiple sclerosis using the MSIS-29 via the web portal of the UK MS Register. PLoS ONE. (2013) 8:e55422. doi: 10.1371/journal.pone.0055422

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Nickerson M, Marrie RA. The multiple sclerosis relapse experience: patient-reported outcomes from the North American Research Committee on Multiple Sclerosis (NARCOMS) Registry. BMC Neurol. (2013) 13:119. doi: 10.1186/1471-2377-13-119

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ. (2015) 350:g7818. doi: 10.1136/bmj.g7818

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Jones KH, Ford DV, Jones PA, John A, Middleton RM, Lockhart-Jones H, et al. How people with multiple sclerosis rate their quality of life: an EQ-5D survey via the UK MS register. PLoS ONE. (2013) 8:e65640. doi: 10.1371/journal.pone.0065640

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Riazi A. Multiple Sclerosis Impact Scale (MSIS-29): reliability and validity in hospital based samples. J Neurol Neurosurg Psychiatry. (2002) 73:701–4. doi: 10.1136/jnnp.73.6.701

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Watson TM, Ford E, Worthington E, Lincoln NB. Validation of mood measures for people with multiple sclerosis. Int J MS Care. (2014) 16:105–9. doi: 10.7224/1537-2073.2013-013

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Krupp LB, LaRocca NG, Muir-Nash J, Steinberg AD. The fatigue severity scale. Application to patients with multiple sclerosis and systemic lupus erythematosus. Arch Neurol. (1989) 46:1121–3. doi: 10.1001/archneur.1989.00520460115022

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Hobart JC, Riazi A, Lamping DL, Fitzpatrick R, Thompson AJ. Measuring the impact of MS on walking ability: the 12-Item MS Walking Scale (MSWS-12). Neurology. (2003) 60:31–6. doi: 10.1212/WNL.60.1.31

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Learmonth YC, Motl RW, Sandroff BM, Pula JH, Cadavid D. Validation of patient determined disease steps (PDDS) scale scores in persons with multiple sclerosis. BMC Neurol. (2013) 13:37. doi: 10.1186/1471-2377-13-37

PubMed Abstract | CrossRef Full Text | Google Scholar

51. National Multiple Sclerosis Society: Clinical Outcome Measures [online]. Available online at: https://www.nationalmssociety.org/For-Professionals/Researchers/Resources-for-Researchers/Clinical-Study-Measures (accessed April 22, 2020).

52. ClinicSpeak[online]. Available online at: http://www.clinicspeak.com (accessed April 20, 2020).

53. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. (2009) 42:377–81. doi: 10.1016/j.jbi.2008.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Rudick RA, Miller D, Bethoux F, Rao SM, Lee J-C, Stough D, et al. The multiple sclerosis performance test (MSPT): an iPad-based disability assessment tool. J Vis Exp. (2014) 2014:51318. doi: 10.3791/51318

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Butzkueven H, Chapman J, Cristiano E, Grand'Maison F, Hoffmann M, Izquierdo G, et al. MSBase: an international, online registry and platform for collaborative outcomes research in multiple sclerosis. Mult Scler. (2006) 12:769–74. doi: 10.1177/1352458506070775

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Bonavita S, Tedeschi G, Atreja A, Lavorgna L. Digital triage for people with multiple sclerosis in the age of COVID-19 pandemic. Neurol Sci. (2020) 41:1007–9. doi: 10.1007/s10072-020-04391-9

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Floodlight Open [online]. (2020). Available online at: https://floodlightopen.com/en-US (accessed March 13, 2020).