Digital endpoints in clinical trials of Alzheimer’s disease and other neurodegenerative diseases: challenges and opportunities

Alzheimer’s disease (AD) and other neurodegenerative diseases such as Parkinson’s disease (PD) and Huntington’s disease (HD) are associated with progressive cognitive, motor, affective and consequently functional decline considerably affecting Activities of Daily Living (ADL) and quality of life. Standard assessments, such as questionnaires and interviews, cognitive testing, and mobility assessments, lack sensitivity, especially in early stages of neurodegenerative diseases and in the disease progression, and have therefore a limited utility as outcome measurements in clinical trials. Major advances in the last decade in digital technologies have opened a window of opportunity to introduce digital endpoints into clinical trials that can reform the assessment and tracking of neurodegenerative symptoms. The Innovative Health Initiative (IMI)-funded projects RADAR-AD (Remote assessment of disease and relapse—Alzheimer’s disease), IDEA-FAST (Identifying digital endpoints to assess fatigue, sleep and ADL in neurodegenerative disorders and immune-mediated inflammatory diseases) and Mobilise-D (Connecting digital mobility assessment to clinical outcomes for regulatory and clinical endorsement) aim to identify digital endpoints relevant for neurodegenerative diseases that provide reliable, objective, and sensitive evaluation of disability and health-related quality of life. In this article, we will draw from the findings and experiences of the different IMI projects in discussing (1) the value of remote technologies to assess neurodegenerative diseases; (2) feasibility, acceptability and usability of digital assessments; (3) challenges related to the use of digital tools; (4) public involvement and the implementation of patient advisory boards; (5) regulatory learnings; and (6) the significance of inter-project exchange and data- and algorithm-sharing.


Introduction
Digital endpoints in clinical trials are being investigated increasingly in large-scale international projects. The rapid advancement of technological developments allows entirely new approaches to assessing activities of daily living (ADL), sleep and fatigue, motor, cognitive, social, neuropsychiatric, and autonomous body functions with potential for both trials and clinical practice. The appeal lies in the objective, immediate and continuous measurement in both clinical and home settings, the reduction of visits to research or clinic facilities, the accessibility for under-served populations, the potential for better stratification and more personalised therapies, and the possibility to support otherwise time-intense clinical decisions with Artificial Intelligence (AI). This is of specific importance for Alzheimer's disease (AD), but also other neurodegenerative diseases, such as Parkinson's disease (PD) and Huntington's disease (HD), with a predominantly slow progression over years as well as cognitive impairment and fluctuations, which reduce the validity of data from self-rated or one-time assessments.
Functional decline is a significant indicator of progression of neurodegenerative diseases. A range of questionnaires have been developed to assess ADL (1). However, many of these instruments lack sensitivity to change in early stages of a disease and therefore have a limited utility as outcome measures in clinical trials (2,3). This is of specific importance considering recent developments in diseasemodifying drugs for the treatment of AD, such as aducanumab and lecanemab (4) that are targeting early cognitive impairment and emphasise the need for highly sensitive methods. Similar restrictions apply to standard mobility and neuropsychological testing and the query of social skills, sleep, fatigue, neuropsychiatric symptoms, and autonomous body functions with self-and informant-rating questionnaires. Standard assessments are intermittent, costly, and partly rely on subjective information, which is especially problematic in later stages of a neurodegenerative disease. The common goal of the Innovative Health Initiative (IMI)-funded projects RADAR-AD, IDEA-FAST and Mobilise-D is to define digital endpoints relevant for neurodegenerative diseases that provide reliable, objective, and sensitive evaluation of disability, ADL, and health-related quality of life.
RADAR-AD (EC Grant No.806999; www.radar-ad.org) aims to identify and validate remote monitoring technologies (RMTs) to assess functional impairment in all stages of Alzheimer's disease. The study includes wearables and smartphone apps in the main study (n = 232) and passive at-home sensors in a sub-study (n = 45). The RMTs measure a wide range of cognitive and functional domains, including spatial navigation, activity, sleep, speech, driving behaviour, and gait (5).
IDEA-FAST (EC Grant No. 853981; www.idea-fast.eu) aims to identify digital parameters in patients with PD and HD, and immunemediated disorders, which are related to fatigue, sleepiness, and sleep quality. A pilot study (6,7) has informed the design of a larger clinical observational study using different devices concurrently to capture data on ADL-related activities, sleep, physiological and cognitive/ psychological variables. In the latter study, up to 2000 participants (PD n = 500; HD n = 200) will be recruited at up to 24 sites across Europe.
Mobilise-D (EC Grant No. 820820; www.mobilise-d.eu) (8) aims to validate a suite of digital mobility outcomes to directly monitor mobility performance continuously over a 7 day duration using a single wearable device in PD (n = 600) and other diseases associated with mobility impairment (chronic obstructive pulmonary disease, multiple sclerosis, proximal femoral fracture) (9,10).
In this article, we will draw from the findings and experiences of these different IMI projects in discussing (1) the value of remote technologies to assess neurodegenerative diseases; (2) feasibility, acceptability and usability of digital assessments; (3) challenges related to the use of digital tools; (4) public involvement and the implementation of patient advisory boards to guide clinical trials in terms of protocol design, ethical issues, and selection and applicability of digital tools; (5) regulatory learnings; and (6) the significance of inter-project exchange and data-and algorithm-sharing ( Figure 1).

The value of remote technologies to assess neurodegenerative diseases
Technological advances in the last decade opened a window of opportunity to introduce digital endpoints into clinical trials. RMTs could provide a useful, objective way to measure decline by collecting data that correspond to various functional domains that are clinically relevant. They assess functional ability either passively (i.e., not requiring any interaction with the device, such as is the case with gait measures) or interactively (i.e., requiring an active interaction with the device such as when assessing functional abilities involving cognition). The benefit of RMTs as compared to standard assessments, is that they Frontiers in Neurology 03 frontiersin.org are objective and can collect data in the real world continuously. They are ideally placed to potentially measure subtle functional changes that are prevalent among individuals in the early, preclinical stages of neurodegenerative diseases, where current methods of cognitive assessments lack the necessary sensitivity (11) and to continuously track changes during the course of a disease. The RMTs used in the different consortia are listed in Table 1. In the three consortia, different functional domains were measured. Mobility, for example, was evaluated in various ways within the IMI projects. Mobilise-D applied both supervised (in the presence of study staff) and unsupervised testing using a standardised protocol. In addition to that, home mobility was evaluated using an inertial measurement unit (IMU) for 7 days at different time points. In RADAR-AD, mobility was evaluated using a supervised standardised protocol as well, and home mobility using a wrist-worn IMU for 8 consecutive weeks. In both RADAR-AD and IDEA-FAST, heart rate was measured using a wearable. Another functional domain assessed was sleep. IDEA-FAST and RADAR-AD both made use of an app to actively collect data on sleep, asking the participants daily about their fatigue, sleep pattern and quality. Moreover, sleep was measured passively: RADAR-AD made use of a portable EEG device, which a subset of participants wore every night for a month, while IDEA-FAST used a bed sensor with a force-sensitive piezo-electric film, placed under the mattress. Cognition has been addressed in a supervised standardised way in all consortia. Cognitive data was evaluated remotely using several smartphone apps in RADAR-AD and a web-based application of CANTAB in IDEA-FAST and both consortia collected passive information on smartphone use, including keyboard metrics and GPS location tracking.
Future clinical trials will profit from these recent technological developments, which promise improved sensitivity and specificity of endpoint measures, better external validity, and the need of fewer visits to research or clinical facilities and smaller sample sizes due to more detailed datasets per participant. Findings and experiences of RADAR-AD, IDEA-FAST, and Mobilise-D in (1) remote technologies to assess neurodegenerative diseases, (2) feasibility, acceptability and usability of digital assessments, (3) challenges related to the use of digital tools, (4) regulatory learnings and public involvement, and (5) data, algorithm and platform sharing.

Challenges related to the use of digital tools
The use of RMTs can present challenges with respect to a range of aspects including the validity of measurements, related to sensitivity and specificity (e.g., differentiating sensor information in multiperson households), data quality, e.g., choosing the right time granularity (12), data missingness, which is often due to technical and software issues (13), and subsequent analysis. The use of Artificial Intelligence (AI) to combine and analyse RMT signals brings a multitude of challenges itself (14), including privacy and security concerns (15), gaining informed consent (16), and ethical challenges. These can be addressed by creating regulatory frameworks and promoting public-private partnerships (17). Ensuring equity and inclusion when deploying digital tools is another important challenge. Connectivity and broadband access, device variability/obsolescence and digital literacy are "digital determinants of health" that impact equitable access to digital healthcare and the outcomes from and experience with digital tools (18). To date, 37% of the world's population has never used the internet. In the European Union, the percentage of older people (aged 65-74) using the internet varies greatly from 25% in Bulgaria to 94% in Denmark and we face a growing age gap in smartphone ownership in emerging economies around the globe (19). Even if a smartphone or PC is available in a household, access might still be restricted due to financial or technical reasons (20). Digital health studies have developed approaches such as "bring-your-own-device studies" (21), providing funding for internet connectivity, or using sensors that are not (continuously) connected to the internet to help address these challenges. Collocation and sharing of best practices across projects will help address these challenges.

Feasibility, acceptability, and usability of digital assessments
It becomes increasingly important to consider the feasibility, acceptance, usability, and ecological validity of digital endpoints in realworld settings. Few studies report on these factors and ageing populations are not well represented in RMT research (22), but are explored in RADAR-AD, IDEA-FAST and Mobilise-D in collaboration with patients and carers. In studies involving wearables and smartphone apps, acceptance to use devices and adherence to protocol are in general positive when they are reported (23,24). For example, the comfort and  Frontiers in Neurology 05 frontiersin.org acceptability of a wearable sensor to monitor mobility in the Mobilise-D study was very high (23). However, many studies to date lack information on acceptability, adherence and usability (24). Overall, passive devices/apps requiring little or no interaction with a device show higher feasibility, acceptability and usability than interactive devices and are the most researched to date (25). Research in PD reports that the successful implementation of digital technologies is primarily driven by familiarity with the technology and ease of use, costs, motor symptoms hampering the use, experiencing beneficial effects, and feeling safe whilst using the technology (26). In AD, acceptance and adherence can be facilitated by familiarising participants with the devices and providing personal support, lowering technical demands, co-designing solutions and involving relevant stakeholders, introducing participants to the devices at the earliest stages of the disease, and increasing the perception of effectiveness and safety. Barriers mainly include technology anxiety, system failures, and lack of access (27, 28). However, if these factors are addressed, adherence is generally high (85.7%) in older adults (29). Some of these barriers became apparent in RADAR-AD. For example, engaging with RMTs led to some participants feeling discouraged, as they acted as a reminder for their declining cognition. Cognitive impairment also led to missing data, e.g., participants removed their wearables before going to bed, meaning sleep hygiene could not be tracked. Study partners are essential when it comes to reducing or overcoming (cognitive) barriersthey help with charging/handling RMTs, provide emotional support, and remind participants to keep wearing/using RMTs. Overall, study partners are vital in the adherence and usability of digital tools in neurodegenerative diseases (Muurling et al., submitted) 1 . In RADAR-AD and IDEA-FAST, participants reported adjustments to daily routines; specifically, acclimating to wearing two wrist-worn wearable devices, using their phone more, and adjusting personal schedules to complete their daily app-based tasks on time. Ergonomic challenges were reported due to the physical design of watches (i.e., watch straps not fitting well or feeling limited in their movements). Similar findings have been collated within multiple systematic reviews on digital tool use in older adults (28,30,31). Participants reported individual preferences for the display of the wearable screen (e.g., matching the clock face of their usual watch) and for device feedback (e.g., cognition and activity tracking), which facilitated integration into daily routines. Lack of, or inaccurate device feedback, small screens and small fonts also contributed towards the challenges faced by participants. In the IDEA-FAST pilot study, participants moreover mentioned skin irritations due to adhesive patches, constant worry about the device and insecurities regarding its proper functioning. Also, participants reported being less willing to wear devices that were very visible, complicated to use, or that had to be manipulated at impractical times, e.g., right before sleeping. The roadmap towards translating RMT use from research to clinical practice has to continue to evolve, together with patient and stakeholder involvement, as the benefits and challenges are evaluated (32).

Public involvement and the implementation of patient advisory boards
Public Involvement (PI) is about involving people affected by the condition in all aspects of the research process as partners rather than as research participants (33,34). PI not only provides the patients' perspective on what research is important and which unmet needs should be addressed, but it is also about understanding and anticipating what aspects of the research may be difficult to manage by the participants, may raise concerns, and how these issues could be addressed. It also involves reflecting about future issues, challenges, and benefits of the project, if and when the results are eventually implemented in the real world. Involving people from minority ethnic groups and other under-served populations is crucial but still remains a challenge (33).
All three consortia involved patients and, in the case of RADAR-AD, also carers in special advisory boards. They provided strategic input to various aspects of consortium activities throughout the projects, including: study protocols and participant-facing documents; digital health technology in general and digital assessments and outcomes in particular; feasibility, usability and acceptability of digital outcome assessment and how it can contribute to improved care; consultation around health technology assessment and regulatory acceptance of digital outcomes; ethical considerations, recruitment and retention strategies; and involvement in promotion activities about the impact and benefits of results. RADAR-AD and IDEA-FAST also collaborated with patient organisations and in IDEA-FAST, two additional groups consisting of patients, consortium members and representatives from patient organisations were formed to develop and review the project activities and to support the design of the two clinical studies.

Regulatory learnings
If digital endpoints are to be used in clinical trials aimed to achieve a market authorisation for medicinal products, it is of paramount importance that the endpoints are accepted by the regulatory authorities. In recent years, the use of RMT-based assessments has increased dramatically (35). However, the number of digital endpoint measures that are qualified is still limited (36) and there are no approved primary or secondary digital endpoints for use in clinical trials in AD or PD yet (35,37). In RADAR-AD, a regulatory strategy was developed early on, including an extensive evaluation of all qualification opinions and advices and scientific advices of the EMA to gain insight in the types of tools that are intended to be used in clinical trials for supporting/submitting applications for obtaining market authorization (registration trials) (36). The EMA recommendations evolved mainly around the relevance, precision, and accuracy of novel endpoints; validation with current gold standards and clinically meaningful legacy endpoints, including those that matter most to patients ("daily-relevant data"); sensitivity and specificity; good compliance and acceptability; and guarantee of optimal data security and privacy. The RADAR-AD consortium had an initial meeting with the Innovation Task Force in 2020 and is currently in the process of having a Qualification Advice discussion with EMA. The Mobilise-D consortium had two consecutive EMA Frontiers in Neurology 06 frontiersin.org qualification advices in 2020 (38,39) and a letter of support was published on the EMA website (40, 41) following each qualification advice. Mobilise-D has furthermore interacted with the Food and Drug Administration (FDA). The IDEA-FAST consortium had two meetings with the EMA between 2020 and 2022. The first meeting with the Innovation Task Force was to discuss the general concepts of developing digital endpoints for fatigue and sleep. The second meeting was to discuss the study design and data analytic plan of a clinical study to identify these digital endpoints which was given general support by the Scientific Advice Working Party. It is highly recommended for similar consortia to develop a regulatory strategy early on, to ensure that what is being developed will also be accepted in drug trials. It is important to plan for multiple Health Authority meetings utilising Innovation Task Force and EMA Qualification advice meetings as well as meetings with other major Health Authorities, as appropriate. Early advice on study design prior to protocol finalisation/study initiation would be highly desirable. Further development of clear guidance for the use of digital technologies in registration trials could remove some of the regulatory hurdles that currently complicate the development and use of novel improved endpoints (42).

The significance of inter-project exchange and data-and algorithm-sharing
To extend and generalise individual project findings and foster deeper understanding of digital outcomes across neurodegenerative diseases, inter-project exchange and data sharing has gained significance. The full value of data collected in large research programmes can only be realised by enabling a wider set of analytics than is possible through individual consortia. This need is only heightened by the current rapidly expanding popularity in AI and Machine Learning research which relies on large datasets. Sharing resources allows for more rapid research to be undertaken, leading to greater efficacy in terms of advancing state-of-the-art than could be otherwise be achieved working on the data in isolation. For example, the sharing of speech data through DementiaBank (43) has enabled a wide range of different machine learning approaches to be compared and assessed on a common database (44). In such a rapidly growing area of research it is also important to conduct replication analysis and robust generalised testing of proposed digital phenotypes. Sharing and open sourcing algorithms enables these vitally important verification steps.
The sharing of data requires careful considerations to preserve the privacy of participants in a manner that not only meets ethical and statutory requirements, but also meets participants' expectations regarding distribution of their data. Entire IMI-projects have developed around this topic. For example, the European Platform for Neurodegenerative Diseases (EPND, www.epnd.org) aims to accelerate the discovery of diagnostics and treatments for neurodegenerative diseases by removing barriers to data and sample sharing (45). This includes sharing of digital data, by building a robust and secure data sharing infrastructure and funding a case study of prospective digital (bio)marker data collection. EPND aims to build connections to existing data platforms and facilitate the discoverability of resources; provide secure, private cloud-based workspaces where researchers can perform and save analyses; collaborate with other permissioned users; and develop ethical, legal, and regulatory principles guiding platform design and discovery and sharing of data.
The access to and reuse of research data generated by Horizon 2020 projects is available through the Open Research Data Pilot (ORD Pilot), which is in line with the FAIR (Findable, Accessible, Interoperable, Reusable) principles 2 and ensures open access to publications and research data (curated and raw data) including access to, e.g., specialised software or software code, algorithms, and analysis protocols. This allows to build on previous research findings, foster collaboration, promote innovation, and improve transparency in research (46). New projects can be greatly strengthened by reusing infrastructure, such as RADAR-base, and sharing algorithms between consortia that use similar RMTs, such as RADAR-CNS, in the case of RADAR-AD.
We argue that sustainability should be plannable and funded beyond the duration of a project, ideally via IMI-funded platforms, to guarantee a lasting impact and allow following projects to profit from the large data volumes produced by RMTs, previous experiences, including cross-learning about device selection and barriers/ facilitators of using digital health technology, especially for studies that are targeting similar demographics and conditions.

Conclusion
Technological advances and collaboration between IMI-funded and other consortia bring new opportunities to develop and introduce digital endpoints into clinical trials that can revolutionise the assessment and tracking of neurodegenerative symptoms. The digitalization of endpoints allows for objective, immediate and continuous measurement in both clinical and home settings, the reduction of visits to research or clinic facilities, greater accessibility for under-served populations, better stratification and more personalised interventions, and AI-supported clinical decisions.

Funding
The RADAR-AD project has received funding from the Innovative Medicines Initiative