Potential effectiveness of digital therapeutics specialized in executive functions as adjunctive treatment for clinical symptoms of attention-deficit/hyperactivity disorder: a feasibility study

Introduction The role of digital therapeutics (DTx) in the effective management of attention deficit/hyperactivity disorder (ADHD) is beginning to gain clinical attention. Therefore, it is essential to verify their potential efficacy. Method We aimed to investigate the improvement in the clinical symptoms of ADHD by using DTx AimDT01 (NUROW) (AIMMED Co., Ltd., Seoul, Korea) specialized in executive functions. NUROW, which consists of Go/No-go Task- and N-Back/Updating-based training modules and a personalized adaptive algorithm system that adjusts the difficulty level according to the user’s performance, was implemented on 30 Korean children with ADHD aged 6 to 12 years. The children were instructed to use the DTx for 15 min daily for 4 weeks. The Comprehensive attention test (CAT) and Childhood Behavior Checklist (CBCL) were used to assess the children at baseline and endpoint. In contrast, the ADHD-Rating Scale (ARS) and PsyToolkit were used weekly and followed up at 1 month, for any sustained effect. Repeated measures ANOVA was used to identify differences between the participants during visits, while t-tests and Wilcoxon signed-rank tests were used to identify changes before and after the DTx. Results We included 27 participants with ADHD in this analysis. The ARS inattention (F = 4.080, p = 0.010), hyperactivity (F = 5.998. p < 0.001), and sum (F = 5.902, p < 0.001) significantly improved. After applying NUROW, internalized (t = −3.557, p = 0.001, 95% CI = −3.682-−0.985), other (Z = −3.434, p = 0.001, effect size = −0.661), and sum scores (t = −3.081, p = 0.005, 95% CI = −10.126-−2.022) were significantly changed in the CBCL. The overall effect was confirmed in the ARS sustained effect analysis even after 1 month of discontinuing the DTx intervention. Discussion According to caregivers, the findings indicate that DTx holds potential effect as an adjunctive treatment in children with ADHD, especially in subjective clinical symptoms. Future studies will require detailed development and application targeting specific clinical domains using DTx with sufficient sample sizes. Clinical trial registration: KCT0007579.


Introduction
Attention-Deficit/Hyperactivity Disorder (ADHD) is one of the most common neurodevelopmental disorders that significantly impacts children's lifestyles and performance in various settings, including home and school (1,2). According to the Diagnostic and Statistical Manual of Mental Disorders: 5th Edition Text Revision (DSM-5-TR), the clinical presentation of ADHD can be predominantly inattentive, predominantly hyperactive/impulsive, and combined (1,3). Patients with ADHD tend to prefer and seek for small, immediate rewards rather than large, temporally delayed rewards (4). Due to such changes, patients with ADHD struggle to focus and frequently distracted, making difficulties in maintaining appropriate academic performances and face problems with interpersonal relationships (5,6). In addition, patients with ADHD may face comorbid mood and anxiety symptoms, which lead to behavioral changes such as depression, anxiety, and sleep disturbances, along with difficulties in social cognition, interpersonal skills, and low self-esteem (7)(8)(9)(10)(11)(12)(13). These ADHD symptoms can persist into adulthood in 60% of cases, making ADHD a chronic condition that burdens patients and caregivers (14). With approximately one in four children and adolescents receiving mental health services, raising children with ADHD may also result in psychological difficulties and conflicts (2,15,16).
Baseline treatment for ADHD involves pharmacological interventions, including stimulants and non-stimulants (17, 18). However, numerous risks are associated with using these medications, including increased heart rate and blood pressure and potential growth reduction (17). Also, there are still some concerns about the potential abuse risk of stimulants to children and adolescents with ADHD (19-21). In addition, medications are limited in their use as they can only be effective when administered properly (17,22). There are several behavioral intervention options, including cognitivebehavioral therapy, behavioral parent training, and training teachers in classroom applications of contingency management techniques (23). However, issues, such as variation among behavioral therapists and the inability to attend all therapy sessions because of financial and time constraints, can be challenging for patients and their parents (24).
Digital therapeutics (DTx) have developed a new paradigm for ADHD treatment: easy therapeutic access, minimal side effects, and low risk of abuse or misuse to overcome such obstacles. DTx is a therapeutic intervention based on scientific evidence and powered by high-quality software programs for preventing, managing and treating medical disorders and diseases (25). It targets mental and behaviormodifiable conditions, including major mood disorders, anxiety disorders, substance use disorders, autism spectrum disorder, and ADHD (25)(26)(27). The FDA has approved EndeavorRx (Akili Interactive Labs, Boston, MA, United States), the first digital therapy indicated for ADHD (28). Several studies evaluating the efficacy of DTx targeting specific cognitive functions for the treatment of patients with ADHD have been conducted (29,30).
A significant boost to the experiments and applications of telemedicine and DTx in clinical psychiatry has risen due to the pandemic. Using such telematics tools and DTx during the lockdown has provided potential in both the healthcare and social sectors, which can truly revolutionize healthcare system in the near future by integrating the treatments that has been already known. Since the pandemic, numerous studies and interests in these new digital approaches have been steadily increased examining their advantages and disadvantages, as well as prejudices and expectations of healthcare professionals and families toward these approaches (31-33). These efforts to develop innovative treatments tailored to individuals, respecting and trying to meet the needs of both the patients and health professionals as much as possible. Despite the increasing number of studies on DTx, trials are still in their early stages in clinical psychiatry. Although some DTx and smartphone applications target ADHD worldwide, there is a dearth of DTx, that targets the executive functions of ADHD (30,(34)(35)(36). In this study, we used AimDT01 (NUROW) (AIMMED Co., Ltd., Seoul, Korea), a prototype smartphone application developed to target executive functions in patients with ADHD, such as improving attention and working memory. NUROW, a combined word for "neuron" and "arrow, " was designed by a Korean healthcare company to target different fields of working memory by providing training modules in the form of video games. Each training module includes three distinct executive function domains: (i) the Go/No-go task to enhance sustained attention and inhibition control, (ii) the N-Back/Updating training module to improve working memory, and (iii) the memorization training technique to improve attention in participants with ADHD. NUROW is expected to improve the clinical need for adjuvant treatment options, given that behavioral intervention, and pharmacological treatment are essential for providing overall benefits to participants with ADHD (35,37). The objective of this study was to assess the potential effectiveness of DTx (NUROW) in treating clinical symptoms of ADHD in children. The evaluation involved measuring changes in (1) subjective clinical symptoms, including ARS and CBCL, and (2) objective neuropsychological domains, such as CAT and PsyToolkit.

Method Participants and study design
Participants aged 6 to 12 years were recruited from the outpatient clinic of the Chungnam National University Sejong Hospital. Patients were recruited from hospital advertisements and online flyers, and the study was conducted from December 2020 to February 2022. During the medical examination and treatment in the psychiatric outpatient clinic, if the patient is considered a potential candidate to participate in the study, the clinician has recommended the patients and their caregivers participate in the study. A clinician (HJK) screened 35 Frontiers in Psychiatry 03 frontiersin.org participants and diagnosed children with ADHD using the Kiddie Schedule for Affective Disorders and Schizophrenia Present and Lifetime Version (38).
Overall study design is presented in Figure 1. Participants taking ADHD medications were not permitted to alter their dosage for 2 months to control for any potential pharmacological effects during the study. Children taking antidepressants, antipsychotics, or anxiolytics were excluded from the study due to the possibility of neurocognitive side effects that could alter the baseline. Participants with intellectual disabilities, autism spectrum disorders, organic brain diseases, congenital disorders, or any incurable neurological conditions, including seizures or sensory disorders, were excluded. After screening, 30 participants with ADHD were enrolled in the study. During the study, 2 more participants were dropped out due to follow-up loss, leaving with final 30 participants to complete the study. Furthermore, participants who did not meet the standard recommended usage time of NUROW, specifically less than 50%, were considered noncompliant and therefore excluded from the analysis.
After a thorough explanation and comprehension of the study protocol, all participants and their caregivers provided written consent after full explanation and understanding of the study protocol. During the study, no participants and their caregivers received any financial assistance (e.g., cash). However, neuropsychological tests for assessing the participants were provided free during the study. The Institutional Review Board approved this study and was registered with the Clinical Research Information Service (KCT0007579).

Digital therapeutic interventions (AimDT01, NUROW)
NUROW is a preliminary trial product of a digital therapeutic smartphone application specializing in executive functions, such as attention and working memory. It was developed by the Korean company AIMMED Co., Ltd., in collaboration with psychiatrists and cognitive science specialists. It aims to improve the ability to pay attention for a long duration, avoid distraction, and focus on multiple signals simultaneously.
NUROW is compatible with the iOS and Android operating systems. In this study, the participants were instructed to use NUROW for 15 min daily, 5 days weekly, for 4 successive weeks. Prior to the study, participants were required to log into the NUROW system using their parents' e-mail addresses or social network services accounts. After filling in the information of the participant's name, gender, date of birth, and school grade level, the participants' caregivers were given the option to click on ADHD symptom-related questionnaires and the participant's academic achievement level. In Frontiers in Psychiatry 04 frontiersin.org addition to personal information, application usage log data, including the date and duration of use, were collected for 4 weeks. Each of the three main training modes in NUROW is named after one of the three main characters in the novel Journey to the West (Sun Wukong or Monkey King, Zhu Bajie or Pigsy, Sha Wujing or Sandy), each with a tutorial session. Journey to the West is a Chinese novel by Wu Cheng and published in the 16th century. It is regarded as one of East Asia's most classic Chinese novels and a well-known characterdriven story (39).
Each training module was designed to develop distinct working memory domains. The participants were permitted to choose any of the three main training modules and were instructed to spend at least 15 min daily on any training module (Supplementary Figure S1). The Sun Wukong or Monkey King training mode targets sustained attention and inhibition control using the Go/No-go Task. The user is instructed to tap the screen (catching a fist) when a typical signal appears and persists until another signal appears. The Zhu Bajie or Pigsy training mode is an N-Back/Updating-based training module designed to improve working memory by asking the participant to count the number of times a specific character appears on the screen while playing the pressing game. Finally, Sha Wujing or Sandy mode requires the user to memorize the sequence in which the characters appear on the screen, which aims to improve attention.
A real-time reflection of the user's difficulty level in each training module is enabled by an adaptive algorithm system in NUROW. NUROW automatically adjusts the difficulty level of each training session based on the user's performance at a certain level. For example, if a user feels a level is familiar and easy, NUROW shortens the response time, making the level relatively difficult. Parents were notified by phone calls if the NUROW was not used for 48 consecutive hours and on the 28 th (4 th week) of study participation at the end of the study.

Assessment Attention deficit/hyperactivity disorder-Rating Scale
Parents of children with ADHD were requested to complete the ARS online. ARS was assessed weekly from baseline to the endpoint and at the 1-month follow-up. The ARS is one of the commonly used ADHD evaluation scales (40,41). The ARS was based on the diagnostic criteria for ADHD, consisting of 18 questionnaires assessing for inattention and hyperactivity/impulsivity. The ARS yielded three domains: inattention, hyperactivity, and sum scores. High ARS scores indicated a diminished ability for attention and impulse control.

Childhood Behavior Checklist
The CBCL is a standardized checklist for parents to assess child behavioral problems. The CBCL was assessed twice at the beginning and end of the study. The CBCL consists of questionnaires regarding children's behavior in the previous 6 months and combines them into nine subscales for behavioral problem. Internalizing (e.g., anxiety, withdrawal, and somatic symptoms) and externalizing (e.g., aggression and rule-breaking) behaviors are included in CBCL questionnaires. In addition, the CBCL includes mood, anxiety, and somatic disorders (42).

Computerized Comprehensive Attention Test
Using the computerized CAT developed by the Korean Academy of Child and Adolescent Psychiatry, various measures of attention were evaluated (43). CAT is a modified continuous-performance assessment for Korean children and adolescents that uses auditory and visual stimuli. The CAT has six subscales: visual selective, auditory selective, sustained attention to the response, Flanker's task/effect, divided attention, and working memory.
The omission error, commission error, mean reaction time, and standard deviation (SD) of reaction time (response time variability) were measured for each subscale (44). CAT was supervised by a welltrained clinical psychologist and was assessed at the beginning and end of treatment. In sustained attention, participants were asked to respond to all shape stimuli except the "x" shape to measure their ability to inhibit responses while other stimuli were presented. Auditory and visual stimuli were presented to participants every 2 s for 3 min 20 s. They were instructed to respond only to the same pair of stimuli as before to measure divided attention.

PsyToolkit
PsyToolkit is an open-access experimental service for conducting cognitive and psychological laboratory and internet-based experiments and surveys (45). In this study, cognitive neuropsychological tests, such as the Corsiblock, Stroop test, Simon's task, and Wisconsin card sorting test (WCST), were conducted via web-based version to measure reaction time and error rate. Each participant with ADHD used PsyToolkit from baseline to endpoint weekly and 1 month after digital therapeutic intervention.
Corsiblock aims to assess spatial short-term working memory by instructing the user to memorize a sequence of flashing blocks and click them in the same order (46). The Stroop test is a popular neuropsychological measure of executive function, selective attention, cognitive flexibility, and processing speed (47). Participants are asked to identify the color of the printed material of the given color word. Simon's task is a well-established neuropsychological test that evaluates the speed and accuracy of response to various stimuli (48). The WCST is a widely used neuropsychological test for evaluating executive function (49).

Statistical analyzes
The primary objective of this study was to investigate the potential effect of NUROW on subjective clinical symptoms through the assessment of ARS and CBCL. The secondary objective was to examine the potential effect on objective neuropsychological domains through the assessment of CAT and PsyToolkit. ARS and PsyTookit results were analyzed using repeated-measures analysis of variance (ANOVA) with within-participants visit factors. According to the normality test, Wilcoxon's signed-rank tests or paired t-tests were

Result
To ensure reliable analysis, we calculated the application log information regarding the usage time. We carefully reviewed the total usage time for each child throughout the study and identified participants who demonstrated poor compliance, defined as having a total usage time below 50% of the standard recommended usage time for NUROW. Three children were categorized as poor compliant participants. The remaining 27 children with ADHD (90% of the total subjects) were shown to have played NUROW compliantly. Table 1 presents the demographic and primary clinical characteristics of the participants. In this study, patients with ADHD were found to be taking methylphenidate and atomoxetine at mean dosages of 28.89 mg and 28.13 mg, respectively.

Attention deficit/hyperactivity disorder-Rating Scale
In this study, repeated-measures ANOVA was performed to determine differences in ARS within hospital visits. Mean

Computerized CAT
The results showed that there were no statistically significant differences in the omission and commission errors of selective visual attention and auditory selective attention, between the baseline and endpoint. However, there was a significant difference in the mean reaction time of selective visual attention between the baseline and endpoint (t = −2.422, p = 0.022). The sustained attention to response showed no significant differences in omission and commission errors, along with reaction time and reaction time SD between the baseline and endpoint. The Flanker's test showed a significant difference in the commission errors between the baseline and endpoint (t = −2.869, p = 0.008), but no significant differences in omission errors, reaction   Table 2 and Figure 2. We speculated that NUROW has a potential effect on reducing reaction times in patients with ADHD. However, PsyToolkit's analysis could not confirm the clinically meaningful and interpretable results.

Discussion
This study indicates that NUROW significantly improved our primary outcome measure, a clinical symptom of ADHD, especially inattentiveness and hyperactivity. To the best of our knowledge, this is the first study to evaluate the feasibility and possible clinical effects of    (29,30,35,(50)(51)(52)(53). ARS is a parent-reported measure; it indicates that caregivers of children with ADHD subjectively experienced positive changes in their child's behavior after using NUROW. The CBCL, another parent-reported measure, also indicated a significant difference after using DTx. After using NUROW, anxiety, depression, somatic, internalizing, additional problems, and total CBCL scores declined in this study. Although the participants' diagnoses other than ADHD were not confirmed in this study, fidgeting and restlessness in the behavior of  Frontiers in Psychiatry 11 frontiersin.org children with ADHD may have overlapped with anxiety items in the CBCL (6,9,10,54,55), indicating that the use of DTx helped behavioral aspects of ADHD. For somatic symptoms on the CBCL, it is known that children with ADHD have comorbidities of somatic disorders (6,56).
Our findings indicate the potential efficacy of DTx in reducing subjective ADHD symptoms, despite the absence of financial assistance for participants and their caregivers in the current study. NUROW may be an adjunctive treatment option for some of the difficulties faced by current interventions, such as medications and behavioral therapy. During the study, no adverse effects of NUROW, including dizziness, nausea, or headache, were observed. Furthermore, as part of a feasibility study, the current research classified participants based on their usage time compared to the recommended usage time, with 90% of the participants demonstrating reasonable compliance (over 50% of the recommended usage time) until the completion of the study. Hence, NUROW was regarded as a feasible digital therapeutic option for individuals with ADHD.
Although it was possible to confirm a significant improvement trend following DTx intervention in the clinical and behavioral domains (ARS and CBCL), there was insufficient evidence that NUROW had a significant change on the neurocognitive domains (CAT and PsyToolkit). Although the NUROW applied in this study was developed specifically for executive function, the lack of a significant therapeutic effect in the neurocognitive domain is a topic that requires further investigation. Various studies on the psychiatric application of serious games, including DTx, have been conducted recently; in particular the potential of DTx in improving neurocognitive symptoms in ADHD, major neurocognitive disorder, and depression (26)(27)(28)(29)(30). However, it seems too early to draw a conclusion on the cognitive function improvement effect in this study. Although there was no significant effect on the neurocognitive domain, some attention-related measures (e.g., reaction time) from PsyToolkit showed statistically significant improvements when DTx was used. It is well-known that a learning effect occurs when cognitive tests are repeated routinely (56,57). From the results of this study, PsyToolkit was assessed weekly, and computerized CAT was assessed at the baseline and endpoint. Although cognitive tests were performed repeatedly, there was no significant effect on cognitive tasks. Only the incompatible error rate sub-item of Simon's task in PsyToolkit decreased with statistical significance. To interpret these results, we must understand the study's design methodology. Although potential confounding factors, such as psychiatric medications, were regulated to the greatest extent, the study design and sample size were insufficient to accurately evaluate the direct effects of DTx on the neurocognitive domain. Nevertheless, the ability to confirm the potential effect in the clinical and behavioral domains is an encouraging result, and it will serve as the foundation for future research and development of DTx.
Clinically, it is essential to confirm whether there is a sustained effect after treatment. In this regard, the study's confirmation of the sustained effect even after 1 month is an additional strength. ARS, a measure that confirmed the potential therapeutic efficacy of NUROW by evaluating the level of clinical improvement at the end of the intervention and 1 month later, showed no statistical difference between the two-time points. This has positive implications for the potential long-term effectiveness of NUROW, particularly in clinical and behavioral domains. Our study results have found that there were other domains, such as Corsiblock test, Stroop task, and WCST, found to have sustained effect of DTx even after 1 month of discontinuation. These results suggest that DTx has also showed possibility of persistent effects on neurocognitive domains. However, as previously mentioned, it would be limited to discuss the sustained effect in terms of PsyToolkit because it was difficult to confirm its clear impact of PsyToolkit during DTx intervention. A feasibility study of DTx in multiple sclerosis patients explored both the therapeutic and sustained effects of DTx (58). Several previous studies have examined the efficacy of DTx in treating ADHD. However, none have examined whether DTx has a sustained effect after discontinuation, to our knowledge. Recent studies have focused on the efficacy of digital therapeutics (DTx) when in use, leaving the effects of sustained use largely unexplored (29,30,35,50,51,53). Our study aims to address this gap, investigating the possible effects of DTx and the potential for sustained effects even after the participants have stopped using the DTx. This will allow us to gain a comprehensive understanding of the potential effects of DTx and how they might be sustained. We believe that our findings could open up new possibilities for the use of DTx in the future.
However, there are certain limitations to this study. First, this study was conducted with a relatively small sample size; however, it has its value as a feasibility study. Even with a relatively small sample size, an appropriate analysis technique was used to evaluate clinical changes in the participants. Although current study lacks a sham control to compare the effectiveness of DTx, our findings hold some tangible improvements in clinical measures, indicating that current study holds solid evidence over previous case control studies (35,53). Second, a formal IQ evaluation was not performed during the preliminary evaluation. However, during the screening process, a child-adolescent psychiatrist evaluated intellectual disability. Third, it can be difficult to interpret the results independent of the medication effects. However, this study protocol specifically included only children who maintained the same dosage and were not receiving other medications except ADHD medications, such as antidepressants and anxiolytics. Fourthly, the current study did not gather any background information about the parents. Lastly, because a significant number of self-reported evaluations have been performed to analyze the effect, the placebo effect must be considered. Despite specific challenges, our study aimed to maintain a reasonable level of participant compliance despite the absence of financial incentives.
In conclusion, our findings suggest that there are tangible differences in DTx, such as NUROW, in children with ADHD, particularly in alleviating the clinical symptoms, including inattentiveness and hyperactivity reported by parents. As this study aimed to evaluate the potential efficacy and feasibility of NUROW, the findings provide preliminary evidence for the potential effects of DTx in improving attentional aspects and objective subclinical symptoms in children with ADHD. It is necessary to conduct well-designed research with a large sample size to analyze the effect of DTx.

Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement
The studies involving human participants were reviewed and approved by the Institutional Review Board of Chungnam National University Sejong Hospital (Approval Number: 2020-090038-007).
Frontiers in Psychiatry 12 frontiersin.org Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.

Author contributions
TS, H-JK, and C-HC conceived and designed the study and performed statistical analyzes. TS, JY, H-JL, H-JK, and C-HC wrote the first draft of the manuscript. TS, K-YC, J-LK, H-JK, and C-HC participated in data collection. C-HC has full access to all the data in the study and is responsible for the integrity of the data and the accuracy of the data analysis. All authors edited all versions of the manuscript and were involved in interpreting the results and read, commented on, and approved the final version of the manuscript.