Tai Hui Sun1,2Ji Won Yeom1,2Kwang-Yeon Choi3Jeong-Lan Kim3
Heon-Jeong Lee1,2
Hyun-Jin Kim3
Chul-Hyun Cho1,2,4*- 1Department of Psychiatry, Korea University Anam Hospital, Seoul, Republic of Korea
- 2Department of Psychiatry, Korea University College of Medicine, Seoul, Republic of Korea
- 3Department of Psychiatry, Chungnam National University College of Medicine, Daejeon, Republic of Korea
- 4Department of Biomedical Informatics, Korea University College of Medicine, Seoul, Republic of Korea
Introduction: The role of digital therapeutics (DTx) in the effective management of attention deficit/hyperactivity disorder (ADHD) is beginning to gain clinical attention. Therefore, it is essential to verify their potential efficacy.
Method: We aimed to investigate the improvement in the clinical symptoms of ADHD by using DTx AimDT01 (NUROW) (AIMMED Co., Ltd., Seoul, Korea) specialized in executive functions. NUROW, which consists of Go/No-go Task- and N-Back/Updating-based training modules and a personalized adaptive algorithm system that adjusts the difficulty level according to the user’s performance, was implemented on 30 Korean children with ADHD aged 6 to 12 years. The children were instructed to use the DTx for 15 min daily for 4 weeks. The Comprehensive attention test (CAT) and Childhood Behavior Checklist (CBCL) were used to assess the children at baseline and endpoint. In contrast, the ADHD-Rating Scale (ARS) and PsyToolkit were used weekly and followed up at 1 month, for any sustained effect. Repeated measures ANOVA was used to identify differences between the participants during visits, while t-tests and Wilcoxon signed-rank tests were used to identify changes before and after the DTx.
Results: We included 27 participants with ADHD in this analysis. The ARS inattention (F = 4.080, p = 0.010), hyperactivity (F = 5.998. p < 0.001), and sum (F = 5.902, p < 0.001) significantly improved. After applying NUROW, internalized (t = −3.557, p = 0.001, 95% CI = −3.682-−0.985), other (Z = −3.434, p = 0.001, effect size = −0.661), and sum scores (t = −3.081, p = 0.005, 95% CI = −10.126-−2.022) were significantly changed in the CBCL. The overall effect was confirmed in the ARS sustained effect analysis even after 1 month of discontinuing the DTx intervention.
Discussion: According to caregivers, the findings indicate that DTx holds potential effect as an adjunctive treatment in children with ADHD, especially in subjective clinical symptoms. Future studies will require detailed development and application targeting specific clinical domains using DTx with sufficient sample sizes.
Clinical trial registration: KCT0007579.
Introduction
Attention-Deficit/Hyperactivity Disorder (ADHD) is one of the most common neurodevelopmental disorders that significantly impacts children’s lifestyles and performance in various settings, including home and school (1, 2). According to the Diagnostic and Statistical Manual of Mental Disorders: 5th Edition Text Revision (DSM-5-TR), the clinical presentation of ADHD can be predominantly inattentive, predominantly hyperactive/impulsive, and combined (1, 3). Patients with ADHD tend to prefer and seek for small, immediate rewards rather than large, temporally delayed rewards (4). Due to such changes, patients with ADHD struggle to focus and frequently distracted, making difficulties in maintaining appropriate academic performances and face problems with interpersonal relationships (5, 6). In addition, patients with ADHD may face comorbid mood and anxiety symptoms, which lead to behavioral changes such as depression, anxiety, and sleep disturbances, along with difficulties in social cognition, interpersonal skills, and low self-esteem (7–13). These ADHD symptoms can persist into adulthood in 60% of cases, making ADHD a chronic condition that burdens patients and caregivers (14). With approximately one in four children and adolescents receiving mental health services, raising children with ADHD may also result in psychological difficulties and conflicts (2, 15, 16).
Baseline treatment for ADHD involves pharmacological interventions, including stimulants and non-stimulants (17, 18). However, numerous risks are associated with using these medications, including increased heart rate and blood pressure and potential growth reduction (17). Also, there are still some concerns about the potential abuse risk of stimulants to children and adolescents with ADHD (19–21). In addition, medications are limited in their use as they can only be effective when administered properly (17, 22). There are several behavioral intervention options, including cognitive-behavioral therapy, behavioral parent training, and training teachers in classroom applications of contingency management techniques (23). However, issues, such as variation among behavioral therapists and the inability to attend all therapy sessions because of financial and time constraints, can be challenging for patients and their parents (24).
Digital therapeutics (DTx) have developed a new paradigm for ADHD treatment: easy therapeutic access, minimal side effects, and low risk of abuse or misuse to overcome such obstacles. DTx is a therapeutic intervention based on scientific evidence and powered by high-quality software programs for preventing, managing and treating medical disorders and diseases (25). It targets mental and behavior-modifiable conditions, including major mood disorders, anxiety disorders, substance use disorders, autism spectrum disorder, and ADHD (25–27). The FDA has approved EndeavorRx (Akili Interactive Labs, Boston, MA, United States), the first digital therapy indicated for ADHD (28). Several studies evaluating the efficacy of DTx targeting specific cognitive functions for the treatment of patients with ADHD have been conducted (29, 30).
A significant boost to the experiments and applications of telemedicine and DTx in clinical psychiatry has risen due to the pandemic. Using such telematics tools and DTx during the lockdown has provided potential in both the healthcare and social sectors, which can truly revolutionize healthcare system in the near future by integrating the treatments that has been already known. Since the pandemic, numerous studies and interests in these new digital approaches have been steadily increased examining their advantages and disadvantages, as well as prejudices and expectations of healthcare professionals and families toward these approaches (31–33). These efforts to develop innovative treatments tailored to individuals, respecting and trying to meet the needs of both the patients and health professionals as much as possible. Despite the increasing number of studies on DTx, trials are still in their early stages in clinical psychiatry. Although some DTx and smartphone applications target ADHD worldwide, there is a dearth of DTx, that targets the executive functions of ADHD (30, 34–36). In this study, we used AimDT01 (NUROW) (AIMMED Co., Ltd., Seoul, Korea), a prototype smartphone application developed to target executive functions in patients with ADHD, such as improving attention and working memory. NUROW, a combined word for “neuron” and “arrow,” was designed by a Korean healthcare company to target different fields of working memory by providing training modules in the form of video games. Each training module includes three distinct executive function domains: (i) the Go/No-go task to enhance sustained attention and inhibition control, (ii) the N-Back/Updating training module to improve working memory, and (iii) the memorization training technique to improve attention in participants with ADHD. NUROW is expected to improve the clinical need for adjuvant treatment options, given that behavioral intervention, and pharmacological treatment are essential for providing overall benefits to participants with ADHD (35, 37). The objective of this study was to assess the potential effectiveness of DTx (NUROW) in treating clinical symptoms of ADHD in children. The evaluation involved measuring changes in (1) subjective clinical symptoms, including ARS and CBCL, and (2) objective neuropsychological domains, such as CAT and PsyToolkit.
Method
Participants and study design
Participants aged 6 to 12 years were recruited from the outpatient clinic of the Chungnam National University Sejong Hospital. Patients were recruited from hospital advertisements and online flyers, and the study was conducted from December 2020 to February 2022. During the medical examination and treatment in the psychiatric outpatient clinic, if the patient is considered a potential candidate to participate in the study, the clinician has recommended the patients and their caregivers participate in the study. A clinician (HJK) screened 35 participants and diagnosed children with ADHD using the Kiddie Schedule for Affective Disorders and Schizophrenia Present and Lifetime Version (38).
Overall study design is presented in Figure 1. Participants taking ADHD medications were not permitted to alter their dosage for 2 months to control for any potential pharmacological effects during the study. Children taking antidepressants, antipsychotics, or anxiolytics were excluded from the study due to the possibility of neurocognitive side effects that could alter the baseline. Participants with intellectual disabilities, autism spectrum disorders, organic brain diseases, congenital disorders, or any incurable neurological conditions, including seizures or sensory disorders, were excluded. After screening, 30 participants with ADHD were enrolled in the study. During the study, 2 more participants were dropped out due to follow-up loss, leaving with final 30 participants to complete the study. Furthermore, participants who did not meet the standard recommended usage time of NUROW, specifically less than 50%, were considered noncompliant and therefore excluded from the analysis.
Figure 1. Schematic research flow of using digital therapeutic (NUROW) on Korean children with ADHD (A) Overall study design of digital therapeutics application and participant flow. (B) Neuropsychological assessment framework for evaluating effects of digital therapeutics in participants with ADHD. ADHD, Attention Deficit/Hyperactivity Disorder; DTx, digital therapeutics; CAT, Comprehensive Attention Test; CBCL, Childhood behavioral checklist; ARS, ADHD-Rating Scale.
After a thorough explanation and comprehension of the study protocol, all participants and their caregivers provided written consent after full explanation and understanding of the study protocol. During the study, no participants and their caregivers received any financial assistance (e.g., cash). However, neuropsychological tests for assessing the participants were provided free during the study. The Institutional Review Board approved this study and was registered with the Clinical Research Information Service (KCT0007579).
Digital therapeutic interventions (AimDT01, NUROW)
NUROW is a preliminary trial product of a digital therapeutic smartphone application specializing in executive functions, such as attention and working memory. It was developed by the Korean company AIMMED Co., Ltd., in collaboration with psychiatrists and cognitive science specialists. It aims to improve the ability to pay attention for a long duration, avoid distraction, and focus on multiple signals simultaneously.
NUROW is compatible with the iOS and Android operating systems. In this study, the participants were instructed to use NUROW for 15 min daily, 5 days weekly, for 4 successive weeks. Prior to the study, participants were required to log into the NUROW system using their parents’ e-mail addresses or social network services accounts. After filling in the information of the participant’s name, gender, date of birth, and school grade level, the participants’ caregivers were given the option to click on ADHD symptom-related questionnaires and the participant’s academic achievement level. In addition to personal information, application usage log data, including the date and duration of use, were collected for 4 weeks.
Each of the three main training modes in NUROW is named after one of the three main characters in the novel Journey to the West (Sun Wukong or Monkey King, Zhu Bajie or Pigsy, Sha Wujing or Sandy), each with a tutorial session. Journey to the West is a Chinese novel by Wu Cheng and published in the 16th century. It is regarded as one of East Asia’s most classic Chinese novels and a well-known character-driven story (39).
Each training module was designed to develop distinct working memory domains. The participants were permitted to choose any of the three main training modules and were instructed to spend at least 15 min daily on any training module (Supplementary Figure S1). The Sun Wukong or Monkey King training mode targets sustained attention and inhibition control using the Go/No-go Task. The user is instructed to tap the screen (catching a fist) when a typical signal appears and persists until another signal appears. The Zhu Bajie or Pigsy training mode is an N-Back/Updating-based training module designed to improve working memory by asking the participant to count the number of times a specific character appears on the screen while playing the pressing game. Finally, Sha Wujing or Sandy mode requires the user to memorize the sequence in which the characters appear on the screen, which aims to improve attention.
A real-time reflection of the user’s difficulty level in each training module is enabled by an adaptive algorithm system in NUROW. NUROW automatically adjusts the difficulty level of each training session based on the user’s performance at a certain level. For example, if a user feels a level is familiar and easy, NUROW shortens the response time, making the level relatively difficult. Parents were notified by phone calls if the NUROW was not used for 48 consecutive hours and on the 28th (4th week) of study participation at the end of the study.
Assessment
Attention deficit/hyperactivity disorder-Rating Scale
Parents of children with ADHD were requested to complete the ARS online. ARS was assessed weekly from baseline to the endpoint and at the 1-month follow-up. The ARS is one of the commonly used ADHD evaluation scales (40, 41). The ARS was based on the diagnostic criteria for ADHD, consisting of 18 questionnaires assessing for inattention and hyperactivity/impulsivity. The ARS yielded three domains: inattention, hyperactivity, and sum scores. High ARS scores indicated a diminished ability for attention and impulse control.
Childhood Behavior Checklist
The CBCL is a standardized checklist for parents to assess child behavioral problems. The CBCL was assessed twice at the beginning and end of the study. The CBCL consists of questionnaires regarding children’s behavior in the previous 6 months and combines them into nine subscales for behavioral problem. Internalizing (e.g., anxiety, withdrawal, and somatic symptoms) and externalizing (e.g., aggression and rule-breaking) behaviors are included in CBCL questionnaires. In addition, the CBCL includes mood, anxiety, and somatic disorders (42).
Computerized Comprehensive Attention Test
Using the computerized CAT developed by the Korean Academy of Child and Adolescent Psychiatry, various measures of attention were evaluated (43). CAT is a modified continuous-performance assessment for Korean children and adolescents that uses auditory and visual stimuli. The CAT has six subscales: visual selective, auditory selective, sustained attention to the response, Flanker’s task/effect, divided attention, and working memory.
The omission error, commission error, mean reaction time, and standard deviation (SD) of reaction time (response time variability) were measured for each subscale (44). CAT was supervised by a well-trained clinical psychologist and was assessed at the beginning and end of treatment. In sustained attention, participants were asked to respond to all shape stimuli except the “x” shape to measure their ability to inhibit responses while other stimuli were presented. Auditory and visual stimuli were presented to participants every 2 s for 3 min 20 s. They were instructed to respond only to the same pair of stimuli as before to measure divided attention.
PsyToolkit
PsyToolkit is an open-access experimental service for conducting cognitive and psychological laboratory and internet-based experiments and surveys (45). In this study, cognitive neuropsychological tests, such as the Corsiblock, Stroop test, Simon’s task, and Wisconsin card sorting test (WCST), were conducted via web-based version to measure reaction time and error rate. Each participant with ADHD used PsyToolkit from baseline to endpoint weekly and 1 month after digital therapeutic intervention.
Corsiblock aims to assess spatial short-term working memory by instructing the user to memorize a sequence of flashing blocks and click them in the same order (46). The Stroop test is a popular neuropsychological measure of executive function, selective attention, cognitive flexibility, and processing speed (47). Participants are asked to identify the color of the printed material of the given color word. Simon’s task is a well-established neuropsychological test that evaluates the speed and accuracy of response to various stimuli (48). The WCST is a widely used neuropsychological test for evaluating executive function (49).
Statistical analyzes
The primary objective of this study was to investigate the potential effect of NUROW on subjective clinical symptoms through the assessment of ARS and CBCL. The secondary objective was to examine the potential effect on objective neuropsychological domains through the assessment of CAT and PsyToolkit. ARS and PsyTookit results were analyzed using repeated-measures analysis of variance (ANOVA) with within-participants visit factors. According to the normality test, Wilcoxon’s signed-rank tests or paired t-tests were conducted on the CBCL and CAT. Wilcoxon’s signed-rank test or paired t-test were used to analyze potential sustained effect of NUROW by ARS and PsyToolkit between the endpoint and 1 month later. All analyzes results were considered statistically significant when p < 0.05. IBM SPSS Statistics (version 26.0; SPSS Inc., Chicago, IL, United States) was used to analyze data.
Result
To ensure reliable analysis, we calculated the application log information regarding the usage time. We carefully reviewed the total usage time for each child throughout the study and identified participants who demonstrated poor compliance, defined as having a total usage time below 50% of the standard recommended usage time for NUROW. Three children were categorized as poor compliant participants. The remaining 27 children with ADHD (90% of the total subjects) were shown to have played NUROW compliantly. Table 1 presents the demographic and primary clinical characteristics of the participants. In this study, patients with ADHD were found to be taking methylphenidate and atomoxetine at mean dosages of 28.89 mg and 28.13 mg, respectively.
Table 1. Demographic and clinical characteristics of participants with attention-deficit/hyperactivity disorder (ADHD) (n = 27) at baseline.
Attention deficit/hyperactivity disorder-Rating Scale
In this study, repeated-measures ANOVA was performed to determine differences in ARS within hospital visits. Mean(±SD) scores of ARS inattention in the baseline and endpoint were 20.22 (±5.639) and 18.26 (±5.34), respectively, with statistically significant difference (F = 4.080, p = 0.010). Mean (±SD) scores of ARS hyperactivity in the baseline and endpoint were 19.59 (±6.28) and 17.19 (±6.06), respectively, with statistically significant difference (F = 5.998, p < 0.001). Mean (±SD) score of ARS sum scores in the baseline and endpoint were 39.81 (±11.48) and 35.44 (±11.04), respectively, showing a significant decrease after DTx intervention (F = 5.902, p < 0.001; Table 2; Figure 2).
Table 2. Result of a repeated measures analysis of variance on the differences of ADHD-Rating Scale and PsyToolkit measures in patients with ADHD during digital therapeutics (NUROW) intervention.
Figure 2. Results using a repeated measures analysis of variance on (A) ADHD-Rating Scale, (B) Corsiblock Test, (C) Stroop Test, (D) Simon Task, and (E) Wisconsin Card Sorting Test measures in patients with ADHD during digital therapeutics (NUROW) intervention. CRT, Congruent Reaction Time; IRT, Incongruent Reaction Time; CER, Compatible Error Rate; IER, Incompatible Error Rate; TER, Total Error Rate; PER, Perseveration Error Rate; NER, Non-Perseveration Error Rate.
Childhood Behavior Checklist
Since we analyzed a total of 11 subdomains for CBCL Problem and 8 subdomains for CBCL DSM-5 Diagnosis, we applied Bonferroni correction for multiple comparisons. Therefore, the statistical significance of the analysis results for subdomains (excluding Sum) of CBCL Problem was p < 0.0045 (0.05/11), and the statistical significance of the analysis results for subdomains of CBCL DSM-5 Diagnosis was p < 0.0063 (0.05/8). For CBCL Problem, internalized (t = −3.557, p = 0.001, 95% CI = −3.682-−0.985), other (Z = –3.434, p = 0.001, effect size = −0.661), and sum scores (t = −3.081, p = 0.005, 95% CI = −10.126-−2.022) were significantly changed after applying NUROW. Regarding the CBCL DSM-5 diagnosis, no subdomain showed a significant change after using DTx (Table 3).
Table 3. Changes in Childhood behavior checklist (CBCL) and Computerized Comprehensive Attention Test (CAT) measures in children with ADHD using digital therapeutics (NUROW).
Computerized CAT
The results showed that there were no statistically significant differences in the omission and commission errors of selective visual attention and auditory selective attention, between the baseline and endpoint. However, there was a significant difference in the mean reaction time of selective visual attention between the baseline and endpoint (t = −2.422, p = 0.022). The sustained attention to response showed no significant differences in omission and commission errors, along with reaction time and reaction time SD between the baseline and endpoint. The Flanker’s test showed a significant difference in the commission errors between the baseline and endpoint (t = −2.869, p = 0.008), but no significant differences in omission errors, reaction time, and reaction time SD. There was no significant difference between the baseline and endpoint in divided attention in terms of omission and commission errors and reaction time and reaction time SD. For spatial working memory measure, 4 sub-items were measured: correct forward response, forward memory span, backward correct response, and backward memory span. Only backward correct response (Z = –2.720, p = 0.007) and backward memory span (t = 2.954, p = 0.007) were improved after DTx, with statistical significance (Table 2; Figure 2).
PsyToolkit
For Corsiblock, there was no significant difference between visits (F = 0.773, p = 0.555). In the Stroop test, congruent reaction time and incongruent reaction time were significantly different within the visits [congruent (F = 3.539, p = 0.009) and incongruent (F = 5.017, p = 0.006)], while there was no significant difference within the visits for error rate [compatible error rate (F = 1.685, p = 0.159) and incompatible error rate (F = 1.463, p = 0.242)]. For the Simon task, congruent and incongruent reaction times were significantly different within the visits [congruent (F = 4.256, p = 0.009) and incongruent (F = 3.912, p = 0.014)], while there was no significant difference within the visits for error rate [compatible error rate (F = 1.715, p = 0.172), incompatible error rate (F = 0.449, p = 0.773]). In the WCST, there was no significant difference in the hospital visits for total error rate (F = 0.388, p = 0.817), perseveration error rate (F = 0.298, p = 0.857), or non-perseveration error rate (F = 0.898, p = 0.468). All mean values and error rates are listed in Table 2 and Figure 2. We speculated that NUROW has a potential effect on reducing reaction times in patients with ADHD. However, PsyToolkit’s analysis could not confirm the clinically meaningful and interpretable results.
Sustained effect: 1-month follow-up
Sustained effects in ARS and PsyToolkit were investigated between the endpoint of the digital therapeutic session and 1 month later. For ARS, 1-month follow-up mean(±SD) scores for inattention, hyperactivity, and sum were 17.81(±5.55), 17.22(±6.59), and 35.04(±11.56), respectively (Table 4), while there was no significant difference in all the three measures (Inattention: t = −1.210, p = 0.237; Hyperactivity: Z = −0.020, p = 0.984, Sum: t = −0.942, p = 0.346), suggesting a potential, clinically meaningful sustained effect after the discontinuation of NUROW.
Table 4. Results of comparative analysis of ADHD-Rating Scale and PsyToolkit neuropsychological measures between immediately after digital therapeutic intervention (Week 4) and follow-up after 1 month to confirm the sustained effect of digital therapeutics (NUROW).
In the Corsiblock test of PsyToolkit, the 1-month follow-up mean(±SD) score was 3.37(±1.88), with no significant difference (Z = -0.475, p = 0.634). For the Stroop task of PsyToolkit, the 1-month follow-up mean(±SD) score of congruent reaction time, incongruent reaction time, compatible error rate, and incompatible error rate were 957.10(±236.68), 846.93(±365.61), 20.36(±18.26) and 124.02(±318.62), respectively, with no statistically significant difference. In the Simon task of PsyToolkit, 1-month follow-up mean(±SD) scores of congruent reaction time, incongruent reaction time, compatible error rate, and incompatible error rate were 1206.91(±464.60), 1384.42(±470.05), 13.09(±18.68), and 12.52(±17.11), respectively, showing a significant difference only in the incompatible error rate 1 month after applying DTx (Z = -2.127, p = 0.003). For WCST, the 1-month follow-up mean(±SD) score of total error rate, perseveration error rate, and non-perseveration error rate were 29.20(±12.09), 16.23(±5.06), and 12.96(±8.21), respectively. No statistically significant difference was found for the WCST, suggesting possible sustained effect.
Discussion
This study indicates that NUROW significantly improved our primary outcome measure, a clinical symptom of ADHD, especially inattentiveness and hyperactivity. To the best of our knowledge, this is the first study to evaluate the feasibility and possible clinical effects of adjunctive DTx for individuals with ADHD in South Korea. Our results are consistent with previous studies involving other DTx for ADHD (29, 30, 35, 50–53). ARS is a parent-reported measure; it indicates that caregivers of children with ADHD subjectively experienced positive changes in their child’s behavior after using NUROW. The CBCL, another parent-reported measure, also indicated a significant difference after using DTx. After using NUROW, anxiety, depression, somatic, internalizing, additional problems, and total CBCL scores declined in this study. Although the participants’ diagnoses other than ADHD were not confirmed in this study, fidgeting and restlessness in the behavior of children with ADHD may have overlapped with anxiety items in the CBCL (6, 9, 10, 54, 55), indicating that the use of DTx helped behavioral aspects of ADHD. For somatic symptoms on the CBCL, it is known that children with ADHD have comorbidities of somatic disorders (6, 56). Our findings indicate the potential efficacy of DTx in reducing subjective ADHD symptoms, despite the absence of financial assistance for participants and their caregivers in the current study. NUROW may be an adjunctive treatment option for some of the difficulties faced by current interventions, such as medications and behavioral therapy. During the study, no adverse effects of NUROW, including dizziness, nausea, or headache, were observed. Furthermore, as part of a feasibility study, the current research classified participants based on their usage time compared to the recommended usage time, with 90% of the participants demonstrating reasonable compliance (over 50% of the recommended usage time) until the completion of the study. Hence, NUROW was regarded as a feasible digital therapeutic option for individuals with ADHD.
Although it was possible to confirm a significant improvement trend following DTx intervention in the clinical and behavioral domains (ARS and CBCL), there was insufficient evidence that NUROW had a significant change on the neurocognitive domains (CAT and PsyToolkit). Although the NUROW applied in this study was developed specifically for executive function, the lack of a significant therapeutic effect in the neurocognitive domain is a topic that requires further investigation. Various studies on the psychiatric application of serious games, including DTx, have been conducted recently; in particular the potential of DTx in improving neurocognitive symptoms in ADHD, major neurocognitive disorder, and depression (26–30). However, it seems too early to draw a conclusion on the cognitive function improvement effect in this study. Although there was no significant effect on the neurocognitive domain, some attention-related measures (e.g., reaction time) from PsyToolkit showed statistically significant improvements when DTx was used. It is well-known that a learning effect occurs when cognitive tests are repeated routinely (56, 57). From the results of this study, PsyToolkit was assessed weekly, and computerized CAT was assessed at the baseline and endpoint. Although cognitive tests were performed repeatedly, there was no significant effect on cognitive tasks. Only the incompatible error rate sub-item of Simon’s task in PsyToolkit decreased with statistical significance. To interpret these results, we must understand the study’s design methodology. Although potential confounding factors, such as psychiatric medications, were regulated to the greatest extent, the study design and sample size were insufficient to accurately evaluate the direct effects of DTx on the neurocognitive domain. Nevertheless, the ability to confirm the potential effect in the clinical and behavioral domains is an encouraging result, and it will serve as the foundation for future research and development of DTx.
Clinically, it is essential to confirm whether there is a sustained effect after treatment. In this regard, the study’s confirmation of the sustained effect even after 1 month is an additional strength. ARS, a measure that confirmed the potential therapeutic efficacy of NUROW by evaluating the level of clinical improvement at the end of the intervention and 1 month later, showed no statistical difference between the two-time points. This has positive implications for the potential long-term effectiveness of NUROW, particularly in clinical and behavioral domains. Our study results have found that there were other domains, such as Corsiblock test, Stroop task, and WCST, found to have sustained effect of DTx even after 1 month of discontinuation. These results suggest that DTx has also showed possibility of persistent effects on neurocognitive domains. However, as previously mentioned, it would be limited to discuss the sustained effect in terms of PsyToolkit because it was difficult to confirm its clear impact of PsyToolkit during DTx intervention. A feasibility study of DTx in multiple sclerosis patients explored both the therapeutic and sustained effects of DTx (58). Several previous studies have examined the efficacy of DTx in treating ADHD. However, none have examined whether DTx has a sustained effect after discontinuation, to our knowledge. Recent studies have focused on the efficacy of digital therapeutics (DTx) when in use, leaving the effects of sustained use largely unexplored (29, 30, 35, 50, 51, 53). Our study aims to address this gap, investigating the possible effects of DTx and the potential for sustained effects even after the participants have stopped using the DTx. This will allow us to gain a comprehensive understanding of the potential effects of DTx and how they might be sustained. We believe that our findings could open up new possibilities for the use of DTx in the future.
However, there are certain limitations to this study. First, this study was conducted with a relatively small sample size; however, it has its value as a feasibility study. Even with a relatively small sample size, an appropriate analysis technique was used to evaluate clinical changes in the participants. Although current study lacks a sham control to compare the effectiveness of DTx, our findings hold some tangible improvements in clinical measures, indicating that current study holds solid evidence over previous case control studies (35, 53). Second, a formal IQ evaluation was not performed during the preliminary evaluation. However, during the screening process, a child-adolescent psychiatrist evaluated intellectual disability. Third, it can be difficult to interpret the results independent of the medication effects. However, this study protocol specifically included only children who maintained the same dosage and were not receiving other medications except ADHD medications, such as antidepressants and anxiolytics. Fourthly, the current study did not gather any background information about the parents. Lastly, because a significant number of self-reported evaluations have been performed to analyze the effect, the placebo effect must be considered. Despite specific challenges, our study aimed to maintain a reasonable level of participant compliance despite the absence of financial incentives.
In conclusion, our findings suggest that there are tangible differences in DTx, such as NUROW, in children with ADHD, particularly in alleviating the clinical symptoms, including inattentiveness and hyperactivity reported by parents. As this study aimed to evaluate the potential efficacy and feasibility of NUROW, the findings provide preliminary evidence for the potential effects of DTx in improving attentional aspects and objective subclinical symptoms in children with ADHD. It is necessary to conduct well-designed research with a large sample size to analyze the effect of DTx.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Institutional Review Board of Chungnam National University Sejong Hospital (Approval Number: 2020-090038-007). Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.
Author contributions
TS, H-JK, and C-HC conceived and designed the study and performed statistical analyzes. TS, JY, H-JL, H-JK, and C-HC wrote the first draft of the manuscript. TS, K-YC, J-LK, H-JK, and C-HC participated in data collection. C-HC has full access to all the data in the study and is responsible for the integrity of the data and the accuracy of the data analysis. All authors edited all versions of the manuscript and were involved in interpreting the results and read, commented on, and approved the final version of the manuscript.
Funding
This work was supported by National Research Foundation (NRF) of Korea grants funded by the Ministry of Science and Information and Communications Technology (MSIT), Government of Korea (NRF-2020R1C1C1007463 and NRF-2021R1A5A8032895), Information and Communications Technology (ICT) and Future Planning for Convergent Research in the Development Program for R&D Convergence over Science and Technology Liberal Arts (NRF-2022M3C1B6080866), and Institute of Information and communications Technology Planning and Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2023-00224823). This work was supported by a grant from Korea University (K2225691).
Acknowledgments
We express our deep gratitude to the children with ADHD and their caregivers, who faithfully participated in the study. We would like to thank Editage (www.editage.co.kr) for English language editing.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2023.1169030/full#supplementary-material
References
1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders fifth edition text revision–DSM-5-TR. Washington, DC: American Psychiatric Association (2022).
2. Bitsko, RH, Claussen, AH, Lichstein, J, Black, LI, Jones, SE, Danielson, ML, et al. Mental health surveillance among children–United States, 2013–2019. MMWR Suppl. (2022) 71:1–42. doi: 10.15585/mmwr.su7102a1
3. Barkley, RA, and Poillion, MJ. Attention deficit hyperactivity disorder: a handbook for diagnosis and treatment. Behav Disord. (1994) 19:150–2. doi: 10.1177/019874299401900205
4. Sonuga-Barke, EJ. The dual pathway model of AD/HD: an elaboration of neuro-developmental characteristics. Neurosci Biobehav Rev. (2003) 27:593–604. doi: 10.1016/j.neubiorev.2003.08.005
5. Daley, D, and Birchwood, J. ADHD and academic performance: why does ADHD impact on academic performance and what can be done to support ADHD children in the classroom? Child Care Health Dev. (2010) 36:455–64. doi: 10.1111/j.1365-2214.2009.01046.x
6. De la Côte-Sainte-Catherine, C. ADHD and comorbid disorders in childhood psychiatric problems, medical problems, learning disorders and developmental coordination. Clin Psychiatry. (2015) 1:1–9. doi: 10.21767/2471-9854.100005
7. Daviss, WB. A review of co-morbid depression in pediatric ADHD: Etiologies, phenomenology, and treatment. J Child Adolesc Psychopharmacol. (2008) 18:565–71. doi: 10.1089/cap.2008.032
8. Hvolby, A. Associations of sleep disturbance with ADHD: implications for treatment. ADHD Atten Defic Hyperact Disord. (2015) 7:1–18. doi: 10.1007/s12402-014-0151-0
9. Jensen, PS, Hinshaw, SP, Kraemer, HC, Lenora, N, Newcorn, JH, Abikoff, HB, et al. ADHD comorbidity findings from the MTA study: comparing comorbid subgroups. J Am Acad Child Adolesc Psychiatry. (2001) 40:147–58. doi: 10.1097/00004583-200102000-00009
10. Leirbakk, MJ, Clench-Aas, J, and Raanaas, RK. ADHD with co-occurring depression/anxiety in children: the relationship with somatic complaints and parental socio-economic position. J Psychol Abnorm. (2015) 4:137. doi: 10.4172/2329-9525.1000137
11. Kim, W-P, Kim, H-J, Pack, SP, Lim, J-H, Cho, C-H, and Lee, H-J. Machine learning–based prediction of attention-deficit/hyperactivity disorder and sleep problems with wearable data in children. JAMA Netw Open. (2023) 6:e233502–2. doi: 10.1001/jamanetworkopen.2023.3502
12. Harpin, V, Mazzone, L, Raynaud, J, Kahle, J, and Hodgkins, P. Long-term outcomes of ADHD: a systematic review of self-esteem and social function. J Atten Disord. (2016) 20:295–305. doi: 10.1177/1087054713486516
13. Sibley, MH, Evans, SW, and Serpell, ZN. Social cognition and interpersonal impairment in young adolescents with ADHD. J Psychopathol Behav Assess. (2010) 32:193–202. doi: 10.1007/s10862-009-9152-2
14. Sibley, MH, Swanson, JM, Arnold, LE, Hechtman, LT, Owens, EB, Stehli, A, et al. Defining ADHD symptom persistence in adulthood: optimizing sensitivity and specificity. J Child Psychol Psychiatry. (2017) 58:655–62. doi: 10.1111/jcpp.12620
15. Moen, ØL, Hedelin, B, and Hall-Lord, ML. Family functioning, psychological distress, and well-being in parents with a child having ADHD. SAGE Open. (2016) 6:26767. doi: 10.1177/2158244015626767
16. Peters, K, and Jackson, D. Mothers’ experiences of parenting a child with attention deficit hyperactivity disorder. J Adv Nurs. (2009) 65:62–71. doi: 10.1111/j.1365-2648.2008.04853.x
17. Cortese, S. Pharmacologic treatment of attention deficit–hyperactivity disorder. N Engl J Med. (2020) 383:1050–6. doi: 10.1056/NEJMra1917069
18. Coghill, D, Banaschewski, T, Cortese, S, Asherson, P, Brandeis, D, Buitelaar, J, et al. The management of ADHD in children and adolescents: bringing evidence to the clinic: perspective from the European ADHD guidelines group (EAGG). Eur Child Adolesc Psychiatry. (2021):1–25. doi: 10.1007/s00787-021-01871-x
19. Faraone, SV, Rostain, AL, Montano, CB, Mason, O, Antshel, KM, and Newcorn, JH. Systematic review: nonmedical use of prescription stimulants: risk factors, outcomes, and risk reduction strategies. J Am Acad Child Adolesc Psychiatry. (2020) 59:100–12. doi: 10.1016/j.jaac.2019.06.012
20. Kollins, SH. Comparing the abuse potential of methylphenidate versus other stimulants: a review of available evidence and relevance to the ADHD patient. J Clin Psychiatry. (2003) 64:14–8.
21. Mucci, F, Carpita, B, Pagni, G, Vecchia, AD, Bjedov, S, Pozza, A, et al. Lifetime evolution of ADHD treatment. J Neural Transm. (2021) 128:1085–98. doi: 10.1007/s00702-021-02336-w
22. Weiss, M, Childress, A, Mattingly, G, Nordbrock, E, Kupper, RJ, and Adjei, AL. Relationship between symptomatic and functional improvement and remission in a treatment response to stimulant trial. J Child Adolesc Psychopharmacol. (2018) 28:521–9. doi: 10.1089/cap.2017.0166
23. Antshel, KM, and Olszewski, AK. Cognitive behavioral therapy for adolescents with ADHD. Child Adolesc Psychiatr Clin. (2014) 23:825–42. doi: 10.1016/j.chc.2014.05.001
24. Goldman, LS, Genel, M, Bezman, RJ, and Slanetz, PJ. Diagnosis and treatment of attention-deficit/hyperactivity disorder in children and adolescents. JAMA. (1998) 279:1100–7. doi: 10.1001/jama.279.14.1100
25. Patel, NA, and Butte, AJ. Characteristics and challenges of the clinical pipeline of digital therapeutics. NPJ Digit Med. (2020) 3:159. doi: 10.1038/s41746-020-00370-8
26. Kim, H-J, Lee, S, Jung, D, Hur, J-W, Lee, H-J, Lee, S, et al. Effectiveness of a participatory and interactive virtual reality intervention in patients with social anxiety disorder: longitudinal questionnaire study. J Med Internet Res. (2020) 22:e23024. doi: 10.2196/23024
27. Cho, C-H, Lee, T, Lee, J-B, Seo, JY, Jee, H-J, Son, S, et al. Effectiveness of a smartphone app with a wearable activity tracker in preventing the recurrence of mood disorders: prospective case-control study. JMIR Ment Health. (2020) 7:e21283. doi: 10.2196/21283
28. Waltz, E. First video game to treat disease gains FDA okay. Nat Biotechnol. (2020) 38:1224–5. doi: 10.1038/s41587-020-0726-6
29. Kollins, SH, DeLoss, DJ, Cañadas, E, Lutz, J, Findling, RL, Keefe, RS, et al. A novel digital intervention for actively reducing severity of paediatric ADHD (STARS-ADHD): a randomised controlled trial. Lancet Digit Health. (2020) 2:e168–78. doi: 10.1016/S2589-7500(20)30017-0
30. Ha, S, Han, JH, Ahn, J, Lee, K, Heo, J, Choi, Y, et al. Pilot study of a mobile application-based intervention to induce changes in neural activity in the frontal region and behaviors in children with attention deficit hyperactivity disorder and/or intellectual disability. J Psychiatr Res. (2022) 146:286–96. doi: 10.1016/j.jpsychires.2021.11.018
31. Gabellone, A, Marzulli, L, Matera, E, Petruzzelli, MG, Margari, A, Giannico, OV, et al. Expectations and concerns about the use of telemedicine for autism Spectrum disorder: a cross-sectional survey of parents and healthcare professionals. J Clin Med. (2022) 11:3294. doi: 10.3390/jcm11123294
32. Bemanalizadeh, M, Yazdi, M, Yaghini, O, and Kelishadi, R. A meta-analysis on the effect of telemedicine on the management of attention deficit and hyperactivity disorder in children and adolescents. J Telemed Telecare. (2021):110451. doi: 10.1177/1357633X211045186
33. Adams, L, Adamo, N, Hollocks, MJ, Valmaggia, L, Brewster, A, Watson, J, et al. Examining clinicians’ concerns delivering telemental health interventions directly to autistic individuals during COVID-19. Res Autism Spectr Disord. (2022) 94:101956. doi: 10.1016/j.rasd.2022.101956
34. Huh, KY, Oh, J, Lee, S, and Yu, K-S. Clinical evaluation of digital therapeutics: present and future. Healthc Inf Res. (2022) 28:188–97. doi: 10.4258/hir.2022.28.3.188
35. Jang, S, Kim, J-J, Kim, S-J, Hong, J, Kim, S, and Kim, E. Mobile app-based chatbot to deliver cognitive behavioral therapy and psychoeducation for adults with attention deficit: a development and feasibility/usability study. Int J Med Inform. (2021) 150:104440. doi: 10.1016/j.ijmedinf.2021.104440
36. Park, DE, Shin, YJ, Park, E, Choi, IA, Song, WY, and Kim, J. Designing a voice-bot to promote better mental health: UX design for digital therapeutics on ADHD patients, extended abstracts of the 2020 CHI conference on human factors in computing systems, pp. 1–8. (2020).
37. Rajeh, A, Amanullah, S, Shivakumar, K, and Cole, J. Interventions in ADHD: a comparative review of stimulant medications and behavioral therapies. Asian J Psychiatr. (2017) 25:131–5. doi: 10.1016/j.ajp.2016.09.005
38. Kim, YS, Cheon, KA, Kim, BN, Chang, SA, Yoo, HJ, Kim, JW, et al. The reliability and validity of kiddie-schedule for affective disorders and schizophrenia-present and lifetime version-Korean version (K-SADS-PL-K). Yonsei Med J. (2004) 45:81–9. doi: 10.3349/ymj.2004.45.1.81
40. DuPaul, GJ. Parent and teacher ratings of ADHD symptoms: psychometric properties in a community-based sample. J Clin Child Psychol. (1991) 20:245–53. doi: 10.1207/s15374424jccp2003_3
41. So, YK, Noh, JS, Kim, YS, Ko, SG, and Koh, YJ. The reliability and validity of Korean parent and teacher ADHD rating scale. J Korean Neuropsychiatr Assoc. (2002) 41:283–9.
42. Achenbach, TM. The child behavior checklist and related instruments In: ME Maruish, editor. The use of psychological testing for treatment planning and outcomes assessment. Mahwah: Lawrence Erlbaum Associates Publishers (1999)
43. Yoo, H-IK, Lee, J-S, Kang, S-H, Park, E-H, Jung, J-S, Kim, B-N, et al. Standardization of the comprehensive attention test for the Korean children and adolescents. J Korean Acad Child Adolesc Psychiatry. (2009) 20:68–75.
44. Seo, J-M, Lee, J-S, Kim, S-Y, and Kim, H-W. Diagnostic significance of comprehensive attention test in children and adolescents with attention-deficit hyperactivity disorder. J Korean Acad Child Adolesc Psychiatry. (2011) 22:246–52. doi: 10.5765/jkacap.2011.22.4.246
45. Stoet, G. PsyToolkit: a novel web-based method for running online questionnaires and reaction-time experiments. Teach Psychol. (2017) 44:24–31. doi: 10.1177/0098628316677643
46. Berch, DB, Krikorian, R, and Huha, EM. The Corsi block-tapping task: methodological and theoretical considerations. Brain Cogn. (1998) 38:317–38. doi: 10.1006/brcg.1998.1039
47. Kingma, A, La Heij, W, Fasotti, L, and Eling, P. Stroop interference and disorders of selective attention. Neuropsychologia. (1996) 34:273–81. doi: 10.1016/0028-3932(95)00101-8
48. Simon, JR. The effects of an irrelevant directional cue on human information processing. Adv Psychol. (1990) 65:31–86. doi: 10.1016/S0166-4115(08)61218-2
50. Gallen, CL, Anguera, JA, Gerdes, MR, Simon, AJ, Cañadas, E, and Marco, EJ. Enhancing neural markers of attention in children with ADHD using a digital therapeutic. PLoS One. (2021) 16:e0261981. doi: 10.1371/journal.pone.0261981
51. Choi, E, Yoon, E-H, and Park, M-H. Game-based digital therapeutics for children and adolescents: their therapeutic effects on mental health problems, the sustainability of the therapeutic effects and the transfer of cognitive functions. Front Psych. (2022) 13:986687. doi: 10.3389/fpsyt.2022.986687
52. Davis, NO, Bower, J, and Kollins, SH. Proof-of-concept study of an at-home, engaging, digital intervention for pediatric ADHD. PLoS One. (2018) 13:e0189749. doi: 10.1371/journal.pone.0189749
53. Oh, S, Choi, J, Han, DH, and Kim, E. Effects of game-based digital therapeutics on attention deficit hyperactivity disorder in children and adolescents as assessed by parents or teachers: a systematic review and meta-analysis. Eur Child Adolesc Psychiatry. (2023) 2023:1–13. doi: 10.1007/s00787-023-02174-z
54. Reimherr, FW, Marchant, BK, Gift, TE, and Steans, TA. ADHD and anxiety: clinical significance and treatment implications. Curr Psychiatry Rep. (2017) 19:1–10. doi: 10.1007/s11920-017-0859-6
55. Read, KL, Settipani, CA, Peterman, J, Kendall, PC, Compton, S, Piacentini, J, et al. Predicting anxiety diagnoses and severity with the CBCL-A: improvement relative to other CBCL scales? J Psychopathol Behav Assess. (2015) 37:100–11. doi: 10.1007/s10862-014-9439-9
56. Bartels, C, Wegrzyn, M, Wiedl, A, Ackermann, V, and Ehrenreich, H. Practice effects in healthy adults: a longitudinal study on frequent repetitive cognitive testing. BMC Neurosci. (2010) 11:1–12. doi: 10.1186/1471-2202-11-118
57. Calamia, M, Markon, K, and Tranel, D. Scoring higher the second time around: meta-analyses of practice effects in neuropsychological assessment. Clin Neuropsychol. (2012) 26:543–70. doi: 10.1080/13854046.2012.680913
Keywords: attention-deficit/hyperactivity disorder, executive function, potential effectiveness, feasibility, digital therapeutics, adjunctive treatment, clinical symptoms
Citation: Sun TH, Yeom JW, Choi K-Y, Kim J-L, Lee H-J, Kim H-J and Cho C-H (2023) Potential effectiveness of digital therapeutics specialized in executive functions as adjunctive treatment for clinical symptoms of attention-deficit/hyperactivity disorder: a feasibility study. Front. Psychiatry. 14:1169030. doi: 10.3389/fpsyt.2023.1169030
Edited by:
Heleen Riper, VU Amsterdam, NetherlandsReviewed by:
Emilia Matera, University of Bari Aldo Moro, ItalyKounseok Lee, Hanyang University Seoul Hospital, Republic of Korea
Hong Jun Jeon, Konkuk University Medical Center, Republic of Korea
Jae-Hon Lee, Western University, Canada
Copyright © 2023 Sun, Yeom, Choi, Kim, Lee, Kim and Cho. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Chul-Hyun Cho, david0203@korea.ac.kr; david0203@gmail.com