Evaluating attention deficit hyperactivity disorder symptoms in children and adolescents through tracked head movements in a virtual reality classroom: The effect of social cues with different sensory modalities

Introduction

Attention deficit hyperactivity disorder (ADHD) is a common neurodevelopmental disorder characterized by inattention, hyperactivity, and impulsivity, observed in multiple daily environments (American Psychiatric Association, 2013). ADHD is associated with impairments in academic and social functioning, and if not treated properly, can be accompanied by a range of comorbid psychiatric disorders (Biederman et al., 1991; Biederman et al., 2004; Spencer et al., 2007). The current gold standard for the clinical assessment of children and adolescents with ADHD is mainly based on subjective reports from the patients themselves or their caregivers and teachers about their behavior at home and school. Such reports may be inaccurate or biased, and a lack of understanding of ADHD students’ behavior in a classroom environment may lead to more inaccurate evaluations. Thus, the lack of an objective means of evaluating ADHD is a significant drawback.

Efforts have been made to incorporate objective neuropsychological tests, such as the continuous performance test (CPT), to evaluate sustained attention (Corkum and Siegel, 1993; Hall et al., 2016; Bloch et al., 2017). In particular, virtual reality CPT (VR-CPT) has been widely used in the field of ADHD for the assessment, treatment, and rehabilitation of patients (Pollak et al., 2010; Parsons et al., 2017). Previous studies have reported that VR tasks used for patients with ADHD better reflect the participants’ performance in daily life compared to traditional computerized neuropsychological tests, suggesting the potential use of VR technology for the objective assessment of attentional symptoms (Rizzo et al., 1999; Rizzo et al., 2004; Rizzo et al., 2006; Parsons, 2015; Bashiri et al., 2017). However, prior attempts to assess ADHD symptoms based on CPTs have mostly focused on the attention and impulsivity aspects of the disorder, with limited assessment of the hyperactivity and motor behavior components (Teicher et al., 2008; Ohashi et al., 2010).

The US FDA has started to focus on real-world data from novel digital sources/monitoring and neurobehavioral testing, suggesting a trend toward emphasizing the role of technology in assisting and augmenting clinical care (Keshav et al., 2019). Furthermore, the development of technologies that can quantify behavior provides us with a means to use them to retrieve patient-generated data in VR settings. In this study, we utilized a head-mounted display (HMD) to automatically track and record real-time body movements of participants in the virtual world. Several studies have used head movement to represent hyperactivity in individuals with ADHD (Hong et al., 2021). Previous research has shown that head movement toward target stimuli is an especially important indicator of attentional orientation, suggesting that further research on this is warranted (Teicher et al., 1996; Lavine et al., 2002; Seyama, 2006; Frischen et al., 2007). Our approach includes these objective locomotor measures provided by the HMD, allowing a more comprehensive approach to the assessment of ADHD compared to most CPT-based studies. Although capturing the movement itself is not clinically valid or useful, it certainly opens new fields for the study and validation of this approach.

While the coronavirus disease (COVID-19) pandemic has triggered research on the behavior of students in virtual environments for the general pediatric population, there have been few studies on children and adolescents with ADHD. This field of study is critical in the current situation, in which a large proportion of education is being replaced with remote online learning. It is important to understand how a teacher giving instructions on the screen influences attention performance in both individuals with and without ADHD and how they differ in an online classroom environment. VR technology has the advantage that it can be used to construct desired environments, including various stimuli that may influence attention performance. Thus, it provides the participant with a more ecologically valid environment for assessment compared to the conventional CPT conducted in a quiet, isolated room. In addition, these environments and stimuli can be easily manipulated and fully controlled by the experimenter, allowing assessment of the impact of each stimulus on patients’ behavior (Hong et al., 2021).

Generally, children are required to utilize their attention in the presence of various environmental stimuli and interactions with teachers. This study focused on how specific features of a VR program may affect the attention performance of patients with ADHD. For example, the wide variety of contexts that can be simulated within VR allows us to compare the effects of different degrees of social cues or stimuli of different sensory modalities in a differential manner. We implemented the social component by adding a teacher avatar that could move and vocalize to mediate the attention of participants in the VR classroom setting. Previous studies have shown that gestures or gaze cues from a teacher in a VR classroom environment can evoke attention shifts, leading to similar motor responses in the student, such as gaze and head movements (Mansfield et al., 2003; Bayliss et al., 2004; Seyama, 2006). Attention has also been shown to improve in the presence of social cues (e.g., different social distances, gestures, and verbal instructions) from the avatar teacher (Eom et al., 2019). These results suggest that social factors may significantly influence attention. If social cues are found to have effect of the attention deficit of ADHD individuals, training involving enhancement in perceiving social stimuli could be implemented to help individuals to perform with better attention in real-life situations.

To implement stimuli with multiple sensory modalities, we created several conditions for the participants to perform the evaluation, each with a stepwise increase in the presentation of sensory modalities. Prior studies suggest that the hyperactivity domain is associated with the sensory modality of the stimulus (e.g., visual or auditory) and that it may be differentially affected in each modality in individuals with ADHD (Schmidt et al., 2019). Investigation of the effect of multisensory stimuli in the sustained attention of ADHD individuals may provide direction for designing new intervention content specifically tailored and adjusted to be more efficient for students with ADHD. For example, if ADHD individuals are found to function better in the presence of multisensory cues in assisting attention, virtual or online tools for their education may be designed to integrate this aspect. On the other hand, if they are found to have difficulties in utilizing cues of multiple sensory modalities, it may be more beneficial to help ADHD students concentrate by using simple cues from a single sensory modality.

Based on this background, head-movement data based on a head-tracking unit on an HMD device were collected by our VR system, and its potential as an objective index of hyperactivity was examined. We also investigated how specific features in a VR classroom environment influence the attention of children and adolescents. While most previous studies on VR classrooms have focused on an individual’s regulation, control, and management of cognitive processes, we created a VR environment in which a virtual teacher alerts the participants toward the stimuli of the VR-CPT via visual (gesturing) and/or verbal instructions. In this proof-of-concept study, we hypothesized the following: First, head movement will show a significant correlation with the current standard clinical symptom measurements, and thus, would be a valid digital phenotype that may represent ADHD severity objectively. Second, differently simulated social or sensory cues from a teacher avatar will affect head movement in both ADHD and healthy control (HC) participants in the VR classroom environment. Finally, children with and without ADHD will exhibit different patterns in response to various cues simulated in our VR setting. We believe that if particular cognitive or behavioral profiles under different stimuli conditions in ADHD and HC groups can be established through our study, our VR system may serve as a useful tool for clinicians to understand the characteristics of patients with ADHD in greater depth in terms of their responses to environmental stimuli.

Materials and methods

Participants and procedures

Twenty children and adolescents with ADHD and 17 HCs participated in this study. The inclusion criteria were as follows: (1) ADHD diagnosis according to the DSM-V criteria, (2) age of 6–17 years, and (3) allowance of common comorbid disorders, such as depressive disorder, anxiety disorder, and oppositional defiant disorder. The exclusion criteria were as follows: (1) an intelligence quotient (IQ) < 70, (2) past or current serious medical or neurological diseases, and (3) comorbid autism spectrum disorder, psychotic spectrum disorder, or bipolar disorder.

All participants with ADHD were taking medications, such as stimulants for ADHD symptoms, aripiprazole for irritability, and/or escitalopram for depression or anxiety. As methylphenidates can affect the performance of attentional tasks, those taking methylphenidates were required to stop medication for at least 48 h before participating in the study. Psychiatric interviews with HCs were also conducted to rule out any past or current medical and/or psychiatric disorders, and psychotropic medication usage.

All participants underwent a practice session to familiarize themselves with the VR-CPT. Following the practice session, the participants were presented with the four VR conditions for the task. The order of the VR-CPT conditions was counterbalanced to control for systematic order effects. After going through all four VR conditions, the participants filled out a debriefing questionnaire regarding their VR experience. This study was approved by the Institutional Review Board of Yonsei University College of Medicine, Gangnam Severance Hospital, and informed consent was obtained from all participants and their guardians prior to participation.

Materials

Diagnostic procedures and assessment of clinical symptoms

The diagnosis of ADHD and comorbid disorders was established based on the Mini-International Neuropsychiatric Interview for Children and Adolescents (Version 6.0; Sheehan et al., 1998; Sheehan et al., 2010) for both ADHD and HC participants. IQs were obtained using the short form of the Korean version of the Wechsler Intelligence Scale for Children, Third Edition (K-WISC III; Kwak et al., 2002).

Attention deficit hyperactivity disorder rating scale–IV, Korean Version (ADHD-RS-IV)

The severity of ADHD symptoms was assessed using the Korean version of the ADHD Rating Scale (ADHD-RS) IV (So et al., 2002). This 18-item Likert scale was filled out by parents, who rated the symptom criteria of their children with ADHD on a severity scale ranging from 0 (not present) to 3 (severe). Higher scores on the ADHD-RS indicate higher severity of symptoms, such as inattention, impulsivity, and hyperactivity.

Korean child behavior checklist

The child behavior checklist (CBCL) is an assessment tool completed by parents to measure the behavioral problems of children. The Korean CBCL (K-CBCL) is a standardized and validated instrument used to evaluate the behavior of children aged 6–18 years [Cronbach’s α = 0.62–0.95, (Oh et al., 1997; Achenbach and Ruffle, 2000; Lee et al., 2015)]. The K-CBCL comprises 119 items, which provide a raw score and normalized T-score based on sex and age (M = 50, SD = 10), with higher T-scores indicating more problematic behavior.

Virtual reality continuous performance test

We developed a system that captures and quantifies the motor activity of participants while they are engaged in a CPT task. The virtual reality continuous performance test (VR-CPT) utilized was based on the two-button AX-CPT version of the CPT (Eom et al., 2019), which lasted 15 min. CPT attention measures and the degree of head motion of each participant were automatically recorded and calculated using a VR system. In this task, the participants were required to press the right arrow on the keyboard as fast as they could upon the appearance of an X after an A. In response to stimuli presented otherwise, the participants were required to press the left arrow key. A total of 24 target stimuli, each with a duration of 100 ms, were presented. Interstimulus intervals varied from 1,500 to 2,500 ms.

Task conditions

We constructed four experimental conditions for the VR-CPT, each providing social cues of different degrees preceding the appearance of the to-be-detected stimuli in a virtual classroom environment so that the cues may direct the participants’ attention toward the stimuli beforehand (Figure 1). In the control condition, participants were instructed to complete the two-button AX-CPT, with letters appearing on a white screen in the center, without a virtual classroom environment or a virtual teacher present. This task is essentially equivalent to the traditional computerized CPT, with the exception of wearing an HMD for the baseline measurement of head movements. In the other three VR conditions, the participants were presented with an image of the upper body and face of a male teacher avatar in a virtual classroom setting and asked to complete the CPTs that appeared on the screen to the left or right of the avatar teacher. The three VR conditions differed in how the participants were directed toward the screens to the right or left of the teacher avatar. In the first VR condition (no-cue condition), participants conducted the task without any cues from the virtual teacher; this condition was implemented to determine the effects of social cues from the presence of the avatar teacher alone. Half of the trials were presented on the left screen and the remaining half on the right screen so that the user would have to turn their head once between the two screens. Whether the stimulus was first presented on the right or left screen was counterbalanced across participants according to the program’s protocol.

FIGURE 1

Figure 1. Virtual reality (VR) classroom environment. VR classroom condition presented to the participants. (A) No-cue condition, in which the teacher avatar was presented in the center of the visual field without presenting any social cues. (B) The visual cue (hand gesture) provided by the teacher avatar in the visual cue condition and the visual/audio cue condition. The images are presented in chronological order; the teacher avatar gestures toward one screen, then the stimulus appears on the pointed screen.

In the second VR condition (visual cue condition), the teacher avatar chose the left or right screen and directed the participants to look at the screen by pointing. In the third VR condition (visual/audio cue condition), the teacher avatar both pointed and provided a verbal (audio) cue, saying “Look at me,” when he wanted the participant to attend to him (i.e., in between stimuli), and “Look at this,” as he pointed to each screen. The selected screens were changed eleven times under these two conditions. The number of trials appearing on each side of the screen was equal and presented in a counterbalanced order across participants, according to the program’s protocol. The other task variables were equally controlled for throughout all four conditions.

Dependent measures

Head movement data

Head telemetry was measured in degrees (°) of movement in three rotation axes – yaw (Y, horizontal, head turns), pitch (P, vertical, head nods), and roll (R, tilt, lateral head inclinations), and movement was calculated by subtracting the current frame values of every 1/60 s from the previous frame values (i.e., Y_t−Y_t−1, P_t−P_t−1,R_t−R_t−1). The participants needed to turn from screen to screen 11 times in the visual and visual/audio social cue conditions, but required only a single head turn in the no-cue condition, whereas the control condition required no head turns. To eliminate this variability, we marked all timestamps in a log file and excluded yaw movements that occurred at the timestamp of the stimuli presented. The following calculations were performed using these values:

The absolute values of movement (△Y,△P,△R) were indexed by dividing each by the total head movements of that time frame (THM_t), defined as the sum of the absolute values of movement for all three axes (e.g., THM_t = △Y_t + △P_t + △R_t). The sum of these values was defined as the total magnitude of head movement in each axis during each VR session.

$Yaw\left(Y\right)=\sum \frac{\mathrm{△}{Y}_t}{THM}$

$Pitch\left(P\right)=\sum \frac{\mathrm{△}{P}_t}{THM}$

$Roll\left(R\right)=\sum \frac{\mathrm{△}{R}_t}{THM}$

In addition, considering that only horizontal movement would be needed to direct one’s point of view from one screen to another, the sum of the vertical and tilt movements in each time frame was calculated to represent the task-irrelevant head movement of the time frame and again summed up as the task-irrelevant head movement during the entire task.

$Task-irrelevantheadmovement=\sum \frac{\mathrm{△}{P}_t+\mathrm{△}{R}_t}{THM}$

The VR system for the VR-CPT consisted of a desktop computer with the Microsoft Windows 7 operating system, a high-end graphics card (NVIDIA GeForce GTX 970) with 16GB memory, a monitor, keyboard, speaker, and an Oculus Rift Stereo-HMD with a tracker (DK2 HMD; Oculus lift, Oculus VR, Irvine, CA), and a screen resolution of 960 × 1080 pixels per eye. The VR-CPT was designed using a commercial three-dimensional development platform (Vizard 5.1; WorldViz, Santa Barbara, CA, United States).

Statistical analysis

For ADHD-RS and CBCL, data with values higher than 1.5 interquartile ranges above the 3rd quartile or lower than 1.5 interquartile ranges below the 1st quartile were excluded as outliers. One participant from the ADHD group had abnormally measured head movement data and was excluded in analyses regarding the data. Independent sample t-tests and chi-square tests were used to compare the ADHD and HC groups in terms of demographics, clinical characteristics, and attention variables. Because the ADHD-RS total score and CBCL attention problems subscale did not follow normal distribution when appropriate outliers were excluded, a Spearman’s r bivariate correlation test was conducted to analyze the associations between head movement data and ADHD symptom severity. Most head movement data satisfied normality via the Shapiro-Wilk test and the Q-Q plots were close to the line, and thus linear mixed models were constructed to compare differences in task-irrelevant head movements according to the four conditions, both within each group and between the groups. Post-hoc analyses were corrected using the false discovery rate (FDR) correction method. SPSS for Windows (version 23.0; SPSS, Chicago, IL, United States) was used for the t-tests, chi-square tests, and the correlation analysis and the SAS version 9.4 (SAS Institute, Cary, NC, United States) was used for the development of linear mixed models. All analyses were considered statistically significant if two-tailed p < 0.05.

Results

Participant demographics and clinical characteristics

The demographic and clinical characteristics of the ADHD and HC participants were compared using independent t-tests, as shown in Table 1. For accurate comparison, outliers were excluded from analyses of simulator sickness questionnaire results, ADHD-RS scores, and CBCL attention problem scores, as few of the participants showed high scores in the extreme for these categories. There were no significant differences in the mean age, IQ, or sex ratio between the two groups. As expected, the two groups had statistically significant differences in ADHD-RS scores [t (35) = 4.56, p < 0.001] and CBCL attention problem subscale scores [t (34) = 2.18, p = 0.016].

TABLE 1

Table 1. Demographic and clinical characteristics.

Correlation analysis between head movements during VR-CPT and ADHD symptom severity

Spearman’s correlation tests were used to examine the association between head movement measurements and scores from the ADHD questionnaire results (Table 2). Head movement data were averaged across all four conditions. One participant’s ADHD-RS total score and two participant’s CBCL subscale score was emitted from the ADHD subgroup due to abnormally high results. One additional participant was excluded from the ADHD group because of abnormally measured head movement data. In the ADHD group, there were significant correlations between average yaw (i.e., the average of relatively task-relevant head movement, r = –0.492, p = 0.038), average task-irrelevant head movement (r = 0.489, p = 0.040), and the summary index of ADHD-RS in the ADHD group (Figure 2), while no such correlation was observed in the HC group. No significant correlations were found with the CBCL attention problems subscale for both groups.

TABLE 2

Table 2. Correlation analyses between head movement measurements averaged across all conditions, ADHD-RS total score.

FIGURE 2

Figure 2. Correlation analyses in the ADHD group between head movement measurements and the ADHD-RS total score. (A) Correlation with average pitch, (B) correlation with average yaw, (C) correlation with average roll, (D) correlation with average task-irrelevant movement. ADHD-RS total scores of the ADHD group had statistically significant correlations with yaw movement (i.e., task-relevant head movement) and task-irrelevant head movement averaged across all four conditions. Linear fit lines are provided for results with p < 0.05.

Comparison of head movement in the setting of different social cue conditions

To investigate whether social cues presented from a teacher avatar affect the head movement and task performance of participants in a differential manner during the VR-CPT, linear mixed models were constructed for each group with each VR condition as the condition factor and task-irrelevant head movement as dependent variables. One participant from the ADHD group was excluded because of abnormally measured head movement data (ADHD group, n = 19; HC group, n = 17). Chronological age and IQ were included in the analyses as covariates to disentangle their effects on the participants’ performance. Age was specifically included as a covariant because a strong correlation was found between age and task-irrelevant movement in the ADHD group.

We found a significant main effect of condition (F₃₄ = 0.10, p < 0.001) and group-by-condition interactions (F₃₄ = 223.84, p = 0.031), but no significant effect of group (F₃₄ = 3.33, p = 0.752). Post-hoc analyses with FDR correction are shown in detail in Table 3. There were significant differences between all conditions (all p < 0.001, except between the visual and visual/audio cue conditions in the HC group, p = 0.016), apart from showing no significant difference between the visual and visual/audio cue conditions in the ADHD group (p = 0.615). When corrected for multiple analyses, there was no significant difference observed in the post-hoc analysis of group-by-condition interactions (Table 4).

TABLE 3

Table 3. Post-hoc analysis results for the linear mixed model on the main effect of condition.

TABLE 4

Table 4. Post-hoc analysis and Bayesian factor computation results for the post-hoc analyses on the group-by-condition interactions of the linear mixed model.

Bayesian factors were computed for each insignificant result. For the main effect of condition between the visual and visual/audio cue conditions in the ADHD group, the computed Bayesian factor was 0.321, indicating presence of moderate evidence supporting the null hypothesis (i.e., there is no difference in task-irrelevant head movement between the two conditions in the ADHD group). For group-by-condition interactions, the Bayesian factors are presented in Table 4. Most interactions had anecdotal evidence for either the null or the alternative hypothesis, indicating that results are largely insignificant.

Therefore, in terms of the overall trend, there was a decreasing trend in task-irrelevant head movement in the presence of the virtual classroom environment and again with visual and/or audio cues added (Figure 3). However, the magnitude of difference in task-irrelevant movement according to changes of condition was not different in the two groups.

FIGURE 3

Figure 3. Comparison of task-irrelevant head movement between conditions by group. The main effects of the condition and group-by-condition interactions were statistically significant, while the main effects of group were not. According to the post-hoc analyses performed with FDR correction, the healthy control (HC) group showed statistically significant differences between all conditions; however, the ADHD group did not show a significant difference of task-irrelevant head movement between the visual and visual/audio cue conditions.

Discussion

Our study examined the impact of different contexts simulated within a VR classroom on the head movements of children and adolescents with and without ADHD. Task-irrelevant head movements in the ADHD group showed a significant correlation with the current standard ADHD clinical scale. In addition, by utilizing digital technology to strategically alter various components, the participants were exposed to different sensory modalities of social cues. The head movement of children with ADHD seemed to decrease in response to a virtual avatar teacher and their social cues compared to when they were absent. While visual cues seemed to positively influence the attention of both ADHD and HC individuals, the effect of an additional audio cue is debatable. Overall, these results imply that virtual environments provide researchers with a platform on which rigorous experimental control and manipulation can be applied while simultaneously enabling the collection of objective behavioral data. Our results also demonstrate that the alteration of physical or social components in VR may be used as an effective method for optimizing students’ attention.

Relationship between head movement during virtual reality-continuous performance test and attention deficit hyperactivity disorder symptom severity

Our results showed a significant positive correlation between task-irrelevant head movement during the VR-CPT with visual/audio social cues and ADHD-RS scores, the most commonly used standard scale to measure ADHD symptoms in clinical practice. This result supports our first hypothesis that head movement data in a virtual environment can effectively represent the degree of ADHD symptoms, reflecting real-life performance reported via ADHD-RS scales. This correlation was not observed in the HC group, possibly implying the specificity of this mode of measurement in the ADHD group. However, these results should be interpreted with our small sample size in consideration. Future replicative studies with a larger sample size would be needed to solidify our findings.

Discovering and defining objective means to quantify the symptoms of ADHD is important, as it may compensate for the lack of objectivity in the current method, which relies on self-or parent-reported ADHD symptom scales. This finding adds to the small but growing body of literature utilizing automated measurement of body and head movement as one such method. However, in our study, no significant differences were observed in task-irrelevant movements between the two groups. This result is inconsistent with previously published meta-analytic results, indicating a higher level of movement in children with ADHD (Parsons et al., 2007; Diaz-Orueta et al., 2014; Nolin et al., 2016). The reason for these inconsistent results is unclear, but may be because VR generally enhances the attention level of children with ADHD through motivation and entertainment (Rizzo et al., 2004; Dovis et al., 2012; Larson et al., 2014), and the inclusion or exclusion of distractions causes or diminishes performance differences between ADHD and HC participants (Mansfield et al., 2003; Rizzo et al., 2004; Rizzo et al., 2006; Mazurek and Engelhardt, 2013; Eom et al., 2019). Our VR setting was relatively simple without significant distractions, in addition to an HMD that blocked distractions from the outside real world (Rizzo et al., 2006; Bailenson et al., 2008), and thus, may have been an ideal environment for ADHD participants to concentrate on the task more easily. The different objective measures used in the assessment of hyperactivity may, in part, explain the inconsistency of the results among studies. Perhaps more sophisticated activity indicators will be more effective in discriminating between the two groups. Other objective locomotion measures, such as actigraphy and infrared motion tracking systems with computerized attention testing, have been shown to be useful for evaluating inattention and hyperactivity/impulsivity symptoms in children and adults with ADHD (Teicher et al., 1996; García Murillo et al., 2015; Hall et al., 2015; Lee et al., 2018). Regardless, our setting of the VR environment was sufficient to analyze the patterns of head movement in both groups, reflecting their real-life performance reported via the ADHD-RS scales.

In addition, the VR classroom environment itself seemed to decrease the hyperactivity of ADHD and HC participants, as reflected by the decrease in the task-irrelevant head movement of participants during the no-cue condition compared to the control condition. VR-CPT has been seen to captivate the attention of children with ADHD more effectively than traditional CPTs in many studies (Rizzo et al., 2004; Dovis et al., 2012; Mazurek and Engelhardt, 2013), as well as having a motivating effect on children (Rizzo et al., 2004; Dovis et al., 2012; Larson et al., 2014). Participants with ADHD often report VR-CPTs as being more engageable than the conventional 2D computerized version (Dovis et al., 2012; Larson et al., 2014). Previous studies have also implied that students can learn more by being in the center of a teacher’s field of view rather than in the periphery by being closer to the teacher (Bailenson et al., 2008). Because our VR setting placed the teacher avatar directly in front of the participant’s point of focus, it may have generated the effect of being in the place of a student sitting in the middle of the front row of a classroom, in the proximity of the teacher. This finding suggests that VR environments have the potential to alter the physical dynamics of learning environments and can be utilized to better treat or educate patients with ADHD.

Effect of the teacher avatar’s cues on the attention of attention deficit hyperactivity disorder and healthy control participants

We explored whether head movement varied depending on the addition of visual cues only and both visual and audio cues provided by a teacher avatar and found that head movement could be affected by these social cues in a virtual classroom environment. In this study, we found no significant group differences in task-relevant or task-irrelevant head movement between children with ADHD and HCs; however, significant differences across conditions were found, which we attributed to the different VR conditions.

Regardless of ADHD diagnosis, previous studies have revealed that social cues tend to improve students’ attention performance. For example, in a classroom environment, the synchronous behavior of the teacher and students can increase rapport and result in better learning outcomes (Zhou, 2012; Gaastra et al., 2016). Coherent results were found in a study investigating the movements of teachers and students using machine learning algorithms, which showed that synchronous nonverbal behavior led to increased attention and higher learning scores (Won et al., 2014). Taken together, the cues of the teacher avatar in our VR classroom environment may have contributed to the attention level of participants by promoting them to carry out head movements synchronous to the teacher’s guidance provided by gestural and vocal cues.

Group-by-condition interaction for the pairs of conditions were not statistically significant, against our third hypothesis that the two groups would respond differently to the two conditions. However, as we have stated in section 4.1, significant correlation between task-irrelevant head movement and ADHD-RS was only observed in the ADHD group, and not in the HC group, possibly indicating that task-irrelevant head movement may represent clinically significant hyperactivity/inattention in those with ADHD diagnosis only. Therefore, it may be questionable to say that this represents clinically significantly increased attention for the HC group, and the interpretation of the group-by-condition interaction should be done with care. Here, we focused on the difference in task-irrelevant head movement by condition in the ADHD group and propose some theories based on previous studies that may provide a reasonable explanation for why there was no significant decrease in the amount of task-irrelevant head movement when audio cues were added to the visual cue condition in the ADHD group.

First, it is possible that the results were influenced by the poor multisensory integration ability of the ADHD participants. While synchronous auditory stimuli may act synergistically with visual stimuli by increasing connectivity between the low-level visual and auditory areas of the brain (Romei et al., 2009; Lewis and Noppeney, 2010; Beer et al., 2011), the synchronicity itself is based on the integrity of the multisensory integration process influenced by an individual’s attention level. Children and adolescents with ADHD who are attention-deficient by definition may have difficulties in integrating multisensory stimuli (i.e., visual and audio cues simultaneously) to search for stimuli requiring attentional focus (Romei et al., 2009; Tang et al., 2016).

Our second theory is based on the use of top-down (i.e., endogenous) and bottom-up (i.e., exogenous) orientation in head movement (Frischen et al., 2007). Bottom-up orientation refers to external stimuli in one direction leading to an unconscious draw of attention toward that direction. In contrast, top-down orientation is a goal-driven process that is triggered by one’s selection to focus on a certain stimulus. While the teacher’s raised forearm in one direction likely triggered some bottom-up orientation, the audio (verbal) cue in our study was provided by the teacher in the center of the two screens, lacking any localization effect. This may have driven the receiver to interpret what the sound means and determine focus. Thus, the results of our study imply that children with ADHD effectively utilize bottom-up orientation but cannot utilize top-down orientation at the same time to improve their focus, consistent with reports from previous studies (Egeland et al., 2010; Petrovic and Castellanos, 2016).

Lastly, while it was our intention for both visual and visual/audio cues to be perceived as social cues, it is possible that the visual cue may have acted as a simple “pointer” and hence had partial properties as a non-social cue (Frischen et al., 2007; Seernani et al., 2021), unlike the verbal cues given from the center of the visual field. In the presentation of valid and invalid cues prior to stimuli in spatial cueing paradigms, children with ADHD have been observed to be able to use non-social cues, but not social cues, to orient their attention (Marotta et al., 2014). The disproportionately impaired orienting ability of children with ADHD may explain how the presence of only visual cues could cause significant changes in task-irrelevant head movements.

To conclude, it is unclear within our study which theory provides the best explanation for the absence of head movement difference between the visual vs visual/audio cue conditions in the ADHD group. Revealing the actual reason behind our results and clarifying whether the two groups are different in each condition would require additional investigation, including assessment of other body movements (Teicher et al., 1996; García Murillo et al., 2015; Hall et al., 2015; Lee et al., 2018) and/or addressing whether the presence of the two modalities together would have a larger impact on the attention performance and body movements of children with ADHD compared to each single modality (visual or audio).

Limitations

This study has several limitations. First, our VR was designed to be relatively simple, and may not demonstrate ADHD-related differences to the same extent as in real-life situations. More work is needed to better understand how attention performance is influenced in these simulated situations compared to an individual’s behavior in real life.

Second, the absence of distractions, which inevitably exist in a real classroom environment, is an important limitation. Deeper insight into the underlying mechanism of how non-social and social stimuli affect the attention-orienting process of ADHD may be obtainable if a more sophisticated distractor setting is created. Third, post-hoc power analysis for our study revealed limited power, implying that our sample size may not have been large enough to provide substantial experimental power. Also, the male-to-female ratio was high, which may be a limitation in generalizing our results to female patients. Fourth, even though participants in the ADHD group were asked to withhold methylphenidate 48 h prior to the study, its effect cannot be completely ruled out. Although it would have been preferable to require children to participate after a longer period of medication abstinence, this may have exposed our study participants to an unnecessary burden. Additionally, the comorbidities included in our inclusion criteria may have influenced the results. Fifth, while previous exposure to video games and VR (Maneuvrier et al., 2020), or self-representation of oneself as an avatar (Friehs et al., 2022) may influence CPT performance within VR environments, we did not control for these factors in our study.

Although the use of VR improves the ecological validity of attention task performance, our study setting was still based on the traditional CPT format, making it difficult to represent real-life attentional capacity (Parsons, 2015). Further research with a novel task that better reflects real-life classroom environments is necessary to overcome this limitation. Another useful future investigation would be the analysis of eye movement using an eye-tracking device, given that participants may have moved their eyes without moving their heads to carry out the CPT task (Lev et al., 2020). However, the use of head-mounted eye trackers may pose practical difficulties, including low tolerance of children to these devices, which may be even more troublesome for individuals with ADHD. Finally, pitch, yaw, and roll were moderately correlated in our data, although they were examined separately in our study. A possible alternative approach would be to include body movements to reflect hyperactivity more precisely.

Conclusion

We quantified head movements in children and adolescents with and without ADHD using automated measurement, which showed different tendencies depending on the presence or absence of social and non-social stimuli. These findings suggest the possibility of using automated head motion measurement in a VR environment as a reliable objective method to quantify the hyperactivity component of ADHD symptoms to supplement the current subjective manner of ADHD symptom measurement. This proof-of-concept study not only provides a new, relatively ecologically valid way of conducting studies aimed at understanding how various cues affect the attention of children in a classroom environment, but may also improve the understanding of nonverbal behavior in ADHD and assist with designing online educational systems for those affected. Further research may reveal a more extensive potential of VR technology in identifying and longitudinally monitoring ADHD symptoms, as well as providing a clinical aid as a digital phenotype to stratify severity.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Institutional Review Board of Yonsei University College of Medicine, Gangnam Severance Hospital. Written informed consent to participate in this study was provided by the participants or their legal guardian/next of kin.

Author contributions

EK and KK designed this study. HE and YH collected the data. JYY, YJC, and HSL performed statistical analyses. JYY and YJC wrote the manuscript. EK, JYY, JK, HSL, and BS contributed to the discussion and reviewed and edited the manuscript. All authors have read and approved the final version of the manuscript.

Funding

This project was funded by the Korean Neuropsychiatric Association’s Jeon Sangbae Child and Adolescent Psychiatry Research Fund Award of 2016.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Achenbach, T. M., and Ruffle, T. M. (2000). The Child Behavior Checklist and related forms for assessing behavioral/emotional problems and competencies. Pediatr. Rev. 21, 265–271. doi: 10.1542/pir.21-8-265

PubMed Abstract | CrossRef Full Text | Google Scholar

American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders. Washington, DC: American Psychiatric Association.

Google Scholar

Bailenson, J. N., Yee, N., Blascovich, J., Beall, A. C., Lundblad, N., and Jin, M. (2008). The use of immersive virtual reality in the learning sciences: digital transformations of teachers, students, and social context. J. Learn. Sci. 17, 102–141. doi: 10.1080/10508400701793141

CrossRef Full Text | Google Scholar

Bashiri, A., Ghazisaeedi, M., and Shahmoradi, L. (2017). The opportunities of virtual reality in the rehabilitation of children with attention deficit hyperactivity disorder: a literature review. Korean J. Pediatr. 60, 337–343.

Google Scholar

Bayliss, A. P., di Pellegrino, G., and Tipper, S. P. (2004). Orienting of attention via observed eye gaze is head-centred. Cognition 94, B1–B10. doi: 10.1016/j.cognition.2004.05.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Beer, A. L., Plank, T., and Greenlee, M. W. (2011). Diffusion tensor imaging shows white matter tracts between human auditory and visual cortex. Exp. Brain Res. 213, 299–308.

Google Scholar

Biederman, J., Monuteaux, M. C., Doyle, A. E., Seidman, L. J., Wilens, T. E., Ferrero, F., et al. (2004). Impact of executive function deficits and attention-deficit/hyperactivity disorder (ADHD) on academic outcomes in children. J. Consult. Clin. Psychol. 72, 757–766.

Google Scholar

Biederman, J., Newcorn, J., and Sprich, S. (1991). Comorbidity of attention deficit hyperactivity disorder with conduct, depressive, anxiety, and other disorders. Am. J. Psychiatry 148, 564–577.

Google Scholar

Bloch, Y., Aviram, S., Segev, A., Nitzan, U., Levkovitz, Y., Braw, Y., et al. (2017). Methylphenidate reduces state anxiety during a continuous performance test that distinguishes adult ADHD patients from controls. J. Atten. Disord. 21, 46–51. doi: 10.1177/1087054712474949

PubMed Abstract | CrossRef Full Text | Google Scholar

Corkum, P. V., and Siegel, L. S. (1993). Is the continuous performance task a valuable research tool for use with children with attention-deficit-hyperactivity disorder? J. Child Psychol. Psychiatry 34, 1217–1239.

Google Scholar

Diaz-Orueta, U., Garcia-Lopez, C., Crespo-Eguilaz, N., Sanchez-Carpintero, R., Climent, G., and Narbona, J. (2014). AULA virtual reality test as an attention measure: convergent validity with Conners’ Continuous Performance Test. Child Neuropsychol. 20, 328–342. doi: 10.1080/09297049.2013.792332

PubMed Abstract | CrossRef Full Text | Google Scholar

Dovis, S., van der Oord, S., Wiers, R. W., and Prins, P. J. M. (2012). Can motivation normalize working memory and task persistence in children with attention-deficit/hyperactivity disorder? The effects of money and computer-gaming. J. Abnorm. Child Psychol. 40, 669–681. doi: 10.1007/s10802-011-9601-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Egeland, J., Johansen, S. N., and Ueland, T. (2010). Do low-effort learning strategies mediate impaired memory in ADHD? J. Learn. Disabil. 43, 430–440. doi: 10.1177/0022219409355473

PubMed Abstract | CrossRef Full Text | Google Scholar

Eom, H., Kim, K. K., Lee, S., Hong, Y. J., Heo, J., Kim, J. J., et al. (2019). Development of virtual reality continuous performance test utilizing social cues for children and adolescents with attention-deficit/hyperactivity disorder. Cyberpsychol. Behav. Soc. Netw. 22, 198–204. doi: 10.1089/cyber.2018.0377

PubMed Abstract | CrossRef Full Text | Google Scholar

Friehs, M. A., Dechant, M., Schäfer, S., and Mandryk, R. L. (2022). More than skin deep: about the influence of self-relevant avatars on inhibitory control. Cogn. Res. Principles Implications 7, 1–19. doi: 10.1186/s41235-022-00384-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Frischen, A., Bayliss, A. P., and Tipper, S. P. (2007). Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychol. Bull. 133, 694–724.

Google Scholar

Gaastra, G. F., Groen, Y., Tucha, L., and Tucha, O. (2016). The effects of classroom interventions on off-task and disruptive classroom behavior in children with symptoms of attention-deficit/hyperactivity disorder: a meta-analytic review. PLoS One 11:e0148841. doi: 10.1371/journal.pone.0148841

PubMed Abstract | CrossRef Full Text | Google Scholar

García Murillo, L., Cortese, S., Anderson, D., di Martino, A., and Castellanos, F. X. (2015). Locomotor activity measures in the diagnosis of attention deficit hyperactivity disorder: meta-analyses and new findings. J. Neurosci. Methods 252, 14–26. doi: 10.1016/j.jneumeth.2015.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Hall, C. L., Valentine, A. Z., Groom, M. J., Walker, G. M., Sayal, K., Daley, D., et al. (2015). The clinical utility of the continuous performance test and objective measures of activity for diagnosing and monitoring ADHD in children: a systematic review. Eur. Child Adolesc. Psychiatry 25, 677–699.

PubMed Abstract | Google Scholar

Hall, C. L., Valentine, A. Z., Groom, M. J., Walker, G. M., Sayal, K., Daley, D., et al. (2016). The clinical utility of the continuous performance test and objective measures of activity for diagnosing and monitoring ADHD in children: a systematic review. Eur. Child Adolesc. Psychiatry 25, 677–699. doi: 10.1007/s00787-015-0798-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Hong, N., Kim, J. J., Kwon, J. H., Eom, H., and Kim, E. (2021). Effect of distractors on sustained attention and hyperactivity in youth with attention deficit hyperactivity disorder using a mobile virtual reality school program. J. Attent. Disord. 26, 358–369. doi: 10.1177/1087054720986229

PubMed Abstract | CrossRef Full Text | Google Scholar

Keshav, N. U., Vogt-Lowell, K., Vahabzadeh, A., and Sahin, N. T. (2019). Digital attention-related augmented-reality game: significant correlation between student game performance and validated clinical measures of attention-deficit/hyperactivity disorder (ADHD). Children 6:72. doi: 10.3390/children6060072

PubMed Abstract | CrossRef Full Text | Google Scholar

Kwak, K. J., Park, H. W., and Kim, C. T. (2002). A study for the standardization of Korean WISC-3 (1). Korean J. Dev. Psychol. 15, 19–33.

Google Scholar

Larson, E. B., Feigon, M., Gagliardo, P., and Dvorkin, A. Y. (2014). Virtual reality and cognitive rehabilitation: a review of current outcome research. Neurorehabilitation 34, 759–772.

Google Scholar

Lavine, R. A., Sibert, J. L., Gokturk, M., and Dickens, B. (2002). Eye-tracking measures and human performance in a vigilance task. Aviat. Space Environ. Med. 73, 367–372.

PubMed Abstract | Google Scholar

Lee, S. J., Shin, M. S., Kim, B. N., Yune, H., Shin, Y., Kim, Y., et al. (2015). Clinical utility of the Korean version of CBCL6-18 in the diagnosis of attention-deficit hyperactivity disorder. Korean J. Clin. Psychol. 34, 829–850.

Google Scholar

Lee, W. H., Cho, S. H., Park, H. K., Cho, S. H., Lim, Y. H., and Kim, K. R. (2018). “Movement measurement of attention-deficit/hyperactivity disorder (ADHD) patients using IR-UWB radar sensor,” in Proceedings of 2018 6th IEEE International Conference on Network Infrastructure and Digital Content, IC-NIDC 2018, (Piscataway, NJ: IEEE), 214–217. doi: 10.3390/s19030688

PubMed Abstract | CrossRef Full Text | Google Scholar

Lev, A., Braw, Y., Elbaum, T., Wagner, M., and Rassovsky, Y. (2020). Eye tracking during a continuous performance test: utility for assessing ADHD patients. J. Attent. Disord. 26, 245–255.

Google Scholar

Lewis, R., and Noppeney, U. (2010). Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J. Neurosci. 30, 12329–12339. doi: 10.1523/JNEUROSCI.5745-09.2010

PubMed Abstract | CrossRef Full Text | Google Scholar

Maneuvrier, A., Decker, L. M., Ceyte, H., Fleury, P., and Renaud, P. (2020). Presence promotes performance on a virtual spatial cognition task: impact of human factors on virtual reality assessment. Front. Virtual Real. 1:571713. doi: 10.3389/frvir.2020.571713

CrossRef Full Text | Google Scholar

Mansfield, E., Farroni, T., and Johnson, M. (2003). Does gaze perception facilitate overt orienting? Visual Cogn. 10, 7–14.

Google Scholar

Marotta, A., Casagrande, M., Rosa, C., Maccari, L., Berloco, B., and Pasini, A. (2014). Impaired reflexive orienting to social cues in attention deficit hyperactivity disorder. Eur. Child Adolesc. Psychiatry 23, 649–657. doi: 10.1007/s00787-013-0505-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Mazurek, M. O., and Engelhardt, C. R. (2013). Video game use in boys with autism spectrum disorder, ADHD, or typical development. Pediatrics 132, 260–266.

Google Scholar

Nolin, P., Stipanicic, A., Henry, M., Lachapelle, Y., Lussier-Desrochers, D., Rizzo, A., et al. (2016). ClinicaVR: classroom-CPT: a virtual reality tool for assessing attention and inhibition in children and adolescents. Comput. Hum. Behav. 59, 327–333.

Google Scholar

Oh, K., Lee, H., Hong, K., and Ha, E. (1997). Korean Version of Child Behavior Checklist (K-CBCL). Seoul: Chung Ang Aptitude Publishing Co.

Google Scholar

Ohashi, K., Vitaliano, G., Polcari, A., and Teicher, M. H. (2010). Unraveling the nature of hyperactivity in children with attention-deficit/hyperactivity disorder. Arch. Gen. Psychiatry 67, 388–396.

Google Scholar

Parsons, T. D. (2015). Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front. Hum. Neurosci. 9:660. doi: 10.3389/fnhum.2015.00660

PubMed Abstract | CrossRef Full Text | Google Scholar

Parsons, T. D., Bowerly, T., Buckwalter, J. G., and Rizzo, A. A. (2007). A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods. Child Neuropsychol. 13, 363–381. doi: 10.1080/13825580600943473

PubMed Abstract | CrossRef Full Text | Google Scholar

Parsons, T. D., Gaggioli, A., and Riva, G. (2017). Virtual reality for research in social neuroscience. Brain Sci. 7:42.

Google Scholar

Petrovic, P., and Castellanos, F. X. (2016). Top-down dysregulation—from ADHD to emotional instability. Front. Behav. Neurosci. 10:70. doi: 10.3389/fnbeh.2016.00070

PubMed Abstract | CrossRef Full Text | Google Scholar

Pollak, Y., Shomaly, H. B., Weiss, P. L., Rizzo, A. A., and Gross-Tsur, V. (2010). Methylphenidate effect in children with ADHD can be measured by an ecologically valid continuous performance test embedded in virtual reality. CNS Spectr. 15, 125–130. doi: 10.1017/s109285290002736x

PubMed Abstract | CrossRef Full Text | Google Scholar

Rizzo, A. A., Bowerly, T., Buckwalter, J. G., Klimchuk, D., Mitura, R., and Parsons, T. D. (2006). A virtual reality scenario for all seasons: the virtual classroom. CNS Spectr. 11, 35–44. doi: 10.1017/s1092852900024196

PubMed Abstract | CrossRef Full Text | Google Scholar

Rizzo, A. A., Buckwalter, J. G., Bowerly, T., van der Zaag, C., Humphrey, L., Neumann, U., et al. (2004). The virtual classroom: a virtual reality environment for the assessment and rehabilitation of attention deficits. Cyber Psychol. Behav. 3, 483–499.

Google Scholar

Rizzo, A. A., Buckwalter, J. G., Neumann, U., Chua, C., Rooyen, V. A. N. A., Larson, P., et al. (1999). Virtual environments for targeting cognitive processes: an overview of projects at the University of Southern California. Cyber Psychol. Behav. 2, 89–100. doi: 10.1089/cpb.1999.2.89

PubMed Abstract | CrossRef Full Text | Google Scholar

Romei, V., Murray, M. M., Cappe, C., and Thut, G. (2009). Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Curr. Biol. 19, 1799–1805.

Google Scholar

Schmidt, S. L., Simões, E. D. N., and Novais Carvalho, A. L. (2019). Association between auditory and visual continuous performance tests in students with ADHD. J. Attent. Disord. 23, 635–640.

Google Scholar

Seernani, D., Ioannou, C., Damania, K., Hill, H., Foulsham, T., Smyrnis, N., et al. (2021). Social and non-social gaze cueing in autism spectrum disorder, attention-deficit/hyperactivity disorder and a comorbid group. Biol. Psychol. 162:108096. doi: 10.1016/j.biopsycho.2021.108096

PubMed Abstract | CrossRef Full Text | Google Scholar

Seyama, J. (2006). Effect of image orientation on the eye direction aftereffect. Psychol. Res. Psychol. Forschung 70, 367–374.

Google Scholar

Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., et al. (1998). The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J. Clin. Psychiatry 59, (Suppl. 20), 11980.

PubMed Abstract | Google Scholar

Sheehan, D. V., Sheehan, K. H., Shytle, R. D., Janavs, J., Bannon, Y., Rogers, J. E., et al. (2010). Reliability and validity of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID). J. Clin. Psychiatry 71, 313–326.

Google Scholar

So, Y.-K., Noh, J.-S., Kim, Y.-S., Ko, S.-G., and Koh, Y.-J. (2002). The reliability and validity of korean parent and teacher ADHD rating scale. J. Korean Neuropsychiatr. Assoc. 41, 283–289.

Google Scholar

Spencer, T. J., Biederman, J., and Mick, E. (2007). Attention-deficit/hyperactivity disorder: diagnosis, lifespan, comorbidities, and neurobiology. Ambul. Pediatr. 7(1 Suppl.), 73–81.

Google Scholar

Tang, X., Wu, J., and Shen, Y. (2016). The interactions of multisensory integration with endogenous and exogenous attention. Neurosci. Biobehav. Rev. 61, 208–224.

Google Scholar

Teicher, M. H., Ito, Y., Glod, C. A., and Barber, N. I. (1996). Objective measurement of hyperactivity and attentional problems in ADHD. J. Am. Acad. Child Adolesc. Psychiatry 35, 334–342.

Google Scholar

Teicher, M. H., Polcari, A., and McGreenery, C. E. (2008). Utility of objective measures of activity and attention in the assessment of therapeutic response to stimulants in children with attention-deficit/hyperactivity disorder. J. Child Adolesc. Psychopharmacol. 18, 265–270.

Google Scholar

Won, A. S., Bailenson, J. N., and Janssen, J. H. (2014). Automatic detection of nonverbal behavior predicts learning in dyadic interactions. IEEE Trans. Affect. Comput. 5, 112–125.

Google Scholar

Zhou, J. (2012). The effects of reciprocal imitation on teacher–student relationships and student learning outcomes. Mind Brain Educ. 6, 66–73. doi: 10.1111/j.1751-228X.2012.01140.x

CrossRef Full Text | Google Scholar