Comparative Evaluation of Child Behavior Checklist-Derived Scales in Children Clinically Referred for Emotional and Behavioral Dysregulation

Background: We recently developed the Child Behavior Checklist-Mania Scale (CBCL-MS), a novel and short instrument for the assessment of mania-like symptoms in children and adolescents derived from the CBCL item pool and have demonstrated its construct validity and temporal stability in a longitudinal general population sample.

Objective: The aim of this study was to evaluate the construct validity of the 19-item CBCL-MS in a clinical sample and to compare its discriminatory ability to that of the 40-item CBCL-dysregulation profile (CBCL-DP) and the 34-item CBCL-Externalizing Scale.

Methods: The study sample comprised 202 children, aged 7–12 years, diagnosed with DSM-defined attention deficit hyperactivity disorder (ADHD), conduct disorder (CD), oppositional defiant disorder (ODD), and mood and anxiety disorders based on the Diagnostic Interview Schedule for Children. The construct validity of the CBCL-MS was tested by means of a confirmatory factor analysis. Receiver operating characteristics (ROC) curves and logistic regression analyses adjusted for sex and age were used to assess the discriminatory ability relative to that of the CBCL-DP and the CBCL-Externalizing Scale.

Results: The CBCL-MS had excellent construct validity (comparative fit index = 0.97; Tucker–Lewis index = 0.96; root mean square error of approximation = 0.04). Despite similar overall performance across scales, the clinical range scores of the CBCL-DP and the CBCL-Externalizing Scale were associated with higher odds for ODD and CD, while the clinical range scores of the CBCL-MS were associated with higher odds for mood disorders. The concordance rate among the children who scored within the clinical range of each scale was over 90%.

Conclusion: CBCL-MS has good construct validity in general population and clinical samples and is therefore suitable for both clinical practice and research.

Introduction

The accurate identification of childhood psychopathology represents an important step in formulating early intervention strategies that could improve prognosis. Yet, the task of developing instruments for the assessment of psychiatric conditions in children is challenging because of high comorbidity (1–3) and significant overlap in clinical features (2). Our group has a long-standing interest in childhood emotional and cognitive dysregulation and its relevance to the formal diagnostic categories of attention deficit hyperactivity disorder (ADHD) and bipolar disorder (BD) (4–7). This motivates research into developing instruments for the assessment of childhood behavioral problems that can be easily used in research and clinical care.

We recently developed the Child Behavior Checklist (CBCL)-Mania Scale (MS) (8) that derives from the CBCL. The CBCL is a 118-item parent report instrument that is widely used because of its sound psychometric properties and transcultural validity (9, 10). The CBCL-MS uses only 19 CBCL items (Table S1 in Supplementary Material) chosen to map onto the criteria for mania outlined in DSM-IV and DSM5 (11, 12), while also acknowledging the predictive value and high prevalence of psychotic symptoms during acute mood episodes (13). The psychometric properties of the CBCL-MS were evaluated in a general population sample of 2,230 youth assessed at ages 11, 13, and 16 years (8). The scale was shown to have a four-factor structure corresponding to distraction/disinhibition, psychotic symptoms, increased libido, and sleep problems, which remained stable across all three assessment waves (8). A recent study based on a sample of 474 children and adolescents from Brazil has provided further support for the construct validity of the CBCL-MS in the general population (14).

The objective of this paper is to determine the usefulness of the CBCL-MS in clinical settings and evaluate its performance against two other popular CBCL-derived scales, namely the CBCL-Externalizing Scale (15) and the CBCL-dysregulation profile (CBCL-DP) (16–19).

The CBCL-Externalizing Scale (Table S1 in Supplementary Material) was developed by combining the delinquent behavior and aggressive behavior CBCL subscales (15, 20) and has been used to identify BD (21), oppositional defiant disorder (ODD), and anxiety disorders in clinically enriched samples of youth (22, 23). This scale has been criticized however for being too long, having poor ecological validity (24), and poor concordance with formal psychiatric diagnoses (25, 26). The CBCL-DP (Table S1 in Supplementary Material), also known as the CBCL-pediatric bipolar disorder profile (CBCL-PBD), was developed by combining the CBCL subscales for attention problems, aggressive behavior, and anxiety/depression (16–19, 27, 28). Several studies report a significant and temporally stable association between high CBCL-DP scores in childhood and BD, ADHD, conduct disorder (CD), and ODD (16, 24, 28, 29). However, other reports have not supported these findings (5, 30–33).

The present study examined the psychometric properties of the CBCL-MS in a sample of 202 clinically referred children and compared its discriminative ability for multiple psychiatric diagnoses to that of the CBCL-DP and the CBCL-Externalizing Scale.

Materials and Methods

Participants

Details of the study sample are shown in Table 1. The sample comprised 202 children aged 7–12 years (M = 9.04, SD = 1.30; 87.5% male) that had been referred for evaluation to the Mount Sinai Childhood Behavior Disorders Research Team for disruptive behaviors and/or suspected ADHD as part of three separate studies (34–36). Exclusion criteria of the original studies included any medical/neurological condition, psychosis, and pervasive developmental disorders.

TABLE 1

Table 1. Baseline characteristics of the sample by study of origin.

Formal diagnoses were based on parental reports using the Diagnostic Interview Schedule for Children (DISC) version 2.1 in 111 children (37) and version 2.3 in 91 children (38). The diagnoses considered included mood disorders, mainly major depressive disorder and dysthymia, ADHD, CD, ODD, and anxiety disorders. In total, 23 children were diagnosed with a mood disorder (13%), 56 with an anxiety disorder (31%), 154 with ADHD (85%), 52 with CD (29%), and 127 with ODD (70%). Nineteen children did not receive any diagnosis (10.4%), while 133 (73%) were comorbid for two or more disorders. The range of the full scale IQ of the analyses sample was 60–139, with 7% (n = 13) of children having IQ scores <70.

Parents of all children completed the CBCL (20) during clinic visits. Cumulative scores ≥210 on the attention problems, aggressive behavior, and anxious/depressed CBCL scales upon standardization (T scores) were considered significantly elevated scores for the CBCL-DP (27, 28). Total standardized T scores ≥70 (2 SDs above the mean) were considered significantly elevated scores for the CBCL-MS and the CBCL-Externalizing Scale. Full scale, verbal, and performance IQ were assessed using the Wechsler Intelligence Scale for Children-Revised (WISC-R) in 96 children and the WISC-III in 106 children (39, 40). The differences observed in the cognitive abilities of children across samples were fully accounted for by the shift from WISC-R to WISC-III (Table 1).

Statistical Analysis

We performed confirmatory factor analyses (CFA) to examine whether the four-factor structure of the CBCL-MS previously described in a general population sample could be validated in referred children. Goodness of fit was determined using four indices, the comparative fit index (CFI) (cutoff values above 0.95 indicate good fit), the Tucker–Lewis index (TLI) (cutoff values above 0.95 indicate good fit), the root mean square error of approximation (RMSEA) (cutoff values below 0.06 indicate good fit), and the relative (also called normed) Chi-Square (χ² divided by the degrees of freedom of the model; cutoff values below 2 indicate good fit) (41, 42). The fit of the CFA model was estimated using the weighted least squares, mean, and variance (WLSMV) estimator. Minor model modifications were performed using Mplus’ modification indices by allowing correlations between the unique variances of some individual items within the same factors. Such model modifications do not alter the substantive conclusions regarding the factor structure yet improve model fit by increasing the proportion of the variance explained (43). We bootstrapped the CFA model to obtain more reliable estimates for the 95% confidence interval (CI) of the factor loadings of individual items on their respective factors (44).

The discriminative ability of the CBCL-MS, CBCL-DP, and the CBCL-Externalizing Scale for DSM-based diagnoses was assessed using receiver operating characteristic (ROC) curves. Areas under the curve (AUC) were compared across scales for each DSM diagnosis using the Stata command roccomp. The total scores of scales were used as continuous variables for the ROC curves; the CBCL-DP scores were divided by three to ensure identical range of scores for the three scales. Differences in total mean scores of the scales were compared between cases with ADHD, CD, ODD, anxiety disorders, or mood disorders and non-cases with a series of t-tests. Upon identification of children with significantly elevated scores on CBCL-MS, CBCL-DP, or CBCL-Externalizing Scale, we ran a series of logistic regression models to assess age-, sex-, and sample-adjusted odds ratios (ORs) and 95% CIs with respect to multiple diagnostic outcomes. Non-cases were used as the reference category for each respective regression model. Additional ROC–AUC and t-tests were performed to examine the discriminative ability of the items that are unique to the CBCL-MS and those that are shared between the CBCL-MS and the other two scales. Analyses were performed using Stata/SE 14.0 (StataCorp, College Station, TX, USA) and Mplus v.6 (www.statmodel.com).

Results

Factor Structure and Internal Consistency of the CBCL-MS

The results of the CFA are shown in Table 2. Standardized factor loadings yielded a four-factor structure representing distraction/disinhibition, increased libido, sleep problems, and psychotic symptoms. The four-factor structure of the CBCL-MS showed excellent fit to the data as all four fit indices were well within the recommended cutoffs (CFI = 0.97; TLI = 0.96; RMSEA = 0.04; and relative Chi-square = 1.32). The internal consistency of the CBCL-MS was also high (Cronbach’s alpha = 0.83). For the individual factors, internal consistency rates were 0.78 for distraction/disinhibition; 0.68 for psychotic symptoms; 0.83 for increased libido (upon removal of item 96 “thinks of sex too much” as the results suggested significant increase of internal consistency); and 0.69 for troubled sleep.

TABLE 2

Table 2. Factor loadings, SE, and bootstrapped 95% confidence interval of the CBCL-MS items.

Discriminative Ability of the Three Scales

Figures 1A–C illustrate the mean score differences of each scale across diagnostic categories. A series of independent samples’ t-tests showed that the mean differences in the scores of the CBCL-MS, the CBCL-DP, and the CBCL-Externalizing Scale between cases and non-cases were all statistically significant (all p values <0.01).

FIGURE 1

Figure 1. (A–C) Mean (±2*SE) CBCL-Mania Scale, CBCL-dysregulation profile (CBCL-DP), and CBCL-Externalizing Scale T-scores across diagnostic categories.

The results of the ROC curve analyses for the CBCL-MS, the CBCL-DP, and the CBCL-Externalizing Scale with respect to psychiatric outcomes are shown in Table 3. For mood disorders, the highest AUC was observed for the CBCL-MS (AUC = 0.82; 95% CI 0.71–0.93), followed by the CBCL-Externalizing Scale (AUC = 0.79; 95% CI 0.68–0.89) and then the CBCL-DP (AUC = 0.78; 95% CI 0.64–0.92); pair-wise comparisons showed that these AUC values were not significantly different (p = 0.30). The CBCL-MS achieved sensitivity rates of 70% and specificity rates of 71%. CBCL-Externalizing Scale achieved sensitivity and specificity rates of 80 and 59%, respectively, and the respective values for the CBCL-DP were 64 and 67%. Comparisons of the extracted AUC values suggest that the three scales have similar discriminative power for anxiety disorders and ADHD (p values >0.05). However, the CBCL-Externalizing Scale appears to have increased discriminative power for CD (p < 0.001) and ODD (p = 0.02).

TABLE 3

Table 3. Discriminative abilities of the CBCL-MS, the CBCL-DP, and the CBCL-Externalizing Scale total and associated factor scores for various psychiatric disorders.

Associations between the Clinical Range of the Three Scales and Multiple Psychiatric Disorders

A series of logistic regressions was performed to obtain sex, age, and sample of origin adjusted ORs (95% CI) for children with elevated scores on the three scales with respect to multiple psychiatric diagnoses. Results are summarized in Table 4. Overall, 57 (34%) children were found to have CBCL-MS scores ≥70, 53 (34%) had CBCL-DP scores ≥210, and 85 (45%) had scores ≥70 on the CBCL-Externalizing Scale. The Goodman and Kruskal’s gamma for the distributions between dichotomized scores on the CBCL-MS and both scores on the CBCL-DP (γ = 0.90, p < 0.001) and the CBCL-Externalizing Scale (γ = 0.94, p < 0.001), and also between scores on the CBCL-DP and the CBCL-Externalizing Scale (γ = 0.93, p < 0.001), suggested that there was great overlap in the children identified as having elevated scores by all three scales. Mean total scores ≥70 on the CBCL-MS were associated with a sevenfold increase in the risk of being diagnosed with a mood disorder (OR = 7.1, 95% CI 2.2–22.7) or CD (OR = 7.2; 95% CI 3.2–15.9), a sixfold increase for ODD (OR = 6.4, 95% CI 2.4–17.3), and a fourfold increase for anxiety disorders (OR = 4.1, 95% CI 1.9–8.6). However, scoring high only on the distraction/disinhibition (OR = 3.3, 95% CI 1.2–9.0) scale of the CBCL-MS, but not on the total CBCL-MS score (OR = 2.3, 95% CI 0.8–6.7), was significantly associated with ADHD.

TABLE 4

Table 4. Cross-sectional associations between the CBCL-MS, CBCL-DP, CBCL-Externalizing Scale, and associated factors with multiple diagnostic outcomes.

Total scores ≥70 on the CBCL-Externalizing Scale were associated with diagnoses of mood disorder (OR = 5.9, 95% CI 1.8–19.2), anxiety disorder (OR = 2.9, 95% CI 1.4–5.7), CD (OR = 15.8, 95% CI 6.3–39.7), or ODD (OR = 13.1, 95% CI 4.8–35.9), but not ADHD (OR = 2.7, 95% CI 1.0–7.6). Finally, CBCL-DP scores ≥210 were strongly associated with ODD (OR = 10.7, 95% CI 3.1–37.6). High scores on the CBCL-DP were weaker, yet also significantly associated with mood disorders (OR = 3.6, 95% CI 1.0–12.2), anxiety disorders (OR = 3.2, 95% CI 1.5–6.8), ADHD (OR = 4.0, 95% CI 1.1–15.1), and CD (OR = 3.6, 95% CI 1.6–7.9).

Discriminative Ability of Items Unique to the CBCL-MS

Ten CBCL-MS items were not shared with the other two scales (Table S1 in Supplementary Material). The mean scores of these 10 items were significantly higher for children with a mood disorder (M = 7.76, SD = 3.98) in comparison to those with a diagnosis of ADHD, CD, ODD, or anxiety disorder (M = 4.05, SD = 2.71), p < 0.001. Additional sensitivity analyses for the ROC–AUC values of individual items that are unique to the CBCL-MS showed that items 40 (“hears sound or voices that are not there”), 59 (“plays with own sex parts in public”), and 76 (“sleeps less than most kids”) could not individually discriminate between mood and non-mood disorders. For the remaining items, the individual ROC–AUC values ranged from 0.60 (95% CI 0.51–0.70) for item 70 (“sees things that are not there”) to 0.71 (95% CI 0.59–0.82) for item 34 (“feels others are out get him/her”).

Discussion

The results of this study suggest that CBCL-derived scales have comparable overall performance in the assessment of children referred for problems with behavioral and emotional self-regulation. Within this context, the CBCL-MS may have some advantages over the other scales. First, the factor structure of the CBCL-MS appears robust as it is identical in clinically referred and general population samples of children (8). Second, although its discriminative ability for mood disorders was comparable to that observed for the CBCL-DP and the CBCL-Externalizing Scale, the sensitivity/specificity achieved (70/71%) was more balanced relative to those of the other scales. Third, having just 19 items, the CBCL-MS is a short and versatile instrument in comparison to both the CBCL-DP (40 items) and the CBCL-Externalizing Scale (34 items) while retaining high internal consistency (84%).

The CBCL-MS showed the strongest association with mood disorders in terms of the OR obtained after adjustments for relevant covariates compared to the CBCL-DP and the CBCL-Externalizing Scale. This may reflect the fact that the items comprising the CBCL-MS were selected to map onto DSM diagnostic criteria for mania (8). In contrast, the CBCL-DP was only weakly associated with mood disorders, which conforms with findings suggesting that the CBCL-DP is not necessarily related to BD, but rather to CD, ODD, and ADHD (32). Moreover, the CBCL-MS is the only scale of the three to take into account extended (psychotic) symptoms of BD in addition to core symptoms. A study in a representative community sample of adolescents and young adults has demonstrated that up to 27% of youth with mood or anxiety disorders also displayed one or more psychotic symptoms (45). Accordingly, psychotic symptoms were associated with a 12-fold increased risk of having received a diagnosis of a mood disorder in this sample; moreover, the highest ROC–AUC of the individual items unique to the CBCL-MS was obtained for one of the items loading on the psychotic symptoms factor (“feels others are out to get him/her”).

The ROC–AUCs observed for the three scales were significant for multiple diagnostic outcomes and ranged from 72 to 82% for the CBCL-MS, 70 to 84% for the CBCL-DP, and 69 to 88% for the CBCL-Externalizing Scale. This was expected since the three scales have several items in common (Table S1 in Supplementary Material). Notably, however, the addition of items from the Thought Problems CBCL-subscale may have contributed to the improved discriminability of the CBCL-MS for mood disorders.

It is also worth noting that while the CBCL-MS and the CBCL-DP were initially developed to screen for pediatric BD, they appear to have high discriminatory power for several other diagnostic entities. It is our view that this reflects the fact that these instruments tap into dimensions of poor self-regulation that are relevant to multiple psychiatric diagnoses. Affective dysregulation and attentional dysfunction are also common in children with BD, ADHD, CD, ODD, and disruptive mood dysregulation disorder (DMDD) (2, 46, 47). This is consistent with the high levels of comorbidity observed (1–3, 48, 49). Still, the absence of specificity for DSM-diagnoses does not diminish the importance of the instruments evaluated here, as they can contribute toward the identification of pluripotential early high-risk phenotypes (50). Screening in general population samples could also serve as a two-step approach to identify children who would benefit from clinical referral and detailed clinical assessments (51).

Strengths and Limitations

This is the first study to compare the commonly examined CBCL-DP and CBCL-Externalizing Scale directly with the newly developed CBCL-MS in a sample of clinically referred children with various psychiatric outcomes. Clinical diagnoses were ascertained using established structured instruments, which are well-validated in clinical samples of this age group (38, 52). Moreover, clinical diagnoses and CBCL assessments were captured almost contemporaneously; they are, therefore, free from recall or attribution biases. The number of patients with BD within the analysis sample was small (n = 11). The CFA conducted to assess the construct validity of the CBCL-MS was performed for the whole sample so that the extracted factors reflect the underlying (latent) trait variance of the entire sample of children referred for emotional and behavioral dysregulation. The item pool of the CBCL-MS covers behavioral manifestations of mania, such as inattention/distractibility, hyperactivity, loudness, over-talkativeness, and disrupted sleep that are shared across different diagnostic entities. It would indeed be interesting in a future study with larger numbers of cases with different diagnoses to replicate the findings of this study by assessing the measurement invariance of the factors across diagnostic groups. Still, we believe that for the purpose of this study, the sample size was adequate as it was within the recommended limits of several rules of thumb reviewed in Velicer and Fava (53), e.g., 10 cases for each item in the instrument being used. Most importantly, the estimated SEs of the factor loadings were almost identical to the bootstrap-corrected confidence intervals, suggesting a stable factor structure for the CBCL-MS in our sample.

There are two inherent limitations of the CBCL, which are inevitably reflected in all CBCL-based screening scales. First, behavioral ratings alone are unlikely to yield high levels of accuracy in case identification for any mental disorder. Second, the CBCL items do not fully capture episodicity, which is considered a salient predictor of conversion to BD particularly in young people (29, 54). It is therefore possible that adding more refined information about episodicity may further enhance the predictive value of the scales considered. Moreover, the results of this study are based on a relatively small sample, and it is therefore possible that some of the analyses might have been underpowered. Although we demonstrate the stability of the factor structure of the CBCL-MS in a new independent sample, the longitudinal stability of these factors in clinical samples has yet to be determined.

Conclusion

The three CBCL-derived scales considered here performed similarly, and the brevity and robust factor structure of the CBCL-MS are distinct benefits. All three scales seem to identify children with difficulties in self-regulation that render them vulnerable to adverse psychiatric outcomes.

Ethics Statement

Ethical approval was provided by the Institutional Review Board of the Icahn School of Medicine at Mount Sinai. Written consent was sought from parents, and assent was sought from children participating in the original studies based on detailed information about the study protocols.

Author Contributions

EP conducted the data analyses and contributed to writing the manuscript. KS and A-CB collected the data and contributed to data analysis and manuscript writing. JN, JH, and SF contributed to study conception, data analysis, and manuscript writing.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

This study was supported by departmental funds by the Icahn School of Medicine at Mount Sinai.

Supplementary Material

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fpsyt.2016.00146

References

1. Cummings CM, Caporino NE, Kendall PC. Comorbidity of anxiety and depression in children and adolescents: 20 years after. Psychol Bull (2014) 140:816–45. doi:10.1037/a0034733

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Egger HL, Angold A. Common emotional and behavioral disorders in preschool children: presentation, nosology, and epidemiology. J Child Psychol Psychiatry (2006) 47:313–37. doi:10.1111/j.1469-7610.2006.01618.x

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Goldstein BI. Recent progress in understanding pediatric bipolar disorder. Arch Pediatr Adolesc Med (2012) 166:362–71. doi:10.1001/archpediatrics.2011.832

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Elmaadawi AZ, Jensen PS, Arnold LE, Molina BS, Hechtman L, Abikoff HB, et al. Risk for emerging bipolar disorder, variants, and symptoms in children with attention deficit hyperactivity disorder, now grown up. World J Psychiatry (2015) 5:412–24. doi:10.5498/wjp.v5.i4.412

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Halperin JM, Rucklidge JJ, Powers RL, Miller CJ, Newcorn JH. Childhood CBCL bipolar profile and adolescent/young adult personality disorders: a 9-year follow-up. J Affect Disord (2011) 130:155–61. doi:10.1016/j.jad.2010.10.019

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Kitsune GL, Kuntsi J, Costello H, Frangou S, Hosang GM, McLoughlin G, et al. Delineating ADHD and bipolar disorder: a comparison of clinical profiles in adult women. J Affect Disord (2016) 192:125–33. doi:10.1016/j.jad.2015.12.024

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Sonuga-Barke EJ, Halperin JM. Developmental phenotypes and causal pathways in attention deficit/hyperactivity disorder: potential targets for early intervention? J Child Psychol Psychiatry (2010) 51:368–89. doi:10.1111/j.1469-7610.2009.02195.x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Papachristou E, Ormel J, Oldehinkel AJ, Kyriakopoulos M, Reinares M, Reichenberg A, et al. Child Behavior Checklist-Mania Scale (CBCL-MS): development and evaluation of a population-based screening scale for bipolar disorder. PLoS One (2013) 8:e69459. doi:10.1371/journal.pone.0069459

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Achenbach TM, Dumenci L, Rescorla L. DSM-oriented and empirically based approaches to constructing scales from the same item pools. J Clin Child Adolesc Psychol (2003) 32:328–40. doi:10.1207/S15374424JCCP3203_02

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Achenbach TM, Rescorla L. ASEBA School-Age Forms & Profiles. Burlington, VT: Aseba (2001).

Google Scholar

11. APA. Diagnostic and Statistical Manual of Mental Disorders, Text Revision (DSM-IV-TR). Washington, DC: American Psychiatric Association (2000).

Google Scholar

12. APA. Diagnostic and Statistical Manual of Mental Disorders, (DSM-5^®). Washington, DC: American Psychiatric Association (2013).

Google Scholar

13. Carvalho AF, McIntyre RS, Dimelis D, Gonda X, Berk M, Nunes-Neto PR, et al. Predominant polarity as a course specifier for bipolar disorder: a systematic review. J Affect Disord (2014) 163:56–64. doi:10.1016/j.jad.2014.03.035

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Zappitelli MC, Pereira ML, Bordin IA. Child Behavior Checklist-Mania Scale as a screening tool to identify children at risk for bipolar disorder. J Child Adolesc Psychopharmacol (2015) 25:448–9. doi:10.1089/cap.2014.0129

CrossRef Full Text | Google Scholar

15. Achenbach TM, Edelbrock CS. Manual for the Child Behavior Checklist and Revised Behavior Profile. Burlington, VT: University of Vermont Department of Psychiatry (1983).

Google Scholar

16. Biederman J, Martelon M, Faraone SV, Woodworth KY, Spencer TJ, Wozniak JR. Personal and familial correlates of bipolar (BP)-I disorder in children with a diagnosis of BP-I disorder with a positive child behavior checklist (CBCL)-severe dysregulation profile: a controlled study. J Affect Disord (2013) 147:164–70. doi:10.1016/j.jad.2012.10.028

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Biederman J, Petty CR, Monuteaux MC, Evans M, Parcell T, Faraone SV, et al. The child behavior checklist-pediatric bipolar disorder profile predicts a subsequent diagnosis of bipolar disorder and associated impairments in ADHD youth growing up: a longitudinal analysis. J Clin Psychiatry (2009) 70:732–40. doi:10.4088/JCP.08m04821

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Faraone SV, Althoff RR, Hudziak JJ, Monuteaux M, Biederman J. The CBCL predicts DSM bipolar disorder in children: a receiver operating characteristic curve analysis. Bipolar Disord (2005) 7:518–24. doi:10.1111/j.1399-5618.2005.00271.x

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Uchida M, Faraone SV, Martelon M, Kenworthy T, Woodworth KY, Spencer TJ, et al. Further evidence that severe scores in the aggression/anxiety-depression/attention subscales of child behavior checklist (severe dysregulation profile) can screen for bipolar disorder symptomatology: a conditional probability analysis. J Affect Disord (2014) 165:81–6. doi:10.1016/j.jad.2014.04.021

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Achenbach TM. Integrative Guide for the 1991 CBCL/4-18, YSR, and TRF Profiles. Burlington, VT: Department of Psychiatry, University of Vermont (1991).

Google Scholar

21. Youngstrom EA, Findling RL, Calabrese JR, Gracious BL, Demeter C, Bedoya DD, et al. Comparing the diagnostic accuracy of six potential screening instruments for bipolar disorder in youths aged 5 to 17 years. J Am Acad Child Adolesc Psychiatry (2004) 43:847–58. doi:10.1097/01.chi.0000125091.35109.1e

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Biederman J, Monuteaux MC, Kendrick E, Klein KL, Faraone SV. The CBCL as a screen for psychiatric comorbidity in paediatric patients with ADHD. Arch Dis Child (2005) 90:1010–5. doi:10.1136/adc.2004.056937

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Hudziak JJ, Copeland W, Stanger C, Wadsworth M. Screening for DSM-IV externalizing disorders with the child behavior checklist: a receiver-operating characteristic analysis. J Child Psychol Psychiatry (2004) 45:1299–307. doi:10.1111/j.1469-7610.2004.00314.x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Kahana SY, Youngstrom EA, Findling RL, Calabrese JR. Employing parent, teacher, and youth self-report checklists in identifying pediatric bipolar spectrum disorders: an examination of diagnostic accuracy and clinical utility. J Child Adolesc Psychopharmacol (2003) 13:471–88. doi:10.1089/104454603322724869

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Doucette A. Child and adolescent diagnosis: the need for a model-based approach. In: Beutler LE, Malik ML, editors. Rethinking the DSM: A Psychological Perspective. Washington, DC: American Psychological Association (2002). p. 201–20.

Google Scholar

26. Kasius MC, Ferdinand RF, Berg H, Verhulst FC. Associations between different diagnostic approaches for child and adolescent psychopathology. J Child Psychol Psychiatry (1997) 38:625–32. doi:10.1111/j.1469-7610.1997.tb01689.x

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Biederman J, Petty CR, Day H, Goldin RL, Spencer T, Faraone SV, et al. Severity of the aggression/anxiety-depression/attention child behavior checklist profile discriminates between different levels of deficits in emotional regulation in youth with attention-deficit hyperactivity disorder. J Dev Behav Pediatr (2012) 33:236–43. doi:10.1097/DBP.0b013e3182475267

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Masi G, Muratori P, Manfredi A, Pisano S, Milone A. Child behaviour checklist emotional dysregulation profiles in youth with disruptive behaviour disorders: clinical correlates and treatment implications. Psychiatry Res (2015) 225:191–6. doi:10.1016/j.psychres.2014.11.019

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Meyer SE, Carlson GA, Youngstrom E, Ronsaville DS, Martinez PE, Gold PW, et al. Long-term outcomes of youth who manifested the CBCL-pediatric bipolar disorder phenotype during childhood and/or adolescence. J Affect Disord (2009) 113:227–35. doi:10.1016/j.jad.2008.05.024

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Holtmann M, Buchmann AF, Esser G, Schmidt MH, Banaschewski T, Laucht M. The child behavior checklist-dysregulation profile predicts substance use, suicidality, and functional impairment: a longitudinal analysis. J Child Psychol Psychiatry (2011) 52:139–47. doi:10.1111/j.1469-7610.2010.02309.x

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Mbekou V, Gignac M, MacNeil S, Mackay P, Renaud J. The CBCL dysregulated profile: an indicator of pediatric bipolar disorder or of psychopathology severity? J Affect Disord (2014) 155:299–302. doi:10.1016/j.jad.2013.10.033

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Volk HE, Todd RD. Does the child behavior checklist juvenile bipolar disorder phenotype identify bipolar disorder? Biol Psychiatry (2007) 62:115–20. doi:10.1016/j.biopsych.2006.05.036

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Youngstrom E, Youngstrom JK, Starr M. Bipolar diagnoses in community mental health: Achenbach child behavior checklist profiles and patterns of comorbidity. Biol Psychiatry (2005) 58:569–75. doi:10.1016/j.biopsych.2005.04.004

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Halperin JM, Newcorn JH, McKay KE, Siever LJ, Sharma V. Growth hormone response to guanfacine in boys with attention deficit hyperactivity disorder: a preliminary study. J Child Adolesc Psychopharmacol (2003) 13:283–94. doi:10.1089/104454603322572615

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Halperin JM, Trampush JW, Miller CJ, Marks DJ, Newcorn JH. Neuropsychological outcome in adolescents/young adults with childhood ADHD: profiles of persisters, remitters and controls. J Child Psychol Psychiatry (2008) 49:958–66. doi:10.1111/j.1469-7610.2008.01926.x

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Marks DJ, Miller SR, Schulz KP, Newcorn JH, Halperin JM. The interaction of psychosocial adversity and biological risk in childhood aggression. Psychiatry Res (2007) 151:221–30. doi:10.1016/j.psychres.2006.07.010

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Shaffer D, Fisher P, Piacentini J, Schwab-Stone M, Wicks J. Diagnostic Interview Schedule for Children (DISC-2.1P)-Parent Version. New York, NY: N. Y. S. P. Institute (1989).

Google Scholar

38. Shaffer D, Fisher P, Dulcan MK, Davies M, Piacentini J, Schwab-Stone ME, et al. The NIMH diagnostic interview schedule for children version 2.3 (DISC-2.3): description, acceptability, prevalence rates, and performance in the MECA Study. Methods for the epidemiology of child and adolescent mental disorders study. J Am Acad Child Adolesc Psychiatry (1996) 35:865–77. doi:10.1097/00004583-199607000-00012

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Wechsler D. Manual for the Wechsler Intelligence Scale for Children, Revised. San Antonio, TX: Psychological Corporation (1974).

Google Scholar

40. Wechsler D. Manual for the Wechsler Intelligence Scale for Children – Third Edition (WISC-III). San Antonio, TX: Psychological Corporation (1991).

Google Scholar

41. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Model (1999) 6:1–55. doi:10.1080/10705519909540118

CrossRef Full Text | Google Scholar

42. Ullman J. Structural equation modeling. In: Tabachnick B, Fidell L, editors. Using Multivariate Statistics. Boston, MA: Allyn and Bacon (2001). p. 653–771.

Google Scholar

43. Bollen KA. Structural Equations with Latent Variables. New York, NY: John Wiley & Sons (1989).

Google Scholar

44. Efron B, Tibshirani RJ. An Introduction to the Bootstrap. New York, NY: Chapman & Hall (1993).

Google Scholar

45. Wigman JT, van Nierop M, Vollebergh WA, Lieb R, Beesdo-Baum K, Wittchen HU, et al. Evidence that psychotic symptoms are prevalent in disorders of anxiety and depression, impacting on illness onset, risk, and severity – implications for diagnosis and ultra-high risk research. Schizophr Bull (2012) 38:247–57. doi:10.1093/schbul/sbr196

CrossRef Full Text | Google Scholar

46. Copeland WE, Angold A, Costello EJ, Egger H. Prevalence, comorbidity, and correlates of DSM-5 proposed disruptive mood dysregulation disorder. Am J Psychiatry (2013) 170:173–9. doi:10.1176/appi.ajp.2012.12010132

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Zepf FD, Holtmann M. Disruptive Mood Dysregulation Disorder. Geneva: International Association for Child and Adolescent Psychiatry and Allied Professions (2012).

Google Scholar

48. Berk M, Hallam K, Lucas N, Hasty M, McNeil CA, Conus P, et al. Early intervention in bipolar disorders: opportunities and pitfalls. Med J Aust (2007) 187:S11–4.

PubMed Abstract | Google Scholar

49. Birmaher B, Axelson D, Strober M, Gill MK, Valeri S, Chiappetta L, et al. Clinical course of children and adolescents with bipolar spectrum disorders. Arch Gen Psychiatry (2006) 63:175–83. doi:10.1001/archpsyc.63.2.175

PubMed Abstract | CrossRef Full Text | Google Scholar

50. McGorry P, van Os J. Redeeming diagnosis in psychiatry: timing versus specificity. Lancet (2013) 381:343–5. doi:10.1016/S0140-6736(12)61268-9

CrossRef Full Text | Google Scholar

51. Severson HH, Walker HM, Hope-Doolittle J, Kratochwill TR, Gresham FM. Proactive, early screening to detect behaviourally at-risk students: issues, approaches, emerging innovations, and professional practices. J Sch Psychol (2007) 45:193–223. doi:10.1016/j.jsp.2006.11.003

CrossRef Full Text | Google Scholar

52. Jensen P, Roper M, Fisher P, Piacentini J, Canino G, Richters J, et al. Test-retest reliability of the diagnostic interview schedule for children (DISC 2.1). Parent, child, and combined algorithms. Arch Gen Psychiatry (1995) 52:61–71. doi:10.1001/archpsyc.1995.03950130061007

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Velicer WF, Fava JL. Effects of variable and subject sampling on factor pattern recovery. Psychol Methods (1998) 3:231–51. doi:10.1037/1082-989X.3.2.231

CrossRef Full Text | Google Scholar

54. Leibenluft E, Rich BA. Pediatric bipolar disorder. Annu Rev Clin Psychol (2008) 4:163–87. doi:10.1146/annurev.clinpsy.4.022007.141216

PubMed Abstract | CrossRef Full Text | Google Scholar