Research Audit on Clinical Utility of Dimensional Disruptive Mood and Behavior Psychopathologies in Child and Adolescent Psychiatry Practice

To investigate the utility of dimensional psychopathologies of disruptive mood and behavior disorders (DBDs) by applying latent profile analysis (LPA) for characterization of youth referred to the tertiary outpatient clinic of child and adolescent psychiatry clinic and pharmacological treatment choices. One hundred fifty-eight children and adolescents with significant DBDs symptoms participated. Core dimensional psychopathologies of DBDs (irritability, callous-unemotional trait, and reactive-proactive aggressive behavior), DSM diagnoses, prescribed medications, and behavioral and emotional problems (Child Behavior Checklist, CBCL) were measured at baseline (clinic intake) and at 3-month follow-up. Latent Profile Analysis (LPA) was applied to characterize the study population based on the levels and interrelations among the core dimensional DBDs psychopathologies. Following LPA, the differences in clinical and treatment features between the latent classes were analyzed. LPA revealed two latent classes based on severity of DBDs symptoms. Class 1 (the moderate group) was characterized by relatively low scores on all trans-diagnostic indicators, whereas class 2 (the severe/critical group) showed higher levels of the dimensional psychopathologies and the majority of CBCL subscales. In addition, the severe/critical group was more often prescribed antipsychotic medications, and also experienced more frequent medication changes (addition, increasing the dose, and trial of different medications). Our findings suggested that application of LPA to a cluster of dimensional DBDs psychopathologies may provide valuable characterization of the youths referred to a tertiary outpatient child and adolescent psychiatric clinic, and offer insight into the providers' decision making on psychotropic medications, by overall severity of these psychopathologies rather than by single categorical diagnosis or single externalizing psychopathology.

Introduction

For the last three decades, there has been a steady and significant increase in the use of mental health care (1) and specifically psychotropic medications for pediatric patients (2). While there has been effort to determine efficacy of psychotropic medications for youths with mental health disorders (3), clinicians continue to face significant difficulties in providing optimal treatment. Indeed, there have been concerns raised about safe and appropriate use of psychotropic medication in pediatric patients (4, 5). One of the difficulties clinicians face is the complexity of patients they encounter where co-morbidities of categorical diagnoses is effectively the norm, specifically, see Figure 1 (6, 7). Theoretically, categorical diagnoses should provide guidance on medication choice [e.g., stimulant medications for Attention-Deficit/Hyperactivity Disorder (ADHD), serotonin selective reuptake inhibitor (SSRI) for depression, antipsychotic medication for schizophrenia, etc.,] (8). However, in reality, categorical diagnoses may have limitation on the identification of subgroups who might be more responsive to a treatment modality (9, 10). Moreover, heterogeneity in how the categorical disorders present at the case/individual level and the different co-morbidities shown by the majority of patients make this approach difficult to implement (8).

FIGURE 1

Figure 1. Mean score of class membership at baseline in each latent class. ARI-SR, ARI-PR, ICU-SR, ICU-PR, PAQ-SR, PAQ-PR, RAQ-SR, and RAQ-PR refer to indicators for LPA of aggression, irritability and callous-unemotional traits reported by youth and their caregivers. ARI, Affective Reactivity Index; AQ, Aggression Questionnaire ICU, Inventory of Callous Unemotional Traits; PR, parent-report; SR, self-report.

Our goal in this study was to assess how clinicians in a real-world setting, specifically a tertiary level outpatient clinic, prescribe psychotropic medications for pediatric mental health patients. In particular, we were interested in determining the extent to which co-occurring externalizing behavior/psychopathologies may influence treatment decision-making, because externalizing behavior/psychopathologies are the most common reason underpinning referral to pediatric mental health clinics (11). Given the very significant concern patients' families and their clinicians have regarding this symptom class, we suspect that co-occurring externalizing behavior might have a significant impact on treatment choice. Importantly, though, there are different forms of externalizing behavior/psychopathologies (12). Specifically, a distinction has been made between instrumental and reactive aggression (13). Instrumental aggression can occur with little or no overt provocation, in order to obtain a personal advantage, whereas reactive aggression implies a defensive response to perceived provocation (14, 15). Instrumental and reactive aggression have been particularly associated with callous-unemotional traits and irritability, respectively (16). Callous-unemotional traits include a relative lack of guilt/remorse and empathy for others (17). Irritability is characterized as an increased propensity to exhibit anger in response to frustration/non-reward relative to one's peers (18). Both callous-unemotional traits and irritability are seen in a variety of pediatric psychiatric diagnoses, including not only externalizing conditions [i.e., Oppositional Defiant Disorder (ODD), Conduct Disorder (CD) and ADHD] (19–21), but also internalizing conditions (particularly irritability) (21).

As such, we examined not only prescription practice as a function of mental health diagnoses but also as a function of externalizing psychopathologies. Given the associations between callous-unemotional (CU) traits, irritability, instrumental and reactive aggression, we applied a novel statistical method of latent profile analysis (LPA) which provides person-centered analysis on various variables to characterize a complex and heterogeneous population (22). We were particularly interested in the extent to which LPA might provide a valid characterization of the disruptive behavior disorder (DBDs) symptoms shown by the patients in this sample—and particularly the extent to which an LPA derived classification might be associated with psychotropic medication choices made by clinicians working with these patients.

To investigate the utility of dimensional psychopathologies, we recruited the youths with emotional and behavioral dysregulation. Emotional and behavioral dysregulation has been recognized as one of the most serious psychopathologies in children (23). It is one of the most common reasons children and adolescents present for mental health services (11, 24–26). The most prominent group of children whose main psychopathology is emotional and behavioral dysregulation are those with disruptive mood and behavior disorders (DBDs), including Attention Deficit Hyperactivity Disorder (ADHD) (20), Oppositional Defiant Disorder (ODD) (27), Conduct Disorder (CD) (28), and Disruptive Mood Dysregulation Disorder (DMDD) (29).

We made several predictions with respect to this research audit of clinical practice. First, we predicted that categorical diagnoses would predict medication choice; that is, (1) youths with ADHD would be more likely to be prescribed on stimulant medications and/or alpha-agonists than youths without ADHD diagnoses; (2) youths with diagnoses of depressive disorder and/or anxiety disorder would be more likely to be prescribed on SSRIs than youths without these diagnoses, and (3) youth with diagnoses of ODD and CD would be more likely to be prescribed on antipsychotic medication (for their aggression) than youths without these diagnoses. Second, and based on previous work (6, 30), we predicted that LPA would identify three categories of youth within the sample: (i) patients showing limited aggression, CU traits and irritability; (ii) patients showing significant aggression (instrumental and reactive) and elevated CU traits; and (iii) patients showing significant reactive aggression and irritability. Third, we predicted that the above LPA-identified categories would significantly predict providers' choice of psychiatric medications. Specifically, we predicted that patients showing significant aggression and elevated CU traits will be more likely to be prescribed on antipsychotic medication (for their aggression) than other psychiatric medications (31) while patients showing significant reactive aggression and irritability would be more likely to be prescribed on SSRIs than other psychiatric medications (32).

Methods

Procedures

Participating youths were recruited from the tertiary outpatient child and adolescent psychiatry clinic at a large academic medical center campus in the Midwest. Trained research assistants contacted the caregivers of eligible youths prior to their initial visit for psychiatric assessment/treatment, and provided information about the study. Inclusion and exclusion criteria for study participants were shown in the Supplementary Material. After obtaining written consent and assent, youths and their caregivers completed symptom profile measurements at baseline. Clinical characterization including psychiatric diagnosis was done through psychiatric interviewed by licensed and board-certified child and adolescent psychiatrists with participants and their parents, to adhere closely to common clinical practice. Participants' active psychiatric medications at the time of each study visit (baseline and 3-month follow-up) and their medications after each study visit (if their medications changed within 2 weeks of the visit by the providers) were recorded. The Institutional Review Board at the participating academic medical center approved the study procedures (UNMC IRB number: 586-18).

Diagnosis and psychiatric medication status were collected at each one of four data collection points [the initial visit (baseline), after the initial visit, the 3-months follow-up, and after the 3-months visit]. Symptom profiles and CBCL were collected at the initial visit and 3 months of follow-up. At the baseline, 158 youths provided information of sociodemographic characteristics and symptom profile measurements. Among them, 136 youths had identifiable information on diagnoses and medication at the initial visit. Twenty-two youths and their parents did not agree on providing information on their diagnoses or medication status. Of the four data collection points (before the initial visit, after the initial visit, before the 3 months visit, and after the 3 months visit), there were 83 youths who provided information of their diagnoses and medication status for all four data collection points, 15 for 3, 36 for 2, and 2 for only one data collection point, respectively. Among them, 15 patients didn't receive any medication (3 for all four data collection points, 3 for 3 and 9 for 2).

Measures

Sociodemographic Information

Upon enrollment, caregivers and youths provided information about the participants' age, gender, and racial ethnic background. Using the electronic medical record, trained research assistants recorded mental health diagnoses at each study visit (baseline and 3-month follow-up).

Medication Recording

Participants' active psychiatric medications at the time of each study visit (baseline and 3-month follow-up) and their medications after each study visit (if their medications changed within 2 weeks of the visit by the providers) were recorded. Participants' medications were recorded from the electronic chart of the clinical visit documentation. Medications were also cross-checked in the medication list of the participant's medical record. Psychiatric medications included those prescribed or managed by psychiatrists or which were specifically prescribed for treating psychiatric symptoms.

Symptom Profile Measurements

Both the participants and their caregivers completed symptom profile measurements at the time of their visit to the clinic for psychiatric assessment/treatment. For youths requiring assistance, a trained research assistant facilitated completion by reading the questionnaires to the child.

The Reactive-Proactive Aggression Questionnaire (RAQ and PAQ, respectively) was administered as self- and parent-report to assess two types of aggressive behavior (33). Both the participant and caregiver completed the Affective Reactivity Index (ARI) to assess irritability and emotion dysregulation (34), and the Inventory of Callous Unemotional Traits (ICU) (17). Also, the Child Behavior Checklist for Ages 6–18 (CBCL/6-18) was administered to caregivers (35) at baseline, and 3-month follow-up visits. A detailed description of the measurements used is shown in the Supplementary Material.

Statistical Analysis

Socio-Demographic and Clinical Characteristics

To characterize the study population, frequency distribution analysis was performed to identify the socio-demographic and clinical characteristics of all the participants.

Medication Choices by Categorical Diagnosis

To assess the medication choices (i.e., stimulant, SSRI, atypical antipsychotic, and alpha-adrenergic agonist) based on the categorical diagnosis, chi-square tests between the youths with and without a specific diagnosis were conducted, to examine the relationship between ADHD and stimulant/alpha-agonist medications, between Depressive Disorder/Anxiety Disorder and SSRI, and between ODD/CD and antipsychotic medications.

Identification and Description of the Latent Classes Derived From Symptom Profile

We conducted latent profile analysis (LPA) to identify distinct classes of individuals based on the unobserved latent profiles that generate patterns of responses on the three indicators (i.e., irritability, aggression, and callous-unemotional traits) (36). Latent profile models were tested using tidy LPA package version 0.2.4 (37) in R version 3.2.3 to find the optimal model. A detailed analysis method is provided in the Supplementary Material.

The optimal model has equal variances across classes and zero co-variances, as this is most in line with classical LPA and the default implementation in MPlus (38).

We standardized each variable using z-scores for clarity of interpretation. Then, LPA was conducted to determine latent, or unobserved, profile membership based on an individual's observed responses on each of the eight indicators including self- and parent- report of ARI, self- and parent- report of ICU, self- and parent-report of proactive AQs, and self- and parent-report of reactive AQs.

Generally, the number of latent profiles is determined through exploratory procedures by evaluating a number of different candidate models based on classification quality, parsimony indexes, and relative goodness of fit of two competing statistical models for the optimal model selection. For the study, a number of different candidate models were evaluated based on classification quality by entropy, parsimony indexes including Akaike Information Criterion (AIC) (39), Bayesian Information Criterion (BIC) (40), and Integrated Complete Likelihood (ICL) (41), and relative goodness of fit of two competing statistical models using Bootstrap Likelihood Ratio Test (BLRT) (42) for the optimal model selection.

The Differences of Symptom Profile and Socio-Demographic/Clinical Characteristics Between the Latent Classes

After the latent profile model was identified, differences in the symptom profiles, socio-demographic and diagnosis, and CBCL subscales were analyzed between the latent groups with chi-square test, t-test, and Mann-Whitney U-test.

Also, a series of repeated measures analysis of variance were conducted to assess whether there were group difference (between latent groups) and two assessment points (baseline and 3-month follow-up) on the eight symptom profile measurements.

Medication Choices by LPA Identified Groups

To assess the medication choices (i.e., stimulant, SSRI, atypical antipsychotic, and alpha-adrenergic agonist) and medication changes for 3 months based on the LPA identified groups, two sample t-test and chi-square tests between the latent classes were conducted.

Additional Analysis 1: Correlation of the Dimensional Psychopathologies

Because the LAP analysis did not divide the groups as we hypothesized (high irritability-low callous-unemotional trait-high reactive aggression vs. high callous-unemotional trait-high proactive/reactive aggression), we conducted additional analyses of the correlation of these psychopathologies in the two groups, to determine the relations between different types of aggression (reactive vs. proactive), irritability, and callous-unemotional trait.

Additional Analysis 2: LPA Analysis With Externalizing and Internalizing Psychopathologies

Due to the significant number of the participants who presented with internalizing diagnoses as their primary diagnoses (see Table 1), we conducted additional LPA analyses including CBCL internalizing psychopathologies in addition to the three disruptive mood and behavior psychopathologies. In the analyses including (1) overall internalizing problems, (2) withdrawn-depression, depressive problems, and anxiety problems, and (3) anxious-depressed score, withdrawn-depressed score, depressed problems, and anxiety problems.

TABLE 1

Table 1. Descriptive statistics on the socio-demographic characteristics of the subjects (N = 158).

Results

Socio-Demographic/Clinical Characteristics and Medication Change in All Participants

A total of 158 youths (51.27% male) who were referred to the outpatient child and adolescent psychiatry clinic participated in this study. Most of the participants were White (76.58%) and non-Hispanic (89.24%). Youths were predominantly junior high school students (48.10%) and an average of 13.50 years of age [Standard Deviation (SD) = 3.32]. The prevalence of mental disorders (N = 136) was as follows: ADHD (49.79%), depressive disorders (46.32%), anxiety disorders (44.12%) and CD/ODD (11.76%); see Table 1.

Prescribed medications from the youths and parents who provided information on this at the initial visit (N=136) were as follows: stimulant (29.41%), SSRI (27.94%), atypical anti-psychotics (17.65%), and alpha-adrenergic agonists (16.18%). Thirty-two youths (23.53%) took more than one medication among stimulant, SSRI, atypical anti-psychotics, and alpha-adrenergic agonists; see Table 1.

Medication Choices by Categorical Diagnoses

Table 2 shows cross-tabulation of medication choices by categorical diagnoses. As predicted, participants with ADHD had a significantly higher number of stimulant (60%) and alpha-adrenergic agonist (29.24%) medications prescribed for them than youths without ADHD diagnosis (Stimulant: 1.4%, X² = 56.11, p < 0.001 and alpha-adrenergic agonists: 4.23%, X² = 15.65, p < 0.001). SSRIs were significantly more often prescribed for depressive youths (44.44%) than youths without depression (13.70%, X² = 15.88, p < 0.001) but not for youths with anxiety diagnoses (anxiety group 33.33%, non-anxiety group 23.68%, X² = 2.55, p = 0.11). Participants with CD/ODD were not significantly more likely to be prescribed atypical antipsychotics than youths without these diagnoses (CD/ODD group 31.25%, non-CD/ODD group 15.83%, X² = 2.30, p = 0.13). However, youths with CD/ODD were significantly more likely to have alpha-adrenergic agonist (43.75%) prescriptions than youths without CD/ODD (12.5%, X² = 10.17, p = 0.001).

TABLE 2

Table 2. Cross tabulation of medication choices and categorical diagnoses (N = 136).

Identification and Description of the Latent Classes Derived From Symptom Profile

We predicted that LPA would identify three categories of youth within the sample: (i) patients showing limited aggression, CU traits and irritability; (ii) patients showing significant aggression (instrumental and reactive) and elevated CU traits; and (iii) patients showing significant reactive aggression and irritability.

Table S1 shows the LPA fit indices is identified by increasing the number of latent classes sequentially to determine the optimal latent model. The two-class solution was selected, by considering the ICL value, the posterior probability and the proportions of members in each class. A description of the model selection is presented in the Supplementary Material.

In a model consisting of two latent classes selected in this study, each individual was assigned to classes based on their model posterior probabilities and adjustments for classification error when estimating the class-specific distributions (43). As a result, 81 (51.27%) of the total youths were allocated for latent class 1 and 77 (48.73%) for latent class 2. The symptom profile characteristics of these classes were examined and class labels were assigned based on these characteristics (Figure 1). Both classes had clinically significant levels of all symptoms, but class 1 comprised significantly lower mean levels of all indicators than class 2 (p < 0.001). Thus, class 1 was labeled the moderate group and class 2 was labeled the severe/critical group; see Figure 1.

The Differences of Symptom Profile and Socio-Demographic/Clinical Characteristics Between the Latent Classes

The severe/critical group showed higher scores in all symptom profiles than the moderate group including irritability, callous-unemotional trait, and aggressive behavior (p < 0.001); see Table 3. Symptom profile changes in follow-up is shown in the Supplementary Material and Tables S2, S3.

TABLE 3

Table 3. Descriptive statistics on the indicator variables in each group (N = 158).

As for the socio-demographic characteristics, the severe/critical group were more likely to be male (X² =8.54, p = 0.004) compared to the moderate group. There was no significant age and racial/ethnic differences between groups. The severe/critical group was more likely to have a diagnosis of ADHD (X² =5.63, p = 0.02) and CD/ODD (X² = 6.00, p = 0.01) than the moderate group; see Table 4.

TABLE 4

Table 4. Differences in socio-demographic and diagnosis between two Latent Classes (N = 158).

In addition, the severe/critical group also showed significantly higher scores on symptom profiles measured by CBCL; see Figure S1 and Table 5.

TABLE 5

Table 5. Differences in CBCL scores between two Latent Classes (N = 158).

Medication Choices by LPA Identified Groups

We examined the use of antipsychotic and SSRI medications in the groups identified by LPA. The severe/critical group (26.98%) were significantly more likely to be prescribed atypical antipsychotic medication compared to the moderate group (9.59%, X² = 7.04, p = 0.01). In addition, the severe/critical group was more prescribed on alpha-adrenergic agonist compared to the moderate group (23.81% vs. 9.59%, X² = 5.04, p = 0.02) (Table 6). However, there is no differences in the SSRI prescription between the severe/critical (22.22%) and the moderate group (32.88%, X² = 1.91, p = 0.17).

TABLE 6

Table 6. Medication choices and changes for three months in each latent group.

In addition, we found that the rate of medication changes in the severe/critical group (0.67) was higher than that of the moderate group (0.49) (t = 2.89, p = 0.004) during the 3-months study period. The severe/critical group was more likely to have additional medications, such as stimulants (36.84%) and atypical antipsychotics (10.52%), but also SSRI (34.21%) or alpha-adrenergic agonists (26.32%) (X² = 5.88, p = 0.02) (Table 6).

Additional Analysis 1: Correlation of the Dimensional Psychopathologies

In the study group as a whole, there were positive associations between all four symptoms (ARI, ICU, RAQ, and PAQ; the correlation coefficient ranges from 0.37 to 0.79, p < 0.01). In the moderate group, only ICU and RAQ did not show positive correlation. In the severe/critical group, ICU-ARI and ICU-RAQ did not show positive correlations (Tables S4–S6). When the degrees of correlation were compared, ARI had significantly stronger correlation with RAQ than ICU (Z-score = 6.88, 3.35, and 4.26 within all participant, the moderate group and the severe/critical group, respectively, p < 0.001) or PAQ (Z-score = 5.31, 2.37, and 3.68, within all participant, the moderate group and the severe/critical group, respectively, p < 0.001 or p < 0.01); see Tables S7–S10.

Additional Analysis 2: LPA Analysis With Externalizing and Internalizing Psychopathologies

The results of the LPA, including other indicators relevant to internalizing symptoms (i.e., internalizing problems, withdrawn-depressed score, depressive problems, anxiety problems, or anxious-depressed score), can be found in Tables S11–S13. Even though other indicators were included for LPA, all yielded repetition of two-class solution (severe/critically ill vs. moderately ill) as the best fitting models; see Tables S11–S13 and Figures S2–S4. Since the entropy values indicate a similar fit for all class solutions, they did not help much in choosing among the models. AIC values indicated a similar fit for the two-, and three-class solutions. However, BIC- and ICL-values were inspected, with these values indicating the two-class solution to perform best. Also, the BLRT p-value was significant for only one-class model, which was the comparison result between the two-class and the one-class models, and other BLRTs failed to reach significance comparing the other classes models (at α = 0.05). Taken together, the two-class model seemed the most relevant to inspect.

Discussion

In this study, we aim to identify the utility of dimensional disruptive behavior and mood psychopathologies in characterizing pediatric patients who were referred for psychiatric assessment/treatment, and its utility in provider's decision making, especially choice of psychotropic medications. To achieve this, we applied LPA to classify children and adolescents by their trans-diagnostic disruptive mood and behavior psychopathologies (i.e., irritability, callous-unemotional trait and reactive/proactive aggression) and assessed whether the subclasses differed significantly in clinical characterization as well as provider's medication treatment choices.

There are three main findings: (1) As we predicted, categorical psychiatric diagnoses of ADHD and depressive disorders were significantly related to being prescribed on stimulant/alpha-agonist medications and SSRIs, respectively. However, there was no further relation between categorical diagnoses and medication choices, especially including prescription of antipsychotic medications with or without diagnoses of ODD/CD. (2) There were two latent groups (critical/severe and moderate) explicitly distinguished by the severity of their irritability, callous-unemotional traits, and aggressive behavior, but not by form of symptom as we had expected. Our hypothesis of distinguishing youths as high irritability/low callous-unemotional trait/high reactive aggression vs. high callous-unemotional trait/high proactive/reactive aggression was not affirmed. (3) Lastly, the latent groups were associated with the medication choices made by the providers; the severe/critical group showed significantly higher rates of prescriptions for atypical antipsychotics and alpha-adrenergic agonists. Also, the severe/critical group showed significantly higher rates of medication changes and being prescribed additional psychiatric medications.

The first hypotheses on the role of categorical diagnoses in medication decision was partially confirmed. Stimulant and alpha-adrenergic agonist have long been used in youths with ADHD (44) and antidepressants, particularly the SSRIs, for youths with depression and anxiety disorders (45, 46). However, diagnosis of anxiety disorders was not related to prescription of SSRIs, which might be due to the fact that the non-anxiety group included significant numbers of depressive youths without comorbid anxiety (n = 30). In addition, we failed to see any relation between prescription of atypical antipsychotic and diagnoses of CD/ ODD, which has been a common practice for their aggressive/violent behavior (47). This might be a type II error, due to the small sample size in this category (n = 16).

Secondly, the results of LPA demonstrated that, contrary to our hypothesis, latent groups were distinguished by severity of overall psychopathologies, rather than by pattern of the externalizing psychopathologies. The subgroups did not reveal the potential dichotomy of high irritability/low callous-unemotional trait/high reactive aggression vs. high callous-unemotional trait/high proactive aggresion as we expected (6, 30). However, it should be also pointed out that the correlation analyses revealed that there was significantly stronger correlation between irritability and reactive aggression, than irritability and proactive aggresison or callous-unemotional trait, which partially affirmed our hypothesis 1 (see Tables S4–S6).

The subgroups (moderate vs. serious/critical) identified by LPA provided intriguing clinical points. Class 1 (the moderate group) was characterized by relatively low scores on all trans-diagnostic indicators and the youths in this group were more likely to be girls, and to be rated higher on the social competence domain, school competence domain, total competence domain among the CBCL subscales. In contrast, those in class 2 (the severe/critical group) showed higher scores on all trans-diagnostic indicators, and were more likely to be boys, to have diagnoses of ADHD and CD. Also, they showed higher scores on almost every subscale of CBCL. This finding might suggest that, the clusters of the external psychopathologies together (irritability, callous-unemotional trait, and reactive/proactive aggression) may provide better explanation of a couple of aspects of the patients referred to the tertiary child and adolescent psychiatry clinic including: (1) Overall severity of the patients driven by all these external psychopathologies and (2) Medication choices (especially atypical antipsychotics as well as frequency of medication changes/addition of psychiatric medications).

Previous studies based on variable-centered approaches including factor analysis have suggested that childhood psychiatric disorders can be divided into internalizing and externalizing categories (48). However, in this sample, the severe/critical group scored significantly higher in all the symptom profiles in the CBCL including internalizing psychopathology (depressive problems, anxious depressive symptoms, and anxiety problems; see Table 5) compared to the moderate group. This indicates that at least for patient clusters differentiated by external psychopathologies (irritability, callous-unemotional trait, and aggressive behavior), internalizing behavior problem were also higher in the group with severe externalizing problems. This may be unique to the patient group referred to the tertiary child and adolescent psychiatry clinic, and future study is warranted by including larger group of children and adolescents with internalizing problems. Rather, the overall severity of disruptive behavior and irritability can better characterize the patient popupation. Recently, the National Institute of Mental Health Research Domain Criteria initiative conducted an LPA using transdiagnostic symptom profiles, including irritability, anxiety, depressive, and ADHD symptoms, on youths with disruptive mood dysregulation disorder, anxiety disorder, ADHD, bipolar spectrum disorder to derive a class or pattern of psychiatric symptoms for youths that cross conventional nosologic boundaries (49). In line with our finding, they observed latent groups showing high or low levels of overall symptomaology but not “pure” classes exhibiting elevations in only one, or in a cluster of, symptom dimension(s). These findings highlight the need to consider the level of symptoms within individuals, rather than focusing solely on the presence of disorder specific symptoms for decision making of clinical characterization and treatement. Future studies are warranted to validate the utility of this approach for a large youth population with mental health issues in general.

We were able to find that the groups identified by the LPA application on the dimensional psychopathologies provided insightful explanation on the clinical practice of psychiatric medications for youths with disruptive mood and behavior symptoms.

First, SSRIs were more prescribed in the moderate group although this was not statistically significant (32.88 vs. 22.22% in the severe/critical group). However, at the same time it is noteworthy that some of the internalizing psychopathologies showed higher scores in the severe/critical group (depressive problems with statistical significance, and internalizing problems, anxiety problems, as well as anxious depressed; see Table 5). It is possible that the providers may tend to focus on depressive/anxiety symptoms when there is lesser degree of disruptive mood/behavior symptoms (although objective scores indicate that the severe/critical group also scores high or even higher on anxiety/depressive symptoms) since disruptive mood/behavior symptoms may tend to display themselves more explicitly during the clinical visit for assessment.

Second, alpha adrenergic agonists and antipsychotics have shown the effectiveness on treatment of comorbid symptoms of ADHD (38, 39), especially aggression, irritability, and low frustration tolerance (40, 41). Thus, it seems clinically justifiable that the providers at the tertiary level clinic choose alpha adrenergic agonists when there were severe symptom presentations, which were mainly driven by disruptive mood and disruptive psychopathologies, even in youths with primary diagnoses of Anxiety and Depressive Disorders (see Table 6). However, overall effectiveness of alpha-adrenergic agent in reducing the severity of symptom presentations driven by disruptive mood and behavior psychopathologies remain to be determined by further studies.

In addition, the providers at this clinic tended to choose antipsychotic medications when youths present more severe/critical symptom presentations overall. Antipsychotic medications have been used for severe aggression especially in the diagnoses of ADHD, ODD, CD, and Autism Spectrum Disorder (50–52). However, since our study indicated that the prescription of atypical antipsychotics was more related to the overall severity of psychiatric symptom presentations, not specific categorical disruptive mood or behavior disorders, the effectiveness of prescribing antipsychotics in this fashion requires further inspection as well.

These findings lead to a speculation that the providers at the tertiary outpatient child psychiatry clinic, with experience in treating very sick patients, have already relied on the overall clinical assessment/impression of the degree of severity of collective psychopathologies as well as functional impairments, rather than pursued amelioration of symptom items based on categorical diagnosis (53). These results show that a dimensional approach to pediatric psychopathologies with latent individual analysis may provide insight into the current clinical practice in this particular real-life setting. Also, since the main decision in routine clinical practice is not generally whether to treat or not, but to match the intensity of the symptom presentation with the level of intervention (54), it may be natural that the intervention strategies correspond to overall dimensional severity of cluster of psychiatric symptoms than individual and/or comorbid diagnoses.

Lastly, it is critical to note that the youths in the severe/critical group experienced more frequent medication changes such as addition of new medications, increase of the dosages of existing medications, and changes to existing medications. This may reflect the real-world struggle of the providers who have to provide treatment for youths with severe level of psychopathologies, facing lack of clear guideline/algorithm or even evidence-based treatment options. Better characterization of the youths with severe psychopathologies by such methods applied in this study, as dimensional psychopathologies instead of categorical diagnoses, and latent profile analyses rather than variable analyses hopefully would guide the future direction of the clinical studies and treatment options.

There are a few caveats to offer. We only examined how the severity of indicators affected decision making for the current prescription and could not provide medication response such as the effectiveness and side effects. As of now we continue to do follow-up visits of these participants at 3, 6, 9, and 12 months, hoping to provide further insight into this question. The sample size in some disorders with low prevalence (i.e., CD/ODD) was not statistically valid in the analysis to identify the medication prediction, which might affect the statistical power. Likewise, it may not have been possible to detect the potential relationship between disorders with low prevalence and class membership. However, there are sufficient data in LPA and statistical analysis for identifying the differences between group separated by the LPA. Lastly, the trans-diagnostic indicators used in this study consisted mainly of assessing the severity of externalizing symptoms. This limits the generalization of our findings into the whole spectrum of childhood psychiatric symptoms by excluding the interpretations of internalizing disorders (i.e., depressive disorders, anxiety disorders, obsessive-compulsive and related disorders, trauma and stress-related disorders, and dissociative disorders) and relevant treatment intervention (i.e., SSRI, Anti-anxiety, Non-SSRI Antidepressant). However, this limitation is mitigated by the additional LPA analyses and their similar result to the main analysis, including internalizing psychopathologies (Tables S11–S13, Figures S2–S4). In future work it will be helpful to integrate the internalizing symptoms such as rumination (55) into our multidimensional approach, which may provide insight into common childhood pathophysiological mechanisms.

Despite some limitations, our results provide the empirical evidence for the utility of dimensional approach to understand childhood psychiatric disorders and treatment intervention. Externalizing psychopathologies (pediatric irritability, callous-unemotional trait, and aggression) seem to differentiate youths referred for psychiatric assessment and treatment at a tertiary child and adolescent psychiatric clinic as severe (high scores in all of these pathologies) and moderate (lower scores in all of these pathologies). Medication choices (antipsychotics, frequency of medication changes, and adding of psychiatric medications) by the providers are better explained by these clusters. We suggest these trans-diagnostic factors/psychopathologies to be taken into clinical practice and future research of child and adolescent psychiatry, in addition to the traditional categorical diagnoses.

Data Availability Statement

The raw de-identified data supporting the conclusions of this article may be available upon request by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by University of Nebraska Medical Center (IRB Number 586-16). Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.

Author Contributions

BS-V, RB, and SH conceived and planned the experiments. WG, AB, and AL carried out the experiments. RE and KC contributed to sample preparation. J-WS, RB, KP, and SH contributed to the interpretation of the results. SH took the lead in writing the manuscript. All authors provided critical feedback and helped shape the research, analysis and manuscript, contributed to the article, and approved the submitted version.

Funding

This work was supported by the National Institute of Mental Health (U01MH120155) and IDeA CTR grant (1U54GM115458).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2022.742148/full#supplementary-material

Abbreviations

ADHD, Attention-Deficit/Hyperactivity Disorder; AIC, Akaike Information Criterion; ARI, Affective Reactivity Index; BIC, Bayesian Information Criterion; BLRT, Bootstrap Likelihood Ratio Test; CBCL, Child Behavior Checklist; CD, Conduct Disorder; CU, callous-unemotional trait; DBDs, disruptive mood and behavior disorders; ICL, Integrated Complete Likelihood; ICU, Inventory of Callous Unemotional Traits; LPA, latent profile analysis; ODD, Oppositional Defiant Disorder; PAQ, Proactive Aggression Questionnaire; RAQ, Reactive Aggression Questionnaire; SD, Standard Deviation; SSRI, serotonin selective reuptake inhibitor.

References

1. Olfson M, Druss BG, Marcus SC. Trends in mental health care among children and adolescents. N Engl J Med. (2015) 372:2029–38. doi: 10.1056/NEJMsa1413512

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Jack RH, Hollis C, Coupland C, Morriss R, Knaggs RD, Butler D, et al. Incidence and prevalence of primary care antidepressant prescribing in children and young people in England, 1998–2017: A population-based cohort study. PLoS Med. (2020) 17:e1003215. doi: 10.1371/journal.pmed.1003215

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Steiner H, Remsing L. Practice parameter for the assessment and treatment of children and adolescents with oppositional defiant disorder. J Am Acad Child Adolesc Psychiatry. (2007) 46:126–41. doi: 10.1097/01.chi.0000246060.62706.af

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Comer JS, Olfson M, Mojtabai R. National trends in child and adolescent psychotropic polypharmacy in office-based practice, 1996-2007. J Am Acad Child Adolesc Psychiatry. (2010) 49:1001–10. doi: 10.1016/j.jaac.2010.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Linares LO, Martinez-Martin N, Castellanos FX. Stimulant and atypical antipsychotic medications for children placed in foster homes. PLoS ONE. (2013) 8:e54152. doi: 10.1371/journal.pone.0054152

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Blair RJR, Leibenluft E, Pine DS. Conduct disorder and callous–unemotional traits in youth. N Engl J Med. (2014) 371:2207–16. doi: 10.1056/NEJMra1315612

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Brodbeck J, Stulz N, Itten S, Regli D, Znoj H, Caspar F. The structure of psychopathological symptoms and the associations with DSM-diagnoses in treatment seeking individuals. Compr Psychiatry. (2014) 55:714–26. doi: 10.1016/j.comppsych.2013.11.001

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Giles LL, Martini DR. Challenges and promises of pediatric psychopharmacology. Acad Pediatr. (2016) 16:508–18. doi: 10.1016/j.acap.2016.03.011

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Bech P, Lucas R, Amir M, Bushnell D, Martin M, Buesching D, et al. Association between clinically depressed subgroups, type of treatment and patient retention in the LIDO study. Psychol Med. (2003) 33:1051–9. doi: 10.1017/S0033291703008249

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Lanza ST, Rhoades BL. Latent class analysis: an alternative perspective on subgroup analysis in prevention and treatment. Prev Sci. (2013) 14:157–68. doi: 10.1007/s11121-011-0201-1

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Avenevoli S, Blader JC, Leibenluft E. Irritability in youth: an update. J Am Acad Child Adolesc Psychiatry. (2015) 54:881–3. doi: 10.1016/j.jaac.2015.08.012

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Bonadio FT, Tompsett C. Who benefits from community mental health care? Using latent profile analysis to identify differential treatment outcomes for youth. J Youth Adolesc. (2018) 47:2320–36. doi: 10.1007/s10964-018-0888-4

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Raine A, Dodge K, Loeber R, Gatzke-Kopp L, Lynam D, Reynolds C, et al. The reactive–proactive aggression questionnaire: differential correlates of reactive and proactive aggression in adolescent boys. Aggress Behav. (2006) 32:159–71. doi: 10.1002/ab.20115

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Deveney CM, Stoddard J, Evans RL, Chavez G, Harney M, Wulff RA. On defining irritability and its relationship to affective traits and social interpretations. Pers Individ Dif. (2019) 144:61–7. doi: 10.1016/j.paid.2019.02.031

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Rosan AM, Costea-Barlutiu C. Associations between callous-unemotional traits, aggression and psychopathology in detained adolescent males. J Cogn Behav Psychotherapies. (2013) 13 397–407.

Google Scholar

16. Fanti KA, Frick PJ, Georgiou S. Linking callous-unemotional traits to instrumental and non-instrumental forms of aggression. J Psychopathol Behav Assess. (2009) 31:285. doi: 10.1007/s10862-008-9111-3

CrossRef Full Text | Google Scholar

17. Frick PJ. The Inventory of Callous-Unemotional Traits. Unpublished Rating Scale. University of New Orleans (2004).

Google Scholar

18. Evans SC, Burke JD, Roberts MC, Fite PJ, Lochman JE, Francisco R, et al. Irritability in child and adolescent psychopathology: an integrative review for ICD-11. Clin Psychol Rev. (2017) 53:29–45. doi: 10.1016/j.cpr.2017.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Barry CT, Frick PJ, DeShazo TM, McCoy M, Ellis M, Loney BR. The importance of callous–unemotional traits for extending the concept of psychopathy to children. J Abnorm Psychol. (2000) 109:335. doi: 10.1037/0021-843X.109.2.335

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Karalunas SL, Fair D, Musser ED, Aykes K, Iyer SP, Nigg JT. Subtyping attention-deficit/hyperactivity disorder using temperament dimensions: toward biologically based nosologic criteria. JAMA psychiatry. (2014) 71:1015–24. doi: 10.1001/jamapsychiatry.2014.763

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Vidal-Ribas P, Brotman MA, Valdivieso I, Leibenluft E, Stringaris A. The status of irritability in psychiatry: a conceptual and quantitative review. J Am Acad Child Adolesc Psychiatry. (2016) 55:556–70. doi: 10.1016/j.jaac.2016.04.014

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Bonadio FT, Dynes M, Lackey J, Tompsett C, Amrhein K. Grouping youth with similar symptoms: a person-centered approach to transdiagnostic subgroups. J Clin Psychol. (2016) 72:676–88. doi: 10.1002/jclp.22274

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Krieger FV, Leibenluft E, Stringaris A, Polanczyk GV. Irritability in children and adolescents: past concepts, current debates, and future opportunities. Braz J Psychiatry. (2013) 35:S32–9. doi: 10.1590/1516-4446-2013-S107

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Brotman MA, Schmajuk M, Rich BA, Dickstein DP, Guyer AE, Costello EJ, et al. Prevalence, clinical correlates, and longitudinal course of severe mood dysregulation in children. Biol Psychiatry. (2006) 60:991–7. doi: 10.1016/j.biopsych.2006.08.042

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Collishaw S, Maughan B, Natarajan L, Pickles A. Trends in adolescent emotional problems in England: a comparison of two national cohorts twenty years apart. J child psychol psychiatry. (2010) 51:885–94. doi: 10.1111/j.1469-7610.2010.02252.x

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Peterson BS, Zhang H, Santa Lucia R, King RA, Lewis M. Risk factors for presenting problems in child psychiatric emergencies. J Am Acad Child Adolesc Psychiatry. (1996) 35:1162–73. doi: 10.1097/00004583-199609000-00014

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Herzhoff K, Tackett JL. Subfactors of oppositional defiant disorder: converging evidence from structural and latent class analyses. J Child Psychol Psychiatry. (2016) 57:18–29. doi: 10.1111/jcpp.12423

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Euler F, Jenkel N, Stadler C, Schmeck K, Fegert JM, Kölch M, et al. Variants of girls and boys with conduct disorder: anxiety symptoms and callous-unemotional traits. J Abnorm Child Psychol. (2015) 43:773–85. doi: 10.1007/s10802-014-9946-x

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Dougherty L, Smith V, Bufferd S, Carlson G, Stringaris A, Leibenluft E, et al. DSM-5 disruptive mood dysregulation disorder: correlates and predictors in young children. Psychol Med. (2014) 44:2339–50. doi: 10.1017/S0033291713003115

PubMed Abstract | CrossRef Full Text | Google Scholar

30. E. Leibenluft. Pediatric irritability: a systems neuroscience approach. Trends Cogn Sci. (2017) 21:277–89. doi: 10.1016/j.tics.2017.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Gadow KD, Brown NV, Arnold LE, Buchan-Page KA, Bukstein OG, Butter E, et al. Severely aggressive children receiving stimulant medication versus stimulant and risperidone: 12-month follow-up of the TOSCA trial. J Am Acad Child Adolesc Psychiatry. (2016) 55:469–78. doi: 10.1016/j.jaac.2016.03.014

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Towbin K, Vidal-Ribas P, Brotman MA, Pickles A, Miller KV, Kaiser A, et al. A double-blind randomized placebo-controlled trial of citalopram adjunctive to stimulant medication in youth with chronic severe irritability. J Am Acad Child Adolesc Psychiatry. (2020) 59:350–61. doi: 10.1016/j.jaac.2019.05.015

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Dodge KA, Coie JD. Social-information-processing factors in reactive and proactive aggression in children's peer groups. J Pers Soc Psychol. (1987) 53:1146. doi: 10.1037/0022-3514.53.6.1146

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Stringaris A, Goodman R, Ferdinando S, Razdan V, Muhrer E, Leibenluft E, et al. The affective reactivity index: a concise irritability scale for clinical and research settings. J Child Psychol Psychiatry. (2012) 53:1109–17. doi: 10.1111/j.1469-7610.2012.02561.x

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Achenbach TM, Dumenci L, Rescorla LA. Ratings of Relations Between DSM-IV Diagnostic Categories and Items of the CBCL/6-18, TRF, and YSR. Burlington, VT: University of Vermont (2001). p. 1–9.

Google Scholar

36. Lanza ST, Cooper BR. Latent class analysis for developmental research. Child Dev Perspect. (2016) 10:59–64. doi: 10.1111/cdep.12163

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Rosenberg JM, Beymer PN, Anderson DJ, Van Lissa C, Schmidt JA. tidyLPA: an R package to easily carry out latent profile analysis (LPA) using open-source or commercial software. J Open Source Softw. (2019) 3:978. doi: 10.21105/joss.00978

CrossRef Full Text | Google Scholar

38. Muthén L, Muthén B. MPlus for Windows 7.31. Los Angeles, CA: Muthén & Muthén (2015).

39. Akaike H. Factor Analysis and AIC, Selected Papers of Hirotugu Akaike. Berlin: Springer (1987). p. 371–86.

Google Scholar

40. Schwarz G. Estimating the dimension of a model. Ann Statist. (1978) 6:461–4. doi: 10.1214/aos/1176344136

CrossRef Full Text | Google Scholar

41. Biernacki C, Celeux G, Govaert G. Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans Pattern Anal Mach Intell. (2000) 22:719–25. doi: 10.1109/34.865189

CrossRef Full Text | Google Scholar

42. G.J. McLachlan. On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture. J R Stat Soc Ser C. (1987) 36:318–24. doi: 10.2307/2347790

CrossRef Full Text | Google Scholar

43. Bakk Z, Vermunt JK. Robustness of stepwise latent class modeling with continuous distal outcomes. Struct Equ Model Multidisciplinary J. (2016) 23:20–31. doi: 10.1080/10705511.2014.955104

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Pliszka S, AACAP AACAP Work Group on Quality Issues. Practice parameter for the assessment and treatment of children and adolescents with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry. (2007) 46:894–921. doi: 10.1097/chi.0b013e318054e724

PubMed Abstract | CrossRef Full Text | Google Scholar

45. B. Birmaher. D. Brent, and AWGoQ Issues, Practice parameter for the assessment and treatment of children and adolescents with depressive disorders. J Am Acad Child Adolesc Psychiatry. (2007) 46:1503–26. doi: 10.1097/chi.0b013e318145ae1c

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Hughes CW, Emslie GJ, Crismon ML, Posner K, Birmaher B, Ryan N, et al. Texas children's medication algorithm project: update from Texas consensus conference panel on medication treatment of childhood major depressive disorder. J Am Acad Child Adolesc Psychiatry. (2007) 46:667–86. doi: 10.1097/chi.0b013e31804a859b

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Findling RL. Atypical antipsychotic treatment of disruptive behavior disorders in children and adolescents. J Clin Psychiatry. (2008) 69(Suppl. 4):9–14.

PubMed Abstract | Google Scholar

48. Krueger RF. The structure of common mental disorders. Arch Gen Psychiatry. (1999) 56:921–6. doi: 10.1001/archpsyc.56.10.921

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Kircanski K, Zhang S, Stringaris A, Wiggins JL, Towbin KE, Pine DS, et al. Empirically derived patterns of psychiatric symptoms in youth: a latent profile analysis. J Affect Disord. (2017) 216:109–16. doi: 10.1016/j.jad.2016.09.016

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Birnbaum ML, Saito E, Gerhard T, Winterstein A, Olfson M, Kane JM, et al. Pharmacoepidemiology of antipsychotic use in youth with ADHD: trends and clinical implications. Curr Psychiatry Rep. (2013) 15:382. doi: 10.1007/s11920-013-0382-3

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Olfson M, Blanco C, Liu S-M, Wang S, Correll CU. National trends in the office-based treatment of children, adolescents, and adults with antipsychotics. Arch Gen Psychiatry. (2012) 69:1247–56. doi: 10.1001/archgenpsychiatry.2012.647

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Toteja N, Gallego JA, Saito E, Gerhard T, Winterstein A, Olfson M, et al. Prevalence and correlates of antipsychotic polypharmacy in children and adolescents receiving antipsychotic treatment. Int J Neuropsychopharmacol. (2014) 17:1095–105. doi: 10.1017/S1461145712001320

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Waszczuk MA, Zimmerman M, Ruggero C, Li K, MacNamara A, Weinberg A, et al. What do clinicians treat: diagnoses or symptoms? The incremental validity of a symptom-based, dimensional characterization of emotional disorders in predicting medication prescription patterns. Compr Psychiatry. (2017) 79:80–8. doi: 10.1016/j.comppsych.2017.04.004

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Krueger RF, Kotov R, Watson D, Forbes MK, Eaton NR, Ruggero CJ, et al. Progress in achieving quantitative classification of psychopathology. World Psychiatry. (2018) 17:282–93. doi: 10.1002/wps.20566

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Grierson AB, Hickie IB, Naismith SL, Scott J. The role of rumination in illness trajectories in youth: linking trans-diagnostic processes with clinical staging models. Psychol Med. (2016) 46:2467–84. doi: 10.1017/S0033291716001392

PubMed Abstract | CrossRef Full Text | Google Scholar