Does screening mode matter? A repeated cross-sectional study of computer self-administered vs. clinician-administered screening of youth substance use in pediatric primary care

Introduction: Universal youth substance use screening in pediatric primary care enables early detection and intervention, which, in turn, can help decrease the risk of problematic substance use. Screening mode [electronic self-administered survey (SA) vs. clinician-administered interview (CA)] may influence whether substance use is reported and, therefore, clinical decisions about whether and how to intervene.

Methods: We performed a secondary data analysis of substance use screening responses collected between 2018 and 2022 from individuals aged 12–20 years who were seen at 314 US pediatric practices utilizing the Comprehensive Health and Decision Information System (CHADIS) online clinical process support system. Patients responded to the Car, Relax, Alone, Forget, Family/Friends, and Trouble (CRAFFT), a well-validated adolescent substance use screening tool that measures past-12-month alcohol, cannabis, and other substance use (“anything else to get high”). We compared substance use rates by screening mode (SA vs. CA) using logistic regression modeling with generalized estimating equations to account for data clustering within practices and patients, controlling for US region, sex, submission year, and patient age in days. We stratified analyses by age group (12–13; 14–15; 16–17; 18–20 years) and sex (male vs. female).

Results: Data represented 201,134 screening responses among N = 130,688 patients. Patients were 50.9% female; 31.3% were from the Northeast, 6.7% from the Midwest, 52.7% from the South, and 9.4% from the West. Of the screening responses, 24.6% were from 12–13-year-olds, 29.5% from 14–15-year-olds, 28.7% from 16–17-year-olds, and 17.2% from 18–20-year-olds. Mode for the screening responses was 74.9% SA and 25.1% CA. Compared with CA screening, SA screening was associated with significantly higher adjusted odds of report of any substance use (adjusted odds ratio, 95% confidence interval by age group: 12–13 years 1.75, 1.43–2.15; 14–15 years 1.21, 1.11–1.33; 16–17 years 1.32, 1.24–1.41; 18–20 years 1.48, 1.39–1.58). Alcohol and cannabis, the most prevalent past-12-month substances used among all age groups, demonstrated similar patterns when examined individually. Report of other substance use only differed by screening mode among 12–13-year-olds, but overall, prevalence was low (0.1%–2.1%).

Conclusion: Electronic self-administered screening was associated with higher rates of reported substance use compared with clinician-administered interviews among youth being seen in primary care, suggesting that self-administered screening may improve substance use detection.

1 Introduction

The burden of substance use among youth between the ages of 12 and 21 years remains a significant public health concern in the United States (1, 2). An estimated one in three high school 12th graders, one in five 10th graders, and one in ten 8th graders reported using substances in the past year (3). The most common substances used by adolescents include alcohol and cannabis (4). Deaths due to drug overdose among youth have almost tripled since the COVID-19 pandemic (5). This trend has been postulated to reflect the increase in risk factors for adolescent substance use problems driven by the pandemic, such as heightened stress, social isolation, reduced school connection, and boredom, along with an increasingly potent and toxic drug supply (1).

Adolescence and young adulthood are important developmental periods for the onset of substance use problems due to evolving neurodevelopment, social influences, and increased reward sensitivity (6–8). Youth who use substances are reported to have higher rates of physical and mental illness, impaired psychological and neurocognitive development, increased risk for academic failure, and reduced overall health and well-being (9, 10). Furthermore, early substance use also correlates with substance use problems later in life. Adolescents who initiate substance use before the age of 14 years are at the highest risk for substance dependence and have a 34% lifetime prevalence of substance use (11). Between the ages of 13 and 21 years, the likelihood of lifetime substance use and dependence decreases by 4%–5% for each year that first-time substance use is delayed, providing further evidence that early substance use initiation leads to greater substance dependence later in life (11). As such, youth-centered approaches to substance use prevention and early intervention are critical tools to promote positive youth developmental and health outcomes while preventing later substance use disorders in adulthood (12).

Pediatricians in primary care settings are uniquely positioned to intervene and prevent the early initiation of substance use during this critical developmental stage (12). Routine pediatric office visits, including well-child visits, are an important venue for providing education, promoting abstinence, and delivering counseling to decrease the risk of problematic use (13). For these reasons, the American Academy of Pediatrics (AAP) recommends universal youth substance use screening in primary care starting at age 11 (14). Regular screenings can help identify substance use and help patients and families receive evidence-based interventions earlier, promoting improved treatment outcomes (15).

Several easy-to-administer, validated tools exist to conduct screening and guide assessment for substance use-related problems (16), including the Car, Relax, Alone, Forget, Family/Friends, and Trouble (CRAFFT). The CRAFFT is an efficient, well-validated, and widely used screening tool designed to identify substance use, substance-related riding/driving risk, and possible substance use disorder among youth ages 12 through 21 (17–19). It is designed to be self-administered (electronic or paper) or administered by a clinician interview. The CRAFFT has been recommended by the AAP's Bright Futures Guidelines for Health Supervision for preventive care screenings and well visits; the Center for Medicare and Medicaid Services and CHIP Services’ Early and Periodic Screening, Diagnosis and Treatment (EPSDT) program; and the National Institute of Alcohol Abuse and Alcoholism (NIAAA) Youth Screening Guide (20) (Table 1).

Table 1

Table 1. The CRAFFT questionnaire (version 2.0) (20).

Screening mode—self- vs. clinician-administered—may influence substance use detection and subsequent decisions about whether and how to intervene. The reliability and validity of self-reported youth substance use data have varied across studies, leading to calls for the implementation of multiple screening modalities to improve detection accuracy (21, 22). In one study, computerized self-administered screening was found to be a valid and efficient modality, with similar sensitivity and specificity as clinician-based screening (23). Furthermore, research has shown that under confidential conditions, youth prefer self-administered questionnaires over clinician-administered questionnaires (24). This may be related to the relative privacy of self-administered screening, as well as the idea that youth may be more open to disclosing socially undesirable behaviors, such as substance use, without face-to-face interactions with clinicians (24). Self-administered screening may be more efficient in practice, as it can allow more time during the visit for the clinician to conduct further assessment and offer counseling (14, 23). Indeed, a recent study demonstrated that standardized substance use screening with adolescents self-administering tablet-based CRAFFT questionnaires resulted in more completed screening, less biased screening responses, and better substance use detection in comparison with other forms of screening, including clinician-administered screening (25). Furthermore, with COVID-19 accelerating the uptake of electronic health interventions in primary care practices across the US, computerized self-administered screening is an increasingly available tool that is suitable for both in-person and telemedicine visits (26).

While previous studies have examined youth reports of substance use between self-administered and clinician-administered modalities across pediatric primary care sites in a single jurisdiction (single city or state), no study to date has compared the administration modes using a national sample. Analyzing national data can both elucidate and account for local and regional differences in attitudes, practices, and policies pertaining to pediatric primary care (27, 28). The Child Health and Development Interactive System (CHADIS), an online clinical process support tool, is in use in 48 states and serves as a unique, population-based data source from which we could draw a national sample of adolescent primary care patients for our study (29). CHADIS is designed to assist in the early identification and management of a wide range of health issues through pre-visit screening and moment-of-care decision support, including CRAFFT screening, patient and provider resources, and tailored counseling for risk behaviors (30). The primary aim of our repeated cross-sectional study is to examine whether self-administered vs. clinician-administered CRAFFT screening mode impacts rates of substance use reported by youth visiting pediatric primary care providers utilizing the CHADIS online clinical process support system. The secondary aim was to examine whether sex moderated the association between screening mode and reported substance use rates across age groups (31). We hypothesized that self-administered screening generates higher rates of substance use reported by youth attending pediatric primary care practices in comparison with clinician-administered screening.

2 Methods

2.1 Study setting and participants

Data for the current study were collected from adolescent patients of US pediatric practices that utilized the CHADIS online clinical process support system between January 2018 to December 2022. This study includes data collected prior to the onset of the COVID-19 pandemic in the U.S. (2018, 2019) and after its onset (2020, 2021, and 2022). Since the pandemic impacted both adolescent substance use patterns and the uptake of telehealth and electronic health interventions in primary care settings, the analysis accounted for the periods of time spanning from pre- to post-pandemic (26, 32).

The analytic sample included patients 12–20 years of age who completed at least 1 response to the CRAFFT screening tool (33). Patients aged 21 years or older were excluded because young adults aged ≥21 can legally purchase and consume alcohol across all US states, and as of 2022, young adults aged ≥21 can legally purchase and consume cannabis across 18 US states (34). Relative to their younger counterparts, the prevalence of any past-12-month substance use among those ≥21 years old is greater, and there is presumptively less stigma to report these legal behaviors, which may attenuate the relative difference in disclosure between screening modalities. The Boston Children's Hospital Institutional Review Board (IRB) deemed this secondary data analysis exempt from full board review. The Biomedical Research Alliance of New York (BRANY) IRB approved this activity for BH and RS of CHADIS, Inc. A data use agreement was established between CHADIS, Inc. and Boston Children's Hospital study teams to allow for the sharing of limited deidentified data sets.

2.2 Screening modality

The CRAFFT was either self-administered, in which questions were administered by patient self-administration via electronic questionnaire before or during a clinical visit, per preferences and standards of the clinician and practice, or delivered by clinician interview verbally to the patient and documented by the clinician. Responses were tabulated for clinician-administered questionnaires and self-administered questionnaires separately. Screening responses collected from anyone besides the clinician or patient, such as a parent or caregiver, were excluded.

Because data from questionnaire submission years 2018–2022 were examined, unique patients may be represented multiple times, for example, due to re-screening at annual well visits or presenting for follow-up visits during this time period. We identified records associated with the same patient collected proximally to each other (within days or even minutes). Multiple records for the same patient within the same calendar day may have been a result of technical glitches (e.g., initial response was un-saved or timed out, prompting a new instance to be created) or standard practices in clinic flow (e.g., assigning the patient a self-administered questionnaire in the waiting room before the visit and then a clinician administering the same questionnaire by interview during the visit). We implemented a decision rule to include only one CRAFFT response per patient per calendar day: if multiple responses collected within the same calendar day represented only patient- or clinician-administered responses, we chose to keep the most recent response of the set; if the multiple responses represented a mix of patient- and clinician-administered responses, we chose to keep the most recent patient self-administered response. We selected the most recent response for two reasons. Firstly, a prior study identified an order effect, whereby more substance use was disclosed on the latter screen, irrespective of screening mode, when adolescent patients completed two substance use screens in the same visits (23). Secondly, because technical glitches may have resulted in duplicates, we assumed that the most recent responses may have been the most accurate. We used the calendar day as a cut point because responses collected ≥1 day apart could reasonably constitute different yet valid responses to the CRAFFT, which asks about the number of days of substance use in the past 12 months. Since this study involved a secondary analysis of CHADIS data, pre hoc sample size calculations were not performed, and the sample size was determined by data availability and the aforementioned inclusion criteria.

2.3 Measures

2.3.1 Sociodemographic variables

Region, sex, submission year, and patient age had to have sufficiently complete data for inclusion in these analyses. Race/ethnicity and health insurance type were not included due to high missingness (>90% for both).

2.3.1.1 Age

Ages were stratified into four age groups (12–13, 14–15, 16–17, and 18–20 years old) to account for increasing levels of substance use prevalence across increasing age groups. Youth aged 16–17 have a higher prevalence of illicit drug use in comparison with those aged 12–13 and 14–15 (35), and young adults aged 18–25 are more likely to initiate or increase substance use than any other age group (36). Age stratification reduces bias by minimizing the effects of age on screening modality and reported substance use. We selected the cut points for the age categories (12–13, 14–15, 16–17, and 18–20), as ages 12–13 represents early adolescence and typically correspond to middle school (37), ages 14–15 typically correspond to the start of high school, ages 16–17 correspond to later high school years (junior, senior), and ages 18–20 correspond to the start of college (38). We also calculated the mean age and standard deviation of patients within each age category in years as a continuous variable. Within each age category, we further examined age in days as a continuous variable.

2.3.1.2 Geographic region

Geographic region was categorized in accordance with the four major US Census Bureau regions (Northeast, Midwest, South, West) (39). States in which offices were located were manually assigned to their respective region. The Northeast was set as the reference category (39).

2.3.1.3 Year

The year of CRAFFT completion was operationalized into a nominal variable with five categories (comparing 2019, 2020, 2021, and 2022 with the reference year 2018).

2.3.1.4 Sex

Sex data were available as male/female in this dataset. We set male as the reference category.

2.3.2 Screening mode

Screening modality was coded into a two-level categorical variable with self-administered or clinician-administered interview, with clinician-administered serving as the reference category.

2.3.3 Substance use variables

The CRAFFT screening tool was used to identify substance use and substance use risk via the CHADIS web-based clinical decision support system among youth aged 12–20 years visiting pediatric primary care settings. This study focused on patient responses to the CRAFFT screen for past-12-month substance use, including alcohol, cannabis, and other drugs (“anything else to get high”). For the measure “any substance use in the past 12 months,” which includes use of alcohol, cannabis, or other drugs, we dichotomized responses to “≥1 days of substance use” vs. “0 days of substance use” (reference level). We also dichotomized responses to alcohol use, cannabis use, and use of other substances to “≥1 days of use” vs. “0 days of substance use” (reference level).

2.4 Statistical analysis

RStudio 2022.07.1+554 was used to conduct all analyses. Descriptive statistics were tabulated for the distributions of categorical variables (region, sex, submission year), stratified by age category (12–13, 14–15, 16–17, 18–20), and the mean age and standard deviation of patients within each age category in years as a continuous variable.

To examine characteristics associated with the outcome of self-administered vs. clinician-administered screening, the proportion of patients undergoing each mode of screening was tabulated for each level of region, sex, and submission year, stratified by age. We then conducted bivariate analyses between each of the region, sex, and submission year, as well as a logistic regression model via generalized estimating equations (GEE) with exchangeable correlation structure, adjusting for region, sex, and submission year and accounting for potential clustering within practice and within patient.

To compare the prevalence of reported substance use by screening mode (self-administered vs. clinician-administered), the proportion of patients reporting use of any substance, alcohol, cannabis, and other drugs by both self-administered and clinician-administered screening was tabulated, stratified by age. Then, unadjusted and adjusted GEE models with exchangeable correlation structure were used to examine whether rates of substance use reported by youth depended on self-administered vs. clinician-administered CRAFFT screening mode. Models were stratified and reported by age category, and both unadjusted and adjusted odds ratios (ORs) and 95% confidence intervals, accounting for data clustering within practice and within patient. Adjusted models controlled for the following potential confounders: region, sex, submission year, and, within each age category, age in days as a continuous variable. Additional sub-analyses with unadjusted and adjusted GEE models were conducted to compare the prevalence of reported substance use by screening mode (self-administered vs. clinician-administered) stratified by both age category and sex. Historically, adolescent males have generally had higher rates of substance use than females (40). Stratifying for both age and sex reduces bias by minimizing the effects of age and sex on reported substance use. Unadjusted and adjusted odds ratios and 95% confidence intervals, accounting for data clustering within practice and within patient, were reported. Adjusted odds ratios controlled for region, sex, submission year, and patient age within each age category in days. We performed subsequent sub-analyses stratified by sex to examine whether sex moderated the association between screening mode and reported substance use rates across age groups. The results of additional sub-analyses are reported in the Supplementary Material. Listwise deletion was performed on missing data for all analyses due to low rates of missingness (<5%) (41).

3 Results

3.1 Sample characteristics

This sample included 201,134 responses from 130,688 unique patients across 314 pediatric practices in 40 US states. The median number of screening responses per patient was 1 (interquartile range, 1–2; full range, 1–18 separate visits; 63% had one screening response, 24.6% had two, 9.0% had three, and 3.5% had four or more). Among unique patients, 50.9% were female, and 31.3% were from the Northeast, 6.7% from the Midwest, 52.7% from the South, and 9.4% from the West. In the analytic sample, 51.5% identified as female; 24.6% aged 12–13 years, 29.5% between 14 and 15, 28.7% between 16 and 17, and 17.2% between 18 and 20. The majority of screens were at practices located in the South (49.1%) and Northeast (34.7%), with a minority from the West (9.4%) and Midwest (6.9%). Most screening records (74.9%) were self-administered, with 25.1% collected via clinician-administered interviews. The proportion of screening that was self-administered increased with older age (Table 2).

Table 2

Table 2. Sociodemographic patient characteristics.

3.2 Characteristics associated with screening mode

There were no sex differences in the likelihood of self-administered screening across age groups, except for among 18–20-year-olds, where females had 1.08 times the adjusted odds (95% CI: 1.01–1.15) of self-administered screening compared with males, controlling for region and submission year (Table 3). In all age groups, compared with youth in the Northeast, those in the South had significantly lower adjusted odds of self-administered screening, which can be attributed to the lower prevalence of self-administered screening in the South relative to all other regions; youth in the West had significantly higher adjusted odds. Youth in the Midwest had higher adjusted odds of self-administered screening in comparison with their Northeastern counterparts in all age groups except for the youngest age category, where there was no difference. Finally, the adjusted odds of self-administered screening increased year on year across all age categories. Unadjusted ORs followed a similar pattern to adjusted ORs.

Table 3

Table 3. Characteristics associated with screening mode, stratified by age group.

3.3 Substance use rates by screening mode

Overall use rates self-reported on screening varied by type of substance (i.e., alcohol, cannabis, and other drugs), with alcohol having the highest prevalence across all age categories. As expected, use rates for all substances increased with age, with the lowest rates reported by 12–13-year-olds and the highest rates reported by 18–20-year-olds (Table 4). There was a low prevalence of use of other substances (“anything else to get high”) (0.1–2.1%) across all age groups.

Table 4

Table 4. Age-Stratified rates of self-reported substance use, by screening mode.

Self-administered screening was associated with higher rates reported of any substance use and alcohol use, compared with clinician-administered screening, across all age strata in both adjusted and unadjusted analyses (Figure 1). The youngest age group had the greatest difference by screening mode for both overall substance use and alcohol use in adjusted and unadjusted analyses (Table 4). Cannabis use followed a similar pattern, with the youngest age group showing the largest difference by screening mode. However, the difference was substantially attenuated in the 14–15 age group and was statistically significant in only the unadjusted analysis (OR: 1.29, 95% CI: 1.17–1.43), while the adjusted analysis was not statistically significant [adjusted OR (AOR) 1.12, 95% CI: 0.99–1.26]. For use of other substances (“anything else to get high”), we found a significant effect only among 12–13-year-old patients in both the adjusted and unadjusted analyses, with self-administered screening yielding a 2.67 times higher adjusted rate of other substance use reported compared with clinician-administered screening (95% CI: 1.55–4.57).

Figure 1

Figure 1. Age-stratified adjusted odds ratios (95% CI) of substance use, by screening mode. ORs from GEE models, adjusting for region, sex, submission year, and age within each age category in days, accounting for data clustering within practice and within patient.

In subsequent analyses stratified by sex, we found the pattern of associations largely similar between sex groups, with a few differences (Supplementary Material). Among 14–15-year-old girls, there was a significant screening mode effect found in both the unadjusted and adjusted analyses for any substance use and any alcohol use, but not for other substance use. For 14–15-year-old girls, cannabis only had significant screening mode effects in the unadjusted analysis (OR: 1.31, 95% CI: 1.13–1.51) but was non-significant in the adjusted analyses. Among 14–15-year-old boys, we found no adjusted significant screening mode effect for any type of substance. However, unadjusted screening mode effects for any type of substance were statistically significant for 14–15-year-old boys. Furthermore, the screening mode effect on self-reported alcohol use became non-significant in adjusted analyses among 12–13-year-old boys but not in 12–13-year-old girls. There was a significant screening mode effect in both adjusted and unadjusted analyses for use of other substances (“anything else to get high”) among 12–13-year-old boys but not in same-age girls.

4 Discussion

In this large, national, multi-year sample of data from substance use screening of youth aged 12–20 being seen in pediatric primary care, self-administered screening was associated with higher rates of reported substance use, compared with clinician-administered screening. The pattern of findings was significant across most age and sex categories and across the substance used.

Our findings of increased reports of substance use on self-administered screening are consistent with the literature, noting that adolescents prefer self-administered screening over clinician interviews (42). This may be due to factors such as social desirability, parental presence, or confidentiality concerns in clinician interviews.

Screening mode effects also varied somewhat by adolescent age group and substance type. In our study, the likelihood of self-administered screening increased as adolescent patients got older, likely reflecting the higher rates of adolescents presenting at visits on their own as they get older (43). Differences in substance use rates between self-administered and clinician-administered screening tended to be largest in the youngest age group (12–13-year-olds) for all substance types, possibly reflecting that this age group is least likely to get confidential time alone with their health care provider (43), thus increasing the disparity in substance use reports between screening modes. Interestingly, we found a more attenuated screening mode effect, after adjustment for region and submission year, on substance use reporting among 14–15-year-olds, particularly among 14–15-year-old boys, compared with the effect seen in other age groups. Clinicians may spend more time with male adolescents, compared with female adolescents, which could have led to higher rates of clinician-administered screening for substance use (43).

Cannabis and other substance use rates were associated with less of a screening mode effect than alcohol among both boys and girls in the 14–15-year-old age group, likely due to low prevalence rates reported on both self-administered and clinician-administered screening for these substances. Surprisingly, we still found a screening mode effect among the oldest age group, 18–20-year-olds, among whom substance use is more common and may be viewed as less stigmatizing or sensitive to disclose.

There was a low prevalence of use of other substances across all age groups. This aligns with existing evidence suggesting that alcohol and cannabis are among the most commonly used substances among adolescents (44), especially in light of the rise in adolescent and young adult substance use post-recreational cannabis legalization (45). Other substances, including illegal drugs, over-the-counter and prescription drugs, and inhalants, remain less common (46). Our results only demonstrated a significant difference between the detection of other substance use by self-administered vs. clinician-administered screening for patients in the 12–13-year-old age category, which may also be attributed to this age group being least likely to receive time alone with their provider (43).

While self-report is among the most common methods of assessing adolescent substance use, it has been deemed by researchers as potentially inaccurate due to carelessness or confusion (47, 48). However, other studies have shown that youth self-administered questionnaires offer a valid and efficient alternative to clinician-administered questionnaires, with one study examining the validity of a Brief Screener for Tobacco, Alcohol, and other Drugs (BSTAD) finding that adolescents ages 12–17 preferred iPad self-administration over interviewer administration (42).

The prevalence of substance use in our pediatric primary care-based study was lower than rates reported in national school-based surveys such as the Monitoring the Future Survey and the Youth Risk Behavior Survey. For example, in the 2021 Youth Risk Behavior Survey, the prevalence of alcohol use among youth in grade 9 (ages 14–15) is 14.7%, while in our study, the prevalence was only 5.4% among 14–15-year-olds (49). This may reflect both sample selection bias (adolescents seen for primary care well visits may not include those who are most vulnerable to substance use, such as those experiencing housing instability) and hesitancy around substance use disclosure in healthcare settings (as opposed to an anonymous survey). Furthermore, with the growing use of patient portal systems in health care during the study period, youth in our study may have underreported substance use due to concerns that parents/guardians could access substance use screening data through their patient portals (50).

We also found that the odds of self-administered screening increased each year in our study period (2018–2022) in all age groups, with particularly large increases seen during the COVID-19 pandemic (2020–2022). While implementation of electronic health record systems in primary care was on a gradually increasing trajectory even prior to the COVID-19 pandemic, the pandemic accelerated their implementation substantially, which may have contributed to the pronounced increases seen in self-administered screening rates from 2020 to 2022 (26). Adolescent males and females were generally similar in their likelihood of self-administered screening vs. clinician-administered, except among youth aged 18–20 years, where there was a small increase in odds of self-administered screening among females in comparison with their male counterparts. These results align with previous research demonstrating that adolescent preferences related to routine preventive screening for reported health risk behaviors (e.g., alcohol, tobacco, drugs, or sexual activity) in terms of paper, provider, or electronic screening, demonstrating that there was no difference in reported preference of screening modality by gender (51).

Furthermore, relative to those in the Northeast, patients in the South had significantly lower odds of reporting self-administered screening, which can be attributed to the lower prevalence of self-administered screening in the South across all years relative to other regions. This finding may reflect potential cultural and practice pattern variations across geographic regions in the United States, with a prior study highlighting that pediatricians in the South, compared with their Northeastern counterparts, were half as likely to report use of a standardized screening instrument and had almost four times the odds of citing patient reluctance to discussing substance use as a barrier to adolescent substance use screening (52).

While our study suggests that self-administered screening may be preferable to clinician-administered screening, uptake of standardized screening using well-validated tools remains limited in the United States. Indeed, a recent national survey of US pediatricians found that only 41% of those surveyed reported using a standardized instrument, with the most frequently reported screening barriers being a lack of treatments for adolescents who screen positive, a lack of on-site substance use counseling providers, and requiring more information on counseling options. Of note, the majority of pediatricians expressed that it was challenging to identify outpatient and inpatient behavioral interventions or medication-based interventions given a positive screen (53). These findings highlight a need to scale up training efforts for clinicians to respond to positive screens, while expanding the availability of standardized screening and referral resources.

5 Strengths and limitations

Our study reports substance use rates in a geographically large sample of pediatric primary care patients, and it is the first to examine the question of appropriate substance use screening modalities with a national sample in this setting. The current study has many strengths, including its large sample size and geographic diversity, which strengthen the external validity of the study. Furthermore, the well-validated CRAFFT clinician interview employs the same standard questions in the same order as the self-administered tool, reducing the possibility of observations being explained by item-level or other differences between screening modalities. The aggregate findings also span between 2018 and 2022 (pre- and post-pandemic years) reflecting the recent, pandemic-driven rise in the use of electronic health interventions, and the GEE models we used in our analyses accounted for clustering within practice and patient, since patients may have returned for repeated visits during the observation period and practices may differ in screening patterns to a greater extent between pediatric practices, rather than within pediatric practices.

There are limitations to consider when interpreting the results of this study. Race, ethnicity, and insurance status variables were largely missing from our dataset; hence, we were unable to understand differences by race, ethnicity, and socioeconomic status (for which insurance status may be a proxy). We were also unable to report on gender identity due to a lack of data availability. We could not ascertain from the data whether clinician interviews were administered in the presence of parents/guardians or whether screening results were visible to parents/guardians in the patient's electronic health record, which may lead youth to underreport substance use, particularly in age groups where parents are more likely to be present at visits (54). While self-administered questionnaires were completed by patients electronically either before or during the visit, clinician-administered questionnaires may have been administered by clinicians either verbally interviewing patients and inputting responses into CHADIS or logging into the CHADIS platform and allowing the patient to self-complete the questionnaire during the visit. Since the latter method of clinician-administered interviews closely resembles self-administration, and we were unable to differentiate between the different ways in which clinicians administered the questionnaires, the effect size observed in our study might be diminished. Likewise, in our systematic removal of multiple responses collected within the same day, it was not possible to know whether we analyzed the “true” response; repeated administration of the same questionnaire may have influenced responses. Another limitation is that we were unable to disaggregate the type of substance use being reported in response to the “anything else to get high” item (i.e., use of other substances). Finally, the 10 states for which CHADIS data were unavailable are among the most rural in the United States (i.e., Arkansas, Iowa, Maine, Minnesota, Mississippi, Nebraska, New Hampshire, North Dakota, South Dakota, and West Virginia), suggesting lower uptake of CHADIS in rural practices. However, we cannot describe how practices included in this dataset compare to practices not included beyond broader state-level differences or determine how generalizable these findings are to youth seen in pediatric practices not included in this dataset.

6 Conclusions and implications

Overall, there was a greater report of substance use through self-administered screening vs. clinician-administered screening among youth presenting to primary care practices. Our findings thus imply that pediatric primary care practices may wish to consider self-administered substance use screening over clinician interviews to facilitate the detection of substance use in youth patients and to inform clinical response. By extent, our findings suggest that there is value in training clinicians in how to respond to positive screens (e.g., brief intervention) while maintaining patient confidentiality (e.g., by leveraging electronic health record features to control parent/guardian access to sensitive health information). Furthermore, our study highlights that self-administered screening has become increasingly prevalent across US pediatric primary care practices in recent years. Given current gaps in standardized substance use screening practices, our study findings provide support for the universal implementation of self-administered screening to support prevention and early intervention (53). For youth, self-administered screening in a confidential setting may be more comfortable than a face-to-face interview with their pediatrician about substance use, a sensitive topic. For providers, whose reluctance to do any screening is often related to time constraints, adopting universal self-administered screening (which patients can do independently in advance) implies an added benefit of preserving important time during the visit. Knowing that the majority of adolescents in this sample had negative substance use screens, providers could use additional time to offer brief preventive care, psychoeducation, or anticipatory guidance during the visit, and when necessary, intervention or referral to treatment—practices that are recommended by leaders in pediatrics (52, 55).

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: The source dataset is proprietary to CHADIS, Inc. (info@chadis.com). The limited deidentified dataset created for analysis is proprietary to CHADIS, Inc., and can be provided upon reasonable request. Permission should be obtained from the corresponding author SH, Boston Children’s Hospital (sion.harris@childrens.harvard.edu), and from CHADIS, Inc. (info@chadis.com).

Ethics statement

The studies involving humans were approved by BRANY and Boston Children's Hospital Institutional Review Boards. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

Author contributions

CG: Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing. MO: Conceptualization, Formal analysis, Methodology, Project administration, Visualization, Writing – original draft, Writing – review & editing. BH: Data curation, Funding acquisition, Investigation, Methodology, Resources, Software, Supervision, Writing – review & editing. RS: Data curation, Funding acquisition, Investigation, Methodology, Resources, Software, Supervision, Writing – review & editing. LS: Investigation, Methodology, Supervision, Writing – review & editing. SH: Conceptualization, Formal analysis, Investigation, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This research was supported by NIDA Award #R44DA046262. Content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Acknowledgments

Earlier findings from these analyses were presented at the AMERSA 46th Annual Conference, 10–12 November 2022, Boston, MA, and at the 19th INEBRIA Conference, 28–29 September 2023, Greensboro, NC. We thank Diana Kennedy, Julia Plumb, and Cynthia Tran for their contributions to formative work on these analyses. We would also like to thank the clinicians and patients who contributed the data used for these analyses.

Conflict of interest

Authors Raymond Sturner and Barbara Howard are employed by company CHADIS, Inc. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence, and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fradm.2025.1694040/full#supplementary-material

References

1. Sarvey D, Welsh JW. Adolescent substance use: challenges and opportunities related to COVID-19. J Subst Abuse Treat. (2021) 122:108212. doi: 10.1016/j.jsat.2020.108212

PubMed Abstract | Crossref Full Text | Google Scholar

2. Ball A, Hadland S, Rodean J, Hall M, Mendoza J, Ahrens K. Trends in substance-related visits among youth to US children’s hospitals, 2016–2021: an analysis of the pediatric health information system database. J Adolesc Health. (2024) 75(1):76–84. doi: 10.1016/j.jadohealth.2024.02.016

PubMed Abstract | Crossref Full Text | Google Scholar

3. Johnston LD, Miech RA, O’Malley PM, Bachman JG, Schulenberg JE, Patrick ME. Monitoring the Future National Survey Results on Drug Use, 1975–2019: Overview, Key Findings on Adolescent Drug Use. Ann Arbor: Institute for Social Research (2020). Available online at: https://eric.ed.gov/?id=ED604018 (Accessed February 9, 2022)

Google Scholar

4. Miech RA, Johnston LD, Patrick ME, O’Malley PM, Bachman JG. Monitoring the Future National Survey Results on Drug Use, 1975–2023: Secondary School Students. Ann Arbor, MI: Institute for Social Research, University of Michigan (2023. Monitoring the Future Monograph Series.

Google Scholar

5. Panchal N. Recent Trends in Mental Health and Substance Use Concerns Among Adolescents. San Francisco: The Kaiser Family Foundation (2024). Available online at: https://www.kff.org/mental-health/issue-brief/recent-trends-in-mental-health-and-substance-use-concerns-among-adolescents/

Google Scholar

6. Schulte MT, Hser YI. Substance use and associated health conditions throughout the lifespan. Public Health Rev. (2013) 35:1–27. doi: 10.1007/BF03391702

Crossref Full Text | Google Scholar

7. Urošević S, Collins P, Muetzel R, Schissel A, Lim KO, Luciana M. Effects of reward sensitivity and regional brain volumes on substance use initiation in adolescence. Soc Cogn Affect Neurosci. (2014) 10(1):106–13. doi: 10.1093/scan/nsu022

PubMed Abstract | Crossref Full Text | Google Scholar

8. Whitesell M, Bachand A, Peel J, Brown M. Familial, social, and individual factors contributing to risk for adolescent substance use. J Addict. (2013) 2013:579310. doi: 10.1155/2013/579310

PubMed Abstract | Crossref Full Text | Google Scholar

9. Layman H, Thorisdottir I, Halldorsdottir T, Sigfusdottir I, Allegrante J, Kristjansson A. Substance use among youth during the COVID-19 pandemic: a systematic review. Curr Psychiatry Rep. (2022) 24:307–24. doi: 10.1007/s11920-022-01338-z

PubMed Abstract | Crossref Full Text | Google Scholar

10. Bugbee BA, Beck KH, Fryer CS, Arria AM. Substance use, academic performance, and academic engagement among high school seniors. J Sch Health. (2019) 89(2):145–56. doi: 10.1111/josh.12723

PubMed Abstract | Crossref Full Text | Google Scholar

11. Jordan CJ, Andersen SL. Sensitive periods of substance abuse: early risk for the transition to dependence. Dev Cogn Neurosci. (2017) 25:29–44. doi: 10.1016/j.dcn.2016.10.004

PubMed Abstract | Crossref Full Text | Google Scholar

12. Matson P, Ridenour T, Ialongo N, Spoth R, Prado G, Hammond C, et al. State of the art in substance use prevention and early intervention: applications to pediatric primary care settings. Prev Sci. (2021) 23:204–11. doi: 10.1007/s11121-021-01299-4

PubMed Abstract | Crossref Full Text | Google Scholar

13. Yonek JC, Velez S, Satre DD, Margolis K, Whittle A, Jain S, et al. Addressing adolescent substance use in an urban pediatric federally qualified health center. J Subst Abuse Treat. (2022) 135:108653. doi: 10.1016/j.jsat.2021.108653

PubMed Abstract | Crossref Full Text | Google Scholar

14. Hagan JF Jr, Shaw JS, Duncan PM. Bright Futures Guidelines and Pocket Guide. Itasca: American Academy of Pediatrics (2017). Available online at: https://www.aap.org/en/practice-management/bright-futures/bright-futures-materials-and-tools/bright-futures-guidelines-and-pocket-guide/?srsltid=AfmBOop3C6en0qs9RPTWECO_y6fgSqxMEdb_HSJ2CyEchfSOVecc5z3a (Accessed March 23, 2025).

Google Scholar

15. Bagley SM, Hadland SE, Schoenberger SF, Gai MJ, Topp D, Hallett E, et al. Integrating substance use care into primary care for adolescents and young adults: lessons learned. J Subst Abuse Treat. (2021) 129:108376. doi: 10.1016/j.jsat.2021.108376

PubMed Abstract | Crossref Full Text | Google Scholar

16. D’Amico EJ, Parast L, Meredith LS, Ewing BA, Shadel WG, Stein BD. Screening in primary care: what is the best way to identify at-risk youth for substance use? Pediatrics. (2016) 138:e20161717. doi: 10.1542/peds.2016-1717

Crossref Full Text | Google Scholar

17. Knight JR, Sherritt L, Shrier LA, Harris SK, Chang G. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. (2002) 156(6):607–14. doi: 10.1001/archpedi.156.6.607

PubMed Abstract | Crossref Full Text | Google Scholar

18. Knight JR, Sherritt L, Harris SK, Gates EC, Chang G. Validity of brief alcohol screening tests among adolescents: a comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcohol Clin Exp Res. (2003) 27(1):67–73. doi: 10.1097/01.ALC.0000046598.59317.3A

Crossref Full Text | Google Scholar

19. Knight JR, Shrier LA, Bravender TD, Farrell M, Vander Bilt J, Shaffer HJ. A new brief screen for adolescent substance abuse. Arch Pediatr Adolesc Med. (1999) 153(6):591–6. doi: 10.1001/archpedi.153.6.591

PubMed Abstract | Crossref Full Text | Google Scholar

20. CRAFFT. About the CRAFFT. Baltimore: CHADIS, Inc. (n.d.). Available online at: https://crafft.org/about-the-crafft/ (Accessed March 23, 2025).

Google Scholar

21. Delaney-Black V, Chiodo L, Hannigan J, Greenwald M, Janisse J, Patterson G, et al. Just say “I don’t”: lack of concordance between teen report and biological measures of drug use. Pediatrics. (2010) 126:887–93. doi: 10.1542/peds.2009-3059

PubMed Abstract | Crossref Full Text | Google Scholar

22. Wade NE, Sullivan RM, Tapert SF, Pelham WE 3rd, Huestis MA, Lisdahl KM, et al. Concordance between substance use self-report and hair analysis in community-based adolescents. Am J Drug Alcohol Abuse. (2023) 49(1):76–84. doi: 10.1080/00952990.2023.2164931

PubMed Abstract | Crossref Full Text | Google Scholar

23. Harris SK, Knight JR, Hook SV, Sherritt L, Brooks TL, Kulig JW, et al. Adolescent substance use screening in primary care: validity of computer self-administered versus clinician-administered screening. Subst Abuse. (2016) 37(1):197–203. doi: 10.1080/08897077.2015.1014615

PubMed Abstract | Crossref Full Text | Google Scholar

24. Knight JR, Harris SK, Sherritt L, Hook SV, Lawrence N, Brooks T, et al. Adolescents’ preferences for substance abuse screening in primary care practice. Subst Abuse. (2007) 28(4):107–17. doi: 10.1300/J465v28n04_03

Crossref Full Text | Google Scholar

25. Ranginwala M, Greiner MV, Fox K, Unkrich J, Beal SJ. Reducing bias in substance use detection in adolescents via standardized substance use screening. Pediatr Open Sci. (2025) 1(1):1–3. doi: 10.1542/pedsos.2024-000157

Crossref Full Text | Google Scholar

26. Chang JE, Lai AY, Gupta A, Nguyen AM, Berry CA, Shelley DR. Rapid transition to telehealth and the digital divide: implications for primary care access and equity in a post-COVID era. Milbank Q. (2021) 99(2):340–68. doi: 10.1111/1468-0009.12509

PubMed Abstract | Crossref Full Text | Google Scholar

27. Shipman SA, Lan J, Chang Ch, Goodman DC. Geographic maldistribution of primary care for children. Pediatrics. (2011) 127(1):19–27. doi: 10.1542/peds.2010-0150

PubMed Abstract | Crossref Full Text | Google Scholar

28. Gentili M, Harati P, Serban N, O’Connor J, Swann J. Quantifying disparities in accessibility and availability of pediatric primary care across multiple states with implications for targeted interventions. Health Serv Res. (2018) 53(3):1458–77. doi: 10.1111/1475-6773.12722

PubMed Abstract | Crossref Full Text | Google Scholar

29. CHADIS The Patient Engagement and Online Screening - Assessment Solution with Clinical Process Support. Baltimore: CHADIS (n.d.). Available online at: https://site.chadis.com/ (Accessed March 23, 2025).

Google Scholar

30. Howard BJ, Sturner R. Use of an online clinical process support system as an aid to identification and management of developmental and mental health problems. Curr Dev Disord Rep. (2017) 4(4):108–17. doi: 10.1007/s40474-017-0124-z

PubMed Abstract | Crossref Full Text | Google Scholar

31. McHugh RK, Votaw VR, Sugarman DE, Greenfield SF. Sex and gender differences in substance use disorders. Clin Psychol Rev. (2018) 66:12–23. doi: 10.1016/j.cpr.2017.10.012

PubMed Abstract | Crossref Full Text | Google Scholar

32. Pelham WE, Tapert SF, Gonzalez MR, McCabe CJ, Lisdahl KM, Alzueta E, et al. Early adolescent substance use before and during the COVID-19 pandemic: a longitudinal survey in the ABCD study cohort. J Adolesc Health. (2021) 69(3):390–7. doi: 10.1016/j.jadohealth.2021.06.015

PubMed Abstract | Crossref Full Text | Google Scholar

33. US Census Bureau. Census.gov. Geographic Levels. Suitland: United States Census Bureau (2021). Available online at: https://www.census.gov/programs-surveys/economic-census/guidance-geographies/levels.html (Accessed March 23, 2025).

34. Martins SS, Levy NS, Bruzelius E, Segura LE. Cannabis legalization in the US. Where do we go from here? Trends Psychiatry Psychother. (2022) 44(Suppl 1):e20220001. doi: 10.47626/2237-6089-2022-0001

PubMed Abstract | Crossref Full Text | Google Scholar

35. Swendsen J, Burstein M, Case B, Conway KP, Dierker L, He J, et al. Use and abuse of alcohol and illicit drugs in US adolescents. Arch Gen Psychiatry. (2012) 69(4):390–8. doi: 10.1001/archgenpsychiatry.2011.1503

PubMed Abstract | Crossref Full Text | Google Scholar

36. Lu W, Lopez-Castro T, Vu T. Population-based examination of substance use disorders and treatment use among US young adults in the National Survey on Drug Use and Health, 2011–2019. Drug Alcohol Depend Rep. (2023) 8:100181. doi: 10.1016/j.dadr.2023.100181

PubMed Abstract | Crossref Full Text | Google Scholar

37. HealthyChildren.org. Stages of Adolescence. Itasca: American Academy of Pediatrics (2019). Available online at: https://www.healthychildren.org/English/ages-stages/teen/Pages/Stages-of-Adolescence.aspx (Accessed June 16, 2025).

Google Scholar

38. LeVos JS. Average Age by School Grade Level: A Guide for Parents. New York: Begin Learning (2024). Available online at: https://www.beginlearning.com/parent-resources/what-grade-are-x-year-olds/ (Accessed June 16, 2025).

Google Scholar

39. Census Regions and Divisions of the United States.

Google Scholar

40. Simon KM, Levy SJ, Bukstein OG. Adolescent substance use disorders. NEJM Evid. (2022) 1(6):EVIDra2200051. doi: 10.1056/EVIDra2200051

PubMed Abstract | Crossref Full Text | Google Scholar

41. Lodder P. To impute or not impute, that's the question. In: Mellenbergh GJ, Adèr HJ, editors. Advising on Research Methods: Selected Topics 2013. Huizen: Johannes van Kessel Publishing (2014). Available online at: http://www.paultwin.com/wp-content/uploads/Lodder_1140873_Paper_Imputation.pdf

Google Scholar

42. Kelly SM, Gryczynski J, Mitchell SG, Kirk A, O’Grady KE, Schwartz RP. Validity of brief screening instrument for adolescent tobacco, alcohol, and drug use. Pediatrics. (2014) 133(5):819–26. doi: 10.1542/peds.2013-2346

PubMed Abstract | Crossref Full Text | Google Scholar

43. Adams SH, Po J, Jane Park M, Irwin CE Jr. Monitoring adolescents’ receipt of time alone from two national surveys. J Adolesc Health. (2021) 68(1):79–85. doi: 10.1016/j.jadohealth.2020.05.002

PubMed Abstract | Crossref Full Text | Google Scholar

44. Hoots BE. Alcohol and other substance use before and during the COVID-19 pandemic among high school students — youth risk behavior survey, United States, 2021. MMWR Suppl. (2023) 72:84–92. doi: 10.15585/mmwr.su7201a10

PubMed Abstract | Crossref Full Text | Google Scholar

45. Stormshak EA, Caruthers AS, Gau JM, Winter C. The impact of recreational marijuana legalization on rates of use and behavior: a 10-year comparison of two cohorts from high school to young adulthood. Psychol Addict Behav J Soc Psychol Addict Behav. (2019) 33(7):595–602. doi: 10.1037/adb0000508

PubMed Abstract | Crossref Full Text | Google Scholar

46. National Institute on Drug Abuse (NIDA) Press Office. Reported use of Most Drugs among Adolescents Remained low in 2024. North Bethesda: National Institutes of Health (NIH) (2024). Available online at: https://www.nih.gov/news-events/news-releases/reported-use-most-drugs-among-adolescents-remained-low-2024 (Accessed August 1, 2025).

Google Scholar

47. Steinhoff A, Shanahan L, Bechtiger L, Zimmermann J, Ribeaud D, Eisner MP, et al. When substance use is underreported: comparing self-reports and hair toxicology in an urban cohort of young adults. J Am Acad Child Adolesc Psychiatry. (2023) 62(7):791–804. doi: 10.1016/j.jaac.2022.11.011

PubMed Abstract | Crossref Full Text | Google Scholar

48. Fan X, Miller BC, Park KE, Winward BW, Christensen M, Grotevant HD, et al. An exploratory study about inaccuracy and invalidity in adolescent self-report surveys. Field Methods. (2006) 18(3):223–44. doi: 10.1177/152822X06289161

Crossref Full Text | Google Scholar

49. Centers for Disease Control and Prevention (CDC). 1991-2021 High School Youth Risk Behavior Survey Data. Atlanta: US Centers for Disease Control and Prevention (n.d.). Available online at: https://youthonline.cdc.gov/ (Accessed September 24, 2025).

Google Scholar

50. Sisk BA, Antes AL, Bereitschaft C, Bourgeois F, DuBois J. Acceptability of adolescent portal access policies to parents and adolescents: a Delphi study. J Adolesc Health. (2025) 76(3):448–54. doi: 10.1016/j.jadohealth.2024.10.021

PubMed Abstract | Crossref Full Text | Google Scholar

51. Jasik CB, Berna M, Martin M, Ozer EM. Teen preferences for clinic-based behavior screens: who, where, when, and how? J Adolesc Health. (2016) 59(6):722–4. doi: 10.1016/j.jadohealth.2016.08.009

PubMed Abstract | Crossref Full Text | Google Scholar

52. Camenga D, Hadland SE, Zoucha K, Somberg CA, Burr WH. Substance Use Screening Practices among Primary Care Pediatricians in the US: Results from the 2021 AAP Periodic Survey. Itasca: American Academy of Pediatrics (2023) Available online at: https://www.aap.org/en/research/pas-abstracts/substance-use-screening-practices-among-primary-care-pediatricians-in-the-us-results-from-the-2021-aap-periodic-survey/ (Accessed August 16, 2023).

Google Scholar

53. Camenga DR, Hadland SE, Somberg C, Burr W. US pediatricians’ attitudes and practices around screening for substance use in adolescents. J Adolesc Health. (2025) 77(3):499–506. doi: 10.1016/j.jadohealth.2025.04.024

PubMed Abstract | Crossref Full Text | Google Scholar

54. Johnson TP. Sources of error in substance use prevalence surveys. Int Sch Res Not. (2014) 2014(1):923290. doi: 10.1155/2014/923290

PubMed Abstract | Crossref Full Text | Google Scholar

55. Levy SJ, Williams JF, Committee on substance use and prevention, Ryan SA, Gonzalez PK, Patrick SW, et al. Substance use screening, brief intervention, and referral to treatment. Pediatrics. (2016) 138(1):e20161211. doi: 10.1542/peds.2016-1211

PubMed Abstract | Crossref Full Text | Google Scholar