Skip to main content


Front. Psychiatry, 26 January 2023
Sec. Computational Psychiatry
Volume 13 - 2022 |

Causal discovery replicates symptomatic and functional interrelations of posttraumatic stress across five patient populations

  • 1Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN, United States
  • 2Institute for Health Informatics, University of Minnesota, Minneapolis, MN, United States
  • 3Department of Neurological Surgery, Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco, San Francisco, CA, United States
  • 4San Francisco Veterans Affairs Medical Center, San Francisco, CA, United States
  • 5Department of Psychiatry and Neurology, University of California, San Francisco, San Francisco, CA, United States
  • 6Department of Medicine, University of Minnesota, Minneapolis, MN, United States
  • 7Department of Medicine, University of California, San Francisco, San Francisco, CA, United States

Introduction: Approximately half of individuals with posttraumatic stress disorder (PTSD) may meet criteria for other psychiatric disorders, and PTSD symptoms are associated with diminished health and psychosocial functioning. However, few studies examine the longitudinal progression of PTSD symptoms concurrent with related symptom domains and functional outcomes, such that may neglect important longitudinal patterns of symptom progression beyond PTSD specifically.

Methods: Therefore, we used longitudinal causal discovery analysis to examine the longitudinal interrelations among PTSD symptoms, depressive symptoms, substance abuse, and various other domains of functioning in five longitudinal cohorts representing veterans (n = 241), civilians seeking treatment for anxiety disorders (n = 79), civilian women seeking treatment for post-traumatic stress and substance abuse (n = 116), active duty military members assessed 0–90 days following TBI (n = 243), and civilians with a history of TBI (n = 43).

Results: The analyses revealed consistent, directed associations from PTSD symptoms to depressive symptoms, independent longitudinal trajectories of substance use problems, and cascading indirect relations from PTSD symptoms to social functioning through depression as well as direct relations from PTSD symptoms to TBI outcomes.

Discussion: Our findings suggest PTSD symptoms primarily drive depressive symptoms over time, tend to show independence from substance use symptoms, and may cascade into impairment in other domains. The results have implications for refining conceptualization of PTSD co-morbidity and can inform prognostic and treatment hypotheses about individuals experiencing PTSD symptoms along with co-occurring distress or impairment.

1. Introduction

Post-traumatic stress disorder (PTSD) is highly co-morbid with internalizing conditions, including major depression and anxiety disorders. Prevalence estimates suggest over half of PTSD patients meet criteria for one such disorder (18). Patients with comorbid PTSD and depression have more suicide attempts and a twofold increase in medical costs, compared with either disorder alone (916). Similarly, comorbid anxiety disorders are associated with more severe PTSD and impairment (7, 9), often emerging in triple comorbidity with PTSD and depression (9, 17). There is debate about interpreting comorbidity among PTSD and other internalizing disorders (18), including whether PTSD symptoms drive internalizing symptoms, or vice versa, or if the relationship is bidirectional (19). Studying the connections between these symptom domains and their impacts on patients’ health and functioning can support more effective, targeted care.

Elucidating the relationship between PTSD and other internalizing conditions has been challenging and hampered by attempts to isolate symptoms of each disorder (20). Many clinical studies of PTSD exclude patients with internalizing comorbidities, or treat such symptoms as confounded, while studies of internalizing disorders often exclude patients with PTSD, or neglect to evaluate traumatic history (20). Cross-sectional studies have identified shared features of PTSD and other internalizing concerns (2125), including network analyses using an adult cohort with depressive symptoms and trauma history, which found that impaired concentration, sleep problems, irritability, and guilt, were shared across disorders (26). Other cross-sectional analyses suggest a negative affectivity factor explains covariance across PTSD, generalized anxiety, and major depression (27, 28), and indicate common genetic and environmental correlates of these concerns (29).

Longitudinal studies including internalizing and PTSD symptoms have produced inconsistent results (30, 31). A study of combat veterans and prisoners of war assessed for depression and PTSD over three occasions found bidirectional relationships between PTSD and depressive symptoms, suggesting a common posttraumatic construct (31). In contrast, a study of urban US civilians assessed over three occasions found PTSD and depression were related bi-directionally, but were distinct constructs (30). Other research identified gender differences in directionality, finding a bi-directional relationship among women, but unidirectional association from PTSD to depression among men (19). Another community-based study found weekly changes in PTSD over a 6 to 12-week period were predicted by pre-treatment anxiety sensitivity (32). Finally, longitudinal studies of war veterans and lung injury survivors suggest anxiety, depressive, and PTSD symptoms emerge as concurrent co-morbidities (9, 33). Therefore, the current study aims to address these discrepancies by demonstrating a consistent and conserved directionality to these symptom domains.

Recent advances in machine-learning support more flexible modeling of longitudinal associations between PTSD and internalizing symptoms. Causal inference algorithms can identify the most plausible network of directional associations, supporting data-driven investigations into the relations among these symptom domains (3436). Unlike prior studies, these algorithms can also incorporate other outcomes that could mediate relations between PTSD and internalizing symptoms, such as alcohol use or functional status. Understanding whether and how PTSD and internalizing symptoms drive health and functioning is also important.

The present study used advanced machine learning methods to investigate the directional associations among posttraumatic and internalizing symptoms along with multiple psychosocial outcomes of clinical significance. We aimed to assess the driving roles of each symptom domain more precisely through applying causal discovery techniques to longitudinal data assessing PTSD, depressive symptoms, and psychosocial outcomes over 8 years of data. We validated our findings using multiple measures of internalizing symptoms in two additional samples from longitudinal studies with distinct age, gender representation, prevalent diagnoses, and treatment settings.

2. Materials and methods

2.1. Overview

The current study focused on interrelations among post-traumatic stress and a breadth of symptom measures using data-driven computational experiments on complete longitudinal clinical data from five studies of distinct populations (Study 1, n = 241; Study 2, n = 79; Study 3, n = 116; Study 4, n = 243; Study 5, n = 43). Study 1 included complete longitudinal data from 8 consecutive years of the Mind Your Heart study [MYH; (37)] and including measures across domains of PTSD, depression, social functioning, physical health, and alcohol use. Study 2 included 4-wave longitudinal data across baseline, 6-month, 12-month, and 18-month follow-ups of participants with a PTSD diagnosis enrolled in a randomized controlled trail of the Coordinated Anxiety Learning and Management [CALM; (38)] intervention. Study 3 included 5-wave longitudinal data across baseline, post-intervention (6 weeks after baseline), 3-month follow-up, 6-month follow-up, and 12-month assessments from women enrolled in a randomized trial of the Seeking Safety intervention for women with post-traumatic stress symptoms and substance abuse, in the Women’s Treatment for Trauma and Substance Use Disorders study [WTTS; (39)]. Study 4 included two-wave longitudinal data from active military service members with and without concussive TBI, assessed at a baseline time-point (immediately post-TBI) then again between 6 and 24 months after baseline (ADAPT; also called “CENC Study 25;” doi: 10.23718/FITBIR/1504245). Study 5 included two-wave longitudinal data from participants with a history of TBI within the past 15 years, collected at a baseline time-point and again between 6 and 12 months after baseline (TEAM-TBI; doi: 10.23718/FITBIR/1518871).

2.2. Study populations

The prospective MYH Study was designed to examine associations between PTSD and health outcomes. Participants were recruited between 2008 to 2010 from the San Francisco and Palo Alto Department of Veterans Affairs (VA) medical centers. Flyers at VA facilities and mailings were used to recruit patients with and without PTSD diagnoses in the previous 5 years (40). Participants were excluded if they had a myocardial infarction 6-month prior, could not walk one block on a treadmill, did not have stable contact information, or planned to move within 3 years. The study was approved by the institutional review boards of the University of California, San Francisco and the San Francisco Veterans Administration and all participants provided written informed consent. A total of 747 enrolled participants completed baseline examinations. Participants completed annual telephone interviews with validated assessments of several health and function domains. The present study included data from 241 participants that completed assessments across 8 years.

The CALM study is a randomized-controlled trial of the CALM Tools for Living intervention for anxiety disorders in primary care (41). Participants were recruited from and received treatment at four U.S. primary care sites between 2006 and 2008. Eligibility criteria included age 18 years or older and meeting criteria for any DSM-IV anxiety disorder (further exclusion criteria reported in; 38). Participants were randomized using stratified permutated block randomization and received either the CALM intervention or treatment-as-usual. Participants assigned to the intervention chose between taking medications, the Tools for Living intervention, or both. The CALM Tools for Living intervention involved eight, hour-long, computerized cognitive-behavioral intervention modules that were supervised by an anxiety specialist clinician. A total of 1004 primary care patients who completed baseline assessments were enrolled (41), and re-assessed on outcomes at 6, 12, and 18 months post-intervention. We identified 79 participants who met diagnostic criteria for PTSD and were assessed on posttraumatic stress symptoms.

The WTTS study compared two manualized interventions for co-occurring substance abuse and posttraumatic stress among women (39). Women with substance abuse disorders and PTSD, or sub-threshold posttraumatic stress, were eligible. Participants were randomly assigned to receive either Seeking Safety or Women’s Health Education interventions to supplement ongoing treatment for substance abuse. Participants were assessed on a range of outcomes at baseline, 2-month later at post-intervention, and at 3, 6, and 12-month follow-ups. Details of the WTTS study design are published elsewhere (39). Of the 353 participants initially enrolled in the WTTS study, our analysis included 116 with complete data on all measures at each assessment point.

The ADAPT study was designed to examine clinical outcomes of active duty US military with mild TBI via clinical assessments and magnetic resonance imaging over a longitudinal assessment period. The study prospectively enrolled 591 active duty US military participants between 0 and 90 days post-injury, who completed longitudinal follow-up assessments between 6 and 24 months post-injury (with later follow-ups planned but not completed). Our analysis included 243 of these participants with complete 6–24 month follow-up data on clinical assessment measures.

The TEAM-TBI study enrolled 95 participants with a history of TBI in the past 15 years and administered a large battery of baseline and 6–12 month follow-up assessments to refine strategies for evaluating TBI outcomes. Of the original sample, 43 participants were included who had valid intake and 6–12 month follow-up assessments on clinical measures.

2.3. Measures

Measures included in each study are presented in Table 1. All studies included measures of PTSD symptoms, depressive symptoms, and alcohol/substance use concerns. The MYH, CALM, and WTTS studies additionally included assessments of participants’ perceptions of their health and physical functioning. The ADAPT and TEAM-TBI studies included assessments of post-TBI outcomes. Beyond these measures, specific assessments were also identified for inclusion based on factors unique to each study (e.g., employment problems in the WTTS study, dizziness-related impairment in the TEAM-TBI study).


Table 1. Measures included in each study sample.

2.4. Analyses

Greedy Fast Causal Inference [GFCI; (34, 35)] analysis was performed to determine the network structure among post-traumatic stress and related outcomes in each dataset, summarized in Figure 1. GFCI uses a combination of goodness-of-fit statistics, conditional independence tests, and mathematical decision rules to search across all possible directed acyclic graphs (DAGs), including DAGs with unmeasured variables, to find the collection of DAGs most consistent with the data. DAGs are used to represent the structure of models, including Structural Equation Models [SEMs; (42)] and Bayesian Networks [BNs; (43)], and GFCI is analogous to searching through the space of all possible SEMs or BNs. This collection of DAGs is represented as a mixed ancestral graph (MAG). The lines connecting the nodes in MAGs can have a combination of different endpoints, e.g., arrowheads, arrow tails, and circles, along with different line types, capturing information about the entire set of represented DAGs.


Figure 1. Study procedural diagram. Data from five separate patient cohorts from different clinical studies were mined for this study, spanning the range of civilians, veterans, and active-duty military populations with posttraumatic stress, substance abuse disorders, traumatic brain injury (TBI), or a combination of multiple of these as the primary diagnosis, with some compared to healthy controls.

The DAG space is extremely large and impossible to search exhaustively for the number of variables in the primary dataset. GFCI thus selects an initial set of DAGs using a depth-first search guided by artificial intelligence (AI) to optimize a comparative goodness-of-fit statistic. It then uses AI to modify these graphs to maintain consistency with conditional independence and dependence statements supported by the data. These modifications can include the addition of unobserved factors. The resulting MAG thus represents a set of DAGs which both outperform all alternative theories and are as consistent with the data as possible. This makes GFCI a robust analytic approach that maximizes information gain from highly multidimensional data without many of the shortcomings of traditional clinical prediction models.

Greedy Fast Causal Inference was run using Tetrad version 6.6.0. The primary settings and parameters were left as their defaults: Bayesian Information Criterion (BIC) score with penalty discount 2, and Fisher Z test with alpha 0.01. The upper bound on maximum degree was removed by setting it to −1. This was a default value for older versions of Tetrad and prioritizes accuracy over runtime. The algorithm’s search was augmented with background knowledge to restrict causal arrows pointing backward in time. Graph stability was assessed from 1,000 bootstrap samples, applying the same GFCI analysis to each, and aggregating the resulting 1,000 graphs into a table summarizing the proportion of all possible relationships. The edges in the graph were compared to those in the table to confirm that they were not regularly changed or destroyed.

2.4.1. Effect size estimation

To obtain effect-size estimates, each MAG was converted to a path analysis input (44). Each directional relation was represented as a regression path, whereas non-directional relations in the MAGs were represented as covariances in the path model. Covariances among exogenous measures were allowed in the path analysis model, while covariances among endogenous measures were restricted. The resulting models thus paralleled the robust causal network represented in the MAGs, while preserving non-directional relations among variables of interest. The path models were run using the “lavaan” package in R (45) using the maximum likelihood estimation procedure with robust standard-errors. Standardized associations and confidence intervals were interpreted to obtain effect-size and uncertainty estimates associated with relations depicted in the MAGs.

3. Results

Table 2 shows demographic and diagnostic characteristics of participants in each sample selected for analysis for the present study. Considering demographics, participants in the MYH study were the oldest, on average, while participants in the TEAM-TBI were the youngest. Participants in MYH, ADAPT, and TEAM-TBI studies were mostly men (over 75%); participants in CALM were mostly women; and all participants enrolled in WTTS were women. Across studies, the most prevalent participant racial-ethnic identification was White and non-Hispanic/Latinx, ranging from 40.5 to 88.4% of the study samples. The next most commonly endorsed participant race-ethnicity was Black and non-Hispanic/Latinx, ranging from 4.7 to 41.4% of the study samples. The MYH, CALM, and ADAPT studies included larger proportions of participants identifying a Hispanic or Latinx ethnicity. Educational attainment also varied across studies, with the majority of MYH participants having high school degrees, the majority of CALM and TEAM-TBI participants completing some post-secondary, and most ADAPT participants having a 4-year college degree. Educational attainment was not assessed in the WTTS study. The majority of participants in all studies except WTTS reported being married, while a slight majority of WTTS participants reported they were divorced or separated. Diagnostic and TBI status similarly varied across studies, yet diagnostic information was not collected consistently. Notably, all participants in the CALM sample met criteria for both PTSD and Generalized Anxiety Disorder; all participants in WTTS met criteria for a substance use disorder; and all participants in the TEAM-TBI study had a previous TBI.


Table 2. Sociodemographics and select clinical characteristics of each study sample.

Supplementary Tables 15 present correlations among focal study measures included in the GFCI for each sample. Across studies, statistically significant associations were observed among measures of post-traumatic stress, depressive symptoms, TBI symptoms, and social and health functioning. Meanwhile, measures of substance abuse symptoms were not statistically significantly associated with other assessment instruments. This pattern was observed across measures of substance use problems.

3.1. GFCI analysis results

Figures 26 present patterns of associations among selected PTSD symptom and related assessments identified via GFCI in each study, and which showed stability across the bootstrapped analyses. Across studies, directed edges were observed from PTSD symptoms to depressive symptoms, while neither directed nor undirected edges were observed between PTSD symptoms and measures of substance use problems. As such, longitudinal variability in depressive symptoms was explained by PTSD symptoms, and this directional association was preserved independent of the potential associations between depressive symptoms and other measures. Meanwhile, longitudinal variability in substance use measures was only independently explained by prior values on those measures, such that potential associations between substance use problems and other measures added no explanatory value independent of the associations between prior and subsequent scores on substance use measures. These edges were consistent across bootstrap resamples in all studies (see Supplementary Tables 610).


Figure 2. Greedy Fast Causal Inference (GFCI) graph for the Mind Your Heart study (MYH) sample (n = 241). Causal links between post-traumatic stress disorder (PTSD) symptoms and depression over the course of 8 years in the MYH sample, leading to social functioning, which are all independent of overall health and alcohol use problems. Colored arrows indicate the level and directionality of the causal edges, with + or – along the edges indicating whether these links cause increases or decreases in scores on the connected measures.


Figure 3. Greedy Fast Causal Inference (GFCI) graph for the Coordinated Anxiety Learning and Management (CALM) sample (n = 79). First independent replication of causal links between post-traumatic stress disorder (PTSD) symptoms and depression over the course of 18 weeks in the CALM sample, leading to social functioning, which are all independent of overall health, physical health, and alcohol use problems. Colored arrows indicate the level and directionality of the causal edges, with + or – along the edges indicating whether these links cause increases or decreases in scores on the connected measures.


Figure 4. Greedy Fast Causal Inference (GFCI) graph for the WTTS sample (n = 116). Second independent replication of causal links between post-traumatic stress disorder (PTSD) symptoms and depression over the course of 12 months in the WTTS sample, which are both independent of drug and alcohol use problems. Colored arrows indicate the level and directionality of the causal edges, with + or – along the edges indicating whether these links cause increases or decreases in scores on the connected measures.


Figure 5. Greedy Fast Causal Inference (GFCI) graph for the CENC-ADAPT sample (n = 243). Third independent replication of causal links between post-traumatic stress disorder (PTSD) symptoms and depression over the course of 6–24 months follow-up in the CENC-ADAPT sample, leading to severity of traumatic brain injury (TBI) outcome, which are all independent of drug and alcohol use problems. Colored arrows indicate the level and directionality of the causal edges, with + or – along the edges indicating whether these links cause increases or decreases in scores on the connected measures.


Figure 6. Greedy Fast Causal Inference (GFCI) graph for the TEAM-TBI sample (n = 43). Fourth independent replication of causal links between post-traumatic stress disorder (PTSD) symptoms and depression over the course of 6–12 months follow-up in the TEAM-TBI sample, leading to neurocognitive dysfunction, which are all independent of alcohol use problems. Colored arrows indicate the level and directionality of the causal edges, with + or – along the edges indicating whether these links cause increases or decreases in scores on the connected measures.

Additional directed edges were noteworthy across studies. PTSD symptoms tended to explain measures of social functioning through depressive symptoms in both the MYH and CALM studies. While the correlational analyses suggested PTSD symptoms were moderately to strongly negatively associated with social functioning in both studies, the present results indicated that this negative association was accounted for by depressive symptoms. Depressive symptoms thus proximally explained longitudinal variability in social functioning, such that associations of social functioning with other measures provided no further explanatory value, while PTSD symptoms distally explained longitudinal variability in social functioning through uniquely explaining depressive symptoms. Conversely, PTSD symptoms indirectly explained longitudinal variability in TBI symptoms/functioning through depression in the ADAPT study while PTSD symptoms showed a direct edge toward TBI symptoms in the TEAM-TBI study independent of depressive symptoms. While these findings generally suggest PTSD symptoms have a directed association toward TBI symptoms (either directly or indirectly through depressive symptoms), this pattern was somewhat less stable across studies and bootstrap resamples. Supplementary Table 10 includes details on the instability of this association across bootstraps in the TEAM-TBI study.

Details on other measures and edges included in the GFCI’s but not shown in Figures 26 are included in Supplementary Tables 611. These edges showed less stability across time and bootstrap resamples, or shared no edges with other domains, and therefore were not selected for presentation in the primary figures.

3.2. Test for Simpson’s paradox

We pooled and normalized the data across the 5 different studies to test the Simpson’s paradox, which identifies relationships in data when pooled together that might not be represented in the individual studies themselves (46). We normalized each variable by calculating a z-score within each study for each domain that was measured. Given that different depression and substance use questionnaires were used across the different studies, this was necessary so that the variance in the pooled matrix is not explained by this scaling discrepancy (Figure 7). We see in the pooled matrix (Figure 7A) that alcohol use is weekly correlated with the other outcomes, with more correlations to PTSD and depression, however, these are not strong correlations that the other measures in the matrix have with each other, and confirms the lack of strong correlations in each individual study (Figure 7B).


Figure 7. Comparison of correlation matrix from pooled sample across all five studies (A), compared to individual correlation matrices for each study (B). Data across all five studies were combined to test whether different relationships would emerge with the pooled sample. We found that the patterns within each dataset were mostly conserved in the pooled dataset. *Indicates common outcomes across all studies.

4. Discussion

This is the first study to use a data-driven approach to identify and replicate a unidirectional relationship from PTSD to internalizing symptoms across multiple, distinct samples with varying diagnostic and demographic characteristics. Our findings provide insight into trajectories of patients with co-morbid diagnoses, with implications for treating internalizing symptoms in the context of PTSD. These results highlight the importance of measuring trauma exposure and presence of PTSD in clinical research studies of internalizing disorders and controlled clinical trials for internalizing distress.

Studies evaluating associations of PTSD and internalizing symptoms over time have had conflicting results (9, 17, 2730). This research has found unidirectional relations between PTSD and internalizing symptoms [though directionality was not consistent; (17, 29)]; bi-directional associations (9, 27, 28, 30); and differential associations by population characteristics (e.g., gender) (17). We expand on this work by using machine learning methods that offer more flexible analyses of the associations among PTSD and internalizing symptoms, along with potential confounders. Previous research relies on testing competing a priori hypotheses for these relations; for instance, by specifying competing models and then assessing their correspondence to the data (27, 28). This research is often limited to a single sample with few assessment points across which to establish conserved symptom relations (1923, 2729). In contrast, the present study used a data-driven approach that allowed for nuanced relations to emerge across multiple timepoints in a hypothesis generation stage, and then reproduced across two additional samples. Our finding of a unidirectional association with PTSD explaining concurrent internalizing symptoms, after accounting for prior internalizing symptoms, was consistent across three cohorts representing separate studies of primarily men or women, and observational and clinical trial samples across both veteran and civilian cohorts. Beyond these results, our mechanistic analyses suggested a bridging role of hyperarousal and cognitive and affective disruption between PTSD and depressive concerns more specifically.

Our findings raise questions about the interpretation of internalizing symptoms that emerge in the context of PTSD. Such symptoms may not be accurately characterized as involving co-morbidity among distinct conditions. Instead, our results suggest these symptoms represent a downstream outcome of PTSD, which may be addressed through treating features bridging between posttraumatic stress and internalizing concerns. Consistent with our exploratory analysis of potential mechanisms, a study of Israeli veterans suggested centrality of the DSM-V negative alterations in cognitions and mood (NACM) symptom cluster in bridging among PTSD, depressive symptoms, and moral injury (25). Similarly, consistent with our findings on the centrality of hyperarousal across PTSD and depressive symptom groups, other research suggests anxiety sensitivity and avoidance of inner experiences broadly may be transdiagnostic processes underlying PTSD and internalizing symptomatology (32, 47, 48). Further longitudinal research should consider the mechanisms connecting hyperarousal, NACM, and other internalizing symptoms that could play a transdiagnostic role in comorbidities.

We also report findings that use data collected under the DSM-IV, despite the DSM-V being available for these assessments. This is not an unusual problem with longitudinal data mining studies in the PTSD field given the evolving nature of the diagnostic criteria. In general, both instruments have strong agreement in determining if someone has low, medium, high symptoms. Previous efforts to produce an equipercentile manipulation with loglinear smoothing to compute a “crosswalk” between PCL-IV and PCL-V scores has been done and are in strong agreement with measuring the range of symptom severity (49).

Our findings affirm the need to assess for posttraumatic stress in the context of research on anxious and depressive disorders. This appears especially pressing for clinical trials, as there is a lack of information about the co-occurrence of PTSD in much of this research (18). For example, we were unable to find a study with a focus on internalizing symptoms and sufficient sample size that also assessed for PTSD across the NDA and NIDA repositories. This issue may be less a result of actively excluding patients with co-morbidities, but more a product of limited assessments of co-morbid symptomology in such studies (50). Conversely, studies may exclude more severe or sub-threshold manifestations of either PTSD or internalizing symptoms (51), inadvertently restricting information about possible co-morbid conditions. A lack of information about co-morbidity may drive inaccurate care when a trauma-informed approach is not taken, and may neglect crucial mechanisms that could support clients’ recovery.

While our primary analyses concerned the relation between PTSD and internalizing symptoms, our exploratory analyses generated other hypotheses pertaining to PTSD and psychosocial outcomes. Specifically, our GFCI analyses on the MYH cohort suggested social functioning is indirectly affected by PTSD symptoms through depressive symptoms. This finding aligns with studies showing that treating interpersonal problems in PTSD can ameliorate comorbid major depression (52, 53). Conversely, our findings suggest changes in PTSD symptoms provide little information about changes in alcohol use, physical functioning, and overall health over time, and vice versa. Individuals with alcohol use or physical health concerns may thus benefit from combined treatment approaches. The alcohol use findings are noteworthy as previous studies have documented co-morbidities among PTSD and alcohol abuse, yet rarely examine whether PTSD symptoms drive alcohol abuse longitudinally (54).

Our finding on the independence of substance abuse from PTSD and depression underscores the importance of other drivers of substance use that are outside the realm of the “self-medication hypothesis.” Koob and Volkow stressed the importance of reward circuitry, which suggests that people use substances to seek out pleasurable reward sensations, and not so much to self-medicate mood/anxiety states (55). This could also be further explained regarding current, rather than past substance use, as is the case with the AUDIT. Clinical experience provides insights about this in the treatment of patients with PTSD that have a history of substance use disorder, but after going through treatment, are now actively abstaining from using alcohol, and therefore the AUDIT may not be picking up on their historical use and the association of chronic PTSD symptoms that have evolved if most of them are not endorsing current use. From a clinical perspective, there is a fair amount of literature describing the co-occurrence of diagnosed PTSD and alcohol or substance use disorders, but fewer studies looking at longitudinal relations among symptoms of those disorders specifically. The nature of the longitudinal relationships identified in our findings and continuous self-report measures may differ from those documented in cross-sectional studies with categorically defined diagnostic interviews. From a measurement standpoint, one of the caveats of the alcohol use measures included in these studies is that they are asking about drinking problems in general, versus drinking specifically to cope with post-traumatic stress symptoms or to avoid/suppress other inner experiences (54). Given that, our findings are important to replicate in future research integrating PTSD symptoms and psychosocial outcomes.

Our results are subject to several limitations. While GFCI can infer “causality” from repeated measures based on statistical criteria, the MYH study was observational. Typically, causality is confirmed with data from randomized controlled trials. We cannot randomize patients to develop psychiatric disorders, but machine learning techniques can be applied to repeated measures of PTSD, internalizing symptoms, and psychosocial outcomes collected during intervention trials to overcome this. Our validation cohorts were drawn from clinical trials, but only a subset of participants had both PTSD and internalizing symptoms assessed. Therefore, replication with thorough measurement of both symptom domains will be important. Second, though we conducted our analyses in samples of largely male veterans, women with substance use disorders, and a subset of patients enrolled in an anxiety trial, validation in populations with diversity in other experiences and identities is crucial. Third, patients were excluded due to missing data on repeated assessments, although findings based on fewer time-points and excluding fewer patients were consistent with those on all time-points. Finally, our samples largely represented patients with PTSD, so replication with other salient clinical concerns would be valuable.

Despite these limitations, our finding that PTSD symptoms drove internalizing symptoms in three separate studies suggests an etiological connection from trauma symptomology to internalizing distress. This supports prior work suggesting evolution of other internalizing symptoms secondary to PTSD could represent a manifestation of the original response to traumatic stress (7, 9, 28). Our results underscore the need for integrated therapies that treat mechanisms underlying a constellation of patients’ symptoms, rather than addressing single disorders. Finally, this work highlights the importance of longitudinal data incorporating simultaneous assessment of PTSD, internalizing distress, and patient-centered outcomes in both observational and trial settings.

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: Data are available to access from federal repositories managed by the National Institutes of Mental Health (NIMH), Drug Abuse (NIDA), and Neurological Disorders and Stroke (NINDS). Data from clinical trials funded by these agencies are available to qualified researchers by setting up a data use agreement (DUA/DUC) which requires affiliation with an institution with a federal wide assurance (FWA). Data and/or research tools used in the preparation of this manuscript were obtained from the National Institute of Mental Health (NIMH) Data Archive (NDA). NDA is a collaborative informatics system created by the National Institutes of Health to provide a national resource to support and accelerate research in mental health. Dataset identifier(s): Coordinated Anxiety Learning and Management (CALM) study (Study 2146 in NDA). This manuscript reflects the views of the authors and may not reflect the opinions or views of the NIH or of the Submitters submitting original data to NDA. Data and/or research tools used in the preparation of this manuscript were obtained and analyzed from the controlled access datasets distributed from the Department of Defense (DOD)- and NIH-supported Federal Interagency Traumatic Brain Injury Research (FITBIR) Informatics Systems. FITBIR is a collaborative biomedical informatics system created by the Department of Defense and the National Institutes of Health to provide a national resource to support and accelerate research in TBI. Dataset identifier(s): CENC Study 25: Assessment of long term outcome and disability in active-duty military prospectively examined following concussive TBI: the ADAPT study (doi: 10.23718/FITBIR/1504245), and Targeted evaluation, action, and monitoring of TBI (doi: 10.23718/FITBIR/1518871). This manuscript reflects the views of the authors and may not reflect the opinions or views of the DOD, NIH, or of the Submitters submitting original data to FITBIR Informatics System. The information reported here results from secondary analyses of data from clinical trials conducted by the National Institute on Drug Abuse (NIDA). Specifically, data from Women’s Treatment for Trauma and Substance Use Disorders study (Study NIDA-CTN-0015) were included. NIDA databases and information are available at ( Additional data from the Mind Your Heart study is available upon request from the Principal Investigator, BC ( Requests to access NDA, FITBIR, and NIDA Data Share datasets should be directed to;; and

Ethics statement

The studies involving human participants were reviewed and approved by University of Minnesota Institutional Review Board, STUDY00003370. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

JN, EK, SM, AF, TN, and BC conceived of the study. BC and TN collected or oversaw collection of MYH primary data. JN, TK, and BP performed data query and wrangling of MYH and NDA datasets. EK performed GFCI on datasets. JN, EK, BP, SM, AF, TN, and BC were involved in interpreting results. JN, EK, SM, TK, BP, AF, TN, and BC wrote the manuscript. All authors contributed to the article and approved the submitted version.


This project was supported by NIH/NIMH grant R01MH116156 (JN). The funders had no role in the design and conduct of the study, data collection, management, analysis, interpretation, or manuscript preparation.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at:


1. Rytwinski N, Scur M, Feeny N, Youngstrom E. The co-occurrence of major depressive disorder among individuals with posttraumatic stress disorder: a meta-analysis. J Trauma Stress. (2013) 26:299–309.

Google Scholar

2. Kessler R, Sonnega A, Bromet E, Hughes M, Nelson C. Posttraumatic stress disorder in the national comorbidity survey. Arch Gen Psychiatry. (1995) 52:1048–60. doi: 10.1002/jts.21814

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Caramanica K, Brackbill R, Liao T, Stellman S. Comorbidity of 9/11-related PTSD and depression in the world trade center health registry 10-11 years postdisaster. J Trauma Stress. (2014) 27:680–8. doi: 10.1001/archpsyc.1995.03950240066012

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Breslau N, Davis G, Peterson E, Schultz L. Psychiatric sequelae of posttraumatic stress disorder in women. Arch Gen Psychiatry. (1997) 54:81–7. doi: 10.1002/jts.21972

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Flory J, Yehuda R. Comorbidity between post-traumatic stress disorder and major depressive disorder: alternative explanations and treatment considerations. Dialogues Clin Neurosci. (2015) 17:141–50. doi: 10.1001/archpsyc.1997.01830130087016

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Knowles K, Sripada R, Defever M, Rauch S. Comorbid mood and anxiety disorders and severity of posttraumatic stress disorder symptoms in treatment-seeking veterans. Psychol Trauma Theory Res Pract Policy. (2019) 11:451–8. doi: 10.31887/DCNS.2015.17.2/jflory

CrossRef Full Text | Google Scholar

7. Spinhoven P, Penninx B, van Hemert A, de Rooij M, Elzinga B. Comorbidity of PTSD in anxiety and depressive disorders: prevalence and shared risk factors. Child Abuse Negl. (2014) 38:1320–30. doi: 10.1037/tra0000383

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Dai W, Kaminga A, Tan H, Wang J, Lai Z, Wu X, et al. Comorbidity of post-traumatic stress disorder and anxiety in flood survivors. Medicine. (2017) 96:e7994. doi: 10.1016/j.chiabu.2014.01.017

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Ginzburg K, Ein-Dor T, Solomon Z. Comorbidity of posttraumatic stress disorder, anxiety and depression: a 20-year longitudinal study of war veterans. J Affect Disord. (2010) 123:249–57. doi: 10.1097/MD.0000000000007994

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Cougle J, Resnick H, Kilpatrick DG. PTSD, depression, and their comorbidity in relation to suicidality: cross-sectional and prospective analyses of a national probability sample of women. Depress Anxiety. (2009) 26:1151–7. doi: 10.1016/j.jad.2009.08.006

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Nichter B, Norman S, Haller M, Pietrzak R. Psychological burden of PTSD, depression, and their comorbidity in the U.S. veteran population: suicidality, functioning, and service utilization. J Affect Disord. (2019) 256:633–40. doi: 10.1002/da.20621

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Tanielian T, Jaycox L R and Corporation, California Community Foundation, Rand Health. Invisible Wounds of War: Psychological and Cognitive Injuries, their Consequences, and Services to Assist Recovery. Santa Monica, CA: RAND Center for Military Health Policy Research (2008). p. 453. doi: 10.1016/j.jad.2019.06.072

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Alexandre P, Hwang S, Roth K, Gallo J, Eaton W. Costs of depression from claims data for medicare recipients in a population-based sample. J Health Hum Serv Adm. (2016) 39:72–94.

PubMed Abstract | Google Scholar

14. Walker E, Katon W, Russo J, Ciechanowski P, Newman E, Wagner A. Health care costs associated with posttraumatic stress disorder symptoms in women. Arch Gen Psychiatry. (2003) 60:369–74.

Google Scholar

15. Stapleton J, Asmundson G, Woods M, Taylor S, Stein M. Health care utilization by United Nations peacekeeping veterans with co-occurring, self-reported, post-traumatic stress disorder and depression symptoms versus those without. Mil Med. (2006) 171:562–6. doi: 10.1001/archpsyc.60.4.369

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Sareen J, Cox B, Stein M, Afifi T, Fleet C, Asmundson G. Physical and mental comorbidity, disability, and suicidal behavior associated with posttraumatic stress disorder in a large community sample. Psychosom Med. (2007) 69:242–8. doi: 10.7205/MILMED.171.6.562

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Galatzer-Levy I, Nickerson A, Litz B, Marmar C. Patterns of lifetime PTSD comorbidity: a latent class analysis. Depress Anxiety. (2013) 30:489–96. doi: 10.1097/PSY.0b013e31803146d8

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Dekel S, Ein-Dor T, Rosen J, Bonanno G. Differences in cortisol response to trauma activation in individuals with and without comorbid PTSD and depression. Front Psychol. (2017) 8:797. doi: 10.1002/da.22048

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Horesh D, Lowe S, Galea S, Aiello A, Uddin M, Koenen K. An in-depth look into PTSD-depression comorbidity: a longitudinal study of chronically-exposed Detroit residents. J Affect Disord. (2017) 208:653–61. doi: 10.3389/fpsyg.2017.00797

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Yehuda R, Vermetten E, McFarlane A. Understanding depression as it occurs in the context of post-traumatic stress disorder. Depress Res Treat. (2012) 2012:178261. doi: 10.1016/j.jad.2016.08.053

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hurlocker M, Vidaurri D, Cuccurullo L, Maieritsch K, Franklin C. Examining the latent structure mechanisms for comorbid posttraumatic stress disorder and major depressive disorder. J Affect Disord. (2018) 229:477–82. doi: 10.1155/2012/178261

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Elhai J, Contractor A, Tamburrino M, Fine T, Cohen G, Shirley E, et al. Structural relations between DSM-5 PTSD and major depression symptoms in military soldiers. J Affect Disord. (2015) 175:373–8. doi: 10.1016/j.jad.2017.12.076

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Biehn T, Elhai J, Seligman L, Tamburrino M, Armour C, Forbes D. Underlying dimensions of DSM-5 posttraumatic stress disorder and major depressive disorder symptoms. Psychol Inj Law. (2013) 6:290–8. doi: 10.1016/j.jad.2015.01.034

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Contractor A, Durham T, Brennan J, Armour C, Wutrick H, Christopher Frueh B, et al. DSM-5 PTSD’s symptom dimensions and relations with major depression’s symptom dimensions in a primary care sample. Psychiatry Res. (2014) 215:146–53. doi: 10.1007/s12207-013-9177-4

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Levi-Belz Y, Greene T, Zerach G. Associations between moral injury, PTSD clusters, and depression among Israeli veterans: a network approach. Eur J Psychotraumatol. (2020) 11:1736411. doi: 10.1016/j.psychres.2013.10.015

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Afzali M, Sunderland M, Teesson M, Carragher N, Mills K, Slade T. A network approach to the comorbidity between posttraumatic stress disorder and major depressive disorder: the role of overlapping symptoms. J Affect Disord. (2017) 208:490–6. doi: 10.1080/20008198.2020.1736411

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Byllesby B, Charak R, Durham T, Wang X, Elhai J. The underlying role of negative affect in the association between PTSD, major depressive disorder, and generalized anxiety disorder. J Psychopathol Behav Assess. (2016) 38:655–65. doi: 10.1016/j.jad.2016.10.037

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Byllesby B, Elhai J, Tamburrino M, Fine T, Cohen G, Sampson L, et al. General distress is more important than PTSD’s cognition and mood alterations factor in accounting for PTSD and depression’s comorbidity. J Affect Disord. (2017) 211:118–23. doi: 10.1007/s10862-016-9555-9

CrossRef Full Text | Google Scholar

29. Schür R, Schijven D, Boks M, Rutten B, Stein M, Veldink J, et al. The effect of genetic vulnerability and military deployment on the development of post-traumatic stress disorder and depressive symptoms. Eur Neuropsychopharmacol. (2019) 29:405–15. doi: 10.1016/j.jad.2017.01.014

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Horesh D, Lowe S, Galea S, Uddin M, Koenen K. Gender differences in the long-term associations between posttraumatic stress disorder and depression symptoms: findings from the Detroit neighborhood health study. Depress Anxiety. (2015) 32:38–48. doi: 10.1016/j.euroneuro.2018.12.009

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Dekel S, Solomon Z, Horesh D, Ein-Dor T. Posttraumatic stress disorder and depressive symptoms: joined or independent sequelae of trauma? J Psychiatr Res. (2014) 54:64–9. doi: 10.1002/da.22267

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Gutner C, Nillni Y, Suvak M, Wiltsey-Stirman S, Resick P. Longitudinal course of anxiety sensitivity and PTSD symptoms in cognitive-behavioral therapies for PTSD. J Anxiety Disord. (2013) 27:728–34. doi: 10.1016/j.jpsychires.2014.03.003

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Bienvenu O, Colantuoni E, Mendez-Tellez P, Shanholtz C, Dennison-Himmelfarb C, Pronovost P, et al. Cooccurrence of and remission from general anxiety, depression, and posttraumatic stress disorder symptoms after acute lung injury: a 2-year longitudinal study. Crit Care Med. (2015) 43:642–53. doi: 10.1016/j.janxdis.2013.09.010

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Anker J, Kummerfeld E, Rix A, Burwell S, Kushner M. Causal network modeling of the determinants of drinking behavior in comorbid alcohol use and anxiety disorder. Alcohol Clin Exp Res. (2019) 43:91–7. doi: 10.1097/CCM.0000000000000752

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Ogarrio J, Spirtes P, Ramsey J. A hybrid causal search algorithm for latent variable models. JMLR Workshop Conf Proc. (2016) 52:368–79. doi: 10.1111/acer.13914

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Entner D, Hoyer P. On causal discovery from time series data using FCI. In: Myllymäki P, Roos T, Jaakkola T editors. Proceedings of the 5th European Workshop on Probabilistic Graphical Models. (Espoo: Helsinki Institute for Information Technology HIIT) (2010).

PubMed Abstract | Google Scholar

37. Kronish I, Edmondson D, Li Y, Cohen B. Post-traumatic stress disorder and medication adherence: results from the mind your heart study. J Psychiatr Res. (2012) 46:1595–9.

PubMed Abstract | Google Scholar

38. Sullivan G, Craske M, Sherbourne C, Edlund M, Rose R, Golinelli D, et al. Design of the coordinated anxiety learning and management (CALM) study: innovations in collaborative care for anxiety disorders. Gen Hosp Psychiatry. (2007) 29:379–87. doi: 10.1016/j.jpsychires.2012.06.011

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Hien D, Wells E, Jiang H, Suarez-Morales L, Campbell A, Cohen L, et al. Multisite randomized trial of behavioral interventions for women with co-occurring PTSD and substance use disorders. J Consult Clin Psychol. (2009) 77:607–19. doi: 10.1016/j.genhosppsych.2007.04.005

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Eswarappa M, Neylan T, Whooley M, Metzler T, Cohen B. Inflammation as a predictor of disease course in posttraumatic stress disorder and depression: a prospective analysis from the Mind Your Heart Study. Brain Behav Immun. (2019) 75:220–7. doi: 10.1037/a0016227

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Bomyea J, Lang A, Craske M, Chavira D, Sherbourne C, Rose R, et al. Course of symptom change during anxiety treatment: REDUCTIONS in anxiety and depression in patients completing the coordinated anxiety learning and management program. Psychiatry Res. (2015) 229:133–42. doi: 10.1016/j.bbi.2018.10.012

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Kline R. Principles and Practice of Structural Equation Modeling. 4th ed. New York, NY: Guilford Press (2016). p. 534. doi: 10.1016/j.psychres.2015.07.056

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Darwiche A. Modeling and Reasoning with Bayesian Networks. Cambridge: Cambridge University Press (2009).

Google Scholar

44. Boker S, McArdle J. Path analysis and path diagrams. In: N Balakrishnan, T Colton, B Everitt, W Piegorsch, F Ruggeri, J Teugels editors. Wiley StatsRef: Statistics Reference Online [Internet] (Hoboken, NJ: John Wiley & Sons, Ltd) (2014).

Google Scholar

45. Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw. (2012) 48:1–36. doi: 10.1002/9781118445112.stat06517

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Kievit R, Frankenhuis W, Waldorp L, Borsboom D. Simpson’s paradox in psychological science: a practical guide. Front Psychol. (2013) 4:513. doi: 10.18637/jss.v048.i02

CrossRef Full Text | Google Scholar

47. Grau P, McDonald J, Clark M, Wetterneck C. The relationship between three ACT core processes of change, PTSD, and depressive symptoms. J Context Behav Sci. (2020) 16:178–82. doi: 10.3389/fpsyg.2013.00513

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Rector N, Szacun-Shimizu K, Leybman M. Anxiety sensitivity within the anxiety disorders: disorder-specific sensitivities and depression comorbidity. Behav Res Ther. (2007) 45:1967–75. doi: 10.1016/j.jcbs.2020.04.004

CrossRef Full Text | Google Scholar

49. Moshier S, Lee D, Bovin M, Gauthier G, Zax A, Rosen R, et al. An empirical crosswalk for the PTSD checklist: translating DSM-IV to DSM-5 using a veteran sample. J Trauma Stress. (2019) 32:799–805. doi: 10.1016/j.brat.2006.09.017

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Ronconi J, Shiner B, Watts B. Inclusion and exclusion criteria in randomized controlled trials of psychotherapy for PTSD. J Psychiatr Pract. (2014) 20:25–37. doi: 10.1002/jts.22438

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Kennedy-Martin T, Curtis S, Faries D, Robinson S, Johnston J. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials. (2015) 16:495. doi: 10.1097/01.pra.0000442936.23457.5b

CrossRef Full Text | Google Scholar

52. Markowitz J, Petkova E, Neria Y, Van Meter P, Zhao Y, Hembree E, et al. Is exposure necessary? A randomized clinical trial of interpersonal psychotherapy for PTSD. Am J Psychiatry. (2015) 172:430–40. doi: 10.1186/s13063-015-1023-4

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Bleiberg K, Markowitz J. Interpersonal psychotherapy for PTSD: treating trauma without exposure. J Psychother Integr. (2019) 29:15–22. doi: 10.1176/appi.ajp.2014.14070908

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Hawn S, Cusack S, Amstadter AB. A Systematic review of the self-medication hypothesis in the context of posttraumatic stress disorder and comorbid problematic alcohol use. J Trauma Stress. (2020) 33:699–708. doi: 10.1037/int0000113

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Koob G, Volkow N. Neurobiology of addiction: a neurocircuitry analysis. Lancet Psychiatry. (2016) 3:760–73. doi: 10.1002/jts.22521

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: post-traumatic stress disorder, depression, substance abuse disorders, longitudinal, machine learning, comorbidity, replication

Citation: Pierce B, Kirsh T, Ferguson AR, Neylan TC, Ma S, Kummerfeld E, Cohen BE and Nielson JL (2023) Causal discovery replicates symptomatic and functional interrelations of posttraumatic stress across five patient populations. Front. Psychiatry 13:1018111. doi: 10.3389/fpsyt.2022.1018111

Received: 12 August 2022; Accepted: 30 December 2022;
Published: 26 January 2023.

Edited by:

Sharain Suliman, Stellenbosch University, South Africa

Reviewed by:

Seth Davin Norrholm, Wayne State University, United States
Emily Belleau, McLean Hospital, United States

Copyright © 2023 Pierce, Kirsh, Ferguson, Neylan, Ma, Kummerfeld, Cohen and Nielson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Jessica L. Nielson,