Differential relationship of observer-rated and self-rated depression and anxiety scales with heart rate variability features

Heart rate variability (HRV) is a known psychophysiological marker for diverse psychiatric symptoms. In this study, we aimed to explore the potential for clinical use of HRV by investigating the interrelationship between HRV indices and clinical measures mainly used to assess depressive and anxious symptoms. Participants who reported depressive and anxious symptoms were designated into the following groups: group 1, clinician-rated and self-rated depression; group 2, only self-rated depression; group 3, clinician-rated and self-rated anxiety; group 4, only self-rated anxiety. Statistical comparisons were performed between these groups to investigate the association between HRV and clinical measures. As a result, HRV variables showed significant correlations only with the clinician-rated assessments. Moreover, both time and frequency domain HRV indices were significantly different between groups 1 and 2, but groups 3 and 4 showed significant differences only in frequency domain HRV indices. Our study showed that HRV is an objective indicator for depressive or anxious symptoms. Additionally, it is considered a potential indicator for predicting the severity or state of depressive symptoms rather than of anxious symptoms. This study will contribute to increasing the diagnostic utility of discriminating those symptoms based on HRV in the future.

1. Introduction

In the 21st century, mental disorders are considered one of the most serious diseases that threaten people’s health and life (1). Many people worldwide are suffering from various mental disorders, with a 12 month prevalence of 17.6% and lifetime prevalence of 29.2% (2). This prevalence has not only significantly increased during the last decade (2–4) but also continues to increase due to circumstances such as the COVID-19 pandemic (5, 6). Mental disorders are one of the leading causes of the global health-related burden (5), imposing a substantial financial strain on the medical system with growing negative social and economic consequences (6, 7). Among the common symptoms of mental disorders, depression and anxiety are associated with emotional distress of patients, symptom severity, prognosis, and recurrence of related disorders (8), while also being the most prevalent complications of psychiatric disorders (9). Therefore, accurate and reliable detection of depressive or anxious symptoms can enable the early detection, intervention, and treatment of psychiatric disorders.

For the most part, psychiatric symptoms are assessed through clinician-rated or self-rated questionnaires in many clinics. The clinician-rated questionnaire is valid and reliable to objectively assess the severity of symptoms; however, it requires complex, time-consuming, and costly endeavors (10, 11). Conversely, although the self-rated questionnaire is an efficient way to save time and cost, the total scores can be over- or under-estimated depending on the character, predisposition, and cognitive ability of the respondents (10, 12), which is a serious problem. There is a controversy on the use of these clinical measures. Several previous studies reported that the severity of depression or anxious symptoms evaluated by both types of measurement was highly concordant, suggesting the interchangeability with each other (13–16). However, some previous studies suggested that evaluation results by those measures showed significant discrepancies, even asserting that only clinician-rated measures have statistical advantages to evaluate the severity of depressive or anxious symptoms (10, 17–19). Contrarily, it was also suggested that both clinician-rated and self-rated measures should be utilized together rather than using a single measure because each measurement provides unique information relevant to clinical prognosis (20). These discrepancies in opinion may result from the following reasons: clinicians’ or patients’ biases in interpreting symptomatology, limited insight of clinicians or patients, cognitive functioning and/or personality traits of patients, and patient demographic characteristics (such as age, sex, education, and occupation) (12, 15, 18). In essence, there are still no definitive and gold-standard assessments to quantify the severity of depressive or anxious symptoms. There is a need for an alternative method that can measure the severity of depression or anxiety in a more stable and reliable way by compensating for the above mentioned limitations of scale-based assessments.

Conversely, various heart rate variability (HRV) indices are considered more objective metrics for describing the health of the body and mind (21, 22). HRV, defined as complex beat-to-beat variations between heartbeats, is regulated by the interaction of the sympathetic and parasympathetic (vagal) components of the autonomic nervous system (ANS) and is recognized as a psychophysiological marker that reliably, objectively, and indirectly reflects the functioning of the ANS (23–25). In particular, dysregulation of the ANS can be quantified using HRV in diverse psychiatric symptoms and disorders (24, 26), and there have been numerous attempts to explore the relationship between the symptoms of depression and anxiety and HRV (27–29). Reduction in HRV is known to be a general indicator for patients with psychiatric disorders rather than healthy controls (30), and this finding is also shown in patients with depressive or anxious symptoms. Accordingly, meta-analyses, which provide more convincing evidence, also showed that reduced HRV is associated with both depression and anxiety. Anxious symptoms might relate to failure to inhibition of cognitive, behavioral, and affective responses, which reduce vagal function and eventually lead to decreased HRV indices (e.g., high-frequency power (HF), standard deviation of all RR intervals (SDRR), and root mean square of successive RR interval differences (RMSSD)) (27). Likewise, depressive symptoms might relate to somatomotor deficits which lead to changed HRV indices such as decreased HF and RMSSD, and increased low-frequency power (LF) and low- to high-frequency ratio (LF/HF) (29). Particularly, Kemp et al. showed that depressive symptoms had a significant negative correlation with HRV in major depressive disorder (MDD) patients, suggesting that HRV can be reflected in the severity of depressive symptoms (29). These findings support that HRV is not only an objective and quantitative psychophysiological marker but also has the potential for screening psychiatric symptoms and disorders.

It is necessary to directly compare the autonomic balance indicator—HRV—with the clinical assessments frequently used for evaluating various psychiatric symptoms, in order to support or increase the possibility of symptom evaluations based on HRV in clinics. A few studies have found that HRV indices have significant correlations with clinician-rated (31–34) and self-rated measures (35) and therefore have the capability to discriminate depressive symptoms in MDD patients. In the case of anxiety, a study demonstrated found no significant correlation between self-rated measures and HRV indices in patients with anxious symptoms (36). However, Bilgin et al. also examined the relationship between the frequency domain measures of HRV and both clinician-rated and self-rated assessments used to determine anxiety levels and identified a significant relationship between the frequency domain HRV indices and the outcomes of those clinical measures (37). Although several studies have been conducted to date, few studies have simultaneously compared the relationship between HRV indices and both clinician-rated and self-rated questionnaires for depressive or anxious symptoms. In other words, existing studies have limitations related to exploring the relationship between HRV indices and using only clinician-rated or self-rated questionnaires, applying only specific domain measures of HRV, or designating the participants who have specific mental disorders. To compensate for these limitations, a more systematic study is needed to precisely examine the relationship between HRV indices and both clinician-rated and self-rated assessments, with respect to the transdiagnostic dimensional approach (38).

As mentioned above, the use of an appropriate measure is required to investigate depressive and anxious symptoms accurately. However, which of the two clinical measures, clinician-rated and self-rated questionnaires, could better reflect the severity of depressive and anxious symptoms is somewhat questionable since each has clear advantages and disadvantages. Moreover, although HRV is considered an objective psychophysiological marker, it is still difficult to be used independently for discriminating the symptoms of depression and anxiety due to the lack of evidence about its relationship with psychological assessments. Therefore, it is required to investigate the interrelationship between physiological changes (i.e., HRV), clinician-rated assessments, and self-rated questionnaires by acquiring and comparing them at the same time. Eventually, it is expected to shed light on the clinical application of HRV as well as examine whether each HRV index is relevant to the objective or subjective clinical measures. Furthermore, a more reliable, objective, and quantitative method can be obtained for identifying depressive and anxious symptoms.

In this study, we hypothesized that the HRV indices would have a significant interrelationship with clinical measures. In particular, as the HRV indices were obtained based on bio-signals, they would show higher correlations with the results of the clinician-rated measure than those of the self-rated measure, which may signify that the HRV indices have objective rather than subjective implications. For this purpose, we simultaneously evaluated depressive or anxious symptoms of patients using clinical-rated and self-rated questionnaires, and acquired electrocardiogram (ECG) data from the patients to extract various HRV indices. Thereafter, the interrelationship among the HRV indices, the clinician-rated assessments, and the self-rated questionnaire was determined.

2. Materials and methods

2.1. Participants

Participants were outpatients from the psychiatric department of Yonsei University Gangnam Severance Hospital (Seoul, South Korea) who reported depressive and/or anxious symptoms from November 2018, to January 2022. During this period, a total of 3,492 new patients visited the psychiatric outpatient clinic of the hospital, regardless of the diagnosis. As shown in Figure 1, most patients were excluded at the first stage according to the following exclusion criteria: (i) patients aged over 50 with a high chance of prevalence of cardiovascular disease (39), (ii) patients diagnosed with a cardiovascular disease within the past 6 months, and (iii) patients who did not complete ECG assessments and/or clinical questionnaires. Additionally, patients who were unable to complete the self-rated questionnaires—such as patients with intellectual disability, cognitive disorders such as dementia, or severe active symptoms of psychotic disorders that cause problems in reality testing— were also excluded. After the first exclusion, 137 patients were evaluated for depressive and anxious symptoms using self-rated questionnaires and clinician-rated assessments. The Korean Quick Inventory of Depressive Symptomatology Self-Report (KQIDS-SR) and State–Trait Anxiety Inventory-State (STAI-S) were used to measure the severity of depressive and anxious symptoms, respectively. To include only patients with obvious symptoms of depression and/or anxiety, 41 patients with total KQIDS-SR scores (40) of ≤15 and 73 patients with total STAI-S scores (41) of ≤61 were excluded. After applying the exclusion criteria, clinician-rated assessments were used for further subdivision. The Hamilton Rating Scale for Depression (HRSD) (42, 43) and Hamilton Anxiety Scale (HAS) (44, 45) were used for patients with symptoms of depression (n = 96) and anxiety (n = 64), respectively. When the total of scores of HRSD or HAS were 24 or higher, patients were considered to have overt symptoms of depression or anxiety. Among the patients excluded based on the self-rated questionnaire results, 40 and 55 patients were not depressed or anxious, respectively, and 1 and 18 patients were considered to have depressive or anxious symptoms only by clinicians, respectively. More detailed descriptions of the group allocation according to the results of clinical measures are provided in Supplementary Tables S1 and S2. All clinical data were retrospectively collected from the Severance Hospital’s Electronic Medical Record system, and informed consent was not obtained. This retrospective study protocol was approved by the institutional review board of Gangnam Severance Hospital (3–2022-0009).

FIGURE 1

Figure 1. Flow diagram of participants’ enrollment, assessment, and allocation. KQIDS-SR, Korean Quick Inventory of Depressive Symptomatology Self-Report; STAI-S, State–Trait Anxiety Inventory-State; HRSD, Hamilton Rating Scale for Depression; HAS, Hamilton Anxiety Scale; ECG, Electrocardiogram.

2.2. Assessment of depression and anxiety

Several clinical assessments were conducted to investigate the severity of depression and/or anxiety in participants. Self-rated questionnaires, such as KQIDS-SR and STAI-S, were used to measure the degree of subjective depression and/or anxiety states. The KQIDS-SR (40) has 16-items that are designed to measure symptoms of depression, including melancholic and atypical symptoms. A KQIDS-SR score of 16 or more represents significant depression. The STAI (41) comprises two sets of 20 items each with a 4-point Likert scale to assess state anxiety (STAI-S) and trait anxiety (STAI-T). Only STAI-S was used in this study, and a STAI-S score of over 62 indicates significant anxiety. If participants were identified with depression and/or anxiety, they were assessed by trained psychiatrists or psychologists to screen the states of depression and/or anxiety through objective verification. In this study, the 17-item HRSD (42, 43) and the 14-item HAS (44, 45) were used. Total scores on the HRSD range from 0 to 54 points, which were classified into four categories: absence of depression (0 to 7), mild depression (8 to 16), moderate depression 17 to 23, and severe depression (above 24). The HAS scores range from 0 to 56, with 0 to 7 indicating absence of anxiety, 8 to 14 indicating mild anxiety, 15 to 23 indicating moderate anxiety, and 24 to 56 indicating severe anxiety.

2.3. Heart rate variability

Participants categorized into the four groups through clinical assessments underwent ECG recording while in a seated position in a quiet and comfortable room. The ECG signals were recorded at 500 Hz (SA-3000P, Medicore Co., Ltd., Seoul, Korea) for 5 min to reliably estimate HRV indices from the acquired signals (24, 46–49). The HRV indices were analyzed using MATLAB 2017b (Mathworks, Inc., MA, United States). At first, raw ECG signals were manually inspected to ensure that they do not contain any unnecessary components such as motion artifacts. After artifact correction, approximately 5–10% of R peaks were discarded from the signals. From artifact-free ECG signals, the electrocardiographic R-peaks used to derive RR intervals were clearly detected by applying peak detection algorithms. Finally, the defined RR intervals were used to calculate several HRV indices (21, 24) as follows: the time domain indices (50)—mean heart rate (Mean HR), mean of all RR intervals (Mean RR), SDRR, standard deviation of differences between adjacent RR intervals (SDSD), RMSSD, and percentage of differences between adjacent RR intervals >20 ms (pNN20); frequency domain indices (51)—normalized HF (nHF, 0.15 to 0.4 Hz), normalized LF (nLF, 0.04 to 0.15 Hz), and LF/HF. The time domain indices of HRV can estimate overall changes in HRV, whereas the frequency domain indices of HRV can estimate the modulation of sympathetic and parasympathetic nervous systems or their balance.

2.4. Statistical analysis

To compare the differences in demographics and clinical characteristics, independent t-tests (for continuous variables) and chi-square tests (for categorical variables) were performed between groups 1 and 2, and between groups 3 and 4. Additional independent t-tests and Mann–Whitney U tests were conducted between the groups to determine how HRV indices differed depending on the discrepancies between clinician-rated and self-rated scores of depression or anxiety. To investigate the group differences in depth, effect sizes (ES) were calculated based on the results of the t-test. A random effects model was used to calculate the standardized mean differences. Likewise, to assess the effectiveness of self-rated measures, the differences in HRV indices were investigated using independent t-tests and Mann–Whitney U tests between patients who were not depressed or anxious and those who were rated as depressed or anxious through self-rated assessments. Moreover, Pearson’s correlation analysis was conducted to explore the relationship between HRV indices and both clinician-rated and self-rated measures of depression or anxiety. Most of the statistical analyses were performed using SPSS version 25 (IBM Corp., Armonk, NY, United States). The calculation of ES was conducted using STATA version 16.0 (StataCorp LLC., College Station, Texas, United States). For all the analyses, statistical significance was defined as a two-tailed value of p of ≤0.05.

3. Results

3.1. Demographical and clinical characteristics of the participants

As mentioned in the Methods section, participants were divided into each group based on the results of self-rated and clinician-rated measures used to assess the states of depression and anxiety. The final groups were constituted as follows: group 1 (n = 28), clinician-rated and self-rated depression; group 2 (n = 68), only self-rated depression; group 3 (n = 44), clinician-rated and self-rated anxiety; group 4 (n = 20), only self-rated anxiety. Groups 1 and 2 comprised participants with depressive symptoms and groups 3 and 4 comprised participants with anxious symptoms. The differences between the groups were that the depressive or anxious states of the participants in groups 2 and 4 were identified using self-rated questionnaires, whereas those in groups 1 and 3 were diagnosed by both self-rated and clinician-rated assessments. Additionally, groups A (no depression group, n = 40) and B (no anxiety group, n = 55) were constituted for further analysis among the excluded participants based on the cut-off values for each self-rated questionnaire.

Demographic and clinical characteristics of participants are presented in Table 1. As expected, there were no variables showing significant differences between groups except for the clinician-rated outcomes. The HRSD score of group 2 was lower than that of group 1 (p < 0.001), indicating that the patients in group 2 were not diagnosed as having severe depression by clinicians. Similarly, the HAS score of group 4 was lower than that of group 3 (p < 0.001), representing that the patients in group 4 were not diagnosed as having severe anxiety by clinicians. Meanwhile, the main type in the DSM-5 diagnosis category assigned to participants was depressive disorders (35.71%, n = 35), followed by trauma-and stressor-related disorders (30.61%, n = 30) including adjustment disorders, and anxiety disorders (15.31%, n = 15). Other categories included somatic symptom disorders (7.14%, n = 7) and substance use and addictive disorders (4.08%, n = 4).

TABLE 1

Table 1. Comparisons of demographic and clinical characteristics between groups 1 and 2, and between groups 3 and 4.

3.2. Correlations of HRV indices with depression and anxiety scales

Pearson’s correlation analysis was used to determine the relationship between HRV indices and clinical assessments for depression or anxiety. As shown in Table 2, the self-rated and clinician-rated assessments for depression were positively correlated (r = 0.214, p = 0.036); however, those assessments for anxiety were not significantly correlated (r = 0.017, p = 0.894). Several HRV indices showed significant correlations with only the clinician-rated assessments—HRSD and HAS. The HRSD score had positive correlations with Mean HR, nLF, and LF/HF (r = 0.262, r = 0.309, r = 0.313; p = 0.010, p = 0.002, p = 0.002, respectively) but had negative correlations with Mean RR and nHF (r = −0.239, r = −0.309; p = 0.019, p = 0.002, respectively). For the HAS score, there were positive correlations between Mean HR and nLF (r = 0.251, r = 0.258; p = 0.045, p = 0.040) but a negative correlation between nHF (r = −0.258; p = 0.040). The self-rated questionnaires, KQIDS-SR and STAI-S, did not show any significant correlations with the HRV indices.

TABLE 2

Table 2. Correlations matrix between HRV indices and clinical measures (clinician-rated and self-rated report).

3.3. Understanding the meaning of HRV indices of depressive or anxious symptoms

Table 3 represents the mean values of HRV indices for each group. The differences in HRV indices between groups 1 and 3 had a similar tendency to those between groups 2 and 4: increase in Mean HR, nLF, and LF/HF; decrease in Mean RR, SDRR, SDSD, RMSSD, pNN20, and nHF. Both the time and frequency domain HRV indices were significantly different between groups 1 and 2, whereas groups 3 and 4 only had significant differences in the frequency domain HRV indices. These results are depicted in Figure 2. The standardized mean difference (ES) and 95% confidence intervals (CI) between groups are represented in the forest plot. The vertical solid black line indicates a point where the ES is zero, which means that there are no significant differences between the groups. Between groups 1 and 2, Mean HR, nLF, and LF/HF indices had positive ES while all other indices had negative ES. The 95% CI of each HRV index between groups 1 and 2 did not include the point where ES is zero, but most of those between groups 3 and 4 did include that point. Similar to the results in Table 3, all HRV indices between groups 1 and 2 showed significant differences. In the same manner, only frequency domain HRV indices between groups 3 and 4 showed significant differences.

TABLE 3

Table 3. Comparison of HRV indices between groups 1 and 2, and between groups 3 and 4.

FIGURE 2

Figure 2. A forest plot showing the effect sizes (ES) and confidence intervals (CI) of HRV indices compared between groups. The vertical solid black line represents a mean difference of zero or no effect. Each black diamond shows the standardized mean difference. The upper and lower range of the line connected to the diamond shows the 95% CI for the ES. Group 1: clinician-rated and self-rated depression; Group 2: only self-rated depression; Group 3: clinician-rated and self-rated anxiety; Group 4: only self-rated anxiety. HRV, heart rate variability; Mean HR, mean heart rate; Mean RR, mean of all RR intervals; SDRR, standard deviation of all RR intervals; SDSD, standard deviation of differences between adjacent RR intervals; RMSSD, root mean square of successive RR interval differences; pNN20, percentage of differences between adjacent RR intervals >20 ms; nHF, normalized high-frequency power; nLF, normalized low-frequency power; LF/HF, low- to high-frequency ratio.

Furthermore, for the purpose of identifying another relationship between HRV indices and clinical assessments from a new perspective, Table 4 and Figure 3 are presented by applying the same method as above to other groups. Thus, new groups A and B of participants with no depression and anxiety were compared to groups 2 and 4, respectively, as mentioned earlier. As shown in Table 4, all HRV indices were not significantly different between groups A and 2 and groups B and 4. Similarly, in Figure 3, the 95% CI of all HRV indices contained the point where ES is zero. These results again revealed that there were no significant differences between groups A and 2 and groups B and 4.

TABLE 4

Table 4. Comparison of HRV indices between groups A and 2, and between groups B and 4.

FIGURE 3

Figure 3. A forest plot showing the effect sizes (ES) and confidence intervals (CI) of HRV indices compared between groups. The vertical solid black line represents a mean difference of zero or no effect. Each black diamond shows the standardized mean difference. The upper and lower range of the line connected to the diamond shows the 95% CI for the ES. Group A: no depression group; Group 2: only self-rated depression; Group B: no anxiety group; Group 4: only self-rated anxiety. HRV, heart rate variability; Mean HR, mean heart rate; Mean RR, mean of all RR intervals; SDRR, standard deviation of all RR intervals; SDSD, standard deviation of differences between adjacent RR intervals; RMSSD, root mean square of successive RR interval differences; pNN20, percentage of differences between adjacent RR intervals >20 ms; nHF, normalized high-frequency power; nLF, normalized low-frequency power; LF/HF, low- to high-frequency ratio.

4. Discussion

The present study evaluated the relationship between HRV and clinical evaluation results in subjects complaining of depressive and anxious symptoms. To increase the availability for clinical use of HRV indices, we sought to determine whether each HRV index is associated with objective or subjective implications of clinical measures. We extracted and utilized the indices of time and frequency domains of HRV reflecting ANS modulation. For clinical evaluations of depressive and anxious symptoms, both objective assessments by clinicians (HRSD and HAS) and subjective assessments by oneself (KQIDS-SR and STAI-S) were performed. As a result, it was confirmed that HRV indices changed significantly according to the severity of depressive and anxious symptoms in line with previous findings: increase in nLF and LF/HF; decrease in SDRR, RMSSD, pNN20, and nHF (27, 29). To our knowledge, few studies to date have simultaneously demonstrated the relationship between HRV, clinician-rated assessment, and self-rated assessment. In particular, various types of HRV indices that are difficult to substitute with each other were used in this study. Moreover, this is the first study to conduct the above analysis on a person who complains of depressive or anxious symptoms regardless of diagnosis at their outpatient clinic visit, rather than on those who have been diagnosed with a specific psychiatric disorder. Consequently, it is considered that HRV can enable estimating the depressive or anxious symptoms in an individual who reported those symptoms, regardless of a specific diagnosis. Additionally, HRV is simple and easy to perform, highly reproducible, and can be measured multiple times at a low cost compared to clinical measures (52, 53). Based on these advantages of HRV, it can be considered a valuable indicator that may have the potential for early detection of psychiatric symptoms and disorders.

The first major finding of this study is that HRV indices have objective implications. Each HRV index was found to be statistically significant when the depressive and anxious symptoms were assessed through both clinician-rated and self-rated assessments rather than self-rated assessment alone. Additionally, there were no significant differences in HRV indices when comparing those who were assessed as having no depressive or anxious symptoms through clinician-rated and self-rated measurements with those who were assessed as having these symptoms through only self-rated measurement. These results support the implication that depressive and anxious symptoms can be evaluated more objectively and accurately through clinician-rated measures and HRV than self-rated measures. In other words, it may be considered that HRV has diagnostic usefulness similar to clinician-rated measures, which show higher diagnostic performance than self-rated measures. Furthermore, statistically significant correlations were found only between HRV indices and clinician-rated assessments. The correlations between HRV indices and self-rated assessments also showed almost coherent tendencies, although they were not significant. These results further support our suggestion that HRV indices can be considered objective indicators similar to clinician-rated measures based on the objective view of experts, rather than self-rated measures based on subjective opinions. For this reason, to measure the symptoms of depression or anxiety more clearly and practically, it could be more effective to apply clinician-rated assessments and HRV indices, although many previous studies had proposed that self-rated measures are capable of evaluating those states (13–16, 35, 37).

The second major finding of this study is that HRV indices are likely to be better indicators of depressive symptoms than anxious symptoms. Through correlation analysis, significant correlations were confirmed between HRV indices and clinician-rated assessments for both depressive and anxious symptoms; however, a stronger relationship appeared in the assessments for depressive symptoms. In other words, observed depression showed high correlations with HRV indices and hence, it may be possible to objectively estimate the severity of depressive symptoms through the HRV indices. Moreover, a meta-analysis based on previous studies examining HRV indices related to depressive symptoms revealed that significant differences were observed in both time and frequency domain HRV indices (54). Our results were not only consistent with past findings but also revealed significant differences in both time and frequency HRV indices according to the observed depressive symptoms that were assessed objectively. Through these results, HRV is regarded as a potential and effective biomarker to objectively predict the severity of depressive symptoms. In contrast, the statistical power was relatively low or absent in our results for anxious symptoms. There are conflicting findings on whether the time or frequency domain HRV indices can reflect some significant changes according to the presence or severity of anxiety states (55). Several previous studies comparing the patients with anxious symptoms to the controls have suggested that the significant differences in HRV indices regarding the symptoms were presented only when a stressful situation such as specific stimuli (e.g., nervousness, tension, apprehension, and worry) was given, but not in resting state (56, 57). These results may indicate that the reactive response to anxiety is reflected in HRV, but significant changes in the resting state are hard to detect using HRV. Our findings also support that it is relatively difficult to estimate the severity of anxious symptoms through HRV in the resting state. Consequently, time domain HRV indices can be used to identify depressive symptoms more precisely. Similarly, although frequency domain HRV indices also appear to be more specific for identifying depressive symptoms, it is more reasonable to perceive that these indices can reflect a broad range of distress symptoms including anxiety.

The current findings should be interpreted with respect to some limitations. First, all groups were constituted using cut-off values that were already designated per each clinical measure. When forming the groups in this study, a high cut-off value was used as a reference because we tried to include only participants with clear clinical symptoms. For this reason, a few cases with relatively mild depressive or anxious symptoms may have been excluded. Second, only one type of clinical evaluation scale for depressive or anxious symptoms was used in this study. There are various types of clinical measures that reflect different aspects of verifying depressive or anxious symptoms (58, 59). As a result, the extent or severity of those symptoms may vary depending on the type of clinical measures. In this study, HAS and STAI were implemented, which are frequently used to evaluate anxious symptoms. The clinician-rated measure, HAS, mainly focuses on somatic symptoms, whereas the self-rated measure, STAI, is consisted of ambiguous items representing a generalized state of malaise (41, 44, 45, 60). The correlation between HAS and STAI may not be significant due to the discrepancies in composition. Conversely, it is suggested that the clinician-rated and self-rated assessments on depression, HRSD and KQIDS-SR, have high coexistence validity and similar sensitivity to symptom change (40). Taken together, it can be estimated that the results of this study could vary if other clinical measures are used to evaluate the symptoms of depression and anxiety. Therefore, exploring the relationship between different types of clinical measures and HRV indices in the same way is worth considering in the future. Third, we did not include the patients according to specific DSM diagnosis. It is noteworthy that we wanted to find the symptom-specific HRV features regardless of the diagnosis, with respect to the transdiagnostic dimensional approach, which is a developing concept accompanying the release of the DSM-5. Fourth, the impact of comorbidity was not considered when interpreting the differences in HRV according to depressive and anxious symptoms. It is generally known that anxious symptoms are known to be common comorbidities in people with depressive symptoms (61). For this reason, our findings should be interpreted carefully considering that many subjects with comorbidity with symptoms of depression and anxiety in this study. Further study is needed to identify the changes in HRV indices associated with pure depressive and anxious symptoms. Lastly, several statistical limitations may affect the generalizability of our findings. Although the correlation was significant, some correlation coefficients were not high. Moreover, we did not correct for multiple comparisons that may possibly result in Type-I errors. Hence, our findings should be interpreted with caution.

5. Conclusion

In summary, this study explored the relationship between HRV and clinical assessments of depressive and anxious symptoms. Particularly, a novelty of this study is that both clinician-rated and self-rated assessments were used to identify the relationship with HRV simultaneously. Moreover, another novelty is that the subjects were not limited to a specific group of mental disorders but included those who showed symptoms of depression or anxiety. Overall, our results suggest that HRV is an objective indicator to reflect the reduction in vagal tone associated with depressive or anxious symptoms, and is especially more effective in representing the state of depression and estimating the severity of depressive symptoms compared with anxious symptoms. This study’s findings contribute to the early detection of and interventions for depressive and anxious symptoms in the future.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary materials, further inquiries can be directed to the corresponding authors.

Ethics statement

The studies involving human participants were conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of Gangnam Severance Hospital, Yonsei University (protocol code 3-2022-0009 and 8th March 2022). Written informed consent for participation was not required due to the retrospective nature of the study.”

Author contributions

JH, BL, and JO: conceptualization, validation, and writing—original draft preparation. JH and JO: methodology, writing—review and editing, and visualization. JH: software and formal analysis. JH, HEK, J-JK, J-HS, EK, JYP, and JO: investigation. J-JK, J-HS, EK, JYP, BL, and JO: resources. HEK, J-JK, J-HS, EK, JYP, and JO: data curation. BL and JO: supervision. J-JK, J-HS, EK, JYP, and JO: project administration. JO: funding acquisition. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the Industrial Technology Innovation Program (no. 20012603, Development of Emotional Cognitive and Sympathetic AI Service Technology for Remote (Non-face-to-face) Learning and Industrial Sites) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2023.1124550/full#supplementary-material

References

1. Zhou, Y, and Sun, Y. Basic study on architectural planning of psychiatric hospital. Sci China Technol Sci. (2010) 53:1755–67. doi: 10.1007/S11431-010-3077-6

CrossRef Full Text | Google Scholar

2. Steel, Z, Marnane, C, Iranpour, C, Chey, T, Jackson, JW, Patel, V, et al. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013. Int J Epidemiol. (2014) 43:476–93. doi: 10.1093/IJE/DYU038

PubMed Abstract | CrossRef Full Text | Google Scholar

3. GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the global burden of disease study 2017. Lancet. (2018) 392:1789–858. doi: 10.1016/S0140-6736(18)32279-7

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Organization WH. The World Health Report 2001—Mental Health: New Understanding, New Hope World Health Organization (2001) Available at: https://apps.who.int/iris/handle/10665/42390.

Google Scholar

5. COVID-19 Mental Disorders Collaborators. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. Lancet. (2021) 398:1700–12. doi: 10.1016/S0140-6736(21)02143-7

PubMed Abstract | CrossRef Full Text | Google Scholar

6. GBD 2019 Mental Disorders Collaborators. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990–2019: a systematic analysis for the global burden of disease study 2019. Lancet Psychiatry. (2022) 9:137–50. doi: 10.1016/S2215-0366(21)00395-3

PubMed Abstract | CrossRef Full Text | Google Scholar

7. GBD 2019 Risk Factors Collaborators. Global burden of 87 risk factors in 204 countries and territories, 1990–2019: a systematic analysis for the global burden of disease study 2019. Lancet. (2020) 396:1223–49. doi: 10.1016/S0140-6736(20)30752-2

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Hartley, S, Barrowclough, C, and Haddock, G. Anxiety and depression in psychosis: a systematic review of associations with positive psychotic symptoms. Acta Psychiatr Scand. (2013) 128:327–46. doi: 10.1111/ACPS.12080

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Smith, L, Jacob, L, Yakkundi, A, McDermott, D, Armstrong, NC, Barnett, Y, et al. Correlates of symptoms of anxiety and depression and mental wellbeing associated with COVID-19: a cross-sectional study of UK-based respondents. Psychiatry Res. (2020) 291:113138. doi: 10.1016/J.PSYCHRES.2020.113138

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Dunlop, BW, Li, T, Kornstein, SG, Friedman, ES, Rothschild, AJ, Pedersen, R, et al. Concordance between clinician and patient ratings as predictors of response, remission, and recurrence in major depressive disorder. J Psychiatr Res. (2011) 45:96–103. doi: 10.1016/J.JPSYCHIRES.2010.04.032

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Zimmerman, M, Posternak, MA, and Chelminski, I. Is it time to replace the Hamilton depression rating scale as the primary outcome measure in treatment studies of depression? J Clin Psychopharmacol. (2005) 25:105–10. doi: 10.1097/01.JCP.0000155824.59585.46

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Corruble, E, Legrand, JM, Zvenigorowski, H, Duret, C, and Guelfi, JD. Concordance between self-report and clinician’s assessment of depression. J Psychiatr Res. (1999) 33:457–65. doi: 10.1016/S0022-3956(99)00011-4

CrossRef Full Text | Google Scholar

13. Broocks, A, Bandelow, B, Pekrun, G, George, A, Meyer, T, Bartmann, U, et al. Comparison of aerobic exercise, clomipramine, and placebo in the treatment of panic disorder. Am J Psychiatry. (1998) 155:603–9. doi: 10.1176/AJP.155.5.603/ASSET/IMAGES/LARGE/AA6F1.JPEG

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Rush, AJ, Bernstein, IH, Trivedi, MH, Carmody, TJ, Wisniewski, S, Mundt, JC, et al. An evaluation of the quick inventory of depressive symptomatology and the Hamilton rating scale for depression: a sequenced treatment alternatives to relieve depression trial report. Biol Psychiatry. (2006) 59:493–501. doi: 10.1016/J.BIOPSYCH.2005.08.022

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Rush, AJ, Carmody, TJ, Ibrahim, HM, Trivedi, MH, Biggs, MM, Shores-Wilson, K, et al. Comparison of self-report and clinician ratings on two inventories of depressive symptomatology. Psychiatr Serv. (2006) 57:829–37. doi: 10.1176/PS.2006.57.6.829

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Rush, AJ, Trivedi, MH, Carmody, TJ, Ibrahim, HH, Markowitz, JC, Keitner, GI, et al. Self-reported depressive symptom measures: sensitivity to detecting change in a randomized, controlled trial of chronically depressed, nonpsychotic outpatients. Neuropsychopharmacology. (2004) 30:405–16. doi: 10.1038/sj.npp.1300614

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Carroll, BJ, Fielding, JM, and Blashki, TG. Depression rating scales: a critical review. Arch Gen Psychiatry. (1973) 28:361–6. doi: 10.1001/ARCHPSYC.1973.01750330049009

CrossRef Full Text | Google Scholar

18. Cuijpers, P, Li, J, Hofmann, SG, and Andersson, G. Self-reported versus clinician-rated symptoms of depression as outcome measures in psychotherapy research on depression: a meta-analysis. Clin Psychol Rev. (2010) 30:768–78. doi: 10.1016/J.CPR.2010.06.001

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Sallee, FR, Sethuraman, G, Sine, L, and Liu, H. Yohimbine challenge in children with anxiety disorders. Am J Psychiatry. (2000) 157:1236–42. doi: 10.1176/APPI.AJP.157.8.1236

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Uher, R, Perlis, RH, Placentino, A, Dernovšek, MZ, Henigsberg, N, Mors, O, et al. Self-report and clinician-rated measures of depression severity: can one replace the other? Depress Anxiety. (2012) 29:1043–9. doi: 10.1002/DA.21993

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Shaffer, F, and Ginsberg, JP. An overview of heart rate variability metrics and norms. Front Public Health. (2017) 5:258. doi: 10.3389/FPUBH.2017.00258/BIBTEX

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Thayer, JF, Hansen, AL, Saus-Rose, E, and Johnsen, BH. Heart rate variability, prefrontal neural function, and cognitive performance: the Neurovisceral integration perspective on self-regulation, adaptation, and health. Ann Behav Med. (2009) 37:141–53. doi: 10.1007/S12160-009-9101-Z

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Bertsch, K, Hagemann, D, Naumann, E, Schächinger, H, and Schulz, A. Stability of heart rate variability indices reflecting parasympathetic activity. Psychophysiology. (2012) 49:672–82. doi: 10.1111/J.1469-8986.2011.01341.X

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Camm, AJ, Malik, M, Bigger, JT, Breithardt, G, Cerutti, S, Cohen, RJ, et al. Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation. (1996) 93:1043–65. doi: 10.1161/01.CIR.93.5.1043

CrossRef Full Text | Google Scholar

25. Sztajzel, J. Heart rate variability: a noninvasive electrocardiographic method to measure the autonomic nervous system. Swiss Med Wkly. (2004) 134:514–22. doi: 10.4414/smw.2004.10321

CrossRef Full Text | Google Scholar

26. Yang, AC, Hong, C-J, and Tasi, S-J. Heart rate variability in psychiatric disorders. Taiwan J Psychiatry. (2010) 24:99–109. doi: 10.29478/TJP.201006.0004

CrossRef Full Text | Google Scholar

27. Chalmers, JA, Quintana, DS, Abbott, MJA, and Kemp, AH. Anxiety disorders are associated with reduced heart rate variability: a meta-analysis. Front Psych. (2014) 5:80. doi: 10.3389/FPSYT.2014.00080/ABSTRACT

CrossRef Full Text | Google Scholar

28. Friedman, BH. An autonomic flexibility–neurovisceral integration model of anxiety and cardiac vagal tone. Biol Psychol. (2007) 74:185–99. doi: 10.1016/J.BIOPSYCHO.2005.08.009

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Kemp, AH, Quintana, DS, Gray, MA, Felmingham, KL, Brown, K, and Gatt, JM. Impact of depression and antidepressant treatment on heart rate variability: a review and meta-analysis. Biol Psychiatry. (2010) 67:1067–74. doi: 10.1016/J.BIOPSYCH.2009.12.012

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Alvares, GA, Quintana, DS, Hickie, IB, and Guastella, AJ. Autonomic nervous system dysfunction in psychiatric disorders and the impact of psychotropic medications: a systematic review and meta-analysis. J Psychiatry Neurosci. (2016) 41:89–104. doi: 10.1503/JPN.140217

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Agelink, MW, Boz, C, Ullrich, H, and Andrich, J. Relationship between major depression and heart rate variability: clinical consequences and implications for antidepressive treatment. Psychiatry Res. (2002) 113:139–49. doi: 10.1016/S0165-1781(02)00225-1

CrossRef Full Text | Google Scholar

32. Agelink, MW, Majewski, T, Wurthmann, C, Postert, T, Linka, T, Rotterdam, S, et al. Autonomic neurocardiac function in patients with major depression and effects of antidepressive treatment with nefazodone. J Affect Disord. (2001) 62:187–98. doi: 10.1016/S0165-0327(99)00202-5

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Chang, HA, Chang, CC, Tzeng, NS, Kuo, TBJ, Lu, RB, and Huang, SY. Generalized anxiety disorder, comorbid major depression and heart rate variability: a case-control study in Taiwan. Psychiatry Investig. (2013) 10:326–35. doi: 10.4306/PI.2013.10.4.326

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Yeh, TC, Kao, LC, Tzeng, NS, Kuo, TBJ, Huang, SY, Chang, CC, et al. Heart rate variability in major depressive disorder and after antidepressant treatment with agomelatine and paroxetine: findings from the Taiwan study of depression and anxiety (TAISDA). Prog Neuro-Psychopharmacol Biol Psychiatry. (2016) 64:60–7. doi: 10.1016/J.PNPBP.2015.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Sun, G, Shinba, T, Kirimoto, T, and Matsui, T. An objective screening method for major depressive disorder using logistic regression analysis of heart rate variability data obtained in a mental task paradigm. Front Psych. (2016) 7:180. doi: 10.3389/FPSYT.2016.00180/BIBTEX

CrossRef Full Text | Google Scholar

36. Yeragani, VK, Srinivasan, K, Pohl, R, Berger, R, Baton, R, and Ramesh, C. Effects of nortriptyline on heart rate variability in panic disorder patients: a preliminary study using power spectral analysis of heart rate. Neuropsychobiology. (1994) 29:1–7. doi: 10.1159/000119054

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Bilgin, S, Arslan, E, Elmas, O, Yildiz, S, Colak, OH, Bilgin, G, et al. Investigation of the relationship between anxiety and heart rate variability in fibromyalgia: a new quantitative approach to evaluate anxiety level in fibromyalgia syndrome. Comput Biol Med. (2015) 67:126–35. doi: 10.1016/J.COMPBIOMED.2015.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Dalgleish, T, Black, M, Johnston, D, and Bevan, A. Transdiagnostic approaches to mental health problems: current status and future directions. J Consult Clin Psychol. (2020) 88:179–95. doi: 10.1037/CCP0000482

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Seo, HS, and Choi, MH. Cholesterol homeostasis in cardiovascular disease and recent advances in measuring cholesterol signatures. J Steroid Biochem Mol Biol. (2015) 153:72–9. doi: 10.1016/J.JSBMB.2015.04.014

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Yoon, JH, Jon, DI, Hong, HJ, and Hong, N. Reliability and validity of the Korean version of inventory for depressive symptomatology. Mood Emot. (2012) 10:131–51. http://www.moodandemotion.org/journal/view.html?spage=131&volume=10&number=3 (Accessed August 19, 2022)

Google Scholar

41. Beckler, K. State-Trait Anxiety Inventory for Adults Consult Psychol Press Inc, Palo Alto, Mind Gard Inc (1983).

Google Scholar

42. Hamilton, M. Development of a rating scale for primary depressive illness. Br J Soc Clin Psychol. (1967) 6:278–96. doi: 10.1111/J.2044-8260.1967.TB00530.X

CrossRef Full Text | Google Scholar

43. Hamilton, M. A RATING SCALE FOR DEPRESSION. J Neurol Neurosurg Psychiatry. (1960) 23:56–62. doi: 10.1136/JNNP.23.1.56

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Guy, W. Ecdeu Assessment Manual for Psychopharmacology. Washington, DC: US Department of Health, Education, and Welfare, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, NIMH Psychopharmacology Branch, Division of Extramural Research Programs (1976).

Google Scholar

45. Hamilton, M. The assessment of anxiety states by rating. Br J Med Psychol. (1959) 32:50–5. doi: 10.1111/J.2044-8341.1959.TB00467.X

CrossRef Full Text | Google Scholar

46. Akselrod, S, Gordon, D, Ubel, FA, Shannon, DC, Barger, AC, and Cohen, RJ. Power Spectrum analysis of heart rate fluctuation: a quantitative probe of beat-to-beat cardiovascular control. Science. (1981) 213:220–2. doi: 10.1126/SCIENCE.6166045

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Hirsch, JA, Bishop, B, Hirsch, J, and Ann, BB. Respiratory sinus arrhythmia in humans: how breathing pattern modulates heart rate. Am J Physiol Heart Circ Physiol. (1981) 241:H620–9. doi: 10.1152/AJPHEART.1981.241.4.H620

CrossRef Full Text | Google Scholar

48. Pagani, M, Lombardi, F, Guzzetti, S, Rimoldi, O, Furlan, R, Pizzinelli, P, et al. Power spectral analysis of heart rate and arterial pressure variabilities as a marker of sympatho-vagal interaction in man and conscious dog. Circ Res. (1986) 59:178–93. doi: 10.1161/01.RES.59.2.178

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Pomeranz, B, Macaulay, RJ, Caudill, MA, Kutz, I, Adam, D, Gordon, D, et al. Assessment of autonomic function in humans by heart rate spectral analysis. Am J Physiol Heart Circ Physiol. (1985) 248:H151–3. doi: 10.1152/AJPHEART.1985.248.1.H151

CrossRef Full Text | Google Scholar

50. Kleiger, RE, Stein, PK, Bosner, MS, and Rottman, JN. Time domain measurements of heart rate variability. Cardiol Clin. (1992) 10:487–98. doi: 10.1016/S0733-8651(18)30230-3

CrossRef Full Text | Google Scholar

51. Ori, Z, Monir, G, Weiss, J, Sayhouni, X, and Singer, DH. Heart rate variability: frequency domain analysis. Cardiol Clin. (1992) 10:499–533. doi: 10.1016/S0733-8651(18)30231-5

CrossRef Full Text | Google Scholar

52. Mazzeo, AT, La Monaca, E, Di Leo, R, Vita, G, and Santamaria, LB. Heart rate variability: a diagnostic and prognostic tool in anesthesia and intensive care. Acta Anaesthesiol Scand. (2011) 55:797–811. doi: 10.1111/J.1399-6576.2011.02466.X

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Chattipakorn, N, Incharoen, T, Kanlop, N, and Chattipakorn, S. Heart rate variability in myocardial infarction and heart failure. Int J Cardiol. (2007) 120:289–96. doi: 10.1016/J.IJCARD.2006.11.221

CrossRef Full Text | Google Scholar

54. Koch, C, Wilhelm, M, Salzmann, S, Rief, W, and Euteneuer, F. A meta-analysis of heart rate variability in major depression. Psychol Med. (2019) 49:1948–57. doi: 10.1017/S0033291719001351

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Cheng, YC, Su, MI, Liu, CW, Huang, YC, and Huang, WL. Heart rate variability in patients with anxiety disorders: a systematic review and meta-analysis. Psychiatry Clin Neurosci. (2022) 76:292–302. doi: 10.1111/PCN.13356

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Levine, JC, Fleming, R, Piedmont, JI, Cain, SM, and Chen, WJ. Heart rate variability and generalized anxiety disorder during laboratory-induced worry and aversive imagery. J Affect Disord. (2016) 205:207–15. doi: 10.1016/J.JAD.2016.07.019

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Wang, SM, Yeon, B, Hwang, S, Lee, HK, Kweon, YS, Lee, CT, et al. Threat-induced autonomic dysregulation in panic disorder evidenced by heart rate variability measures. Gen Hosp Psychiatry. (2013) 35:497–501. doi: 10.1016/J.GENHOSPPSYCH.2013.06.001

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Antony, MM. Assessment of anxiety and the anxiety disorders: an overview. Practitioner’s Guide Empirically Based Measures Anxiety. (2002):9–17. doi: 10.1007/0-306-47628-2_2

CrossRef Full Text | Google Scholar

59. Nezu, AM, McClure, KS, and Nezu, CM. “The Assessment of Depression,” in Treating Depression: MCT, CBT, and Third-Wave Therapies. John Wiley & Sons, Ltd. (2015):24–51.

Google Scholar

60. Keedwell, P, and Snaith, RP. What do anxiety scales measure? Acta Psychiatr Scand. (1996) 93:177–80. doi: 10.1111/J.1600-0447.1996.TB10627.X

CrossRef Full Text | Google Scholar

61. Kaiser, T, Herzog, P, Voderholzer, U, and Brakemeier, EL. Unraveling the comorbidity of depression and anxiety in a large inpatient sample: network analysis to examine bridge symptoms. Depress Anxiety. (2021) 38:307–17. doi: 10.1002/DA.23136

PubMed Abstract | CrossRef Full Text | Google Scholar