Establishment and validation a relapse prediction model for bipolar disorder

Abstract

Background:

The recurrence rate of bipolar disorder (BD) is relatively high. Assessing the risk of relapse in patients with BD can assist in identifying populations at high risk for recurrence, and early feasible interventions can improve patient’ prognoses. Therefore, it is important to establish and validate predictive models for relapse risk in patients with BD.

Method:

We used 303 patients with BD admitted to the Anhui Mental Health Center as a retrospective training cohort and 81 patients from the Wuhu Fourth People’s Hospital as an external validation cohort. We collected a multidimensional assessment of the characteristics of patients eligible for enrollment, including general demographic characteristics, medical history, treatment, and assessment of selected scales. At the same time, they were followed up for 1 year after reaching the recovery standard after treatment. Depending on whether their symptoms returned within a year, patients with BD were divided into recurrent and non-recurrent groups. Recurrence risk factors for BD were selected using univariate and binary logistic regression analyses based on the clinical data of the patients and other pertinent information. A nomogram model was developed to predict the incidence of BD relapse. To further assess the model fit and dependability, calibration curves, working curves of subject attributes, and decision curves were also employed.

Result:

A total of 384 patients with BD were enrolled in this study, of whom 250(65.1%) had non-recurrent episodes and 134(34.9%) had recurrent episodes. Of these, 96 (31.7%) had relapses at the Anhui Mental Health Centre and 38 (46.9%) at the Fourth People’s Hospital of Wuhu City. According to the results of univariate and multivariate logistic regression analyses, the number of prior episodes (odds ratio [OR]: 1.38, 95% confidence interval [CI]: 1.179–1.615), Social Disability Screening Schedule (SDSS) score (OR: 1.303, 95% CI: 1.027-1.652), Pittsburgh Sleep Quality Index (PSQI) (OR: 1.476, 95% CI: 1.29-1.689), Number of visits(OR: 0.768, 95% CI: 0.684-0.863), suicidal behaviors (OR: 5.54, 95% CI: 1.818-16.881) and the electroconvulsive therapy (ECT) (OR: 0.382, 95% CI: 0.156-0.94) were independent risk factors for relapse in patients with BD. An analysis of the receiver operating characteristic curve, calibration curve, and clinical decision curve further revealed that the predictive efficiency and degree of fit between the predicted value of the nomogram and the actual observed value were better.

Conclusion:

This study found that the number of previous episodes, SDSS score, PSQI score and suicidal behaviors were independent risk factors for relapse of BD, while the number of visits and ECT were protective factor. Based on these factors, we developed and validated a nomogram for predicting relapse in patients with BD; that has clinical reference values.

1 Introduction

Bipolar Disorder (BD) is a mental disorder characterized by recurrent manic and depressive symptoms (1). BD is often associated with drug abuse, anxiety, suicide, and other diseases. The proportion of comorbidity is high, seriously impacting patient health and social stability (2, 3). Even if patients with BD, especially hospitalized patients, receive standardized treatment, the disease is characterized by recurrent episodes of mania, depression, or mixed states (4),which places enormous pressure on families and healthcare providers. The treatment effect in patients with BD is often difficult to maintain; therefore, the main clinical feature is symptom recurrence (5). Understanding the long-term response of patients to treatment and the predictive factors for disease recurrence helps clinicians and patients monitor treatment effectiveness and reduces direct and indirect treatment costs. Studies have reported that the annual recurrence rate of patients with BD varies from 39% to 52% during different treatment and maintenance plans (6). Therefore, the prevention and treatment of symptom recurrence are important for disease treatment.

Although the relapse rate in patients with BD is high, there is still a lack of effective clinical prediction models to predict the risk of relapse. However, developing nomogram can effectively help solve this problem; as it integrates several important factors and predicts whether the condition in these patients will relapse after active standardized treatment (7). Therefore, we conducted a retrospective study on patients with BD who received standardized treatment, collected information such as demographic characteristics and general clinical characteristics of patients, constructed and verified a risk prediction model of relapse within 1 year in patients with BD, and predicted the relapse risk of individual patients, to provide a certain reference value for clinical treatment response and prognosis judgment.

2 Materials and methods

2.1 Patients

In this study, 303 patients with bipolar disorder were retrospectively collected from March 2021 to May 2022 at Anhui Mental Health Centre. At the same time, we also collected 81 patients who met the enrolment criteria at the Fourth People’s Hospital of Wuhu City. Patients were followed up for 1 year after reaching the recovery standard after treatment.

2.1.1 Inclusion criteria

(1) BD compliance with the International Classification of Diseases (ICD-10); (2) Unilateral seizures; (3) Age: 14-65 years; (4) A Hamilton Depression Scale (HAMD) (8)score or the Bech Rafaelsen Mania Rating Scale (BRMS) score on the Mania Rating Scale of >75% after treatment with administration of mood stabilizers and antipsychotic drugs; (5) A follow-up period of 1 year; (6) Presence of at least one informant who understands the patient’s situation;

2.1.2 Exclusion criteria

(1) Patients with organic brain lesions or a history of major brain trauma; (2) Neurological disorders; (3) Concomitant conditions, such as schizophrenia and vascular dementia; (4) Illiteracy (inability to complete the questionnaire); (5) Visual, auditory, reading, and writing impairments; (6) Missing visit.

BD relapse was defined as: (1) Patient visit or hospitalization because of mood disorders; (2) Patients met the ICD-10 criteria for manic, hypomanic, or depressive episodes. Patients were divided into relapse and non-relapse groups based on whether they relapsed within 1 year of effective treatment.

The study was approved by the ethical committees of the Anhui Mental Health Centre (approval number: HFSY-IRB-YJ-LWTG-ZXQ( (2023-061-01)) and the Wuhu Fourth People’s Hospital (approval number:[2021]-KY-17).

2.2 Predictive variables and follow-up outcomes

This study collected patient-related information from patient’s medical records and questionnaires. Demographic characteristics and clinical variables including age, sex, duration of illness, and medications taken by the patients were assessed. The Pittsburgh Sleep Quality Index (PSQI) (9) and Social Disability Screening Schedule (SDSS) (10) scales were used to evaluate patients’ social function and sleep at discharge. In addition, we also collected multidimensional characteristics related to patients’ general demographic characteristics, past medical history, and treatment.

Meanwhile, we checked patients’ medical records or telephone follow-ups to investigate the recurrence of BD. We defined a positive outcome based on the recurrence of symptoms within 1 year of discharge for inpatients with BD and a negative outcome based on the lack of symptom recurrence.

2.3 Statistical method

In this study, we analyzed the data using SPSS 26.0 and R 4.3.1. The Kolmogorov-Smirnov test was used to determine the normal distribution of continuous variables, and the double sample t-test was used for normal distribution data, shown in the form of mean ± standard deviation. Non-parametric data were compared between groups using the Mann-Whitney U test and are presented as quartiles (P₂₅–P₇₅). For categorical variables, the chi-square test or Fisher’s exact test was employed to compare groups, and the results were expressed as frequency (%). This study used univariate regression analysis to analyze the predictive variables. To eliminate the impact of multicollinearity on the model, we perform covariance screening when including variables in the model. Variables with a Variance Inflation Factor (VIF) greater than 5 are addressed accordingly. The binary logistic regression included variables with p <0.05, and the regression prediction model was established and transformed into a nomogram model. A calibration curve, receiver operating characteristic (ROC) curve, decision curve analysis (DCA) curve and C-index were used to evaluate and verify the nomogram model. This study defined a significant difference as a two-tailed p < 0.05.

3 Results

3.1 Clinical characteristics

Table 1 demonstrates the comparison of the relevant feature information between the training set and the external validation set. The training set included a total of 303 patients, of which 139 were female (45.9%) and 164 were male (54.1%). A total of 96 (31.7%) patients experienced recurrence and 207 (68.3%) did not. A total of 44 (54.3%) females and 37 (45.7%) males were included in the external validation center. A total of 38 (46.9%) patients in the external validation set experienced recurrence and 43 (53.1%) did not.

Table 2 demonstrates the comparison between the baseline characteristics of the relapse and non-relapse groups of patients with BD in the training set. In the relapse group, the duration of disease was significantly higher than in the non-relapse group (10.313 ± 8.68 vs. 7.784 ± 6.994, p=0.007). Age at onset was significantly lower in the relapse group than in the non-relapse group (22.19 ± 7.32 vs 25.55 ± 9.556, p=0.002). The number of previous episodes was also significantly higher in the relapse group than in the non-relapse group (4.77 ± 3.922 vs. 2.92 ± 2.209, p=0.000). The number of years of education was also significantly higher in the non-relapse group than in the relapse group (11.81 ± 3.319 vs 10.89 ± 3.115, p=0.022). In the comparison of SDSS scores, it was significantly higher in the relapse group than in the non-relapse group (4.33 ± 2.405 VS 2.24 ± 1.582, p=0.000). In the comparison of PSQI scores, the relapse group was significantly higher than the non-relapse group (9.45 ± 3.808 VS 5.24 ± 2.543, p=0.000). In the comparison of the number of recurrence visits, the non-recurrence group was significantly higher than the recurrence group (7.03 ± 4.463 VS 3.81 ± 3.203, p=0.000). In the relapse group, the percentage of patients with suicidal behavior was significantly higher than in the non-relapse group (24.0% vs 10.1%, p=0.001). In addition, the proportion of ECT was significantly higher in the non-relapse group than in the relapse group (35.7% vs 24.0%, p=0.041).

3.2 Results of univariate regression analysis

The results of univariate regression analysis showed that disease duration (odds ratio [OR]: 1.042, p = 0.009), age at onset (OR: 0.953, p = 0.003), number of previous episodes (OR: 1.233, p = 0.03), years of education(OR: 0.915, p = 0.023), SDSS score (OR: 1.75, p = 0.000), PSQI score(OR: 1.499, p = 0.000), Number of visits(OR: 0.768, p = 0.000), working status(OR: 0.367, p = 0.000), housing conditions(OR: 2.321, p = 0.049), mood stabilizer(p = 0.000), suicidal behaviors(OR: 2.791, p = 0.002) and ECT(OR: 0.566, p = 0.042) were the risk factors for relapse in patients with BD, with statistically significant differences (all p < 0.05) (Table 3).

3.3 Multivariate regression analysis results

Multivariate regression analyses showed that the number of previous episodes (OR: 1.38, p = 0.000), Number of visits (OR: 0.806, p = 0.000), SDSS score (OR: 1.303, p = 0.029), PSQI scores (OR: 1.476, p = 0.000), suicidal behaviors (OR: 5.54, p = 0.003) and ECT (OR: 0.382, p =0.036) were statistically significant independent risk factors for relapse within 1 year in patients with BD (all p < 0.05) (Table 4).

3.4 Development of the nomogram model

Based on the logistic regression analyses described above, we developed a nomogram model to predict the risk of recurrence within 1 year in patients with BD (Figure 1). Each clinical factor corresponds to a specific score, and a straight- line point axis was drawn to calculate the total score, which corresponds to a high risk of BD relapse and vice versa.

Figure 1

3.5 Validation of the nomogram model

The nomogram developed in this study have high predictive performance. Calibration curves were plotted from the predicted values of the nomogram model to the actual values. The results in Figures 2, 3 show that the agreement between the predicted and actual values of the nomogram model is high for both the training set and the validation set. In addition, Figure 4 shows that the clinical prediction curves (DCA) have high clinical applicability on both the training set and the validation set. As shown in Figure 5, the area under the ROC curve suggests that the nomogram has high classification and prediction ability in the training set (AUC: 0.924, 95% CI: 0.888 - 0.951). Figure 6 suggests that this nomogram model also has good generalization ability on the external test set (AUC:0.741, 95% CI:0.632 - 0.832).

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

4 Discussion

The high prevalence and relapse rate of BD place great pressure on the rehabilitation and treatment of patients (11). Therefore, establishing a risk prediction model for short-term recurrence is significance for early intervention and treatment guidance in patients with BD. In this study, 384 patients with BD were retrospectively included to analyze the risk factors that may affect short-term recurrence. Based on binary logistic regression analysis, this study found that the number of previous episodes, SDSS score, PSQI score, number of visits, suicidal behaviors and ECT were independent risk factors for short-term relapse in patients with this disease. Nomograms created by combining the above risk factors have a high predictive efficacy. Meanwhile, the performance of the external validation set demonstrated that the nomogram has a good panchromatic ability. The results showed that the model had a high predictive accuracy and clinical application value.

Recurrent relapses of symptoms in patients with BD can further deteriorate the condition and lead to poor prognosis (12), highlighting the importance of identifying factors associated with an increased risk of relapse in patients. Patients with BD have a large heterogeneity in the disease course and symptoms, and it is crucial to develop and validate relapse prediction models and guide individualized treatment. The results of multivariate logistic regression analysis showed that t the number of previous episodes, SDSS score, PSQI score, number of visits, suicidal behaviors and ECT were independent risk factors for relapse within 1 year.

Previous studies have shown that the number of previous episodes of a patient’s symptoms is an important predictor of relapse, which is consistent with our findings (13). On the one hand, prolonged depressive or manic episodes may disrupt the balance of neurotransmitters in the brain, such as dopamine, norepinephrine, and serotonin, making patients more susceptible to recurrent mood swings (14, 15). On the other hand, frequent symptomatic episodes can lead to cognitive decline, particularly impairments in memory, attention, and executive functioning, and may also make it more difficult for patients to effectively recognize and manage early signs of symptoms, thereby increasing the risk of relapse (16). Moreover, the results of this study indicate that the higher the number of previous episodes, the higher the risk of recurrence, and the higher the number of episodes, the more severe and complex the patient’s condition is, which in turn causes the patient’s disease to prolong, thus increasing the patient’s risk of recurrence (17). Our study also found that the number of outpatient visits was a protective factor against relapse in patients with BD. We found that the higher the number of outpatient follow-up visits, the lower the risk of relapse. Patients seek timely medical help based on dynamic changes in clinical symptoms; in contrast, doctors provide standardized medical management, suggesting the importance of patient compliance and standardized outpatient management. In addition, this study found that the higher the PSQI score, the higher the risk of recurrence; the worse the sleep quality, the higher the risk of BD recurrence. Some studies have found that sleep disorders may be precursors of BD (18). Some studies have confirmed that sleep disorders can affect circadian rhythms, change brain electrical activity during sleep, and impair patients’ emotional processing ability (19). Therefore, early maintenance of a good sleep state is of great value for the prognosis of patients with BD. Similarly, as SDSS scores increase, the risk of relapse increases in patients with BD. The SDSS scale is an instrument that responds to social and occupational functioning, and as the SDSS score increases, the poorer the social functioning of the respondent. Previous studies have shown that social dysfunction significantly increases the risk of relapse in patients with bipolar disorder (20).

The present study also found that the presence of suicidal behavior in patients significantly increased the risk of recurrence. It is well known that suicidal behavior usually responds to a more severe stage of illness in people with bipolar disorder, which significantly increases the risk of fluctuations and relapses (21). Previous studies have shown that suicidal behavior is closely associated with damage to the frontal lobe, which plays a critical role in emotion regulation, decision-making, and impulse inhibition (22). A reduction in frontal lobe function in patients exhibiting suicidal behavior may impair their ability to regulate negative emotions and control impulsive actions, thereby heightening the risk of relapse (23). This study also found that patients treated with ECT significantly reduced the risk of relapse in patients with bipolar disorder. ECT can rapidly relieve severe depressive and manic symptoms in patients with bipolar disorder, thereby stabilizing mood and reducing the likelihood of relapse (24). ECT is equally effective in patients with refractory BD, further stabilizing symptoms and thereby significantly reducing the risk of relapse (25).

However, even if patients with BD have the same or similar risk factors, the predicted risk of symptom recurrence is different. The combined prediction of multiple risk factors can improve predictive performance. Nomograms can combine multiple risk factors for joint prediction and have been widely used in statistical analyses of various neurological disorders (26). However, nomograms are rarely used to assess the risk of BD recurrence. In this study, we recruited 384 patients with BD and constructed a prediction model for symptom recurrence within 1 year. According to the multivariate logistic regression analysis results, a line chart scoring system was constructed based on the previous episodes, SDSS score, PSQI score and suicidal behaviors, the number of visits and ECT. The score system has a high predictive efficiency, with a AUC of 0.924 (95% CI: 0.888 - 0.951). Moreover, in order to test the generalisation ability of this nomogram model in this study, the prediction performance of the external test set was kept high (AUC:0.741, 95% CI:0.632 - 0.832). In addition, the calibration curve suggested a high degree of consistency between the predicted and actual values. The model aligns with actual clinical scenarios; in clinical settings, doctors can use column-line graphs to visualize the risk of relapse in patients with BD. In addition, combining the individual specificity of patients for early and comprehensive interventions has greater clinical value for the prognosis of patients with BD.

This study had some limitations. Firstly, this was a retrospective study. In the future, we will conduct a prospective cohort study to improve the reliability and persuasiveness of the study. Secondly, we established a prediction model based on demographic and clinical characteristics, and the input characteristics were relatively simple. In the future, we will further develop multigroup, multi-model, and multidimensional models to improve the model’s accuracy and clinical practice value. Finally, we did not use more cutting-edge AI techniques applied in this study, such as machine learning, deep learning, etc. In the future, we will actively explore the performance of high-performance AI techniques applied in the occurrence and development of BD. Finally, the relatively small sample size may limit the performance and generalizability of the model. In the future, we plan to conduct a multicenter study to expand the sample size and further validate our findings.

5 Conclusion

This study found that the number of previous episodes, SDSS score, PSQI score and suicidal behaviors were independent risk factors for relapse of BD, while the number of visits and ECT were protective factor. Based on these factors, we developed and validated a nomogram for predicting relapse in patients with BD; that has clinical reference values.

References

• 1

  FindlingRLStepanovaEYoungstromEAYoungAS. Progress in diagnosis and treatment of bipolar disorder among children and adolescents: An international perspective. Evidence-Based Ment Health. (2018) 21:177–81. doi: 10.1136/eb-2018-102912

  • CrossRef
  • Google Scholar

• 2

  FennHHBauerMSAltshulerLEvansDRWillifordWOKilbourneAMet al. Medical comorbidity and health-related quality of life in bipolar disorder across the adult age span. J Affect Disord. (2005) 86:47–60. doi: 10.1016/j.jad.2004.12.006

  • CrossRef
  • Google Scholar

• 3

  GamageNSenanayakeSKumbukageMMendisJJayasekaraA. The prevalence of anxiety and its association with the quality of life and illness severity among bipolar affective disorder patients in a developing country. Asian J Psychiatry. (2020) 52:102044. doi: 10.1016/j.ajp.2020.102044

  • CrossRef
  • Google Scholar

• 4

  OrtizAHintzeABurnettRGonzalez-TorresCUngerSYangDet al. Identifying patient-specific behaviors to understand illness trajectories and predict relapses in bipolar disorder using passive sensing and deep anomaly detection: protocol for a contactless cohort study. BMC Psychiatry. (2022) 22:1–8. doi: 10.1186/s12888-022-03923-1

  • CrossRef
  • Google Scholar

• 5

  BirmaherBMerrankoJAGillMKHafemanDKellerM. Predicting personalized risk of mood recurrences inYouths and young adults with bipolar spectrumDisorder. J Am Acad Child Adolesc Psychiatry. (2020) 59:1156–64. doi: 10.1016/j.jaac.2019.12.005

  • CrossRef
  • Google Scholar

• 6

  TohenMZarateCAJr.HennenJKhalsaH-MKStrakowskiSMGebre-MedhinPet al. The McLean-Harvard first-episode mania study: prediction of recovery and first recurrence. Am J Psychiatry. (2003) 160:2099–107. doi: 10.1176/appi.ajp.160.12.2099

  • CrossRef
  • Google Scholar

• 7

  ChangBNiCMeiJXiongCChenPJiangMet al. Nomogram for predicting depression improvement after deep brain stimulation for parkinson's disease. Brain Sci. (2022) 12:841. doi: 10.3390/brainsci12070841

  • CrossRef
  • Google Scholar

• 8

  HamiltonM. A rating scale for depression. J Neurol Neurosurg Psychiatry. (1960) 23:56–62. doi: 10.1136/jnnp.23.1.56

  • CrossRef
  • Google Scholar

• 9

  BuysseDJIiiCFRMonkTHBermanSRKupferDJ. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res. (1989) 28:193–213. doi: 10.1016/0165-1781(89)90047-4

  • CrossRef
  • Google Scholar

• 10

  SchubartCKrummBSchwarzHB. Measurement of social disability in a schizophrenic patient group. Soc Psychiatry&Psychiatric Epidemiol. (1986) 21:1–9. doi: 10.1007/BF00585315

  • CrossRef
  • Google Scholar

• 11

  MillerSDell'OssoBKetterTA. The prevalence and burden of bipolar depression. J Affect Disord. (2014) 169 Suppl 1:S3–11. doi: 10.1016/S0165-0327(14)70003-5

  • CrossRef
  • Google Scholar

• 12

  OssolaPGarrettNSharotTMarchesiC. Belief updating in bipolar disorder predicts time of recurrence. Elife. (2020) 9. doi: 10.7554/eLife.58891

  • CrossRef
  • Google Scholar

• 13

  TundoAMusettiLBenedettiAMassimettiEPergentiniICambialiEet al. Predictors of recurrence during long-term treatment of bipolar I and II disorders. A 4 year prospective naturalistic study. J Affect Disord. (2018) 225:123–8. doi: 10.1016/j.jad.2017.08.013

  • CrossRef
  • Google Scholar

• 14

  MartinoMMagioncaldaP. Tracing the psychopathology of bipolar disorder to the functional architecture of intrinsic brain activity and its neurotransmitter modulation: a three-dimensional model. Mol Psychiatry. (2022) 27:793–802. doi: 10.1038/s41380-020-00982-2

  • CrossRef
  • Google Scholar

• 15

  MandalPKGaurSRoyRGSamkariaAIngoleRGoelA. Schizophrenia, bipolar and major depressive disorders: overview of clinical features, neurotransmitter alterations, pharmacological interventions, and impact of oxidative stress in the disease process. ACS Chem Neurosci. (2022) 13:2784–802. doi: 10.1021/acschemneuro.2c00420

  • CrossRef
  • Google Scholar

• 16

  Sánchez-TorresAMAmorettiSEnguita-GermánMMezquidaGMoreno-IzcoLPanadero-GómezRet al. Relapse, cognitive reserve, and their relationship with cognition in first episode schizophrenia: a 3-year follow-up study. Eur Neuropsychopharmacol. (2023) 67:53–65. doi: 10.1016/j.euroneuro.2022.11.011

  • CrossRef
  • Google Scholar

• 17

  MagalhaesPVDoddSNierenbergAABerkM. Cumulative morbidity and prognostic staging of illness in the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD). Aust N Z J Psychiatry. (2012) 46:1058–67. doi: 10.1177/0004867412460593

  • CrossRef
  • Google Scholar

• 18

  JacksonACavanaghJScottJ. A systematic review of manic and depressive prodromes. J Affect Disord. (2003) 74:209–17. doi: 10.1016/S0165-0327(02)00266-5

  • CrossRef
  • Google Scholar

• 19

  GershonADoDSatyanarayanaSShahSYuenLD. Abnormal sleep duration associated with hastened depressive recurrence in bipolar disorder. J Affect Disord. (2017) 218:374–9. doi: 10.1016/j.jad.2017.05.015

  • CrossRef
  • Google Scholar

• 20

  Gimenez-PalomoAAndreuHOlivierLOchandianoIde JuanOFernandez-PlazaTet al. Clinical, sociodemographic and environmental predicting factors for relapse in bipolar disorder: A systematic review. J Affect Disord. (2024) 360:276–96. doi: 10.1016/j.jad.2024.05.064

  • CrossRef
  • Google Scholar

• 21

  VietaEBerkMSchulzeTGCarvalhoAFSuppesTCalabreseJRet al. Bipolar disorders. Nat Rev Dis Primers. (2018) 4:18008. doi: 10.1038/nrdp.2018.8

  • CrossRef
  • Google Scholar

• 22

  WuYZhangHShenQJiangXYuanXLiMet al. Exploring the neurocognitive correlates of suicidal ideation in major depressive disorder: The role of frontoparietal and default mode networks. J Psychiatr Res. (2024) 177:211–8. doi: 10.1016/j.jpsychires.2024.07.009

  • CrossRef
  • Google Scholar

• 23

  ChengXChenJZhangXWangTSunJZhouYet al. Characterizing the temporal dynamics of intrinsic brain activities in depressed adolescents with prior suicide attempts. Eur Child Adolesc Psychiatry. (2024) 33:1179–91. doi: 10.1007/s00787-023-02242-4

  • CrossRef
  • Google Scholar

• 24

  PerugiGMeddaPToniCMarianiMGSocciCMauriM. The role of electroconvulsive therapy (ECT) in bipolar disorder: effectiveness in 522 patients with bipolar depression, mixed-state, mania and catatonic features. Curr neuropharmacology. (2017) 15:359–71. doi: 10.2174/1570159X14666161017233642

  • CrossRef
  • Google Scholar

• 25

  MeddaPPerugiGZanelloSCiuffaMCassanoGB. Response to ECT in bipolar I, bipolar II and unipolar depression. J Affect Disord. (2009) 118:55–9. doi: 10.1016/j.jad.2009.01.014

  • CrossRef
  • Google Scholar

• 26

  WeiLChangBGengZChenMCaoYYaoLet al. Nomogram for predicting traumatic subdural effusion after mild traumatic brain injury. Front Neurol. (2022) 13:947976. doi: 10.3389/fneur.2022.947976

  • CrossRef
  • Google Scholar