Development and validation of nomograms to predict clinical outcomes of preeclampsia

Background Preeclampsia (PE) is one of the most severe pregnancy-related diseases; however, there is still a lack of reliable biomarkers. In this study, we aimed to develop models for predicting early-onset PE, severe PE, and the gestation duration of patients with PE. Methods Eligible patients with PE were enrolled and divided into a training (n = 253) and a validation (n = 108) cohort. Multivariate logistic and Cox models were used to identify factors associated with early-onset PE, severe PE, and the gestation duration of patients with PE. Based on significant factors, nomograms were developed and evaluated using the area under the curve (AUC) and a calibration curve. Results In the training cohort, multiple gravidity experience (p = 0.005), lower albumin (ALB; p < 0.001), and higher lactate dehydrogenase (LDH; p < 0.001) were significantly associated with early-onset PE. Abortion history (p = 0.017), prolonged thrombin time (TT; p < 0.001), and higher aspartate aminotransferase (p = 0.002) and LDH (p = 0.003) were significantly associated with severe PE. Abortion history (p < 0.001), gemellary pregnancy (p < 0.001), prolonged TT (p < 0.001), higher mean platelet volume (p = 0.014) and LDH (p < 0.001), and lower ALB (p < 0.001) were significantly associated with shorter gestation duration. Three nomograms were developed and validated to predict the probability of early-onset PE, severe PE, and delivery time for each patient with PE. The AUC showed good predictive performance, and the calibration curve and decision curve analysis demonstrated clinical practicability. Conclusion Based on the clinical features and peripheral blood laboratory indicators, we identified significant factors and developed models to predict early-onset PE, severe PE, and the gestation duration of pregnant women with PE, which could help clinicians assess the clinical outcomes early and design appropriate strategies for patients.


Introduction
Preeclampsia (PE), which typically occurs after 20 weeks of gestation, is one of the most severe pregnancy-related diseases.It is characterized by sudden-onset hypertension and is accompanied by at least one of the following complications: proteinuria and maternal organ dysfunction (1).Globally, there are an estimated 4 million women newly diagnosed with PE each year, resulting in the death of more than 70,000 women and 500,000 newborns, making it the leading cause of maternal and perinatal morbidity and mortality (2,3).
The heterogeneity of PE as clinical presentation and outcome varies between different subtypes.Patients with early-onset PE (<34 weeks of gestation) always present more severe clinical complications and an enrichment of metabolism-related pathways in the transcriptional profile compared to those with late-onset PE (≥34 weeks of gestation) (4,5).Patients with PE are also at risk of rapid deterioration and severe disease, including eclampsia, stroke, HELLP (hemolysis, elevated liver enzymes, and low platelets) syndrome, placental abruption, renal function failure, and pulmonary edema, without receiving timely treatment (6).The management of PE consists of monitoring perinatal blood pressure and controlling complications through pharmacological intervention.Currently, timely delivery of the fetus is the only definitive treatment for PE; however, it may cause the babies of women with early-onset PE or severe symptoms to have increased risks of preterm birth, perinatal death, neurodevelopmental delay, and later cardiovascular and metabolic diseases (2).Therefore, early identification of the occurrence of PE, especially early-onset PE and severe PE, and prediction of gestation duration are of utmost importance to minimize adverse perinatal events both in pregnant women and in fetuses.
Three checklists from the International Society for the Study of Hypertension in Pregnancy (ISSHP) (3), the American College of Obstetricians and Gynecologists (ACOG) (7), and the National Institute for Health and Care Excellence (NICE) (8) are broadly used in clinical practice to assess the risk of PE occurrence; however, all risk factors derived from clinical features and their predictive power for PE are weak (9).Recently, increased numbers of biomarkers from peripheral blood have been identified to predict pregnant women with a high risk of PE at an early stage.Soluble fms-like tyrosine kinase 1 (sFlt-1) and placental growth factor (PlGF) are a pair of anti-and pro-angiogenic factors (respectively) found significantly unbalanced in PE (10).The PROGNOSIS trial demonstrated that, in women with a sFlt-1/ PlGF ratio lower than 38, the likelihood of developing PE over the next week could accurately be ruled out, with a 99.3% negative predictive value (11).In addition, a series of novel placental-and endothelial-derived nucleic acid (mainly RNA) and proteins were also discovered for PE, including extravillous trophoblast signature (MMP11, SLC6A2, and IL18BP) (12), the chromosome 19 miRNA cluster (combination of miR-517-5p, miR520a-5p, and miR-525-5p) (13, 14), placental protein 13 (PP13) (15), pregnancy-associated plasma protein A (PAPP-A) (16), and vascular cell adhesion molecule-1 (VCAM-1) (17).However, the efficacy of a single biomarker from peripheral blood in the accurate diagnosis of PE is inadequate, and the majority of studies lacked validation.Hence, the development and validation of a predictive model based on multiple indicators consisting of clinical characteristics and laboratory parameters could be helpful in clinical practice.In the recent decade, several predictive models have been developed based on a series of risk factors.For example, by combining gestational age, chest pain or dyspnea, oxygen saturation, platelet count, and the creatinine and aspartate transaminase concentrations, the fullPIERS model could identify the risk of fatal or life-threatening complications in women with PE within 48 h of hospital admission (18).In addition, another study constructed a machine learning model for the prediction of PE in the first trimester based on the mean arterial blood pressure, uterine artery pulsatility index, PlGF, and PAPP-A (19).However, these models could not provide an exact probability for PE occurrence and the delivery of pregnant women at a certain time.A nomogram is a predictive tool to evaluate the clinical outcomes of patients by quantifying the probability based on easily accessed variables, which is widely used in patients with cancer and other chronic diseases, even in patients with coronavirus disease 2019 (20)(21)(22).
In this study, based on the clinical characteristics and peripheral blood laboratory indicators of patients with PE, we aimed to identify biomarkers for the early diagnosis of PE with early-onset and severe symptoms, predict the gestation duration, and construct a model for each clinical outcome, which could help clinicians recognize and manage patients with PE in the early stage of the disease and improve the clinical prognosis for pregnant women and infants.

Study population
This study retrospectively enrolled patients from Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, China, in January 2017 and December 2022.Eligible populations were diagnosed with PE according to the "Diagnosis and treatment of hypertension and preeclampsia in pregnancy: a clinical practice guideline in China (2020)" (23).The detailed diagnostic criteria for PE were as follows: pregnant women with a systolic blood pressure higher than 140 mmHg and/or a diastolic blood pressure higher than 90 mmHg after 20 weeks of gestation, accompanied by any of the following symptoms: 1) urine protein ≥0.3 g/24 h or a urine protein/creatinine ratio ≥0.3 and 2) any dysfunction of important organs such as the heart, lung, liver, kidney, blood system, digestive system, and nervous system or involvement of placenta-fetus.The exclusion criteria were as follows: 1) patients with preexisting hypertension, immune disorders, and maternal organ dysfunction (such as hematopoietic, hepatic, and renal dysfunction) and 2) patients without complete maternal or infant records.
A total of 361 eligible patients were enrolled.This study was approved by the Ethics Committee of Sir Run Run Shaw Hospital (approval no.2023-0248).

Clinical outcomes
According to the gestational age at PE diagnosis, the patients were classified into an early-onset PE (<34 weeks) and a late-onset PE (≥34 weeks) group (24).The diagnostic criteria for severe PE were as follows: 1) continuously increasing blood pressure (systolic pressure ≥160 mmHg and/or diastolic pressure ≥110 mmHg); 2) persistent headache, visual disturbance, or other central nervous system abnormalities; 3) persistent upper abdominal pain, subcapsular hematoma, or liver rupture; 4) abnormal elevation of the AST and ALT levels; 5) impaired renal function-urinary protein quantification ≥2 g/24 h, oliguria, or serum creatinine level >106 mmol/L; 6) hypoproteinemia with ascites, pleural effusion, or pericardial effusion; 7) PC decreasing continuously and lower than 100 × 10 9 /L, microvascular hemolysis, anemia, elevated LDH level, or jaundice; 8) heart failure; 9) pulmonary edema; and 10) fetal growth restriction, oligohydramnios, intrauterine fetal death, and placental abruption.A PE patient with one of the above symptoms was diagnosed as severe PE (23).The gestation duration was defined as the period between the last menstrual period and delivery.

Statistical analysis
Normality of the continuous variables was assessed using the Shapiro-Wilk test.Normally distributed variables were expressed as the mean ± standard deviation (SD), with significance analyzed using Student's t-test.Non-normally distributed variables were expressed as the median and interquartile range (IQR), with significance analyzed using the Mann-Whitney U test.Categorical variables were expressed as frequency and percentage, with significance analyzed using the chisquare test.
The whole study population was randomly divided into a training cohort and a validation cohort at a 7:3 ratio using a random sampling method ("Caret" R package).In the training cohort, univariate and multivariate logistic regression models (forward) were used to identify significant variables related to early-onset PE and severe PE (p < 0.05), and odds ratios (ORs) and 95% confidence intervals (CIs) were calculated ("glmnet" R package).Univariate and multivariate Cox proportional hazard regression models (forward) were used to identify significant variables related to gestation duration, and hazard ratios (HRs) and 95% CIs were calculated ("survival" R package).The results of the multivariate model were visualized using the "forestplot" R package.The curve of gestation duration was constructed using the Kaplan-Meier method and the log-rank test.
Based on the significant variables in the multivariate model, three nomograms were developed to predict the probability of early-onset PE, severe PE, and delivery at 26, 28, 30, 32, 34, 36, and 38 weeks for each patient with PE ("rms" R package).The predictive performance and the discriminative ability of each nomogram were assessed using 1,000 bootstrap resamples to obtain the concordance index (C-index) and the area under the receiver operating characteristic (ROC) curve (AUC).Calibration curves were used to evaluate the consistency between the predicted probabilities of the nomograms and the actual clinical outcomes.Decision curve analysis (DCA) was performed to evaluate the clinical utility of the nomograms.The above methods were then applied to validate the performance of the nomograms in the validation cohort.
All statistical analyses were performed using R software version 3.6.0.All tests were two-sided, and a p < 0.05 was considered statistically significant.

Patient characteristics
The workflow is shown in Figure 1.After random sampling, a total of 361 patients with PE were allocated to the training cohort (n = 253) and the validation cohort (n = 108).The clinical characteristics and laboratory indicators between the two cohorts were basically balanced (Table 1).In the training cohort, the median age was 32.0 years (range, 29.0-35.0years).A total of 135 patients (53.4%) had multiple gravidity experience, while 188 patients (74.3%) were primiparas.There were 198 patients (78.3%) who had a single pregnancy, 116 patients (45.8%) had a history of abortion, and 221 patients (87.4%) had irregular menstruation.
Based on these three variables, a nomogram was constructed to predict the probability of early-onset PE for each individual patient (Figure 2D).The C-index and AUC of the nomogram were both 0.843 (95%CI = 0.776-0.910),indicating good predictive performance.In addition, the calibration curve demonstrated good consistency between the probabilities predicted by the nomogram and the actual results (Figures 2E, F), and DCA showed that the nomogram offered a net benefit over the "treatall" or "treat-none" strategy (Supplementary Figure S1A).

Discussion
In this study, based on the clinical characteristics and peripheral blood laboratory indicators, we identified a series of risk factors associated with early-onset PE, severe PE, and shorter gestation duration of patients with PE.In addition, three nomograms were developed to predict the probability of early-onset PE, severe PE, and delivery at different gestational weeks for each individual patient with PE, which could help clinicians manage patients with PE in the early stage of the disease and improve the clinical outcomes for pregnant women and infants.
The etiologies and pathogenesis of PE are complex and multisystemic, which involve placental dysfunction, immune system dysfunction, maternal metabolic disorder, and dysregulated endothelial function due to the release of a series of circulating factors including angiogenic proteins, pro-inflammatory cytokines, and small extracellular vesicles (25)(26)(27)(28)(29)(30).Several risk factors of obstetric history have been identified as associated with PE from clinical guidelines, including previous PE, history of parity, gravidity, abortion, and multiple pregnancies, which were considered to lead to a weakened maternal immune tolerance to the placenta, thus increasing the risk of PE (3,7,8).A previous study found that multiple fetal pregnancies were associated with a significantly higher rate of PE than singleton pregnancies, with the rate increasing with the number of fetuses present (31).In this study, multiple gravidities, previous abortion history, and multiple pregnancies were also found to be independent risk factors for inferior clinical outcomes of patients with PE, which is consistent with previous results.Accumulating evidence suggested that impaired maternal metabolic function is associated with PE, which leads to inadequate adaptation to the demands of pregnancy.An altered metabolic function has been proposed to contribute to PE by causing reduced spiral artery remodeling and altered placental metabolic function (28).A previous study evaluated the role of LDH isozymes in the placenta between patients with PE and those with normal pregnancy and found that, compared to placentas from normal pregnancy, the mRNA and activity of LDH-A were increased in placentas from patients with PE, probably as a result of hypoxia (32).In this study, we found that a higher serum LDH level was related to early-onset PE, severe PE, and shorter gestation duration of patients with PE, indicating that LDH could serve as a marker for PE.
In addition, there was an increase of transaminases and hypoalbuminemia in PE patients with poor clinical outcomes, suggesting that an impaired liver function was associated with severe PE and shorter gestation duration of patients with PE.Similarly, several researchers also observed impaired liver function in patients with PE, including elevated AST and ALT and reduced ALB (33,34).This elevation may be due to the systemic inflammatory response caused by placental ischemia, which then resulted in vasoconstriction and endothelial dysfunction and eventual liver dysfunctions.
Platelet activation occurred in the early stage of PE, which may be associated with platelet aggregation and depletion due to injury to the vascular endothelium in patients with PE (35).In the present study, we found that a higher MPV was related to the shorter gestation duration of patients with PE.High levels of MPV represented a high platelet consumption status, and this aggregation and depletion would result in increased blood viscosity and the potential for microthrombosis, which could also lead to placental ischemia and hypoxia and further affect both maternal organ function and fetus growth.Moreover, we found that a prolonged TT was associated with the shorter gestation duration of patients PE, suggesting that the coagulation function disorder could affect the severity of PE.Thrombin time refers to the time it takes for thrombin to convert fibrinogen into fibrin; the prolonged TT reflected the insufficiency of plasma fibrinogen or abnormal structure, or excessive anticoagulant substances in the body.A previous study identified that the levels of fibrinogen were significantly lower in placentas from women with early-onset PE compared with control placentas, indicating that a low fibrinogen level might be involved in the coagulation disorder in PE (36).Although numerous studies have explored a series of risk factors related to PE, none have constructed nomograms to accurately predict the probabilities of early-onset PE, severe PE, and the gestation duration of patients with PE.Here, based on significant clinical features and peripheral blood laboratory indicators, we constructed three nomograms to predict the probability of early-onset PE, severe PE, and delivery at 26, 28, 30, 32, 34, 36, and 38 weeks for pregnant women with PE.The AUC of each nomogram presented good discriminative ability, and each nomogram was validated in the validation cohort.Although there were several flaws in the calibration curves at 26-30 weeks, we believe that the main reason was the improvements in medical condition and early intervention, and thus the delivery events were relatively low.However, the results of the AUC and C-index, as well as the DCA, demonstrated that our model still had reliability for prediction.Compared with other reported biomarkers or risk scores, the model developed in this study could quantify each predictive variable and provide a specific probability of the occurrence of severe and early-onset PE and the delivery time for each individual pregnant woman.Furthermore, this model could also help clinicians assess the clinical outcomes early and design an appropriate strategy for each patient.To our knowledge, this is the first study to construct three different nomograms to predict earlyonset PE, severe PE, and the gestation duration of patients with PE based on clinical characteristics and laboratory parameters.This study has some limitations.Firstly, it is a retrospective study with a relatively small sample size.Therefore, expanding the sample size and assessing the predictive performance of the nomograms in a larger prospective study are needed.Secondly, this study did not examine the serum sFlt-1, PlGF, PP13, PAPP-A, and VCAM-1, which are considered as important markers for PE.

Conclusion
Based on the clinical features and peripheral blood laboratory indicators, we identified significant factors and developed models to predict early-onset PE, severe PE, and the gestation duration of pregnant women with PE, which could help clinicians assess the clinical outcomes early and design an appropriate strategy for each patient.

2
FIGURE 2 Development of a nomogram for predicting early-onset pre-eclampsia (PE).(A) Forest plot showing multiple gravidity experience, lower albumin (ALB), and higher lactate dehydrogenase (LDH) as significantly associated with early-onset PE in multivariable Cox regression analysis.(B, C) Box plot showing the distribution of the levels of ALB (B) and LDH (C) between the early-onset and late-onset PE groups (Wilcoxon test).(D) Nomogram for predicting early-onset PE probability in the training cohort.(E) Receiver operating characteristic (ROC) curve of the nomogram predicting earlyonset PE probability in the training cohort.(F) Calibration curve of the nomogram predicting early-onset PE probability in the training cohort.

3
FIGURE 3 Development of a nomogram for predicting severe pre-eclampsia (PE).(A) Forest plot showing abortion history, prolonged thrombin time (TT), and higher aspartate aminotransferase (AST) and lactate dehydrogenase (LDH) as significantly associated with severe PE in multivariable Cox regression analysis.(B-D) Violin box plots showing the distribution of the levels of TT (B), AST (C), and LDH (D) between the severe and non-severe PE groups (Wilcoxon test).(E) Nomogram for predicting severe PE probability in the training cohort.(F) Receiver operating characteristic (ROC) curve of the nomogram predicting severe PE probability in the training cohort.(G) Calibration curve of the nomogram predicting severe PE probability in the training cohort.

4
FIGURE 4 Development of a nomogram for predicting the gestation duration of patients with pre-eclampsia (PE).(A) Forest plot showing abortion history, gemellary pregnancy, prolonged thrombin time (TT), higher mean platelet volume (MPV) and lactate dehydrogenase (LDH), and lower albumin (ALB) as significantly associated with severe PE in multivariable Cox regression analysis.(B-G) Kaplan-Meier curves of gestation duration according to abortion history (B), gemellary pregnancy (C), MPV (D), TT (E), ALB (F), and LDH (G) (log-rank test).(H) Nomogram for predicting the delivery probability of patients with PE in the training cohort.

5
FIGURE 5 Validation of the nomograms in the validation cohort.(A) Area under the curve (AUC) of the nomogram predicting the delivery probability of patients with pre-eclampsia (PE) at 26, 28, 30, 32, 34, 36, and 38 weeks in the training cohort.(B-H) Calibration curves of the nomogram predicting the delivery probability of patients with PE at 26 (B), 28 (C), 30 (D), 32 (E), 34 (F), 36 (G), and 38 weeks (H) in the training cohort.(I, J) Receiver operating characteristic (ROC) curves of the nomograms predicting early-onset (I) and severe (J) PE probability in the validation cohort.(K, L) Calibration curves of the nomograms predicting early-onset (K) and severe (L) PE probability in the validation cohort.(M) AUCs of the nomogram predicting the delivery probability of patients with PE at 26, 28, 30, 32, 34, 36, and 38 weeks in the validation cohort.(N-T) Calibration curves of the nomogram predicting the delivery probability of patients with PE at 26 (N), 28 (O), 30 (P), 32 (Q), 34 (R), 36 (S), and 38 weeks (T) in the validation cohort.

TABLE 1
Baseline clinical characteristics and laboratory parameters.

TABLE 1 Continued
Data were expressed as n (%), mean (±SD), and median (interquartile range).The p-values compared between the training and the validation cohort.