Developing a nomogram-based scoring model to estimate the risk of pulmonary embolism in respiratory department patients suspected of pulmonary embolisms

Abstract

Objective:

Pulmonary embolisms (PE) are clinically challenging because of their high morbidity and mortality. This study aimed to create a nomogram to accurately predict the risk of PE in respiratory department patients in order to enhance their medical treatment and management.

Methods:

This study utilized a retrospective method to collect information on medical history, complications, specific clinical characteristics, and laboratory biomarker results of suspected PE patients who were admitted to the respiratory department at Affiliated Dongyang Hospital of Wenzhou Medical University between January 2012 and December 2021. This study involved a total of 3,511 patients who were randomly divided into a training group (six parts) and a validation group (four parts) based on a 6:4 ratio. The LASSO regression and multivariate logistic regression were used to develop a scoring model using a nomogram. The performance of the model was evaluated using receiver operating characteristic curve (AUC), calibration curve, and clinical decision curve.

Results:

Our research included more than 50 features from 3,511 patients. The nomogram-based scoring model was established using six predictive features including age, smoke, temperature, systolic pressure, D-dimer, and fibrinogen, which achieved AUC values of 0.746 in the training cohort (95% CI 0.720–0.765) and 0.724 in the validation cohort (95% CI 0.695–0.753). The results of the calibration curve revealed a strong consistency between probability predicted by the nomogram and actual probability. The decision curve analysis (DCA) also demonstrated that the nomogram-based scoring model produced a favorable net clinical benefit.

Conclusion:

In this study, we successfully developed a novel numerical model that can predict the risk of PE in respiratory department patients suspected of PE, which can not only appropriately select PE prevention strategies but also decrease unnecessary computed tomographic pulmonary angiography (CTPA) scans and their adverse effects.

1. Introduction

Pulmonary embolism (PE) is the most common disease that can endanger the life of patients in respiratory and cardiovascular departments, mainly caused by various emboli blocking pulmonary arteries or branches, not only in the West, but also in China (1). It is not only related to respiratory disease but also involves cardiology, gynecology and obstetrics, oncology, hematology, surgery, and other diseases (2–6). The clinical manifestation of PE is a lack of specificity, and it is difficult to distinguish it from other respiratory diseases (3), such as chronic obstructive pulmonary disease. Its misdiagnosis rate and mortality rate are high, so it has become an important medical topic to improve the diagnosis rate and reduce the mortality of PE.

Computed tomographic pulmonary angiography (CTPA) is recommended for diagnosis and risk level assessment of PE (7–9). However, it is time consuming and expensive and even may cause serious side effects in patients. Many researchers have created a variety of risk assessment models (RAM) to predict PE, and their usability has been continuously validated. Hou et al. (10) established a novel PE risk prediction model based on machine learning (ML) methods. Robert-Ebadi et al. verified the feasibility of the simplified Geneva score in a clinic in 2017 (11). Wang et al. (12) established a novel risk assessment model to estimate the probability of PE in postoperative patients. Other studies indicated that initial blood parameters seem to enable further differentiation of patients with suspected PE and elevated d-dimers to raise pre-test probability of PE. Machine-learning-based prediction models might help to further narrow down CT indications in future (13). Other findings showed that syncope, systolic blood pressure, oxygen saturation, white blood cell, neutrophil percentage, and others, are crucial for the feature selection to assess the severity of PE (14). Kirsch et al. (15) demonstrated the ability of good score in predicting PE, which showed Wells score above 4 was significantly associated with PE but the sensitivity and specificity of the score were unreliable. Although many researchers have reported different prediction models to calculate the pretest probability of PE, they are underused and tend to underperform in practice, leading to persistent overuse of CTPA imaging for PE. Therefore, it will be convenient to have a simple and fast risk prediction model to predict the probability of PE occurrence.

Due to the non-specificity of PE in different diseases, many debates about these RAMs are presented, that there are no consensual methods to diagnose PE. In order to accurately diagnose them, it is necessary to develop a potential and appropriate model for patients in respiratory department to predict PE.

The purpose of this study was to develop and validate a numerical model based on electronic medical record (EMR) data for predicting PE in respiratory department patients.

2. Materials and methods

2.1. Study population

In this study, a total of 3,511 patients who were suspected of PE and hospitalized in the respiratory department at Affiliated Dongyang Hospital of Wenzhou Medical University from January 2012 to December 2021 were included. The data of subjects were retrospective collected from our clinical research data platform. The medical records of 3,511 subjects were statistically analyzed after the baseline data was cleared and extracted, and the subjects were randomly divided into a training cohort and a validation cohort with a 6:4 ratio.

The retrospective study was granted ethical approval by the Medical Ethics Committee of Affiliated Dongyang Hospital of Wenzhou Medical University (No: 2022-YX-160), and informed consent was waived. Patient data were processed in a confidential manner, with identifying information removed before analysis. The study was performed in accordance with the principles outlined in the Declaration of Helsinki.

2.2. Study outcomes and data collection

PE was defined in accordance with the criteria of European Society of Cardiology Guidelines (7). The patients who have undergone computed tomographic pulmonary angiography (CTPA) examination are suspected of PE. PE was confirmed by an identified filling defect in the pulmonary artery system in CTPA, including subsegmental PE. Based on the diagnostic conclusions of radiologists, we used this to determine whether PE occurred. The past medical history, complications, individual clinical features, and clinic biomarkers data were collected. Our research flow chart is shown in Figure 1.

Figure 1

2.3. Statistical analysis

The data were analyzed using R Studio software for Windows. Continuous variables were expressed as the mean with standard deviation (SD) or median with interquartile range (IQR) and compared using Student’s t-test or the Mann–Whitney U-test. Categorical variables were expressed as frequency with percentage and compared using the chi-squared test or Fisher’s exact test. All subjects contain 58 variables. To guarantee the reliability of the data, five indicators with missing values greater than 20% were deleted. Use the (mice) package in R software for multiple imputation techniques (16) which were utilized to impute the remaining missing values of the predictor variables. Use the (glmnet) package for the least absolute shrinkage and selection operator (LASSO) analysis which regression model (17–19) was performed to tackle the collinearity of candidate variables to select the optimal predictive variables. The results were expressed as odds ratios (ORs) and 95% confidence intervals (CIs). A two-tailed p-value <0.05 was considered to be statistically significant.

2.4. Model development and validation, and evaluation

By combining the selected predictors from the LASSO analysis, the (rms) package for multivariable logistic regression analysis was performed to develop the PE prediction model. Use the (regplot) package in R software for nomogram. The performance of the model was evaluated in terms of its discrimination, calibration, and clinical utility (20). The performance of the model was assessed using various evaluation metrics, including the (pROC) package for the area under the receiver operating characteristic (ROC) curve (AUC), using the (calibrate) package for calibration curve, and using the (rmda) package for decision curve analysis (DCA). The AUC measured the discrimination capability of the model, while the calibration curve evaluated the consistency between the predicted and actual probabilities of PE. The DCA was used to quantify the net clinical benefits at different thresholds.

3. Results

3.1. Characteristics of the study population

A total of five variables were removed from our data due to excessive amounts of missing information (over 20%) before proceeding with the analysis. We involved 53 variables in which missing data of less than 20% can be found in this study (shown in “Appendix 1”). The data contained missing information for 53 variables, with a range of missing data percentage from 0.00%–19.74%. Multiple imputation techniques were used to complete the missing data. A total of 3,511 subjects suspected of PE were included in the present study, with an incidence rate of 28.36%. The demographic and clinical characteristics of these patients are displayed in Table 1. The sample was randomly divided into two groups, the training cohort (n = 2,128) and the validation cohort (n = 1,383), and their respective characteristics are summarized in Table 2.

3.2. Selected predictors

In 53 variables, six potential predictors were finally selected on the basis of LASSO regression analysis (Figures 2A,B). The optimal predictors incorporated age, temperature, systolic pressure, fibrinogen, D-dimer, and smoke. The final predictive model was generated through a multivariable logistic regression analysis incorporating six predictors that were selected from the LASSO regression analysis (presented in Table 3).

Figure 2

3.3. Construction and validation of the model

Predicting model for PE was visualized by a nomogram, which is shown in Figure 3. The discrimination of the model, as measured by the area under the receiver operating characteristic curve (AUC), was found to be 0.746 (95% CI 0.720–0.765) in the training cohort and 0.724 (95% CI 0.695–0.753) in the validation cohort. This suggests that the predictive model has the ability to effectively differentiate PE from non-PE (Figures 4A,B). The calibration plot in training and validation cohorts (Figures 5A,B) demonstrates good consistency between PE discriminated by the model and the actual occurrence PE. The clinical utility of the nomogram was evaluated through decision curve analysis, which assessed the net benefit and threshold probabilities (shown in Figures 6A,B), indicating that the nomogram had a favorable net benefit across a broad range of threshold probabilities for PE patients in the respiratory department in both the training and validation cohorts.

Figure 3

Figure 4

Figure 5

Figure 6

3.4. Comparison of model discrimination ability

The performance of the nomogram was demonstrated by its AUC of 0.746 (95% CI 0.720–0.765), compared to the discriminative ability of the models which revealed that the nomogram model exhibited superior accuracy in the prediction of clinical outcomes than individual indicators (age, temperature, systolic pressure, fibrinogen, and D-dimer), as shown in Figure 7.

Figure 7

4. Discussion

The purpose of this study was to develop and validate a numerical model based on patient electronic medical record (EMR) data for predicting PE in respiratory department patients. In this study, a novel numerical model was developed incorporating six predictor variables, namely, age, temperature, systolic pressure, fibrinogen, D-dimer, and smoke. All parameters are readily available clinical features and biomarkers in routine health examinations. Notably, receiver operating characteristic (ROC) analysis indicated that our model AUC was 0.746 (95% CI 0.720–0.765) and displayed good discrimination and calibration. The decision curve analysis and clinical impact curve demonstrated that the majority of the threshold probabilities in this model exhibited favorable net benefits.

Our findings indicated that an increase in D-dimer levels was associated with a higher incidence of PE (OR 1.13; 95% CI 1.10–1.16). This result is in accordance with earlier observations (21, 22), which showed that high D-dimer level attributable to the possibility of developing PE. In terms of biomarkers, D-dimer is the only biomarker currently used in routine practice to predict PE, but it is unlikely to have adequate specificity in respiratory department patients at conventional thresholds for positivity. A large sample study from 2000 to 2015 showed increased hospitalization rates and the highest inpatient mortality for PE in elderly patients (23). A retrospective study demonstrated an association between age and the severity of the submassive PE stadium (24). Our model also shows that age (OR 1.01; 95% CI 1.01–1.02) is a high-risk factor for PE, which is similar to previous studies. In our model, two factors were positively associated with the risk of PE, whereas temperature, systolic pressure, fibrinogen, and smoke were negatively associated. A previous study showed a low systolic pressure was connected with a raised risk of PE-related mortality (25, 26). Fibrinogen is a large, complex, fibrous glycoprotein, which is converted into fibrin during the coagulation cascade, yielding the fibrin clot for hemostasis. Additionally, it was discovered that fibrinogen is synthesized as an acute phase reactant by the liver in response to inflammation or ischemia (27). A prospective study assessed the D-dimer and fibrinogen levels in 191 outpatients with suspected PE and observed that patients suffering from PE had a lower fibrinogen and higher D-dimer/fibrinogen (D/F) ratio versus those without PE (28). Our findings corroborated the inverse correlation between D-dimer and fibrinogen, which suggests that heightened coagulation prompts the consumption of fibrinogen and activation of endogenous fibrinolysis, leading to a rise in D-dimer levels. Our data also revealed that two clinical indicators including body temperature and smoke were incorporated into the model to predict PE although smoking has little effect on the whole model. Relevant studies (29) also show that fever and smoke are also factors affecting PE, which is dissimilar from our model. Although the role of tobacco is well established in arterial thrombosis, the evidence of its role in venous thromboembolism (VTE) as a distinct risk factor is less obvious and remains controversial (30–32). The possible reasons for the differences between our research and other studies are that smoking-attributable diseases or other predisposing factors are essential for smoking to convey a risk of VTE. It may also be due to our retrospective real-world research, where the data quality is not as good as prospective research, and there is sampling bias. However, the predictors included in our model differ from those in previous studies (33). Overall, three possible explanations for the discrepant results are as follows: (1) Our clinical research data platform did not have such indicators; (2) Indicators with a missing value rate higher than 20% were excluded; and (3) Following analysis, these indicators were not incorporated into the model.

This retrospective study suggested that a nomogram developed with clinical features and biomarkers to generate personalized evaluations of PE in respiratory department patients may distinguish target at high risk of PE. For example, a respiratory department patient’s total score is 321 points that corresponded to approximately 10.6% risk of PE. The proposed numerical model can assist clinicians in classifying respiratory department patients as either likely or unlikely to have PE, thereby reducing the number of unnecessary CTPA examinations. This model may be helpful for us to identify high-risk patients early, evaluate thrombosis, and implement active and individualized anticoagulation therapy.

The limitations of this study should be acknowledged. First, it is a retrospective study, and five indicators with a missing value rate exceeding 20% were excluded. Furthermore, the sample size was limited, and insufficient variables were recorded, potentially impacting the results. Finally, the data were collected from a single center and may not reflect a larger population.

In conclusion, we successfully developed a novel numerical model that can predict the risk of PE in respiratory department patients suspected of PE, which can not only appropriately select PE prevention strategies but also decrease unnecessary CTPA scans and their adverse effects.

Funding

This study was supported by Zhejiang Provincial Natural Science and Public Welfare Foundation of China under Grant no. LTGY23H200002 and Jinhua Science and Technology Foundation, Zhejiang Province, China (Grant nos. 2022-3-012 and 2022-4-269).

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1.

  Wendelboe AM Raskob GE . Global burden of thrombosis: epidemiologic aspects. Circ Res. (2016) 118:1340–7. doi: 10.1161/CIRCRESAHA.115.306841

  • CrossRef
  • Google Scholar

• 2.

  Ye R Zhao L Wang C Wu X Yan H . Clinical characteristics of septic pulmonary embolism in adults: a systematic review. Respir Med. (2014) 108:1–8. doi: 10.1016/j.rmed.2013.10.012

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3.

  Freund Y Cohen-Aubart F Bloom B . Acute pulmonary embolism: a review. JAMA. (2022) 328:1336–45. doi: 10.1001/jama.2022.16815

  • CrossRef
  • Google Scholar

• 4.

  Lao TT . Pulmonary embolism in pregnancy and the puerperium. Best Pract Res Clin Obstet Gynaecol. (2022) 85:96–106. doi: 10.1016/j.bpobgyn.2022.06.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5.

  Palas M Silva BV Jorge C Almeida AG Pinto FJ Caldeira D . The accuracy of Hestia and simplified PESI to predict the prognosis in pulmonary embolism: systematic review with meta-analysis. TH Open. (2022) 6:e347–53. doi: 10.1055/a-1942-2526

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6.

  Porres-Aguilar M Rosovsky RP Rivera-Lebron BN Kaatz S Mukherjee D Anaya-Ayala JE et al . Pulmonary embolism response teams: changing the paradigm in the care for acute pulmonary embolism. J Thromb Haemost. (2022) 20:2457–64. doi: 10.1111/jth.15832

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7.

  Konstantinides SV Meyer G . The 2019 ESC guidelines on the diagnosis and management of acute pulmonary embolism. Eur Heart J. (2019) 40:3453–5. doi: 10.1093/eurheartj/ehz726

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 8.

  Pang W Zhang Z Wang Z Zhen K Zhang M Zhang Y et al . Higher incidence of chronic thromboembolic pulmonary hypertension after acute pulmonary embolism in Asians than in Europeans: a meta-analysis. Front Med. (2021) 8:721294. doi: 10.3389/fmed.2021.721294

  • CrossRef
  • Google Scholar

• 9.

  Wang X Zhao H Cui N . The role of electrical impedance tomography for management of high-risk pulmonary embolism in a postoperative patient. Front Med. (2021) 8:773471. doi: 10.3389/fmed.2021.773471

  • CrossRef
  • Google Scholar

• 10.

  Hou L Hu L Gao W Sheng W Hao Z Chen Y et al . Construction of a risk prediction model for hospital-acquired pulmonary embolism in hospitalized patients. Clin Appl Thromb Hemost. (2021) 27:107602962110408. doi: 10.1177/10760296211040868

  • CrossRef
  • Google Scholar

• 11.

  Robert-Ebadi H Mostaguir K Hovens MM Kare M Verschuren F Girard P et al . Assessing clinical probability of pulmonary embolism: prospective validation of the simplified Geneva score. J Thromb Haemost. (2017) 15:1764–9. doi: 10.1111/jth.13770

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12.

  Wang MF Li FX Feng LF Zhu CN Fang SY Su CM et al . Development and validation of a novel risk assessment model to estimate the probability of pulmonary embolism in postoperative patients. Sci Rep. (2021) 11:18087. doi: 10.1038/s41598-021-97638-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13.

  Gawlitza J Ziegelmayer S Wilkens H Jagoda P Raczeck P Buecker A et al . Beyond the D-dimer—machine-learning assisted pre-test probability evaluation in patients with suspected pulmonary embolism and elevated D-dimers. Thromb Res. (2021) 205:11–6. doi: 10.1016/j.thromres.2021.07.001

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14.

  Su H Han Z Fu Y Zhao D Yu F Heidari AA et al . Detection of pulmonary embolism severity using clinical characteristics, hematological indices, and machine learning techniques. Front Neuroinform. (2022) 16:1029690. doi: 10.3389/fninf.2022.1029690

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15.

  Kirsch B Aziz M Kumar S Burke M Webster T Immadi A et al . Wells score to predict pulmonary embolism in patients with coronavirus disease 2019. Am J Med. (2021) 134:688–90. doi: 10.1016/j.amjmed.2020.10.044

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  Peyre H Leplege A Coste J . Missing data methods for dealing with missing items in quality of life questionnaires. A comparison by simulation of personal mean score, full information maximum likelihood, multiple imputation, and hot deck techniques applied to the Sf-36 in the French2003 decennial health survey. Qual Life Res. (2010) 20:287–300. doi: 10.1007/s11136-010-9740-3

  • CrossRef
  • Google Scholar

• 17.

  Hu X Shen F Zhao Z Qu X Ye J . An individualized gait pattern prediction model based on the least absolute shrinkage and selection operator regression. J Biomech. (2020) 112:110052. doi: 10.1016/j.jbiomech.2020.110052

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18.

  Narala S Li SQ Klimas NK Patel AB . Application of least absolute shrinkage and selection operator logistic regression for the histopathological comparison of chondrodermatitis nodularis helicis and hyperplastic actinic keratosis. J Cutan Pathol. (2021) 48:739–44. doi: 10.1111/cup.13931

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  Yang S Wu H . A novel PM2.5 concentrations probability density prediction model combines the least absolute shrinkage and selection operator with quantile regression. Environ Sci Pollut Res Int. (2022) 29:78265–91. doi: 10.1007/s11356-022-21318-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20.

  Wu J Zhang H Li L Hu M Chen L Xu B et al . A nomogram for predicting overall survival in patients with low-grade endometrial stromal sarcoma: a population-based analysis. Cancer Commun. (2020) 40:301–12. doi: 10.1002/cac2.12067

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21.

  Damodaram M Kaladindi M Luckit J Yoong W . D-dimers as a screening test for venous thromboembolism in pregnancy: is it of any use?J Obstet Gynaecol. (2009) 29:101–3. doi: 10.1080/01443610802649045

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22.

  Hassanin IM Shahin AY Badawy MS Karam K . D-dimer testing versus multislice computed tomography in the diagnosis of postpartum pulmonary embolism in symptomatic high-risk women. Int J Gynaecol Obstet. (2011) 115:200–1. doi: 10.1016/j.ijgo.2011.05.024

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23.

  Pauley E Orgel R Rossi JS Strassle PD . Age-stratified national trends in pulmonary embolism admissions. Chest. (2019) 156:733–42. doi: 10.1016/j.chest.2019.05.021

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24.

  Keller K Beule J Coldewey M Dippold W Balzer JO . Impact of advanced age on the severity of normotensive pulmonary embolism. Heart Vessel. (2015) 30:647–56. doi: 10.1007/s00380-014-0533-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25.

  Keller K Beule J Balzer JO Dippold W . Blood pressure for outcome prediction and risk stratification in acute pulmonary embolism. Am J Emerg Med. (2015) 33:1617–21. doi: 10.1016/j.ajem.2015.07.009

  • CrossRef
  • Google Scholar

• 26.

  Quezada A Jimenez D Bikdeli B Moores L Porres-Aguilar M Aramberri M et al . Systolic blood pressure and mortality in acute symptomatic pulmonary embolism. Int J Cardiol. (2020) 302:157–63. doi: 10.1016/j.ijcard.2019.11.102

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27.

  Kara H Bayir A Degirmenci S Kayis SA Akinci M Ak A et al . D-dimer and D-dimer/fibrinogen ratio in predicting pulmonary embolism in patients evaluated in a hospital emergency department. Acta Clin Belg. (2014) 69:240–5. doi: 10.1179/2295333714y.0000000029

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28.

  Kucher N Kohler HP Dornhöfer T Wallmann D Lämmle B . Accuracy of D-dimer/fibrinogen ratio to predict pulmonary embolism: a prospective diagnostic study. J Thromb Haemost. (2003) 1:708–13. doi: 10.1046/j.1538-7836.2003.00145.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29.

  Puri C Obah EA . Clomiphene-induced pulmonary embolism. J Community Hosp Intern Med Perspect. (2021) 11:256–7. doi: 10.1080/20009666.2021.1878624

  • CrossRef
  • Google Scholar

• 30.

  Al-Nasser B . Influence of tobacco smoking on perioperative risk of venous thromboembolism. Turk J Anaesthesiol Reanim. (2020) 48:11–6. doi: 10.5152/tjar.2019.08683

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31.

  Enga KF Braekkan SK Hansen-Krone IJ le Cessie S Rosendaal FR Hansen JB . Cigarette smoking and the risk of venous thromboembolism: the Tromsø study. J Thromb Haemost. (2012) 10:2068–74. doi: 10.1111/j.1538-7836.2012.04880.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32.

  Dulicek P Ivanova E Kostal M Sadilek P Beranek M Zak P et al . Analysis of risk factors of stroke and venous thromboembolism in females with oral contraceptives use. Clin Appl Thromb Hemost. (2018) 24:797–802. doi: 10.1177/1076029617727857

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33.

  Zhou Q Xiong XY Liang ZA . Developing a nomogram-based scoring tool to estimate the risk of pulmonary embolism. Int J Gen Med. (2022) 15:3687–97. doi: 10.2147/ijgm.S359291

  • Pubmed Abstract
  • CrossRef
  • Google Scholar