Practical Guidance for Applying Epidemiological Research Methods

40K
views
21
authors
4
articles
Editors
3
Impact
Loading...
4,124 views
8 citations

Objective: To quantify prediction model performance in relation to data preparation choices when using electronic health records (EHR).

Study Design and Setting: Cox proportional hazards models were developed for predicting the first-ever main adverse cardiovascular events using Dutch primary care EHR data. The reference model was based on a 1-year run-in period, cardiovascular events were defined based on both EHR diagnosis and medication codes, and missing values were multiply imputed. We compared data preparation choices based on (i) length of the run-in period (2- or 3-year run-in); (ii) outcome definition (EHR diagnosis codes or medication codes only); and (iii) methods addressing missing values (mean imputation or complete case analysis) by making variations on the derivation set and testing their impact in a validation set.

Results: We included 89,491 patients in whom 6,736 first-ever main adverse cardiovascular events occurred during a median follow-up of 8 years. Outcome definition based only on diagnosis codes led to a systematic underestimation of risk (calibration curve intercept: 0.84; 95% CI: 0.83–0.84), while complete case analysis led to overestimation (calibration curve intercept: −0.52; 95% CI: −0.53 to −0.51). Differences in the length of the run-in period showed no relevant impact on calibration and discrimination.

Conclusion: Data preparation choices regarding outcome definition or methods to address missing values can have a substantial impact on the calibration of predictions, hampering reliable clinical decision support. This study further illustrates the urgency of transparent reporting of modeling choices in an EHR data setting.

30,815 views
2 citations
Recommended Research Topics
Frontiers Logo

Frontiers in Epidemiology

Insights in Clinical Epidemiology: 2023
Edited by Lin Yang
764
views
0
articles
Frontiers Logo

Frontiers in Epidemiology

Causal Methods in Clinical Prediction
Edited by Marco Piccininni
17.1K
views
59
authors
4
articles
Frontiers Logo

Frontiers in Epidemiology

Insights In Research Methods and Advances in Epidemiology: 2022
Edited by Ralph Brinks
28.1K
views
27
authors
6
articles
Frontiers Logo

Frontiers in Epidemiology

Women In Research Methods and Advances in Epidemiology
Edited by Rosie Cornish, Rachael Hughes, Insa Feinkohl
11.9K
views
29
authors
4
articles