The Role of Adherence Thresholds for Development and Performance Aspects of a Prediction Model for Direct Oral Anticoagulation Adherence

Patients who do not sufficiently adhere to their dosing regimens will, ultimately, do not get the full benefit of their medication. For example, if direct oral anticoagulants (DOAC) are not taken continuously, an intervention to improve adherence or maintain persistence will show direct effects on clinical outcomes. Usually, adherent patients are defined by taking ≥80% of their medication. The resulting binary adherence status from this threshold can as well be used for predictive classification. Thus, the threshold can determine the prediction model’s performance to identify patients at risk for poor adherence by this binary adherence status. In this perspective, we propose a plan for model development and performance considering the threshold’s role. Concerning development demands, we extracted predictors from a systematic literature search on DOAC adherence to be used as a core set of candidate predictors. Independently, we investigated how well a future model would technically have to perform by modeling drug intake and thromboembolic events based on a rivaroxaban pharmacokinetic-pharmacodynamic model. Using this simulation framework for different thresholds, we projected the impact of an imperfectly predicted adherence status on the event risk, and how imperfect sensitivity and specificity affect the cost balance if a supporting intervention was offered to patients classified as non-adherent. Our simulation results suggest applying a rather high threshold (90%) for discrimination between patients at low or high risk for non-adherence by a prediction model in order to assure cost-efficient implementation.


Clinical setting (A): M&S of threshold-dependent adherence effects
The following investigations were based on a pharmacokinetic-pharmacodynamic (PKPD) model of rivaroxaban as an exemplary drug among the DOACs (Girgis et al., 2014). In brief, this published model linked drug intake described by a PK model to prothrombin time as an immediate PD effect. For atrial fibrillation (AF) patients with typical covariate characteristics, this model allows simulating of administration regimens and corresponding prothrombin times (PT). We chose a regimen of 20 mg rivaroxaban once daily. We derived poor adherence in terms of insufficient drug intake by randomly omitting intake times according to thresholds (e.g., 80 % adherence corresponded to a regimen with only 80 % of mandatory intakes).
For our purpose, we extended this model to incorporate a clinical effect, which we considered as the composite of stroke and systemic embolism in accordance with pivotal DOAC trials (e.g., Connolly et al., 2009 andPatel et al., 2011). We thus added ( ) . We adjusted these estimates to (i.e. on a daily scale) with an inter-individual variability of 20 % ( ) and , so that simulated yearly cumulative incidences matched empiric results from pivotal DOAC trials (Connolly et al., 2009;Patel et al., 2011) and that the risk for a 10% difference in adherence approximated a hazard ratio of 1.13 (Shore et al., 2014).
Based on these assumptions, a simulation framework was set up for 1,000 virtual patients in each group above and below an adherence threshold (i.e. 2,000 in total), which were followed-up for one year. Levels of adherence were chosen in intervals of ten from 10 % to 100 % adherence. For convenience, the interval corresponding to a certain threshold was added to both groups above and below an adherence threshold. For both sides approaching the threshold, probabilities for the remaining intervals were derived from the empirical normal cumulative density function for adherence levels until the threshold, so that highest probabilities and thus most observations originated from administration regimens around the threshold. In particular, we chose four thresholds for investigation: 60 %, 70 %, 80 %, and 90 %.

Prediction setting (B): M&S of imperfect discrimination
For each threshold defining two groups above and below the respective threshold, we let denote the true classification of an individual (assigning 0 to the group above the threshold and 1 to the group below the threshold) and its classification by the prediction model. Discrimination was thus defined based on conditional probabilities, so that ( ) and ( ). Accordingly, we re-assigned the simulated 'true' group status for various proportions of imperfect discriminations, i.e. estimates for sensitivity and specificity ranging from 0.025 to 1 in equally-spaced intervals of 0.025. This was accomplished by drawing group indicators from a Bernoulli process with response proportions ( for the poorly adhering group below the threshold and for the better adhering group above the threshold). In our simulation framework, we repeated this for 1000 draws to extract and report mean values of the simulations.

Implementation setting (C): M&S of cost savings attributed to an intervention
Focusing on these scenarios mimicking imperfect sensitivity and specificity of an intended prediction model at various adherence thresholds, a practical application of such a model would comprise an intervention targeting poor adherence. Naturally, such an intervention would have to be supplied to patients in need (i.e. with actually poor adherence) and not necessarily to well-adhering patients, where the intervention would solely take up resources. Cost-effectiveness is thus a major concern to be explored. In our M&S framework, we virtually offer an intervention to those patients being identified as poorly adherent (i.e. below the threshold under investigation) by an imperfectly discriminating prediction model. An intervention success rate of 50 % was assumed in all further steps; this fraction of patients with potential benefit was drawn from the subsample with good adherence (i.e. above the respective threshold). The number of events in such a re-classified data set was compared to the number of events in the original data (setting A). In order to study potential cost savings attributed to such a program, we defined savings as the difference between expected benefits and costs, i.e. ( ) ( ). We determined the average costs for stroke and its consequences in the German health care system at 43,129 EUR (Kolominsky-Rabas et al., 2006) and set intervention cost to 100 € as a reasonable estimate within the range of possible interventions (Chapman et al., 2010)