Daniel J. Taylor
Harry Saxton
Ian Halliday1,2
Julian P. Gunn
Paul D. Morris- 1Division of Clinical Medicine, School of Medicine and Population Health, University of Sheffield, Sheffield, United Kingdom
- 2Insigneo Institute for in Silico Medicine, University of Sheffield, Sheffield, United Kingdom
- 3NIHR Sheffield Biomedical Research Centre, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
- 4School of Computer Science, University of Sheffield, Sheffield, United Kingdom
- 5Department of Cardiology and Cardiothoracic Surgery, Sheffield Teaching Hospitals NHS Trust, Sheffield, United Kingdom
Angiography-derived, or “virtual” fractional flow reserve (vFFR) is beginning to replace invasive assessment in some catheterisation laboratories. Despite its incorporation into clinical guidelines, recent clinical outcomes data have cast doubt over its effectiveness relative to invasive assessment. These somewhat unexpected trial results are underpinned by poorer than anticipated agreement between invasive and vFFR. In particular, the widespread use of traditional Bland-Altman analysis fails to account for the phenomenon of worsening agreement at lower FFR values which hinders comparison between studies. We propose a novel approach using quantile regression to derive overall bias and limits of agreement (LOA) to better characterise agreement across the spectrum of coronary disease. This new method may improve understanding of optimal vFFR clinical applications and addresses common statistical deficiencies in current validation practices.
1 Introduction
Virtual fractional flow reserve (vFFR), derived from standard coronary angiography, was first described over ten years ago (1) and offers improved decision-making with reduced procedural burden. Since its conception, several validation trials (2, 3) have provided a body of evidence supporting its ability to predict invasive FFR. These data supported the recent endorsement for its use in the assessment of chronic coronary syndromes (CCS) in the European guidelines (4). For most cases seen in everyday clinical practice, where the significance of lesions is judged from angiography alone, vFFR-guided therapy is superior to angiography for major adverse cardiac events (MACE) (5). However, recent trial outcome data have challenged assumptions about the equivalence of vFFR when compared to the gold-standard physiological assessment – invasive FFR (6). Failure of vFFR-guided therapy to meet non-inferiority for MACE likely resulted from poorer than anticipated agreement between invasive FFR and vFFR. This indicates discrepancies between the techniques which are not necessarily obvious from published statistical analyses. This real-world mismatch may result from worsening agreement at decreasing FFR (grey zone and lower values). Consequently, studies reporting agreement in a cohort containing more non-haemodynamically significant lesions, than would be subject to vFFR assessment in everyday clinical practice, may produce misleading results.
The Bland-Altman plot, first introduced in 1986, remains the gold standard for assessing agreement between two clinical measurement techniques (7). The technique was devised as an easy-to-use tool, specifically targeted at clinical applications, which moves beyond simple regression and correlation. This parametric technique plots the average of two measurements against the difference between them. On the plot, the mean delta gives the overall bias and the 95% limits of agreement (LOA) represent the range within which 95% of the differences between two measurement methods are expected to lie. Both are plotted (Figure 1). The plot therefore gives a more visually intuitive presentation of the data vs. a simple scatter plot. It has proved extremely popular and is now one of the most highly cited statistical manuscripts in existence (over 53,000 at the time of writing). It is widely used in vFFR validation studies (1, 8, 9). However, key assumptions of the Bland-Altman method — including homoscedasticity (equal variance across measurement range) — are routinely violated in vFFR validation datasets, where measurement error frequently varies with disease severity. This is certainly not exclusive to vFFR analyses but is well demonstrated by their application. This results in traditional LOA calculations underestimating accuracy in healthier vessels while overestimating it in more diseased arteries. As a result, true agreement between vFFR and FFR remains poorly understood. While statistical workarounds such as transformation of data may provide a solution (10, 11), these detract from a key principle of the Bland-Altman plots – simplicity and ease of access for those without a more thorough statistical grounding. The lack of, to our knowledge, any vFFR validation study employing these alternative techniques to ensure an appropriate fit of data suggests they do not meet these criteria.
Figure 1. Standard Bland-Altman plot and plot with quantile-derived median bias and limits of agreement. Panel (A) Standard Bland-Altman plot with mean bias −0.01 (95% LOA −0.28 to 0.26). Panel (B) Quantile-derived LOA same data fitted with quantile regression, at pairwise mean (FFR + vFFR)/2 value of 0.80, median bias was +0.03 (95% LOA −0.20 to 0.23). At pairwise mean (FFR + vFFR)/2 value of 0.70, median bias was +0.07 (95% LOA −0.31 to 0.37) and at 0.90 median bias was −0.01 (95% LOA −0.09 to 0.09). Solid line represents overall bias, dashed lines represent 95% LOA. Note the widening LOA at lower pairwise mean (FFR + vFFR)/2 values, consistent with greater variability in agreement at higher disease severity. LOA, limits of agreement; FFR, fractional flow reserve; vFFR, virtual Fractional Flow Reserve.
In this Perspective, we propose a refined statistical framework using quantile regression to generate more appropriate LOA, enhancing the clinical interpretation of vFFR performance across a spectrum of disease severities. Crucially, this technique is easily implemented and understood for those familiar with the original Bland-Altman technique.
2 Why vFFR agreement is unsuited to traditional Bland-Altman assessment
The Bland-Altman plot displays the difference between two paired measurements against the pairwise mean of measurements. The mean bias is then plotted, along with the 95% LOA defining the expected spread of differences (Figure 1). This parametric test is dependent upon two key assumptions: differences must be normally distributed and the variance (spread) of differences should be constant across the range of measurement (homoscedasticity). These assumptions are rarely satisfied for vFFR. Principally, as vFFR quantifies a ratio between 0 and 1, results close to the higher of these values are limited in the maximum possible error. For example, a case producing an average FFR and vFFR result of 0.95 may only have a maximum error of 0.10 (results of 0.90 and 1.00), yet this phenomenon is entirely lost when LOA typically exceed ±0.10. Furthermore, accuracy is inherently dependent upon stenosis severity. In mild disease, anatomical and physiological surrogacy is stronger, resulting in tighter agreement. In contrast, in severe disease or borderline cases, vFFR models are prone to larger error, often due to greater lesion severity or complexity, leading to broader discrepancies. Despite this, Bland-Altman analysis is important for evaluating vFFR performance across studies. When applied correctly, its results are more robust to case selection compared with diagnostic accuracy metrics such as sensitivity and specificity, therefore facilitating inter-study comparison.
3 A new proposal: quantile regression-derived limits of agreement
We propose an enhancement to the traditional Bland-Altman approach by calculating overall bias and LOA using quantile regression (12). Instead of assuming constant variance, quantile regression models the spread of differences across the range of measurements, generating dynamic LOA that expand or contract in a manner which is more likely to appropriately model the typical variations in vFFR studies. A key reason we advocate for this approach reflects an underpinning value of the original Bland-Altman plot – simplicity. Our approach can be easily implemented with basic statistical software (for our study we used the quantreg package in R version 2024.04.2 + 764) and does not require more advanced statistical knowledge about data transformation. The technique could be almost universally applied to vFFR validation studies; even if the data unexpectedly met the assumptions required for traditional Bland-Altman analysis, the effect on overall bias and LOA would be minimal. A key distinction to make, is while the original Bland-Altman reports mean bias, quantile regression reports median bias which is more appropriate for non-parametric data. An example application is shown in Figure 1 with data taken from a recent validation study conducted by our group (13). Panel A shows the traditional Bland-Altman plot with fixed LOA, while Panel B demonstrates our proposed method using quantile regression. The divergence of LOA with increasing disease severity is readily apparent, accurately capturing worsening measurement variability as FFR values decrease. To aid comparison, we propose reporting the median bias and 95% LOA principally at the pairwise average (FFR + vFFR)/2 value of 0.80 (the diagnostic threshold). Authors may also wish to report LOA at 0.70 and 0.90, to quantify the relationship between agreement with disease severity.
4 Discussion
In this article, we have described a simple method of assessing agreement between invasive and vFFR which accounts for commonly encountered issues with data distribution. Our method directly addresses the heteroscedasticity inherent to coronary physiology and virtual modelling. Specifically, the technique considers the phenomenon of poorer agreement with more diseased cases in a visually intuitive way and may also be more appropriate for assessing microvascular resistance (14, 15). vFFR is now a guideline indicated tool in the assessment of intermediate coronary artery stenoses. While evidence supports it's superiority vs. standard angiographic assessment (5), it does not meet non-inferiority vs. invasive physiology (6). This difference is underpinned by agreement with invasive physiology, but traditional Bland-Altman analysis does not fully capture this relationship. While the proposed technique does not always guarantee an optimal data fit, and other non-parametric approaches may be used when fitting LOA to non-parametric data (such as with polynomials) (16), the simplicity in implementing, interpreting and comparing results strikes a balance which also preserves the familiarity of the original Bland-Altman plot. We therefore encourage other authors to consider this statistical technique when performing their own vFFR validation studies.
Data availability statement
Data underlying this work are available upon reasonable request to the corresponding author.
Ethics statement
The studies involving humans were approved by South Yorkshire Health Research Authority Regional Ethics Committee (16/NW/0897, 08/H1308/193). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants' legal guardians/next of kin in accordance with the national legislation and institutional requirements.
Author contributions
DT: Writing – review & editing, Data curation, Visualization, Formal analysis, Conceptualization, Writing – original draft. HS: Conceptualization, Writing – original draft, Writing – review & editing, Formal analysis. IH: Writing – review & editing, Writing – original draft, Supervision. JG: Writing – review & editing, Supervision, Writing – original draft, Data curation. PM: Data curation, Writing – review & editing, Supervision, Writing – original draft.
Funding
The author(s) declare financial support was received for the research and/or publication of this article. This study/research is funded by the National Institute for Health and Care Research (NIHR) Sheffield Biomedical Research Centre (NIHR203321). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. HS is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) via grant reference EP/Z531297/1. PM was funded by the Wellcome Trust [214567/Z/18/Z]. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declare that no Generative AI was used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Morris PD, Ryan D, Morton AC, Lycett R, Lawford PV, Hose DR, et al. Virtual fractional flow reserve from coronary angiography: modeling the significance of coronary lesions: results from the VIRTU-1 (VIRTUal fractional flow reserve from coronary angiography) study. JACC Cardiovasc Interv. (2013) 6(2):149–57. doi: 10.1016/j.jcin.2012.08.024
2. Collet C, Onuma Y, Sonck J, Asano T, Vandeloo B, Kornowski R, et al. Diagnostic performance of angiography-derived fractional flow reserve: a systematic review and Bayesian meta-analysis. Eur Heart J. (2018) 39(35):3314–21. doi: 10.1093/eurheartj/ehy445
3. Cortés C, Carrasco-Moraleja M, Aparisi A, Rodriguez-Gabella T, Campo A, Gutiérrez H, et al. Quantitative flow ratio—meta-analysis and systematic review. Catheter Cardiovasc Interv. (2021) 97(5):807–14. doi: 10.1002/ccd.28857
4. Vrints C, Andreotti F, Koskinas KC, Rossello X, Adamo M, Ainslie J, et al. 2024 ESC guidelines for the management of chronic coronary syndromes: developed by the task force for the management of chronic coronary syndromes of the European Society of Cardiology (ESC) endorsed by the European association for cardio-thoracic surgery (EACTS). Eur Heart J. (2024) 45(36):3415–537. doi: 10.1093/eurheartj/ehae177
5. Xu B, Tu S, Song L, Jin Z, Yu B, Fu G, et al. Angiographic quantitative flow ratio-guided coronary intervention (FAVOR III China): a multicentre, randomised, sham-controlled trial. Lancet. (2021) 398(10317):2149–59. doi: 10.1016/S0140-6736(21)02248-0
6. Andersen BK, Sejr-Hansen M, Maillard L, Campo G, Råmunddal T, Stähli BE, et al. Quantitative flow ratio versus fractional flow reserve for coronary revascularisation guidance (FAVOR III Europe): a multicentre, randomised, non-inferiority trial. Lancet. (2024) 404(10465):1835–46. doi: 10.1016/S0140-6736(24)02175-5
7. Bland JM, Altman D. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. (1986) 327(8476):307–10. doi: 10.1016/S0140-6736(86)90837-8
8. Westra J, Andersen BK, Campo G, Matsuo H, Koltowski L, Eftekhari A, et al. Diagnostic performance of in-procedure angiography-derived quantitative flow reserve compared to pressure-derived fractional flow reserve: the FAVOR II Europe-Japan study. J Am Heart Assoc. (2018) 7(14):e009603. doi: 10.1161/JAHA.118.009603
9. Ninomiya K, Serruys PW, Kotoku N, Zhou J, Kageyama S, Masuda S, et al. Anonymous comparison of various angiography-derived fractional flow reserve software with pressure-derived physiological assessment. Cardiovasc Interv. (2023) 16(14):1778–90. doi: 10.1016/j.jcin.2023.04.026
10. Ludbrook J. Confidence in Altman–Bland plots: a critical review of the method of differences. Clin Exp Pharmacol Physiol. (2010) 37(2):143–9. doi: 10.1111/j.1440-1681.2009.05288.x
11. Bland JM, Altman DG. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. (1995) 346(8982):1085–7. doi: 10.1016/S0140-6736(95)91748-9
12. Koenker R, Hallock KF. Quantile regression. J Econ Perspect. (2001) 15(4):143–56. doi: 10.1257/jep.15.4.143
13. Taylor DJ, Saxton H, Xu X, Yones E, Aubiniere-Robb L, Adikari T, et al. Invasive validation of novel 1D models for computation of coronary fractional flow reserve. Cardiovasc Res. (2025):cvaf168. doi: 10.1093/cvr/cvaf168
14. Taylor DJ, Feher J, Czechowicz K, Halliday I, Hose D, Gosling R, et al. Validation of a novel numerical model to predict regionalized blood flow in the coronary arteries. Eur Heart J Digit Health. (2023) 4(2):81–9. doi: 10.1093/ehjdh/ztac077
15. Taylor DJ, Saxton H, Halliday I, Newman T, Feher J, Gosling R, et al. Evaluation of models of sequestration flow in coronary arteries—physiology versus anatomy? Comput Biol Med. (2024) 173:108299. doi: 10.1016/j.compbiomed.2024.108299
Keywords: coronary artery disease, FFR, virtual physiology, Bland-Altman, agreement
Citation: Taylor DJ, Saxton H, Halliday I, Gunn JP and Morris PD (2025) Rethinking the Bland-Altman plot when quantifying virtual coronary physiology. Front. Cardiovasc. Med. 12:1687923. doi: 10.3389/fcvm.2025.1687923
Received: 18 August 2025; Accepted: 13 October 2025;
Published: 23 October 2025.
Edited by:
Nikos Stalikas, Cardiovascular Center, OLV Aalst, BelgiumReviewed by:
Sadeek Sidney Kanoun Schnur, Royal Cornwall Hospital Trust, United KingdomCopyright: © 2025 Taylor, Saxton, Halliday, Gunn and Morris. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Daniel J. Taylor, ZGFuaWVsLnRheWxvckBzaGVmZmllbGQuYWMudWs=