Diagnostic agreement and limitations of the Banff automated system in kidney transplant biopsies

The Banff classification for renal allograft rejection has evolved over time, increasing in complexity. For non-pathologists conducting retrospective studies, assigning Banff diagnostic categories across different eras presents a significant challenge. The Automated Diagnosis System (ADS) is a publicly available web-based tool designed to standardize Banff category diagnoses based on Banff scoring. We retrospectively evaluated ADS using 1,071 kidney biopsy results from 544 transplant recipients, including 146 ABO-incompatible cases performed at our institution. Overall concordance between ADS and pathologists was 69.8%, with high agreement in non-rejection (97.4%) and rejection (86.3%) cases. Among rejection cases, discrepancies were noted in 27 antibody-mediated rejection (AMR) and 22 T cell–mediated rejection (TCMR) cases. Discrepancies were frequently observed in AMR following ABO-incompatible transplantation and in chronic TCMR, highlighting challenges in standardizing these categories. Despite these limitations, ADS demonstrated acceptable concordance and potential utility for promoting global standardization in rejection diagnosis.

Introduction

Antibody mediated rejection (AMR) is a critical factor influencing the prognosis of kidney allografts (1) but its diagnosis remains complex. Initially defined at the 2001 Banff Conference as a condition characterized by microvascular inflammation (MVI), C4d deposition, and the presence of donor-specific antibodies (DSA) in the serum, the diagnostic criteria for AMR have undergone multiple revisions over the years (1, 2). Notably, C4d-negative AMR has been newly proposed, while ABO-incompatible transplants often demonstrate C4d positivity—defined as diffuse linear staining of peritubular capillaries (PTCs) for C4d in ≥50% of cortical PTCs—regardless of the presence of rejection (3, 4). These complexities make accurate diagnosis challenging, particularly for non-specialists in transplant pathology.

To address these challenges, Yoo et al. developed a web-based diagnostic tool capable of providing AMR diagnoses aligned with the latest Banff criteria (2019) by inputting Banff scores (5). This automated diagnosis system (ADS), free from human bias, has been reported to predict allograft outcomes more accurately than traditional pathologist-based diagnoses, highlighting the potential benefits of diagnostic standardization (6). However, the validation of this tool has predominantly relied on datasets in which 70–80% of cases were deceased donor transplants, while ABO-incompatible transplants—accounting for only 2–4% of the cohort—were underrepresented.

Given the high prevalence of ABO-incompatible transplants in Japan, there is a need to evaluate the applicability of this system in an international context. This study aims to validate the performance of the ADS in a Japanese kidney transplant cohort with a significant proportion of ABO-incompatible cases.

Methods

This study was approved by the Institutional Review Board (IRB) of Tokyo Women’s Medical University (approval number: 2023-0033), and the procedures followed were in accordance with the ethical standards of the local IRB and with the Helsinki Declaration of 1975, as revised in 2013. Pathological data were retrieved from the database of 1,071 allograft kidney biopsies from 544 recipients underwent kidney transplant at Tokyo Women’s Medical University between January 2017 and July 2022, including 146 cases with ABO incompatibility. Banff scores and pathological diagnosis were determined by professional pathologist according to the Banff criteria at each time point. In samples which had more than two diagnoses, the most dominant rejection feature was selected as primary rejection category. The ADS processed each case using input data, including Banff lesion scores, C4d immunofluorescent staining results, and DSA status. Except the presence or absence of thrombotic microangiopathy (TMA), non-rejection-related diagnoses, such as IgA nephropathy and focal segmental glomerulosclerosis (FSGS), were not incorporated for the ADS processing. All cases were classified into one of five AMR categories though the ADS: active AMR (A-AMR), chronic active AMR (CA-AMR), equivocal for diagnosis of AMR, C4d staining without evidence of rejection that is classified as linear staining in ≥50% of cortical PTCs with no histologic evidence of acute tissue injury consistent with AMR, and no evidence of AMR. T-cell mediated rejection (TCMR) were also classified into acute TCMR (A-TCMR) and chronic active TCMR (CA-TCMR). The primary rejection category was selected according to the following priority: A-AMR (including equivocal for diagnosis), CA-AMR, A-TCMR, C-TCMR, chronic-AMR inactive, borderline change for TCMR (BC), interstitial fibrosis and tubular atrophy (IFTA), C4d staining without evidence of rejection, and no evidence of rejection.

Results

Figure 1 presents a Sankey diagram illustrating the relationship between pathologists’ diagnoses and ADS-based reclassification of 1071 biopsy results. Among these, ADS diagnoses were concordant with pathologists’ diagnoses in 747 cases, yielding an overall agreement rate of 73.45%. Among the 886 cases classified as “No Rejection” by pathologists (including 31 cases of IFTA, 33 cases of BC, and 73 cases with other diagnoses), 863 cases were also categorized as “No Rejection” by ADS (including 8 cases of C4d staining without evidence of rejection; 234 cases of IFTA; and 33 cases of BC). While pathologists tend to avoid using mild IFTA as the final diagnosis, the agreement rate for non-rejection cases was adequately high (97.4%), indicating minimal discrepancies.

Figure 1

Figure 1. Sanky diagram showing relationship between pathologist- and automated diagnosis system-diagnoses. (A) Overall comparison. (B) Focused on results related to antibody mediated rejection. A-AMR, Active Antibody Mediated Rejection: CA-AMR, Chronic Active Antibody Mediated Rejection; A-TCMR, Acute T Cell Mediated Rejection I – III; CA-TCMR, Chronic active T cell Mediated Rejection; BC, Borderline Change; IFTA, Interstitial Fibrosis and Tubular Atrophy I – III.

On the other hand, among the 131 cases diagnosed as rejection by pathologists, 113 cases were also classified as rejection by ADS, resulting in an agreement rate of 86.3%. However, complete concordance in rejection type was observed in only 87 cases (66.4%). Furthermore, 18 cases diagnosed as rejection by pathologists were re-classified as non-rejection by ADS (A-AMR: 11 cases, CA-AMR: 2 cases, A-TCMR: 2 cases, CA-TCMR: 3 cases). Conversely, 23 cases diagnosed as no-rejection by pathologists were re-categorized as rejection by ADS (A-AMR: 8 cases, CA-AMR: 2 cases, A-TCMR: 6 cases, CA-TCMR: 7 cases).

Table 1 lists the cases with diagnostic discrepancies for AMR. Fourteen cases diagnosed as AMR by pathologists were not classified as AMR by ADS. The majority of these cases did not meet the threshold for the MVI score. Although C4d deposition was observed in 11 cases, its diagnostic significance was uncertain in 7 cases due to ABO-incompatible transplantation. Some cases included follow-up biopsies after rejection treatment, where AMR diagnosis was made based on clinical history rather than Banff scoring alone. Conversely, 13 cases classified as AMR by ADS had different pathological diagnoses. Nine of these were “for-cause biopsies” refer to biopsies performed when rejection was suspected based on elevated creatinine levels, proteinuria, or other clinical findings, suggesting that pathologists may have exercised caution in determining treatment interventions. Additionally, cases of recurrent glomerulonephritis and BK nephropathy, which were not influenced by the Banff scores input in ADS, were included. Although not examined in this study, ADS allows for the input of diagnoses other than rejection and previous biopsy findings, suggesting that incorporating detailed clinical and histopathological information could help resolve such discrepancies in real-world applications.

Table 1

Table 1. AMRs with different diagnosis between pathologists and ADS.

Regarding TCMR, 8 cases diagnosed as TCMR by pathologists were not classified as TCMR by ADS, while 14 cases were classified as TCMR by ADS but not by pathologists (Table 2). Notably, discrepancies were observed in the diagnosis of CA-TCMR. The classification of CA-TCMR was introduced in Banff 2017, requiring independent evaluation of tubulitis in preserved cortex and cortical IFTA regions (7). Distinguishing CA-TCMR from the mixture of BC or TCMR grade IA/IB and IFTA based solely on Banff scores is challenging, highlighting the need for further discussions to standardize its diagnosis.

Table 2

Table 2. TCMRs with different diagnosis between pathologists and ADS.

Discussion

Overall, ADS showed substantial concordance with pathologists’ diagnoses. However, notable discrepancies in AMR for ABO-incompatible grafts and CA-TCMR highlight ongoing challenges in standardizing these categories. These discrepancies do not necessarily indicate a fundamental flaw in the ADS, but rather reflect the complex and evolving nature of the Banff Classification, as well as the critical role of clinical context in biopsy interpretation. As Van Loon et al. have noted (8), the Banff schema has become increasingly intricate, and pathologists often reach different conclusions by incorporating prior biopsy findings, treatment responses, and clinical impressions—elements not currently captured by the ADS in this study. While the ADS allows entry of “Other diagnosis” inputs in addition to Banff inputs, these appear only as supplementary information in the report and do not directly influence the Banff classification rejection diagnosis logic. In fact, among the 49 cases in which the ADS diagnosis diverged from that of the pathologist in Tables 1 and 2, 22 cases (44.9%) were for-cause biopsies prompted by clinical findings such as proteinuria or elevated serum creatinine, or follow-up biopsies performed after treatment for rejection. This underscores the distinction between rule-based classification and clinically integrated diagnosis, highlighting the ADS’s value as a standardized reference tool rather than a substitute for expert judgment.

ABO-incompatible transplantation is frequently performed in Asian countries such as Japan and Korea, where donor shortages persist, but it is not widely practiced in many international centers, and thus limited data are available globally (8, 9). In a previous study, C4d positivity in peritubular capillaries was observed in 94% (diffuse in 66%) of protocol biopsies without any association with ABMR (10), indicating that the diagnostic value of C4d positivity in the setting of ABO-incompatible transplantation remains debated. While the ADS allows entry of ABO incompatibility as an input, the actual number of ABO-incompatible transplants included in the dataset used to develop the algorithm is unknown, and challenges remain regarding its incorporation into the AMR diagnostic algorithm. In our study, 282 of the 1,017 biopsy specimens (27.7%) were from ABO-incompatible kidney transplants. This proportion underscores the unique clinical context of regions such as Japan, where ABO-incompatible transplantation is common, and highlights important considerations for the practical use of the ADS in these settings.

In the current study, a substantial number of discrepancies between ADS and pathologist diagnoses were also observed for CA-TCMR. The diagnosis of CA-TCMR, introduced in Banff 2017, requires nuanced interpretation of tubulitis in areas with and without interstitial fibrosis. Although the ADS incorporates this distinction into its pseudocode, it is likely that technical limitations remain in the algorithm’s ability to fully replicate the subtle interpretive judgments made by experienced pathologists.

As a limitation, we chose a case-by-case approach to discrepancy analysis rather than a wholesale reclassification of all biopsies using the Banff 2019 criteria. In reviewing detailed Banff scores for discrepant cases, we found that differences arose not only from evolving Banff definitions, but also from clinical history, treatment effects, and the controversial nature of AMR diagnosis in ABO-incompatible transplantation—all of which can significantly influence pathologists’ interpretations.

In clinical practice, diagnoses are informed not only by Banff scores but also by histological nuances, longitudinal clinical information, and treatment history. Thus, cautious interpretation of ADS output is warranted. Nevertheless, ADS offers a practical tool for transplant clinicians, especially non-pathologists, by facilitating Banff-based classification in retrospective and multicenter studies. Its role in reducing inter-institutional variability is particularly valuable. Recently, Banff criteria has been updated (11). The major updates include the concept of microvascular inflammation (MVI); e.g. “MVI, DSA-negative, C4d-negative” are no longer considered AMR, MVI below threshold, C4d- but DSA+ cases are deemed “probable AMR”. Future updates to align with the latest Banff iterations are anticipated.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Institutional Review Board of Tokyo Women’s Medical University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

JM: Investigation, Writing – original draft, Formal Analysis, Data curation. TH: Conceptualization, Funding acquisition, Writing – review & editing, Writing – original draft, Visualization. TS: Writing – review & editing, Supervision, Funding acquisition. YK: Writing – review & editing, Investigation. KU: Resources, Writing – review & editing. MY: Writing – review & editing, Project administration. HK: Project administration, Writing – review & editing. TT: Writing – review & editing, Supervision. HI: Writing – review & editing, Supervision.

Funding

The author(s) declare financial support was received for the research and/or publication of this article. This work was supported by JSPS KAKENHI Grant Numbers 22K08723 and 25K11850.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that Generative AI was used in the creation of this manuscript. We used generative AI for code writing for R software and language correction.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abbreviations

ADS, Automated Diagnosis System; AMR, antibody-mediated rejection; TCMR, T cell–mediated rejection; MVI, microvascular inflammation; DSA, donor-specific antibodiesl; A-AMR, active AMR; CA-AMR, chronic active AMR; A-TCMR, acute T cell–mediated rejection; CA-TCMR, chronic active T cell–mediated rejection; IFTA, interstitial fibrosis and tubular atrophy; BC, borderline change.

References

1. Loupy A, Haas M, Solez K, Racussen L, Glotz D, and Seron D. The banff 2015 kidney meeting report: current challenges in rejection classification and prospects for adopting molecular pathology. Am J Transplant. (2017) 17:28–41. doi: 10.1111/ajt.14107

PubMed Abstract | Crossref Full Text | Google Scholar

2. Mengel M, Loupy A, Haas M, Roufosse C, Naesens M, and Akalin E. Banff 2019 Meeting Report: Molecular diagnostics in solid organ transplantation-Consensus for the Banff Human Organ Transplant (B-HOT) gene panel and open source multicenter validation. Am J Transplant. (2020) 20:2305–17. doi: 10.1111/ajt.16059

PubMed Abstract | Crossref Full Text | Google Scholar

3. Haas M, Sis B, Racusen LC, Solen K, Glotz D, and Colvin RB. Banff 2013 meeting report: inclusion of c4d-negative antibody-mediated rejection and antibody-associated arterial lesions. Am J Transplant. (2014) 14:272–83. doi: 10.1111/ajt.12590

PubMed Abstract | Crossref Full Text | Google Scholar

4. Haas M, Montgomery RA, Segev DL, Rahman MH, Racusen LC, and Bagnasco SM. Subclinical acute antibody-mediated rejection in positive crossmatch renal allografts. Am J Transpl. (2007) 7:576–85. doi: 10.1111/j.1600-6143.2006.01657.x

PubMed Abstract | Crossref Full Text | Google Scholar

5. Yoo D, Goutaudier V, Divard G, Gueguen J, Astor BC, and Aubert O. An automated histological classification system for precision diagnostics of kidney allografts. Nat Med. (2023) 29:1211–20. doi: 10.1038/s41591-023-02323-6

PubMed Abstract | Crossref Full Text | Google Scholar

6. Mengel M and Li XC. Automation of Banff rules for precision diagnosis. Am J Transpl. (2023) 23:1284–5. doi: 10.1016/j.ajt.2023.07.005

PubMed Abstract | Crossref Full Text | Google Scholar

7. Haas M, Loupy A, Lefaucheur C, Roufosse C, Glotz D, and Seron D. The Banff 2017 Kidney Meeting Report: Revised diagnostic criteria for chronic active T cell-mediated rejection, antibody-mediated rejection, and prospects for integrative endpoints for next-generation clinical trials. Am J Transplant Off J Am Soc Transplant Am Soc Transpl Surg. (2018) 18:293–307. doi: 10.1111/ajt.14625

PubMed Abstract | Crossref Full Text | Google Scholar

8. Cho H, Baek CH, Park SK, Kim H, and Go H. Significance of C4d expression in peritubular capillaries concurrent with microvascular inflammation in for-cause biopsies of ABO-incompatible renal allografts. Kidney Res Clin Pract. (2023) 43:82–92. doi: 10.23876/j.krcp.22.221

PubMed Abstract | Crossref Full Text | Google Scholar

9. Lim BJ. C4d Puzzle in ABO-incompatible kidney transplantation. Kidney Res Clin Pract. (2024) 43:6–7. doi: 10.23876/j.krcp.23.139

PubMed Abstract | Crossref Full Text | Google Scholar

10. Setoguchi K, Ishida H, Shimmura H, Shimizu T, Shirakawa H, and Omoto K. Analysis of renal transplant protocol biopsies in ABO-incompatible kidney transplantation. Am J Transpl. (2008) 8:86–94. doi: 10.1111/j.1600-6143.2007.02036.x

PubMed Abstract | Crossref Full Text | Google Scholar

11. Naesens M, Roufosse C, Haas M, Lefaucheur C, Mannon RB, and Adam BA. The Banff 2022 Kidney Meeting Report: Reappraisal of microvascular inflammation and the role of biopsy-based transcript diagnostics. Am J Transpl. (2024) 24:338–49. doi: 10.1016/j.ajt.2023.10.016

PubMed Abstract | Crossref Full Text | Google Scholar