A Validation Study Comparing Risk Prediction Models of IgA Nephropathy

We aimed to validate three IgAN risk models proposed by an international collaborative study and another CKD risk model generated by an extended CKD cohort with our multicenter Chinese IgAN cohort. Biopsy-proven IgAN patients with an eGFR ≥15 ml/min/1.73 m2 at baseline and a minimum follow-up of 6 months were enrolled. The primary outcomes were a composite outcome (50% decline in eGFR or ESRD) and ESRD. The performance of those models was assessed using discrimination, calibration, and reclassification. A total of 2,300 eligible cases were enrolled. Of them, 288 (12.5%) patients reached composite outcome and 214 (9.3%) patients reached ESRD during a median follow-up period of 30 months. Using the composite outcome for analysis, the Clinical, Limited, Full, and CKD models had relatively good performance with similar C statistics (0.81, 0.81, 0.82, and 0.82, respectively). While using ESRD as the end point, the four prediction models had better performance (all C statistics > 0.9). Furthermore, subgroup analysis showed that the models containing clinical and pathological variables (Full model and Limited model) had better discriminatory abilities than the models including only clinical indicators (Clinical model and CKD model) in low-risk patients characterized by higher baseline eGFR (≥60 ml/min/1.73 m2). In conclusion, we validated recently reported IgAN and CKD risk models in our Chinese IgAN cohort. Compared to pure clinical models, adding pathological variables will increase performance in predicting ESRD in low-risk IgAN patients with baseline eGFR ≥60 ml/min/1.73 m2.


INTRODUCTION
Immunoglobulin A nephropathy (IgAN), first described by Berger in 1968, is the most common type of glomerulonephritis and an important cause of end-stage renal disease (ESRD) worldwide (1)(2)(3). Because of the heterogeneous prognostic nature of IgAN, it is important to identify high-risk patients at diagnosis not only for the selection for treatment strategies and clinical trials but also for patient health education (4)(5)(6)(7).
In recent decades, dozens of clinical risk factors, including proteinuria, hypertension, and estimated glomerular filtration rate (eGFR), at the time of renal biopsy have been reported to be associated with worse renal prognosis in IgAN (8). In addition to clinical parameters at baseline, proteinuria or blood pressure during the first 2 years of follow-up after diagnosis also has clear correlations with the prognosis of IgAN (9). Among these risk factors, baseline eGFR was established as the most consistent indicator. An outstanding question in the field is whether combined pathological indicators, such as mesangial hypercellularity, endocapillary hypercellularity, segmental sclerosis, and interstitial fibrosis, can increase the accuracy of clinical indicators for prognosis prediction (9)(10)(11).
To date, several prediction models of IgAN progression have been established based on patients from different populations at different stages of renal function (9,(12)(13)(14)(15)(16)(17). We previously established a clinical model (CLIN model) and a combined model containing both clinical and pathological variables (CLINPATH model), which had good performance in predicting the occurrence of ESRD at 10 years in the validation cohort (18). Later, a large-scale study of a combined multiethnic IgAN cohort performed by Barbour et al. established risk prediction models based on 3,927 IgAN patients (19). The clinical model in this study included proteinuria, blood pressure, and eGFR at renal biopsy. In addition to the clinical indicators, the limited model contained the MESTC histologic score, and the full model included age, medication, and racial/ ethnic characteristics. The authors found that the limited model [area under the curve (AUC) = 0.80; 95% confidence interval (CI), 0.79-0.81] and full model (AUC = 0.82; 95% CI, 0.81-0.82) showed improved performance in predicting the composite outcome (defined as a 50% decline in eGFR or ESRD) compared to the clinical model (AUC = 0.78; 95% CI, 0.77-0.78). In addition, whether risk models of chronic kidney disease (CKD) can be used to predict the prognosis of IgAN is an interesting question. The study by Tangri developed and validated CKD risk models by including 8,391 Canadian CKD patients. Model 3 (C statistic, 0.91; 95% CI, 0.89-0.93), a clinical model, had good performance in predicting disease progression in patients with CKD stages 3 to 5 (20). Both studies by Barbour and Tangri studies have been further assessed and externally validated (21)(22)(23), but the prediction models would still benefit from additional external validation to improve confidence in using them in practice.
The objective of this study was to use our established multicenter Chinese IgAN cohort to conduct an independent external validation study of Barbour's IgAN models and Tangri's CKD model. We also compared the performance of pure clinical models (including clinical variables only) and combined models (including both clinical and pathological variables). We aimed to determine whether pathological parameters independently contribute to clinical models in predicting IgAN prognosis.

Ethics Approval and Consent to Participate
This study was performed in accordance with the Declaration of Helsinki and approved by the Ethics Research Committee of Ruijin Hospital, Medical School of Shanghai Jiaotong University. Written informed consent was collected from all participants prior to inclusion in the study.

Participants
A multicenter collaborative cohort (six nephrology centers from teaching hospitals throughout the country) was established to represent Chinese patients with IgAN. All patients were recruited from six renal centers from 1985 to 2018. The recruitment criteria for the IgAN patients included the following: (1) IgAN was defined by a renal biopsy demonstrating dominant IgA deposition in the mesangium of glomeruli by immunofluorescence microscopy; (2) IgAN was not secondary to systemic diseases, such as Henoch-Schoünlein purpura, systemic lupus erythematosus, and liver disease; (3) the eGFR was ≥15 ml/ min/1.73 m 2 at diagnosis; (4) the minimum follow-up time was 6 months; (5) the age at biopsy was more than 18 years; and (6) an informed consent form was signed.

Clinical and Pathologic Characteristics
All clinical and pathologic variables at the time of renal biopsy and during follow-up were collected. Age at biopsy, mean arterial blood pressure (MAP), serum creatinine (Scr), hemoglobin, eGFR (using the EPI equation), 24-h protein excretion, and renin-angiotensin system blocker (RASB) or glucocorticoid treatments were recorded. The severity of the renal damage was scored according to the Oxford MESTC classification (24). Three recently reported risk prediction models, including the clinical model, limited model, and full model with race/ethnicity (19), and one CKD risk prediction model (20), were used to calculate the risk of renal disease progression in individuals with IgAN.

Outcomes and Definitions
The start of follow-up time was considered the date of renal biopsy. The primary renal outcome of our study was the combined outcome (the first occurrence of either a 50% decline in eGFR from that at biopsy or ESRD). The secondary outcome was defined as ESRD (eGFR < 15 ml/min/1.73 m 2 or the need for dialysis/renal transplantation). Patients were censored at the time of meeting the endpoint criterion or loss to follow-up.

Calculation of Predicted Risk and Risk Groups
To calculate the prediction risk of renal outcomes for each patient, the b coefficients from the original models of Barbour (19) and Tangri (20) were used (Supplementary Table 1). Patients were categorized into four risk groups by the percentiles of linear predictors: low risk: <16th; intermediate risk: 16th to 50th; higher risk: 50th to 84th; and highest risk: > 84th percentile (19).

Statistical Analysis
There are no reliable sample size recommendations for studies that validate prognostic models, but at least 100 events are recommended (25). Continuous data that are normally distributed or had a skewed distribution are expressed as the medians (interquartile range) or mean ± SD, respectively, and categorical data are expressed as the frequencies or percentages (%); probabilities of cumulative renal survival curves were generated by the Kaplan-Meier method. Prediction model performance was assessed using measures of model fit (Nagelkerke R2, Akaike information criterion (AIC), C statistic). Comparisons of the observed and predicted 5-year risk and 2-year risk of renal outcomes were analyzed separately. In addition, survival receiver operating characteristic (ROC) analysis was performed to evaluate the discriminatory ability of the scoring system after 5 years of follow-up. Reclassification improvement was quantified using the net reclassification improvement (NRI). Calibration refers to the agreement between observed outcomes and predictions, which was analyzed by the Hosmer-Lemeshow test in our study. Statistical analysis was performed using the ResourceSelection package (version 0.3-5), rms package (version 5.1-4), pROC package (version 1.16.1), and PredictABEL package (version 1.2-4) with the R statistical programming language (R, version 3.5.3; R Foundation for Statistical Computing, Vienna, Austria). Two-tailed p-values <.05 were considered statistically significant, except where otherwise indicated. The results are presented according to the TRIPOD guidelines for risk prediction models (Supplementary Table 2).

Subject Characteristics
A total of 2,300 IgAN patients were finally enrolled based on the inclusion criteria ( Figure 1). The characteristics of our cohort and the two original cohorts are summarized in Table 1. Our cohort included 1,106 males (48.1%), and the median age was 35 years (IQR, 28-44 years). The median values of baseline eGFR and 24-h proteinuria were 76.9 ml/min/1.73 m 2 and 1.3 g/day, respectively. Among the included patients, 73.7% received RASB treatment and 59.8% received glucocorticoid treatment after diagnosis. During the median follow-up time of 2.5 years, 288 patients (12.5%) had a renal composite outcome, and 214 patients progressed to ESRD (9.3%).

Performance of the IgAN Prediction Tool in Two Renal Outcomes
The goodness of fit and statistics for discrimination for all models at 5 years after biopsy are shown in Tables 2 and 3, respectively.
Using the composite outcome as an endpoint, the clinical model, including eGFR, proteinuria, and MAP, performed well (C statistic, 0.81; R 2 , 0.23). The C statistic and R 2 were not significantly improved after adding pathological indicators in the limited model (C statistic, 0.82; R 2 , 0.27) or medication and other predictors in the full model (C statistic, 0.82; R 2 , 0.27). The AIC was also similar among the clinical, limited, and full models (712. 10 Table 3). Supplementary Figure 1 shows the mean predicted risk probability of the composite outcome against the observed risk over the follow-up period. The full model with race was calibrated well, with a mild underestimation in the low-, intermediate-, and highest-risk groups and mild overestimation in the higher-risk group.
Indeed, the three IgAN models used for predicting ESRD performed better (all C statistics > 0.9) than that used to predict the composite outcome ( Table 2). Compared with the clinical model, the full model with race also demonstrated significant improvement in risk reclassification for predicting 5-year risk, with an NRI of 0.36 (95% CI, 0.15 to 0.58, Table 3). Overall, the three models mildly underestimated the risk within 5 years in the    highest-risk group (Figure 2). In addition, we validated the performance of those models in predicting the 2-year renal outcome and found that they could also effectively predict shortterm prognosis ( Supplementary Tables 3 and 4).

Performance of the CKD Prediction Tool in Two Renal Outcomes
We next evaluated the performance of the CKD risk prediction model predicting different renal outcomes in IgAN patients given the CKD-like nature of IgAN. As a model containing only clinical indicators, CKD model 3 also had excellent performance in predicting ESRD (C statistic, 0.90; 95% CI, 0.86-0.94) and relatively good performance in predicting composite outcomes (C statistic, 0.81; 95% CI, 0.76-0.86) in our IgAN patients ( Table 2). Using ESRD as the renal outcome, the R 2 (0.31) and AIC (450.11) were also similar to those of the above clinical models ( Table 2). The clinical models based on baseline eGFR and various other clinical parameters exhibited good performance. In addition, the difference between the observed and predictive probabilities ( Figure 2) and the ROC curve ( Figure 3) for predicting ESRD at 5 years in the CKD model were similar to those of the above IgAN models. The IgAN models and the CKD model performed better in predicting ESRD than in predicting the composite endpoint. Considering that ESRD is a robust renal outcome, it was used for further analysis.

Subgroup Analysis of the Four Models for Predicting ESRD
A subgroup analysis of the entire cohort was used to evaluate the performance of the four models in patients from different subgroups (Supplementary Figure 2). Either clinical models or   H). The predicted and observed event probability estimates represent the mean predicted probability from risk-prediction model and the mean observed probability from the population divided into quartiles of predicted probability. For those models, risk groups were based on the 16th (lowest risk), 16th to 50th (intermediate risk), 50th to 84th (higher risk), and higher than 84th (highest risk) percentiles of the linear predictor.  (Figure 3).

DISCUSSION
Clinical challenges of IgAN include accurately stratifying patients, helping clinicians to identify high-risk patients to enhance treatment, and avoiding unnecessary hormone and immunosuppressive therapies for low-risk patients. Recently, with the efforts of clinical nephrologists, multiple risk models have been established. Some models include baseline or follow-up clinical parameters, such as eGFR, proteinuria, and blood pressure, and some models add pathological parameters to the clinical models to establish combined models. These predictive models still benefit from other external validations, thereby increasing the confidence in their clinical use. In addition, whether pathological parameters, such as Oxford MEST predictors, can enhance the predictive value of clinical parameters for the prognosis of patients with IgA nephropathy remains controversial.
In this study, we assessed the performance of international IgAN prediction tools and another CKD model by external validation in a large, multicenter Chinese IgAN cohort. Relative to those in the derivation cohort of the international IgAN prediction tools, our follow-up time was shorter, and the incidence of a 50% decline in eGFR or development of ESRD was lower (12.5% versus 17.7%). Compared to the derivation cohort of the CKD model, our cohort had a lower proportion of patients with baseline eGFR < 30 ml/min/1.73 m 2 , and fewer patients progressed to kidney failure/ESRD (9.3% versus 11%) during the follow-up period.
Indeed, we still found that the four models performed well at predicting the 5-year risk of renal outcomes. Compared with composite outcomes, better performance of those tools used for predicting ESRD at 5 years was observed. These models also performed relatively well at predicting short-term prognosis. In addition, we found that the clinical models based on baseline eGFR and various other clinical parameters had good performance, as did the combined model that included clinical and pathological indicators. For IgAN patients at low risk characterized by higher baseline eGFR, adding pathological variables could enhance the discriminatory ability of models that contain only clinical variables. Finally, after application to patients at low risk, the full model had the best performance in predicting ESRD among the four reported models.
Among patients with IgAN, there can be considerable heterogeneity in the risk for progression to kidney failure. Risk factors associated with IgAN progression have gained increasing attention over the last two decades (26)(27)(28)(29). The emerging literature suggests improved patient outcomes with individualized risk prediction models (30)(31)(32)(33)(34). The availability of these risk prediction tools has led to better adherence to treatment guidelines and encouraged individual decision making (32)(33)(34). Despite these benefits, the lack of easily applicable and externally validated models has delayed the widespread integration of risk prediction in all fields of medicine (35,36). We confirmed that the models rely on clinical data and histological markers of IgAN severity to predict the early risk of kidney failure at 5 years. Similar to Barbour (37,38), we also confirmed that a lower estimated eGFR, more severe proteinuria, and male gender predict faster progression to kidney failure. In addition, a higher percentage of tubular injury and segmental sclerosis also predict a higher risk of kidney failure. These markers may enable a better estimate of the underlying processes of disease (39,40). Considering that those laboratory and pathological markers have been associated with the progression of IgAN, risk prediction models integrate them in different combinations. Based on our data, the performances of all models according to ROC analysis were compared. The clinical models and combined models showed similar performance in predicting ESRD after 5 years of follow-up. However, for patients at low risk characterized by higher baseline eGFR (≥60 ml/min/1.73 m 2 ), the pathological variables could add predictive value to clinical variables, likely because the contribution of pathological indicators to ESRD prediction is diminished by the subsequent use of immunosuppressive therapy in high-risk patients characterized by lower eGFR.
Risk prediction models have important implications for clinical practice, research, and public health policy. Different risk thresholds could be used to triage patients for decisionmaking. For example, primary care physicians could manage lower-risk patients without additional testing or treatment of complications, whereas higher-risk patients could receive more intensive testing, intervention, and early nephrology care (41). Furthermore, the risk prediction model could be used to select higher-risk patients for enrollment into clinical trials and for the evaluation of risk-treatment interactions. In addition, the risk prediction model may be useful for identifying high-risk patients for public health interventions, thereby improving the costeffectiveness of medical care.
The strength of our study is that we added a strict primary endpoint of ESRD, which is more relevant than other common endpoints based on declined eGFR or CKD stage. Specifically, for patients at low risk characterized by higher baseline eGFR (≥60 ml/min/1.73 m 2 ), using the full model to predict ESRD at 5 years should be more precise. Additionally, both the CKD model and IgAN model exhibited similar performance in our IgAN cohort. We still need to improve the predictive ability of the models by adding IgAN-specific biomarkers, such as HAA-IgA1 levels. Alternatively, considering the genetic background of IgAN (1,(42)(43)(44)(45), adding genetic risk factors (29,46) involved in disease progression could also be useful for improving risk prediction of disease progression.
Our analysis also has limitations. We did not explicitly model the risk of all-cause mortality in our IgAN population because the number of deaths in our cohort might have been underestimated. In addition, disease duration and treatment information prior to renal biopsy were incomplete, thus we did not involve these data for the analysis. Moreover, although the CKD model from the study by Tangri was evaluated and performed well in our Chinese IgAN cohort, more CKD cohorts are still needed to validate it. Additionally, as this study was a multicenter cohort study, the heterogeneity of the study population is a limitation. The lack of detailed data on systematic therapies from all the centers is another limitation.

CONCLUSION
In summary, we validated recently reported highly accurate predictive models for the progression of IgAN to kidney failure. Especially for patients at low risk characterized by higher baseline eGFR (≥60 ml/min/1.73 m 2 ), an improvement in model performance was observed after adding histological indicators to these clinical indicators.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material. Further inquiries can be directed to the corresponding author.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by the Ethics Research Committee from Ruijin Hospital, Medical School of Shanghai Jiaotong University. The patients/participants provided their written informed consent to participate in this study.

AUTHOR CONTRIBUTIONS
JXi designed and was responsible for the study. YO analyzed the data and drafted the paper. ZZ, GL, HL, FX, LS, ZC, SY, YJ, JXu, MS, HH, WD, ZF, XP, WM, and NC collected the data. JXi revised the paper. All authors approved the final version of the manuscript. All authors contributed to the article and approved the submitted version. in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

ACKNOWLEDGMENTS
We acknowledge the contributions of clinicians at six renal centers caring for these patients. We also acknowledge the participation of the patients involved.