Development and Validation of a Novel Risk Stratification Model for Cancer-Specific Survival in Diffuse Large B-Cell Lymphoma

Diffuse large B-cell lymphoma (DLBCL) is a biologically and clinically heterogenous disease. Identifying more precise and individual survival prognostic models are still needed. This study aimed to develop a predictive nomogram and a web-based survival rate calculator that can dynamically predict the long-term cancer-specific survival (CSS) of DLBCL patients. A total of 3,573 eligible patients with DLBCL from 2004 to 2015 were extracted from the Surveillance, Epidemiology and End Results (SEER) database. The entire group was randomly divided into the training (n = 2,504) and validation (n = 1,069) cohorts. We identified six independent predictors for survival including age, sex, marital status, Ann Arbor stage, B symptom, and chemotherapy, which were used to construct the nomogram and the web-based survival rate calculator. The C-index of the nomogram was 0.709 (95% CI, 0.692–0.726) in the training cohort and 0.700 (95% CI, 0.671–0.729) in the validation cohort. The AUC values of the nomogram for predicting the 1-, 5-, and 10- year CSS rates ranged from 0.704 to 0.765 in both cohorts. All calibration curves revealed optimal consistency between predicted and actual survival. A risk stratification model generated based on the nomogram showed a favorable level of predictive accuracy compared with the IPI, R-IPI, and Ann Arbor stage in both cohorts according to the AUC values (training cohort: 0.715 vs 0.676, 0.652, and 0.648; validation cohort: 0.695 vs 0.692, 0.657, and 0.624) and K-M survival curves. In conclusion, we have established and validated a novel nomogram risk stratification model and a web-based survival rate calculator that can dynamically predict the long-term CSS in DLBCL, which revealed more discriminative and predictive accuracy than the IPI, R-IPI, and Ann Arbor stage in the rituximab era.


INTRODUCTION
Diffuse large B-cell lymphoma (DLBCL) is the most common type of aggressive lymphoma, accounting for 30-40% of all newly diagnosed non-Hodgkin lymphomas (NHL) (1). It is a heterogeneous group of lymphomas in terms of clinical presentation, tumor biology and prognosis (2). The addition of rituximab to conventional CHOP chemotherapeutic regimen (cyclophosphamide, doxorubicin, vincristine, and prednisone) has dramatically improved the treatment outcome of patients with DLBCL (3). However, approximately 30-40% of patients still experience disease relapse or refractory to therapy (4).
The International Prognostic Index (IPI) (5) and its subsequent revisions [Revised-IPI (R-IPI)] (6) and National Comprehensive Cancer Network IPI (NCCN-IPI) (7) remain the most useful prognostications for DLBCL. These scoring systems were based on similar factors: patient age, Eastern Cooperative Oncology Group (ECOG) performance status (PS), Ann Arbor stage, serum lactate dehydrogenase (LDH), and extranodal involvement. IPI is the first, best validated, and still most widely used tool for routine clinical application in DLBCL. However, it was developed far before the introduction of rituximab in the clinical practice, and its discriminating power has decreased in the rituximab era. R-IPI was based on a cohort including more patients with younger age, it may be of limited value for elderly patients. NCCN-IPI has been shown to provide better risk stratification than the IPI, nonetheless, it was still insufficiently accurate to be applied in clinical practice.
Hence, novel more precise prognostic tools are needed to assess patients' prognosis in the rituximab era. A variety of molecular biomarkers and gene-based predictors have good discrimination ability (8)(9)(10)(11)(12)(13)(14). However, these methods are costly, technically challenging in developing countries and require further validation. Therefore, identifying new, easily accessible and more accurate prognostic markers is still needed.
By integrating various significant factors, a nomogram can predict and quantify the probability of individual patient developing a certain clinical event, such as the possibility of disease death or recurrence. In addition, it had shown a more effective predicted ability than traditional staging systems in several cancers. Therefore, the nomogram has become an important instrument for clinical decision making and risk stratification in oncology (15). Although nomogram can improve predictive accuracy, it may be difficult to use in clinical practice due to need to perform manual calculations. The web-based calculator based on nomogram allows to enhances calculation efficiency of CSS rate. Therefore, the current study aimed to develop a predictive nomogram and a web-based survival rate calculator that can dynamically predict the long-term CSS of DLBCL patients and compared the performance of the nomogram with traditional IPI and R-IPI prognostic scoring systems based on a large cohort of patients from the Surveillance, Epidemiology, and End Results (SEER) database.

Data Sources
Study data were retrieved from the SEER database, which collected patient-level clinical, pathological, demographic, therapeutic, and survival information from a total of 18 population-based cancer registries in the United States (http:// seer.cancer.gov/). Patient consent and institutional review board approval were not required but a data use agreement was submitted to SEER database for access to data.

Patient Selection
A case listing session was created from SEER database using SEER*Stat software (Version 8.3.6).
The following information on patient demographics, diagnosis, treatment, and survival were collected for analysis: patient identification, age at diagnosis, sex, race, year of diagnosis, marital status [married, single, or divorced/separated/widowed (DSW)], Ann Arbor stage, histology based on ICD-0-3, primary site, presence of systemic symptoms, IPI, cancer-directed surgery, radiation therapy, chemotherapy, vital status, cause of death, and survival months. In SEER database, Code the pretreatment point value for the IPI score as documented by the clinician in the range 000 (0 points) to 005 (5 points). Use a code in the 990 to 993 range if a risk category is described and points are not stated.
Study inclusion criteria were as follows: (1) Patients diagnosed with DLBCL-NOS according to the International Classification of Diseases for Oncology, third edition (ICD-O-3) histology code 9680 (DLBCL-NOS); (2) Patients diagnosed from 2004 to 2015, because we aimed to build a nomogram for predicting CSS in the rituximab era and 79% of DLBCL patients have received rituximab as first-line treatment from 2002 (16), however, access to the IPI relevant information (IPI score or risk stratification) was available from 2004 in the SEER database; (3) IPI relevant information was known; (4) diagnosed with DLBCL as the primary malignancy; (5) only one malignant tumor present.
The exclusion criteria were as follows: (1) those cases diagnosed without histological confirmation or diagnosed by autopsy or death certificate; (2) no active follow-up or survival time were recorded as 0 month; (3) Patients with missing data on any of these characteristics: primary site, IPI score, Ann Arbor Stage, B symptoms, surgery treatment, marital status and race; (4) patients with primary central nervous system or mediastinal DLBCL. Because they have unique clinical features, prognosis, and management.
A total of 130,896 patients diagnosed with DLBCL-NOS from 1975 to 2016 were extracted from the SEER database. However, only 6,646 patients with available IPI information from 2004 to 2015 were identified. After data screening, a total of 3,573 patients were eventually included in the present study. The process of patient selection was illustrated in Figure 1.

Construction and Validation of the Nomogram
To improve the prediction power of the nomogram, 3,573 patients were randomly assigned to the training cohort (n = 2,504) and the validation cohort (n = 1,069) at a ratio of 7:3. The training cohort was used for construction and internal validation of the nomogram, and the validation cohort was applied for external validation.
In the first step, cox proportional hazard regression models were applied for univariable and multivariable analysis. Significant predictors of CSS determined by multivariate analyses were used to construct the nomogram. Then, the nomogram performance for predicting survival outcomes was evaluated with discrimination and calibration in internal validation and external validation. The discrimination ability was measured by calculating the C-index, which resembles the area under ROC curve (AUC) value. A calibration plot was then used to show the concordance between predicted and observed probabilities for survival. Nomograms were subjected to 1,000 bootstrap resamples for internal validation in the training cohort and validation cohort, respectively.

Nomogram Performance in Risk Stratification
Each patient was assessed a total score predicted by the nomogram. X-tile plots were constructed to assess nomogram score using four groups, and the most effective cutoff points were identified following correction for the use of minimum P statistics by Miller-Siegmund P-value correction (17). Receiver operating characteristic (ROC) curves and Kaplan-Meier survival curves were used to compare the precision of the survival prediction of the nomogram with IPI, R-IPI and Ann Arbor stage.

Statistical Analysis
The endpoint of our study was CSS, which was defined as the time from diagnosis to death attributed to DLBCL. Overall survival (OS) was defined as the time from diagnosis to death from any cause. Patients who were alive at the time of last follow-up were counted as censored observations. Statistical analyses were performed using SPSS version 25.0 (IBM Corp., Armonk, NY, USA), GraphPad Prism version 8.0 (GraphPad, San Diego, CA, USA), and R version 3.6.1 (http://www.R-project.org). Demographic and clinical variables were compared between the training and validation cohorts using Pearson Chi-square test for categorical variables and independent t-test for continuous variables. Survival curves were estimated with the Kaplan-Meier method and compared with a log-rank test stratified according to the prognostic factors. Univariate and multivariate analyses were performed using the Cox regression model. Nomogram construction and validation were performed with Iasonos' guide (18). Nomogram and calibration plots were constructed by R software using the "rms package." The "shiny" and "DynNom" packages were used to develop the web-based survival rate calculator (https://www. shinyapps.io/), which can individually and dynamically predict the survival rates of patients. Statistical tests were two-sided, and P < 0.05 was considered statistically significant.

Baseline Characteristics of Study Population
The baseline characteristics of the patients in the training and validation cohorts are presented in Table 1. The median age of all patients was 63 years (range, 9-97 years). Overall, there were no substantive differences between the two cohorts.

Overall Survival and Cancer-Specific Survival of Patients
In the entire study cohort, the median follow-up time was 49 months (range, 1-155 months). There were 1,342 deaths during the follow-up period, of which 979 were cancer-specific deaths and 363 deaths were from other causes. The 5-year OS rate was 65.3% and 5-year CSS rate was 72.2% for the entire group ( Figure 2).

Univariate and Multivariate Analyses of Cancer-Specific Survival in the Training Cohort
In the univariate analysis, the prognostic factors that affected survival in the training cohort were as follows: Age, sex, marital status, Ann Arbor stage, B symptom, chemotherapy, radiation therapy, and primary site surgery. Multivariate analysis revealed that Age, sex, marital status, Ann Arbor stage, B symptom, and chemotherapy were independent risk factors for CSS ( Table 2).

Construction and Validation of the Nomogram
The significant independent prognostic factors of CSS for DLBCL were used to construct the nomogram. The nomogram exhibited that age made the largest contribution to survival outcome, followed by Ann Arbor stage, chemotherapy, marital status, sex, and B symptom ( Figure 3). C index of the nomogram was calculated to be 0.709 (95% CI, 0.692-0.726) in the training cohort and 0.700 (95% CI, 0.671-0.729) in the validation cohort. Similarly, AUC values of the nomogram for predicting the 1-, 5-, and 10-year CSS rates in the training cohort were 0.741, 0.733, and 0.730, respectively ( Figure 4A). These values were 0.729, 0.712, and 0.732 in the validation cohort ( Figure 4B). In addition, the calibration curves for the probability of the 1-, 5-and 10year CSS showed favorable agreement between the actual observed outcome and the prediction by the nomogram both in the training (Figures 5A-C) and validation cohorts ( Figures 5D-F).
These findings indicated that the discriminatory capacity and accuracy of the nomogram was reasonably clear.

Dynamic Web-Based Survival Rate Calculator
We established a dynamic web-based survival rate calculator (https://qiaofengzhong.shinyapps.io/DynNomapp/) to predict the long-term CSS of patients with DLBCL based on the nomogram. For example, a 66-year-old married man was diagnosed as DLBCL with Ann Arbor stage of IV, presented with B symptoms, if he refused chemotherapy, his 5-year cancer-specific survival rate is approximately 8.6% (95% CI 4.3-17.0%) ( Figure 6).

Nomogram Performance in Risk Stratification
The training and validation cohorts were each stratified into four groups based on the total points predicted by the nomogram for further evaluate the calibration. We defined optimal cutoffs value based on the X-tile plots. According to the cut-off values, patients were classified into low risk (n = 898, 25.1%; score <119), low-intermediate risk (n = 949, 26.6%; score 119-157), high-intermediate risk (n = 844, 23.6%; score 158-187), and high risk (n = 882, 24.7%; score >187).    The novel prognostic model displayed better accuracy than IPI, R-IPI, and Ann Arbor stage in both cohorts ( Figures 8A, E). (Figures 8B, F). R-IPI was unsatisfactory for discriminating between good and very good patient groups in both cohorts ( Figures 8C, G). Ann Arbor stage was unsatisfactory for the stratification of patients with stage I and II, stage III and IV disease in both sets (Figures 8D, H).

IPI was unsatisfactory for the stratification of patients with low and low-intermediate, high and high-intermediate in both sets
These results revealed that the established nomogram risk stratification model indicated greater discriminatory capacity and accuracy in CSS prediction of DLBCL compared with the IPI, R-IPI, and Ann Arbor stage.

DISCUSSION
DLBCL is one of the most common subtypes of NHL with great heterogeneity in clinical behavior and response to treatment. Precision medicine rapidly developed in recent years. Clinicians must establish individualized treatment and follow-up strategies  for patients, which requires more accurate and individual survival prognostic models. Although the IPI and its permutations (R-IPI and NCCN-IPI) are still recognized as good prognostic models for DLBCL, they mainly reflect a stationary state before treatment and cannot consistently predict prognosis for individual patients. A variety of new biomarkers and gene signatures with prognostic significance for risk stratification for DLBCL also has been identified, such as cell of origin (19), C-MYC, BCL-2, or BCL-6 rearrangements (20), MYC/BCL2 expression (21), PD-1 and PD-L1 expression (22,23), absolute lymphocyte/monocyte count (24)(25)(26), which have all shown promise as new predictive factors. However, most of these parameters remain expensive and not readily accessible. Nomograms have become more widely used prognostic tools in recent years as it permits improved predictive accuracy for clinical outcomes compared with the former prognostic indexes through calculating the cumulative effect of each independent variable (18). Nomograms were previously developed for DLBCL (27)(28)(29)(30)(31)(32)(33)(34)(35), although some of these nomograms incorporated clinicopathological parameters into nomogram construction for accurate survival prediction, the generalizable conclusion of these study were drawn based on a relatively small patient cohort, and most of them have not yet been validated in an external cohort. In addition, these nomograms may be not easily applied in daily clinical practice due to the need to perform manual calculations. Therefore, we developed a dynamic web-based survival rate calculator (https:// qiaofengzhong.shinyapps.io/DynNomapp/) to predict the longterm survival of patients with DLBCL based on the nomogram. It can dynamically predict the CSS rate of patients at different time  points. In the present study, a nomogram was developed based on 3,573 cases from the SEER database. We identified six demographic, clinicopathological and therapeutic features including age at diagnosis, sex, marital status, Ann Arbor stage, B symptom, and chemotherapy as independent predictors of CSS in patients with DLBCL, which were used to construct the nomogram. These simple parameters are routinely recorded, easily available in daily clinical practice, and are inexpensive to test. The nomogram exhibited that age made the largest contribution to survival outcome for DLBCL. Age has been demonstrated to be a significant predictive and prognostic factor in previous studies (36,37). The NCCN-IPI additionally emphasized the role of age by allocating three points to very elderly patients. The correlation of age with increased mortality was not surprising as patients of advanced age tend to have comorbidities, poorer ECOG and therapy-related toxicity compared to younger patients, which might contribute to lesseffective outcome. Ann Arbor stage also had a strong prognostic association with CSS for DLBCL. This observation was similar to previous reports and was not unexpected as advanced stage indicates disseminated disease.
Therapeutic factors play an important role in individual survival of cancer patients. It is worth noting that with the inclusion of chemotherapy, which usually convey a tremendous survival benefit in DLBCL, the novel nomogram is only marginally better than conventional R-IPI. One possible reason for this was that the real impact of chemotherapy was underestimated. In the SEER database, the chemotherapy field is coded to "Yes" or "No/Unknown," however, the detailed regimens and cycles were unavailable. Therefore, the findings of our study should be interpreted cautiously.
Interestingly, this study showed that married patients were more likely to have a good survival outcome than those who were unmarried or DSW. Numerous published studies addressed the association between marital status and disease specific survival, with variable results. A systematic review that included a total of 18 studies confirmed that unmarried, DSW patients diagnosed with any of the malignancies are more likely to present with advanced stage disease (38). Although the real mechanisms underlying the association between marital status and cancer specific survival are still largely unknown. There are several possible reasons that may help to explain this. Firstly, married patients may have better health insurance and socioeconomic status compared to single and DSW patients. Secondly, widowed patients tend to be older, which may increase cancer mortality. Thirdly, married patients tend to have better adherence with prescribed treatments than unmarried patients (39). Impaired adherence has been associated with poorer outcomes in patients with cancer. Lastly, patients who are married display less distress, depression, and anxiety than their unmarried counterparts after

Zhong and Shi
A Prognostic Model for DLBCL Frontiers in Oncology | www.frontiersin.org January 2021 | Volume 10 | Article 582567 a diagnosis of cancer, as a partner can share the emotional burden and provide appropriate social support (40). It is worth noting that, the results of Table 2 showed that "single" marital status was not prognostic in the univariate analysis, however, it was of significance in the multivariate analysis, the possible reason was that there may be a certain correlation between marital status and other confounding factors, after the influence of other factors is eliminated by multivariate analysis, it is found that "single" marital status plays an independent role in cancer prognosis.
In addition, the nomogram exhibited that the female had a better survival outcome than male. Multiple randomized trials (RICOVER 60, NHL-B2, MInT, and the MegaCHOEP trials) also have demonstrated superior outcomes in female relative to male, particularly in older women. Two factors may account for this result: (1) elderly females had a statistically significant slower clearance of rituximab, which brought about longer exposure time and higher serum levels of rituximab in the body (41,42).
The B symptoms has prognostic significance. B symptoms tend to correlate with disease that is either more widespread or of a higher histologic grade (44). In line with this, B symptoms was an independent risk factor for poor prognosis in this study. A risk stratification model was generated on the basis of patient's total scores evaluated by the nomogram, which had a favorable level of predictive accuracy compared to the IPI, R-IPI, and Ann Arbor stage. Although the traditional prognostic scoring systems have retained prognostic significance regarding CSS for DLBCL patients in our results, a limitation existed that they simply suggest the general prognostic value of a patient without dynamic survival estimate and risk stratification. In addition, IPI reflects pretreatment factors relating to prognosis, whereas our new prognostic model included covariates both from the baseline and after treatment, thus adding the strength of treatment-responsiveness in prognostication. Moreover, the new prognostic score could divide DLBCL patients into four groups more accurately, which could help decrease over-therapy in lowrisk patients and promote effective therapies in high risk patients to prevent the fatal nature of this disease. Therefore, we have developed a new prediction model in the rituximab era that may provide an accurate risk stratification of individual DLBCL patient to determine the optimal treatment and follow-up  strategies. To our knowledge, this is the first large-cohort, comprehensive retrospective study that has developed a nomogram and a web-based survival rate calculator to predict the CSS in DLBCL and defined optimal cutoff values based on the nomogram score for risk stratification compared to classic prognosis indexes from the SEER database in the rituximab era. There are several unavoidable limitations in the present research. Firstly, the SEER database was lacking of detail information of ECOG PS, LDH, NCCN-IPI, molecular markers, gene expression, chemotherapy regimens, and complications that may affect prognosis. We are unable to further analyze the impact of these potential prognostic factors and compare this nomogram model with NCCN-IPI. Secondly, due to the retrospective nature of the study and the strict inclusion and exclusion criteria used, potential selection biases might be present. Thirdly, although our nomogram model was developed and validated in two independent subgroups of patients from the SEER database, external validation of the predictive model is still necessary. Lastly, although this nomogram model predicted well in patients with lower-risk, its predictive ability was reduced in high-risk patients. Therefore, the findings of this study need to be confirmed by a larger and prospective cohort of DLBCL patients.
In conclusion, we have developed a predictive nomogram and a web-based survival rate calculator by using some clinically common variables from the SEER database that can predict individual and dynamic CSS for DLBCL patients with a high degree of accuracy based on a large number of patients in the rituximab era. The proposed novel nomogram risk stratification model showed a better level of risk discrimination and predictive accuracy than IPI, R-IPI, and Ann Arbor stage, which may help guide individual treatment and follow-up strategies.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
Patient consent and institutional review board approval were not required, but a data use agreement was submitted to SEER database for access to data.

AUTHOR CONTRIBUTIONS
YS conceived the concept, designed the research, drafted and reviewed the article. QZ extracted and analyzed the data and wrote the manuscript. All authors contributed to the article and approved the submitted version.

FUNDING
This work was supported by grants of the Beijing Natural Science Foundation, Grant/Award Number: H2018206591.