Development and validation of nomograms for predicting survival in differentiated thyroid cancer patients with or without radioiodine therapy

Objective This study aimed to establish and validate the nomograms for predicting overall survival (OS) probabilities in differentiated thyroid cancer (DTC) patients who received and did not receive radioiodine therapy (RAI), respectively. Methods In this study, 11, 099 patients diagnosed with DTC in the Surveillance, Epidemiology, and End Results (SEER) database from 2004 to 2016 were selected. Whether they have RAI, they are divided into RAI (n=6427) and non-RAI (n=4672) groups. They were randomly assigned to either a training cohort (RAI: n=4498, non-RAI: n=3263) or a validation cohort (RAI: n=1929, non-RAI: n=1399) using R software to divide the patients in a 7-to-3 ratio randomly. Variables were selected using a backward stepwise method in a Cox regression model to determine the independent prognostic factors, which were then utilized to build two nomograms to predict the 5-, 8-, and 10-year OS probabilities in DTC patients with or without RAI. The concordance index (C‐index), the area under the time-dependent receiver operating characteristics curve (AUC), the net reclassification improvement (NRI), the integrated discrimination improvement (IDI), calibration plotting, and decision-curve analysis (DCA) were used to evaluate the performance of our models. Results The multivariate analyses demonstrated that birth of the year, race, histological type, tumor size, grade, TNM stage, lymph node dissections, surgery, and chemotherapy were risk factors for OS. Compared to the AJCC stage, the C‐index (RAI: training group: 0.911 vs. 0.810, validation group: 0.873 vs. 0.761; non-RAI: training group: 0.903 vs. 0.846, validation group: 0.892 vs. 0.808). The AUC values for the training cohort (RAI: 0.940, 0.933, and 0.942; non-RAI: 0.891, 0.884, and 0.852 for the 5-, 8-, and 10-year OS, respectively) and validation cohort (RAI: 0.855, 0.825, and 0.900, non-RAI: 0.867, 0.896, and 0.899), and the calibration plots of both two models all exhibited better performance. Additionally, the NRI and IDI further showed that they exhibited good 5-, 8-, and 10-year net benefits. Conclusion We have established the prediction models of DTC patients with or without RAI respectively through various variables. The nomogram may be more targeted to guide clinical decisions in the future.


Introduction
Thyroid cancer (TC) is the most prevalent cancer of the endocrine system (1), and its main pathological types are: DTC -including papillary, follicular, and Hürthle cells-medullary thyroid carcinoma (MTC), and undifferentiated thyroid carcinoma (ATC) (2). Papillary thyroid carcinoma (PTC) is the most common histological type of TC (3). The prevalence of TC has multiplied in recent decades, accounting for approximately 1-3% of human cancers (1), and is predicted to reach the fourth most common cancer by 2030 (4). There is broad agreement that DTC demonstrates relatively indolent clinical behavior. Significant improvements in the diagnosis and treatment of DTC have provided an excellent prognosis. In general, overall survival rates for TC are high, especially in patients with DTC, with 10-year survival rates ranging from 80 to 95 percent (5,6). However, some studies in recent years have found an increase in mortality among patients with advanced-stage PTC (7,8). This suggests a renewed focus on the prognosis and survival of thyroid cancer patients.
Thyroidectomy, RAI, and thyroid-stimulating hormone suppression are the primary treatment options for patients with DTC. Selective RAI therapy plays a vital role in removing potentially residual DTC and the treatment of distant metastases (9). To determine if patients need more aggressive treatment, the 2015 American Thyroid Association (ATA) guidelines developed a risk stratification system to identify patients at low, intermediate, or high-risk of thyroid cancer recurrence or death. They strongly recommend RAI for patients with a high risk of recurrence stratum. RAI may be considered for patients in the intermediaterisk stratum and is not routinely recommended after thyroidectomy for ATA low-risk DTC patients and patients with single lesion/ multifocal microcarcinoma (≤1 cm in diameter) (10). However, consideration of specific features of the individual patient that could modulate recurrence risk, disease follow-up implications, and patient preferences are relevant to RAI decision-making (11,12). So not all of the patients with DTC will be treated with RAI, and the indication and modalities of radioiodine treatment are shifting from a standardized practice to a tailored approach.
At present, there are no predictions for the survival of individual DTC patients based on RAI. In this study, based on the SEER database, we analyzed the mortality rates of DTC patients treated with or without radioiodine at 5, 8, or 10 years, which may accurately provide valuable individualized prognostic information. At the same time, these two nomograms can serve as a new reference to assist clinicians in selecting personalized RAI for DTC patients.

Source of data
The data we selected were obtained from the Surveillance, Epidemiology, and End Results (SEER) database (covering 18 registries) using the latest SEER*Stat version 8.3.5 (https:// seer.cancer.gov/). The SEER Program is an authoritative data source of clinical information on cancer incidence and survival. It contains statistical information on cancer for approximately 35% of the U.S. population. All data from the SEER database was free, and we signed the SEER Research Data Agreement for this study to access SEER information using the username 19803-Nov2020. This study was approved by the Institutional Research Committee of the First Affiliated Hospital of Jinan University.

Patient selection
The location codes collected from the SEER database were TC C73.9, diagnosed with TC from 2004 to 2016. There were 148, 349 qualified patients identified in the SEER database, of which 11, 099 were available after applying a strict screening process ( Figure 1). Then according to whether they have RAI, they are divided into RAI (n=6427) and non-RAI (n=4672) groups. For the construction and validation of the nomograms, we randomly distributed 70% of the patients to the training cohort (RAI: n=4498, non-RAI: n=3263) and 30% to the validation cohort (RAI: n=1929, non-RAI: n=1399). The following main variables were evaluated: the age of diagnosis, sex, race, histological type, tumor size, grade, AJCC stage, TNM stage, lymph node dissections (LND), the number of examined lymph nodes (ELN), the number of positive lymph nodes (PLN), the ratio of PLN and ELN (LNR), surgery, chemotherapy, follow-up time and all-cause death.

Nomogram development and statistical analyses
In the RAI group, the eleven pathological and clinical characteristics of the age of diagnosis, race, tumor size, grade, T \M stage, and chemotherapy were used to conduct the analyses, while in non-RAI, age of diagnosis, race, tumor size, grade, TNM stage, PLN, and chemotherapy were used for analysis. The backward stepwise selection method in a Cox regression model was applied to the training cohort to select variables. Using these identified prognostic factors, we then established two nomograms that predicted the 5-, 8-, and 10-year OS probabilities in DTC patients according to whether they have RAI, respectively.
After establishing the nomograms, internal and external validation is performed separately in the training and validation cohorts. We first used the C-index and the receiver operating characteristic curve (ROC). The new model's ability to discriminate is evaluated by assessing the AUC. Calibration plots were used to determine the agreement between the actual results and the predicted probabilities. Then to complement the accuracy and comprehensiveness of the comparison, two relatively new metrics (NRI and IDI) were used.
Finally, the clinical value of the prediction model was determined by quantifying the net benefit at different threshold probabilities using DCA curves. Descriptive statistics were calculated and presented as the frequency (percentage) for categorical variables, the mean ± standard for continuous variables of normal distribution, and the median (interquartile range) for continuous variables of skewness distribution. All statistical analyses were conducted using R software (version 3.5.1; https://www.r-project.org). A two-sided probability value of P ≤ 0.05 was deemed to indicate statistical significance.

Patient characteristics
We evaluated a total of 11, 099 DTC patients from 2004 to 2016. These in groups RAI and non-RAI were divided by R software, respectively, using the popular random split-sample method (with a split ratio of 7: 3) into 4498 and 3263 in the training cohort and into 1929 and 1399 in the validation cohort. Tables 1, 2 summarized the demographic and tumor characteristics of the patients.

Nomogram establishment
Two nomograms based on the selected prognostic factors from the training cohort were developed to predict OS probabilities in DTC patients with or without radioisotope, respectively, at 5, 8, and 10 years (Figures 2A, B). The nomogram for RAI demonstrated that age of diagnosis contributed the most to prognosis, followed by tumor size, M, Grade, race, T, and chemotherapy. In the OS nomogram for non-RAI, age of diagnosis was also the strongest influencing factor, followed by tumor size, chemotherapy, Grade, PLN, T, N, and M. A total score was obtained by adding the scores for each selected variable, and a vertical line is dropped down from the total-points row to estimate the OS for 5, 8, and 10 years (13).

Validation and calibration of the nomograms
The C-index for the OS nomogram of RAI (training group=0.911, validation group=0.873) was higher than those based on the AJCC stage (training group=0.810, verification group=0.761). The AUC values for the training cohort (0.940,  (Figures 3A, B). The C-index for the OS nomogram of non-RAI was also higher for the nomogram than for the AJCC stage in both the training cohort (0.903 vs. 0.846) and the validation cohort (0.892 vs. 0.808). The AUC values for the training cohort (0.891, 0.884, and 0.852 for the 5-, 8-, and 10-year OS, respectively) and validation cohort (0.867, 0.896, and 0.899) indicated the good identification ability of the model also for the OS nomogram of non-RAI (Figures 3C, D).
Analysis of accuracy for RAI showed that the NRI value for the 5- These values being more significant than zero indicate that our nomograms have a better potential for accurately predicting prognosis compared to the AJCC stage system.
The calibration plots of both nomograms showed excellent agreement between the actual observations and the predicted outcomes in both training and validation cohorts (Figures 4, 5) for 5-, 8-, and 10-years OS.
We also performed a DCA, which showed that predicting the OS probabilities applying the two models yielded net benefits in the training and validation cohorts (Figures 6, 7).

Discussion
Although the American Joint Committee on Cancer (AJCC) TNM staging system is beneficial in determining the prognosis of DTC patients (11), it ignores significant risk factors such as age, race, tissue type, and surgical status. With the advancement of precision medicine, the TNM staging system cannot provide comprehensive treatment guidance and prognosis prediction for individual patients. Also, the benefits of RAI application are currently controversial, especially in patients with low-risk DTC and microcarcinoma. Whether RAI can improve survival in these patients is a matter of intense scientific debate (14-18). As a result, although as a classic postoperative adjuvant therapy, RAI is not used for all patients with DTC. It is necessary to separately predict the survival probability of DTC patients based on RAI. A nomogram, which translates complex regression equations into visual graphs, is a more widely used clinical tool that will facilitate patient counseling and personalized treatment. To date, no nomogram has been reported respectively for predicting the probability of survival in DTC patients with or without RAI. Therefore, we constructed two more comprehensive models to better predict DTC patients' prognoses in this study. We analyzed the characteristics of DTC patients treated with or without RAI in the SEER database. Univariate and multifactorial analyses were performed to screen for independent risk factors for line graph modeling. Compared to the traditional AJCC staging system, the 7variable scale based on age of diagnosis, race, tumor size, T, M, Grade, and chemotherapy in RAI and the 8-variable scale based on the age of diagnosis, tumor size, Grade, PLN, TNM, and chemotherapy in non-RAI were able to more accurate assessment and prediction of DTC patients in training and validation cohorts, further providing detailed forecast and follow-up plans for each patient.
The age presented higher coefficients and HR values in both univariate and multivariate analyses. Previous studies have shown that the adverse effect of age on outcome gradually increases with each decade, especially after the age of 40-45 year (19,20). In the eighth edition of the AJCC TNM staging system, the threshold for age at diagnosis was changed from 45 years to 55 years. In our study, age, as an essential independent predictor, was not stratified, but a continuous age line was used to evaluate the prognosis of patients. Although DTC is 2-3 times more common in women than in men (21), in none of our studies was gender found to be an independent predictor of mortality from DTC either in patients receiving RAI or in those not receiving RAI (10). Consistent with the findings of Sunny Patel et al. Several other studies have highlighted that there is no significant difference in mortality between male and female patients (2,22). Also in the AJCC cancer staging system, gender was not seen as an independent prognostic factor.
Nomogram enables us to find that chemotherapy is an independent risk factor for overall survival in patients with DTC. The 2015 ATA guidelines recommend systemic therapies (such as cytotoxic chemotherapy or kinase inhibitors) for loco-regional disease are considered after all surgical and radiation therapy options have been exhausted (10). In our study, all patients in the RAI group had undergone total thyroidectomy, consistent with the treatment regimen recommended by ATA guidelines. We speculate that these patients who require chemotherapy are in poor condition and are likely to have iodine-refractory thyroid cancer, so chemotherapy would be an independent risk factor for their prognosis. However, only one patient in the non-RAI group did not receive surgical treatment, and the rest underwent total thyroidectomy. Due to the lack of relevant information in the database, it is not possible to know why these patients were treated with chemotherapy rather than surgery and RAI. Therefore, in further studies, we will optimize history-taking and deeply explore the additional effects of chemotherapy on DTC patients.
The number of positive lymph nodes and the N-stage was independent risk factors for prognosis in DTC patients who did not receive RAI compared to patients who did. According to the guidelines, the vast majority of patients not recommended for RAI are low-risk patients with no or few lymph node metastases. For DTC patients receiving RAI, the level of risk should be reassessed based on the post-RAI status. The number of positive lymph nodes and stage N1 will be independent risk factors for patients at intermediate to high risk who should have received RAI but do not receive it. During the follow-up, in these patients, lymph node metastases should be closely monitored for real-time assessment of prognosis.
The strengths of this study include that it is a populationbased study with a large sample size and is adjusted for the clinical B C D E F A FIGURE 6 Decision curve analysis curves for RAI. Decision curve analysis of the training cohort (A-C) and validation cohort (D-F) for 5-, 8-, and 10-years cancer-specific survival probability.
demographic characteristics of the patients. These two nomograms can be used not only to predict the survival prognosis of DTC patients with and without RAI respectively, but also to assist in clinical and patient decision-making by assessing prognostic risk based on a patient's current clinical profile when deciding whether to receive RAI. However, this study has several significant limitations. First, several potentially essential parameters and specific information related to prognosis were not available in the SEER database, making it less comprehensive, such as thyroid-stimulating hormone levels, thyroid hormone suppression treatment status, radioiodine treatment dose, etc. Second, our nomogram was internally validated but not externally validated, which may lead to overheating of the new model. Finally, the nomogram was constructed based on data from the SEER database and did not represent the global population. Therefore, we plan to conduct additional prospective studies in the future to test the denomination maps to compensate for these limitations.

Conclusions
To better determine the prognosis of DTC patients, this study establishes and validates, for the first time, 5-, 8-, and 10-year OS rates for DTC patients receiving or not receiving RAI, respectively, based on a large population-based cohort study. Compared with the B C D E F A FIGURE 7 Decision curve analysis curves for non-RAI. Decision curve analysis of the training cohort (A-C) and validation cohort (D-F) for 5-, 8-, and 10-years cancer-specific survival probability.