Prognostic Performance of Different Lymph Node Staging Systems in Patients With Small Bowel Neuroendocrine Tumors

Background: The prognostic significance of the lymph node (LN) classification for small bowel neuroendocrine tumors (SBNETs) remains unknown. The aim of the present study was to evaluate and compare the prognostic assessment of different LN staging systems.

Methods: Patients with SBNETs were identified from the Surveillance, Epidemiology, and End Results (SEER) database. The X-tile program was used to determine the cutoff value of the resected lymph nodes (RLNs), negative lymph nodes (NLNs), lymph node ratio (LNR), and the log odds of positive lymph nodes (LODDS). Survival analyses were performed using Kaplan–Meier curves with log-rank test. Logistic regression analysis was used to evaluate the differences between different periods. Univariate and multivariate Cox proportional hazards models were used to assess the prognostic value of different LN staging systems on cause-specific survival (CSS). The relative discriminative abilities of the different LN staging systems were assessed using the Akaike information criterion (AIC) and the Harrell consistency index (HCI).

Result: A total of 3,680 patients were diagnosed with SBNETs between 1988 and 2014 from the SEER database. A significant difference over time (1988–1999 vs. 2000–2014) was seen in age (P <0.001), tumor differentiation (P <0.001), T stage (P <0.001), and RLN (P <0.001) subgroups. Multivariate Cox survival analysis identified that LN status stratified by the number of RLNs, NLNs, LNR, and LODDS all predicted CSS in patients with SBNETs (all P <0.05), whereas the number of positive lymph nodes (PLNs) failed (P = 0.452). When assessed using categorical variables, LODDS staging systems showed the best prognostic performance (HCI: 0.766, AIC: 7,575.154) in the whole population. Further analysis based on different RLNs after eliminating the missing data showed that when the RLNs are <12, the LODDS (HCI: 0.769, AIC: 1,088.731) maintained the best prognostic performance as well when the RLNs are ≥12 (HCI: 0.835, AIC: 825.692). Among patients with LNR scores of 0 or 1, there was a residual heterogeneity of outcomes that were better stratified and characterized by the LODDS.

Conclusion: LODDS was a better predicator of survival when LN status was stratified as a categorical variable and should be considered when assessing the prognosis of patients with SBNETs to allow a more reliable means to stratify patient survival.

Introduction

Small bowel neuroendocrine tumors (SBNETs) comprise the third largest subgroup of gastrointestinal neuroendocrine tumors (NETs) (1). The incidence of SBNETs has tripled in the past three decades because of the development of radiological endoscopic procedures (2). Historically, the exact staging of SBNETs has been problematic for the characteristically indolent nature (3). The prognosis may vary from slowly to highly aggressive disease with remarkably different outcomes (4). This variability causes significant challenges in medical decision making and treatment. The lymph node (LN) involvement may have a substantial prognostic effect and important therapeutic implications in SBNETs (5). In the staging classification of the European Neuroendocrine Tumor Society (ENETS), the LN staging of SBNETs is classified as N0 or N1 disease depending on the absence or presence of LN involvement (6). The recently updated American Joint Committee on Cancer (AJCC) eighth edition classification stratified the previous N1 disease into N1 [number of positive lymph nodes (PLNs) <12) and N2 (PLNs ≥ 12 or large mesenteric mass) (7). However, the prognostic significance of these categories is not yet established. Therefore, increasing attention has been paid to LN status, which can be evaluated by the number of resected lymph nodes (RLNs), PLNs, negative lymph nodes (NLNs), lymph node ratio (LNR), and the log odds of positive lymph nodes (LODDS). Compared to a single parameter, LNR showed its advantage in pancreatic (8), colon (9), breast (10, 11), and ovarian cancers (12). Another parameter, LODDS, is defined as the log[(PLN + 0.5)/(NLN + 0.5)] (13, 14). A value of 0.5 is added to both numerator and denomination to avoid singularity (15, 16). The LODDS also showed superiority in predicting outcomes in bladder (17), colorectal (18), lung (19), ovarian (20), and esophageal (21) cancers. In this study, we performed a large population-based database investigation of the prognostic values of RLNs, PLNs, NLNs, LNR, and LODDS in SBNETs, which may inform treatment options and prognosis discussions and guide eligibility for clinical trials.

Methods

Patients

Information about patients with SBNETs (site codes: C17.1, C17.2, and C17.3; histologic codes: 8013, 8150 to 8156, 8240 to 8247, 8249, and 9091) diagnosed between 1988 and 2014 according to the International Classification of Disease for Oncology, third edition (ICD-O-3), was obtained from the SEER database, a national cancer registry managed by the United States National Cancer Institute, which collected information related to sociodemography and clinicopathology. Parameters included race, age, year of diagnosis, sex, tumor site and size, tumor extension, tumor differentiation, regional LN removal, regional LN involvement, distant metastasis, surgery, and survival status. Tumor size and extension were used to determine T categories according to the AJCC eighth edition stage classification system. Tumor grade according to the World Health Organization (WHO) classification was not available in the SEER database, only tumor differentiation was retrieved. Eligibility criteria were as follows: SBNETs were the first and only malignant tumor; patient is 18 years of age or older; surgical resection was performed; there is complete LN information; and the survival time is at least one month. Cause-specific survival (CSS) was the primary outcome measure.

Statistical Analysis

Descriptive statistics was used to report the basic clinicopathological characteristics of the patients for total study population and by each diagnosis-year cohort (1988–1999 vs. 2000–2014). The cutoff points of the RLNs, NLNs, LNR, and LODDS were determined by the X-tile program by using the minimum P-values from log-rank chi-square analysis in terms of CSS (22). Survival analyses were performed using Kaplan–Meier curves with log-rank test. Logistic regression analysis was used to evaluate the influence of the different periods. Univariate Cox proportional hazards regression was used to identify potential prognostic factors. Multivariable Cox regression models were built to jointly assess the prognostic ability of the different LN staging schemes and other potential prognostic indicators. The Akaike information criterion (AIC) and the Harrell consistency index (HCI) were used to assess the relative discriminative power of different LN staging systems. An HCI of 0.5 indicated no predictive power, and an HCI of 1 indicated complete differentiation (23). In general, a predictive model with a high HCI indicated a better discriminating ability, while a low AIC indicated a better model fit (24). All analyses were carried out with SPSS version 22.0 and R version 3.6.1. For all analysis, P <0.05 was considered significant, and all tests were two-tailed.

Result

Patient Characteristics

A total of 3,680 qualified participants with SBNETs diagnosed from 1988 to 2014 in the SEER database were enrolled in this study. The program selection details for the SEER database queries were shown in Supplementary Figure 1. The clinicopathological characteristics and distributions of different LN staging systems for the study population were shown in Table 1. The total study population consisted of 1,814 males (49.5%) and 1,866 females (50.7%). The median age at diagnosis was 59.0 years (range, 18–100 years). The median follow-up time was 53.0 months (range, 1–318 months). Overall, 304 (8.3%) SBNETs were located in the duodenum, 3,064 (83.3%) in the ileum, and 312 (8.5%) in the jejunum. According to the AJCC eighth edition staging criteria, patients with T4 disease (43.0%) were the most numerous, followed by patients with T1 disease (9.3%), T2 disease (19.7%), and T3 disease (27.9%). There were 2,963 (80.5%) patients with SBNETs who had LN metastases and 717 (19.5%) patients with no LN metastases. The median numbers of PLNs, RLNs, NLNs, LNR, and LODDS in the whole cohort were 2, 12, 7, 0.23, and −1.18, respectively.

TABLE 1

Table 1. Demographics and clinicopathological characteristics of patients with SBNETs.

Characteristics of the LN Staging Schemes

The threshold of PLN was determined by the AJCC eighth edition staging system. The X-tile software was used to perform log-rank chi-square analysis to estimate the cutoff value of RLNs, NLNs, LNR, and LODDS with the minimum P-values. For the RLN staging system, we used 11 as the best cutoff value and divided the RLNs into two groups as follows: RLNs ≤ 11 and RLNs > 11 (Supplementary Figure 2). For the NLN staging system, we used seven as the best cutoff value and divided the NLNs into two groups as follows: NLNs ≤ 7 and NLNs > 7 (Supplementary Figure 3). For the LNR staging system, we used 0 and 0.4 as the best cutoff values and divided the LNR into three groups as follows: LNR = 0; LNR ≤ 0.04; and LNR > 0.4 (Supplementary Figure 4). For the LODDS staging system, we used −1.3 and −0.3 as the best cutoff values and divided the LODDS into three groups as follows: LODDS ≤ −1.35; −1.35 < LODDS ≤ −0.3; and LODDS > −0.3 (Supplementary Figure 5).

Survival

Kaplan–Meier survival curves and survival data based on different LN staging systems were shown in Figure 1 for all patients. The 5-year CSS of patients without LN involvement was 90.8%, with 86.7% for with PLNs <12 and with 83.9% for those with PLNs ≥ 12 (P <0.001; Figure 1A). The 5-year CSS was 85.3% for patients with RLNs ≤ 11 and 90.3% for those with RLNs > 11 (P <0.001; Figure 1B). The 5-year CSS was 84.5% for patients with NLNs ≤ 7 and 91.1% for those with NLNs > 7 (P <0.001; Figure 1C). The 5-year CSS rates were 90.5, 89.6, and 81.8% for patients in the LNR = 0, LNR ≤ 0.4, and LNR > 0.4 subgroups, respectively (P <0.001; Figure 1D). The 5-year CSS rates were 91.6, 86.9, and 81.5% in the LODDS ≤ −1.3, −1.3 < LODDS ≤ −0.3, and LODDS > −0.3 subgroups, respectively (P <0.001; Figure 1E). These survival disparities remained among the subsets of age (P <0.001; Figure 2A), tumor site (P <0.001; Figure 2D), and tumor differentiation (P <0.001; Figure 2E) but not in the subsets of sex (P = 0.61; Figure 2C). When stratified by year of diagnosis, CSS improved significantly over time (P <0.001; Figure 2B).

FIGURE 1

Figure 1. Kaplan–Meier survival analysis according to PLN classification (A), RLN classification (B), NLN classification (C), LNR classification (D), and LODDS classification (E) for cause-specific survival in patients with SBNETs.

FIGURE 2

Figure 2. Kaplan–Meier survival analysis according to age (A), year of diagnosis (B), sex (C), tumor site (D), and tumor differentiation (E) for cause-specific survival in patients with SBNETs.

Disparities Between Diagnosis-Year Cohorts

To further explore the period-specific differences observed, we performed additional analysis of diagnosis-year cohorts among patients with SBNETs. These significant differences over time were seen in age (P <0.001), tumor differentiation (P <0.001), T stage(P <0.001), and RLNs (P <0.001) subgroups, except in race, sex, tumor site, M stage, PLNs, NLNs, and LNR subgroups (Figure 3). The most common age range at initial diagnosis was ≤ 60 years in the 2000–2014 cohort, whereas it was >60 years in the 1988–1999 cohort (HR 0.680, 95% CI 0.543–0.852, P <0.001). Patients with SBNETs were more likely to have advanced T disease and RLNs ≤ 11 in the 1988–1999 cohort, whereas they were likely to have early T disease and RLNs > 11 in the 2000–2014 cohort.

FIGURE 3

Figure 3. Forest plot showing the logistic regression analyses of the difference between years of diagnosis.

Prognostic Abilities of Different LN Staging Systems in the SEER Database

Univariate Cox survival analysis indicated that the age, year of diagnosis, tumor site, tumor size, tumor differentiation, T stage, M stage, and the number of PLNs, NLNs, RLNs, LNR, and LODDS were closely related to CSS of SBNETs (Table 2). Multivariate Cox analysis was used to assess the independent prognosis of the different LN staging systems separately. Higher numbers of RLNs (HR: 0.729, 95% CI: 0.605–0.878, P <0.001; Supplementary Table 1) and NLNs (HR: 0.689, 95% CI: 0.574–0.828, P <0.001; Supplementary Table 2) were related to a better CSS. But the number of PLNs was not prognostic for CSS (PLNs = 0 as the reference; PLNs <12: HR: 1.124, 95% CI: 0.885–1.427, P = 0.338; PLNs ≥ 12: HR: 0.912, 95% CI: 0.549–1.517, P = 0.724; Supplementary Table 3). The LNR (LNR = 0 as the reference; LNR ≤ 0.4: HR: 0.952, 95% CI: 0.737–1.230, P = 0.709; LNR > 0.4: HR: 1.352, 95% CI: 1.047–1.746, P = 0.021; Table 3) and LODDS (LODDS ≤ −1.3 as the reference; −1.3 < LODDS ≤ −0.3: HR: 1.048, 95% CI: 0.832–1.320, P = 0.693; LODDS > −0.3: HR: 1.490, 95% CI: 1.221–1.818, P <0.001; Table 4) persisted as significant and independent prognostic factors of CSS.

TABLE 2

Table 2. Univariate analysis of prognostic factors influencing CCS in patients with SBNETs.

TABLE 3

Table 3. Multivariate analysis of LNR influencing CCS in patients with SBNETs.

TABLE 4

Table 4. Multivariate analysis of LODDS influencing CCS in patients with SBNETs.

Comparison of Prognostic Values of the Different LN Staging Systems

The LN status was evaluated as a categorical variable to analyze the prognostic discriminating power of different LN staging systems, after controlling for age, year of diagnosis, tumor size, primary tumor site, tumor differentiation, T stage, and M stage data. In the whole population, LODDS staging systems showed the best prognostic performance (HCI: 0.766, AIC: 7,575.154; Supplementary Table 4). To assess whether the ability of the predicted prognosis of different LN staging systems was affected by artificially determined cutoff values, the LN status was modeled as a continuous variable for repeated analysis. The LNR staging system had better prognostic performances (HCI: 0.766, AIC: 7,578.546; Supplementary Table 4). However, tumor differentiation was not available for approximately half of the analyzed population, and 5.8% of the patients had incomplete data for tumor size. An additional analysis was constructed that eliminated these missing data. In this sensitivity model, LODDS still had better prognostic performance (HCI: 0.795, AIC: 2,157.289; Table 5) in the categorical cohort, whereas the LNR staging system had better prognostic performance in the continuous cohort (HCI: 0.794, AIC: 2,161.076). Further analysis based on different RLNs showed that when the RLNs were insufficient (RLNs ≤ 11), the LODDS (HCI: 0.769, AIC: 1,088.731) staging system maintained the best prognostic performance as well as when the RLNs were >11 (LODDS, HCI: 0.835, AIC: 825.692). Then scatter plots were created to evaluate the relationship between LODDS and LNR (Figure 4). The correlation between LNR and LODDS was near, but not completely linear. When patients have different LNR, the LODDS has a one-to-one mapping value for each LNR, and as the LNR increases, the value of LODDS also increases.

TABLE 5

Table 5. Prognostic performance of different lymph node staging systems by stratifying the number of RLNs after eliminating the missing data.

FIGURE 4

Figure 4. Scatter plot presenting the distribution of LODDS and LNR.

Discussion

SBNETs are a heterogeneous group of tumors with a wide spectrum of clinical features and different clinical outcomes (25). Regional LN involvement is one of the main metastatic patterns and the most important prognostic parameter of SBNETs (26). Therefore, accurate staging of LN status can more accurately predict cancer risk and develop postoperative treatment options and surveillance for patients with SBNETs (27). Given this, several alternative staging systems have been proposed to address the shortcomings of the LN classification of the AJCC staging system. Some researchers have evaluated different staging systems with mixed results, supporting the prognostic ability of the LNR staging system (28, 29). In the current study, we first used population-based data to assess the relative discriminative ability of different LN staging systems to predict CSS in patients with SBNETs. We found that LN status stratified by the numbers of RLNs, NLNs, LNR, and LODDS predicted CSS in patients of SBNETs, while the number of PLNs failed. After the categorization based on the data of the current study, the LODDS classification showed a better predictive value.

The importance of LN involvement in determining the prognosis of patients with SBNETs has been demonstrated (29). Our data confirm the important prognostic role of LN status in patients with SBNETs. All LN staging systems stratified patients progressively based on CSS. However, patients with PLNs <12 vs. PLNs ≥ 12 had overlapping survival curves. These findings suggested that the use of PLNs to define pN classification may had some prognostic deficiencies when patients were stratified by specific categorical cutoff values, as reported in a previous study (27).

Several studies had noted that a minimal number of LNs need to be evaluated for accurate stage. For example, Zaidi et al. (30) had advocated that accurate LN staging requires a minimum of eight LNs for examination. In this study, the optimal cut-point of RLNs was 11. A higher number of RLNs was associated with longer CSS. In view of this, we conducted subgroup studies based on different RLNs to analyze the prognostic accuracy of each LN staging system. The PLN staging systems performed relatively poorly when the RLN was insufficient (RLNs ≤ 11), whereas when RLN was >11, the PLN staging systems had a better prognostic value. This had shown that the PLN was significantly correlated with the RLN, especially when the RLN was insufficient, which may cause the missed PLNs, resulting in staging migration (31, 32).

LNR was proposed as a mean to take the total number of LN into account when evaluating LN status to avoid the stage migration phenomenon (33, 34). We noted that LNR was the best-performing model among patients when examined as a continuous variable, regardless of how many LNs were evaluated. However, for patients with either a very low or high LNR, the prognostic discriminatory ability of LNR seemed poor. These findings may be somewhat intuitive, since most clinicians would agree that a patient with one RLN and one NLN has a different prognosis compared with a patient with 12 RLNs and 12 NLNs, even though both patients have an LNR score of 1.

Although LODDS has been previously examined in a large cohort of patients with other cancers, the role of LODDS among patients with SBNETs has not been well-studied. In the present study, we noted that the heterogeneity of LODDS in patients with LNR scores of 0 or 1 indicated that the LODDS has the potential to distinguish patients with the same LNR classification but different prognosis, especially if those LNRs were 0 and 1.

Notably, we also found that LODDS had the highest discrimination in patients who had <12 RLNs. Collectively, these data strongly suggested that LODDS should be the preferred LN staging systems to stratify patients when the RLN was insufficient. Although a more complicated calculation than LNR, LODDS seems to be the most reliable method of patient stratification in N classification.

Several studies indicated that tumor grade may significantly impact the prognosis of SBNETs (35, 36). The current widely used histological grading system for NETs was firstly proposed by ENETS and endorsed by WHO in 2010 (fourth edition) (37, 38). In the SEER database, tumor grade according to the WHO 2010 classification was not available, only tumor differentiation was retrieved. This grading classification based on tumor proliferative activity was assessed by mitotic count and Ki-67 index (38). It has been revised in 2017 for pancreatic neuroendocrine neoplasms, and the fifth edition of the WHO classification of tumors of the digestive system was in press in 2019 (39). The essential change concerned the recognition that well-differentiated NETs may be of high grade, but these neoplasms remained well-differentiated genetically and distinguished from poorly differentiated neuroendocrine carcinoma. Supplementary Table 5 showed the key points of the grading scheme. Nevertheless, the cutoff level of the Ki-67 index differentiating G1 and G2 diseases was still an open issue. Strosberg et al. (36) and Khan et al. (40) suggested that the threshold to classify G1 and G2 should be revised from 2 to 5% in SBNET patients. On the other hand, Cunningham et al. found a cutoff of 1% to be more accurate than a cutoff of 2% in a study of metastatic SBNETs (41). Future studies possibly performed by additional and larger patient cohorts may contribute important data on this topic.

Regional lymphadenopathy was associated with desmoplasia and fibrosis of the mesentery forming a mesenteric mass, which may impact on clinical and staging features in SBNETs (42–44). The updated AJCC 8th edition had incorporated mesenteric masses in staging for midgut NETs, but the prognostic validity of this classification has yet to be validated. In a study of 72 cases of SBNET resections, Gonzalez et al. (45) demonstrated that the presence of a mesenteric mass was significantly associated with lymph nodal and liver metastasis, lymphovascular invasion, T3 or T4 disease, and increased disease progression and death. Malik et al. (46) also found that patients with a mesenteric mass were more likely to have advanced T status and confirmed that mesenteric involvement represented more extensive disease and was also associated with more aggressive treatment. If feasible, surgery was required for the primary tumor, regional LNs, and mesenteric masses. However, large mesenteric masses or encasement of the proximal mesenteric artery and vein may prevent safe resection (47).

This study had several limitations. First, it was a retrospective study based on the SEER database, so there were inevitably some selection biases. The SEER database lacked some clinical information such as vascular invasion, mesenteric mass, and specific locations of LN metastasis. Tumor grade according to WHO classification was not available in the SEER database. However, our findings are credible and widely applicable because the SEER database was based on the US population and clinical practice. Secondly, classification systems had changed in use during the long period and may have potentially exceeded high heterogeneity of the SBNETs. Nevertheless, although age, tumor differentiation, T status, and RLN showed disparities between diagnosis-year cohorts, LODDS showed no significant difference, indicating that the LODDS staging system had excellent applicability even if the period is long. Additionally, the disease-free survival and molecular pathologic characteristics were not available in this study, which may result in a limitation on the survival and LN metastasis analysis.

Conclusion

In conclusion, the current study had demonstrated that LODDS and LNR were more useful and powerful than the PLN classification for prognostic assessment for SBNETs. It should be recommended that LODDS was a better predicator of survival when LN status was stratified as a categorical variable, especially in patients with either a very low or high LNR. As such, LODDS should be considered when assessing the prognosis of patients with SBNETs to allow a more reliable means to stratify patient survival.

Data Availability Statement

Publicly available datasets were analyzed in this study. This data can be found here: https://seer.cancer.gov/data/.

Author Contributions

YY and SJ conceived the concept and designed the study. SJ and LZ collected and analyzed the data. HS and CX prepared the figures. YY and SJ wrote the paper. All authors reviewed and approved the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank SEER for providing open access to the database. All authors indicate no potential conflicts of interests. In addition, SJ wants to express her thanks to YY and sincerely wishes her happiness.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fendo.2020.00402/full#supplementary-material

Abbreviations

LN, lymph node; SBNETs, small bowel neuroendocrine tumors; SEER, Surveillance, Epidemiology and End Results; PLNs, positive lymph nodes; RLNs, resected lymph nodes; NLNs, negative lymph nodes; LNR, lymph node ratio; LODDS, the log odds of positive lymph nodes; CSS, cause-specific survival; AIC, Akaike information criterion; HCI, Harrell consistency index; PLNs, positive lymph nodes; ENETS, the European Neuroendocrine Tumor Society; AJCC, the American Joint Committee on Cancer; ICD-O-3, the International Classification of Disease for Oncology, third edition; WHO, World Health Organization; HR, hazard ratio; CI, confidence interval.

References

1. Niederle MB, Hackl M, Kaserer K, Niederle B. Gastroenteropancreatic neuroendocrine tumours: the current incidence and staging based on the WHO and European Neuroendocrine Tumour Society classification: an analysis based on prospectively collected parameters. Endocr Relat cancer. (2010) 17:909–18. doi: 10.1677/ERC-10-0152

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Lawrence B, Gustafsson BI, Chan A, Svejda B, Kidd M, Modlin IM. The epidemiology of gastroenteropancreatic neuroendocrine tumors. Endocrinol Metab Clin North Am. (2011) 40:1–18. doi: 10.1016/j.ecl.2010.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Sun Y, Lohse C, Smyrk T, Hobday T, Kroneman T, Zhang L. The influence of tumor stage on the prognostic value of Ki-67 index and mitotic count in small intestinal neuroendocrine tumors. Am J Surg Pathol. (2018) 42:247–55. doi: 10.1097/PAS.0000000000000968

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Curran T, Tulin-Silver S, Patel K, Ward S, Schneiderman M, Harpaz N, et al. Prognostic clinicopathologic factors in longitudinally followed patients with metastatic small bowel carcinoid tumors. Pancreas. (2011) 40:1253–7. doi: 10.1097/MPA.0b013e318225483c

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Kim MK, Warner RR, Roayaie S, Harpaz N, Ward SC, Itzkowitz S, et al. Revised staging classification improves outcome prediction for small intestinal neuroendocrine tumors. J Clin Oncol. (2013) 31:3776–81. doi: 10.1200/JCO.2013.51.1477

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Partelli S, Bartsch DK, Capdevila J, Chen J, Knigge U, Niederle B, et al. ENETS consensus guidelines for standard of care in neuroendocrine tumours: surgery for small intestinal and pancreatic neuroendocrine tumours. Neuroendocrinology. (2017) 105:255–65. doi: 10.1159/000464292

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Egner JR. AJCC Cancer Staging Manual. Jama. (2010) 304:1726–7. doi: 10.1001/jama.2010.1525

CrossRef Full Text | Google Scholar

8. La Torre M, Nigri G, Petrucciani N, Cavallini M, Aurello P, Cosenza G, et al. Prognostic assessment of different lymph node staging methods for pancreatic cancer with R0 resection: pN staging, lymph node ratio, log odds of positive lymph nodes. Pancreatology. (2014) 14:289–94. doi: 10.1016/j.pan.2014.05.794

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Parnaby CN, Scott NW, Ramsay G, MacKay C, Samuel L, Murray GI, et al. Prognostic value of lymph node ratio and extramural vascular invasion on survival for patients undergoing curative colon cancer resection. Br J Cancer. (2015) 113:212–9. doi: 10.1038/bjc.2015.211

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Wen J, Yang Y, Liu P, Ye F, Tang H, Huang X, et al. Development and validation of a nomogram for predicting survival on the base of modified lymph node ratio in breast cancer patients. Breast. (2017) 33:14–22. doi: 10.1016/j.breast.2017.01.017

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Tsai J, Bertoni D, Hernandez-Boussard T, Telli ML, Wapnir IL. Lymph node ratio analysis after neoadjuvant chemotherapy is prognostic in hormone receptor-positive and triple-negative breast cancer. Ann Surg Oncol. (2016) 23:3310–6. doi: 10.1245/s10434-016-5319-8

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Ataseven B, Grimm C, Harter P, Prader S, Traut A, Heitz F, et al. Prognostic value of lymph node ratio in patients with advanced epithelial ovarian cancer. Gynecol Oncol. (2014) 135:435–40. doi: 10.1016/j.ygyno.2014.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Huang B, Chen C, Ni M, Mo S, Cai G, Cai S. Log odds of positive lymph nodes is a superior prognostic indicator in stage III rectal cancer patients: a retrospective analysis of 17,632 patients in the SEER database. Int J Surg. (2016) 32:24–30. doi: 10.1016/j.ijsu.2016.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Arslan NC, Sokmen S, Canda AE, Terzi C, Sarioglu S. The prognostic impact of the log odds of positive lymph nodes in colon cancer. Colorectal Dis. (2014) 16:O386–92. doi: 10.1111/codi.12702

PubMed Abstract | CrossRef Full Text | Google Scholar

15. La Torre M, Lorenzon L, Pilozzi E, Barucca V, Cavallini M, Ziparo V, et al. Number of harvested lymph nodes is the main prognostic factor in Stage IIa colorectal cancer patients. J Surg Oncol. (2012) 106:469–74. doi: 10.1002/jso.23101

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Zhou Y-Y, Du X-J, Zhang C-H, Aparicio T, Zananan A, Afchain P, et al. Comparison of three lymph node staging schemes for predicting the outcome in patients with small bowel adenocarcinoma: a population-based cohort and international multicentre cohort study. EBioMedicine. (2019) 41:276–85. doi: 10.1016/j.ebiom.2019.02.043

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Jin S, Wang B, Zhu Y, Dai W, Xu P, Yang C, et al. Log odds could better predict survival in muscle-invasive bladder cancer patients compared with pN and lymph node ratio. J Cancer. (2019) 10:249–56. doi: 10.7150/jca.27399

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Pei JP, Zhang CD, Fan YC, Dai DQ. Comparison of different lymph node staging systems in patients with resectable colorectal cancer. Front Oncol. (2018) 8:671. doi: 10.3389/fonc.2018.00671

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Deng W, Xu T, Wang Y, Xu Y, Yang P, Gomez D, et al. Log odds of positive lymph nodes may predict survival benefit in patients with node-positive non-small cell lung cancer. Lung Cancer. (2018) 122:60–6. doi: 10.1016/j.lungcan.2018.05.016

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Wang J, Li J, Chen R, Lu X. Survival effect of different lymph node staging methods on ovarian cancer: An analysis of 10 878 patients. Cancer Med. (2018) 7:4315–29. doi: 10.1002/cam4.1680

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Cao J, Yuan P, Ma H, Ye P, Wang Y, Yuan X, et al. Log Odds of positive lymph nodes predicts survival in patients after resection for esophageal cancer. Ann Thorac Surg. (2016) 102:424–32. doi: 10.1016/j.athoracsur.2016.03.030

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Camp RL, Dolled-Filhart M, Rimm DL. X-tile: a new bio-informatics tool for biomarker assessment and outcome-based cut-point optimization. Clin Cancer Res. (2004) 10:7252–9. doi: 10.1158/1078-0432.CCR-04-0713

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. (1996) 15:361–87. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Burnhan KP, Anderson DR. Model selection and multi-model inference: a practical Information-theoretic approach[J]. Technometrics. (2002) 45:181. doi: 10.1198/tech.2003.s146

CrossRef Full Text | Google Scholar

25. Pape UF, Jann H, Müller-Nordhorn J, Bockelbrink A, Berndt U, Willich SN, et al. Prognostic relevance of a novel TNM classification system for upper gastroenteropancreatic neuroendocrine tumors. Cancer. (2008) 113:256–65. doi: 10.1002/cncr.23549

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Landry CS, Lin HY, Phan A, Charnsangavej C, Abdalla EK, Aloia T, et al. Resection of at-risk mesenteric lymph nodes is associated with improved survival in patients with small bowel neuroendocrine tumors. World J Surg. (2013) 37:1695–700. doi: 10.1007/s00268-013-1918-8

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Chen L, Song Y, Zhang Y, Chen M, Chen J. Exploration of the exact prognostic significance of lymphatic metastasis in jejunoileal neuroendocrine tumors. Ann Surg Oncol. (2018) 25:2067–74. doi: 10.1245/s10434-018-6511-9

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Wu L, Chen F, Chen S, Wang L. The lymph node ratio optimizes staging in patients with small intestinal neuroendocrine tumors. Neuroendocrinology. (2018) 107:209–17. doi: 10.1159/000491017

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Kim MK, Warner RR, Ward SC, Harpaz N, Roayaie S, Schwartz ME, et al. Prognostic significance of lymph node metastases in small intestinal neuroendocrine tumors. Neuroendocrinology. (2015) 101:58–65. doi: 10.1159/000371807

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Zaidi MY, Lopez-Aguiar AG, Dillhoff M, Beal E, Poultsides G, Makris E, et al. Prognostic role of lymph node positivity and number of lymph nodes needed for accurately staging small-bowel neuroendocrine tumors. JAMA Surg. (2019) 154:134–40. doi: 10.1001/jamasurg.2018.3865

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Le Voyer TE, Sigurdson ER, Hanlon AL, Mayer RJ, Macdonald JS, Catalano PJ, et al. Colon cancer survival is associated with increasing number of lymph nodes analyzed: a secondary survey of intergroup trial INT-0089. J Clin Oncol. (2003) 21:2912–9. doi: 10.1200/JCO.2003.05.062

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Berger AC, Sigurdson ER, LeVoyer T, Hanlon A, Mayer RJ, Macdonald JS, et al. Colon cancer survival is associated with decreasing ratio of metastatic to examined lymph nodes. J Clin Oncol. (2005) 23:8706–12. doi: 10.1200/JCO.2005.02.8852

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Liu H, Deng J, Zhang R, Hao X, Jiao X, Liang H. The RML of lymph node metastasis was superior to the LODDS for evaluating the prognosis of gastric cancer. Int J Surg. (2013) 11:419–24. doi: 10.1016/j.ijsu.2013.03.009

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Inoue K, Nakane Y, Iiyama H, Sato M, Kanbara T, Nakai K, et al. The superiority of ratio-based lymph node staging in gastric carcinoma. Ann Surg Oncol. (2002) 9:27–34. doi: 10.1245/aso.2002.9.1.27

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Landerholm K, Falkmer SE. Ki-67 index and solid growth pattern as prognostic markers in small intestinal neuroendocrine tumors. Neuroendocrinology. (2015) 102:327–34. doi: 10.1159/000434724

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Strosberg JR, Weber JM, Feldman M, Coppola D, Meredith K, Kvols LK. Prognostic validity of the American Joint Committee on Cancer staging classification for midgut neuroendocrine tumors. J Clin Oncol. (2013) 31:420–5. doi: 10.1200/JCO.2012.44.5924

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Rindi G, Klöppel G, Alhman H, Caplin M, Couvelard A, de Herder WW, et al. TNM staging of foregut (neuro)endocrine tumors: a consensus proposal including a grading system. Virchows Arch. (2006) 449:395–401. doi: 10.1007/s00428-006-0250-1

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Bosman FT, Hruban RH, Carneiro F. WHO Classification of Tumours of the Digestive System, 4th ed. WHO Press (2010).

Google Scholar

39. Nagtegaal ID, Odze RD, Klimstra D, Paradis V, Rugge M, Schirmacher P, et al. The 2019 WHO classification of tumours of the digestive system. Histopathology. (2020) 76:182–8. doi: 10.1111/his.13975

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Khan MS, Luong TV, Watkins J, Toumpanakis C, Caplin ME, Meyer T. A comparison of Ki-67 and mitotic count as prognostic markers for metastatic pancreatic and midgut neuroendocrine neoplasms. Br J Cancer. (2013) 108:1838–45. doi: 10.1038/bjc.2013.156

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Cunningham JL, Grimelius L, Sundin A, Agarwal S, Janson ET. Malignant ileocaecal serotonin-producing carcinoid tumours: the presence of a solid growth pattern and/or Ki67 index above 1% identifies patients with a poorer prognosis. Acta Oncol. (2007) 46:747–56. doi: 10.1080/02841860701218659

CrossRef Full Text | Google Scholar

42. Rodríguez Laval V, Pavel M, Steffen IG, Baur AD, Dilz LM, Fischer C, et al. Mesenteric fibrosis in midgut neuroendocrine tumors: functionality and radiological features. Neuroendocrinology. (2018) 106:139–47. doi: 10.1159/000474941

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Laskaratos FM, Walker M, Wilkins D, Tuck A, Ramakrishnan S, Phillips E, et al. Evaluation of clinical prognostic factors and further delineation of the effect of mesenteric fibrosis on survival in advanced midgut neuroendocrine tumours. Neuroendocrinology. (2018) 107:292–304. doi: 10.1159/000493317

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Kasai Y, Mahuron K, Hirose K, Corvera CU, Kim GE, Hope TA, et al. Prognostic impact of a large mesenteric mass >2 cm in ileal neuroendocrine tumors. J Surg Oncol. (2019) 120:1311–7. doi: 10.1002/jso.25727

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Gonzalez RS, Liu EH, Alvarez JR, Ayers GD, Washington MK, Shi C. Should mesenteric tumor deposits be included in staging of well-differentiated small intestine neuroendocrine tumors? Mod Pathol. (2014) 27:1288–95. doi: 10.1038/modpathol.2013.232

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Malik P, Pinto C, Naparst MS, Ward SC, Aronson A, Aalberg JJ, et al. Impact of mesenteric mass in patients with midgut neuroendocrine tumors. Pancreas. (2019) 48:682–5. doi: 10.1097/MPA.0000000000001309

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Nakakura EK. Challenges staging neuroendocrine tumors of the pancreas, jejunum and ileum, and appendix. Ann Surg Oncol. (2018) 25:591–3. doi: 10.1245/s10434-017-6026-9

PubMed Abstract | CrossRef Full Text | Google Scholar