ORIGINAL RESEARCH article

Front. Oncol., 02 September 2020

Sec. Surgical Oncology

Volume 10 - 2020 | https://doi.org/10.3389/fonc.2020.01638

Establishment of Decision Rules and Risk Assessment Model for Preoperative Prediction of Lymph Node Metastasis in Gastric Cancer

  • Department of Gastrointestinal Surgery, The Second Affiliated Hospital of Nanchang University, Nanchang, China

Abstract

Background: Preoperative accurate prediction of lymph node status is especially important for the formulation of treatment plans for patients with gastric cancer (GC). The purpose of this study was to establish decision rules and a risk assessment model for lymph node metastasis (LNM) in GC using preoperative indicators.

Methods: The clinical data of 554 patients who underwent gastrectomy with D2 lymphadenectomy were collected. A 1:1 propensity score matching (PSM) system was used, and the clinical data of the matched 466 patients were further analyzed. The important risk factors for LNM were extracted by the random forest algorithm, and decision rules and nomogram models for LNM were constructed with a classification tree and the “rms” package of R software, respectively.

Results: Tumor size (OR: 2.058; P = 0.000), computed tomography (CT) findings (OR: 1.969; P = 0.001), grade (OR: 0.479; P = 0.000), hemoglobin (Hb) (OR: 1.211; P = 0.005), CEA (OR: 1.111; P = 0.017), and CA19-9 (OR: 1.040; P = 0.033) were independent risk factors for LNM in GC. Tumor size did rank first in the ranking of important factors for LNM in GC and was the first-level segmentation of the two initial branches of the classification tree. The accuracy, sensitivity, specificity, and positive predictive value of the decision rules in diagnosing preoperative LNM in GC were 75.6, 85.7, 73.9, 73.5, and 79.3%, respectively. The accuracy, sensitivity, and specificity of the risk assessment model in predicting preoperative LNM in GC were 79.3, 80.3, and 79.4%, respectively.

Conclusion: Tumor size was the most important factor for evaluating LNM in GC. This decision rules and nomogram model constructed to take into account tumor size, CT findings, grade, hemoglobin, CEA, and CA19-9 effectively predicted the incidence of LNM in preoperative GC.

Introduction

Gastric cancer (GC) is a common malignant tumor of the digestive system with the third highest fatality rate in the world (1). Lymph node metastasis (LNM) is one of the most important prognostic factors for patients with GC (2, 3). D2 gastrectomy is a standard operation for GC, especially advanced gastric cancer (AGC) (4). For early gastric cancer (EGC) patients without LNM, endoscopic mucosal resection, and endoscopic submucosal dissection are widely accepted approaches that can not only preserve gastric function and maintain a high quality of life but also avoid post-operative complications caused by radical gastrectomy (57). The Japanese gastric cancer association (JGCA) proposed that for differentiated T1a EGC without LNM, endoscopic resection or partial resection plus D1/D1+ lymphadenectomy should be considered, but standard D2 lymphadenectomy must be performed in the presence of LNM (8). The National Comprehensive Cancer Network (NCCN) guidelines recommend perioperative chemotherapy or preoperative chemoradiotherapy for lymph node-positive AGC (9). The Japanese gastric cancer treatment guidelines 2018 (5th edition) also proposed that neoadjuvant chemotherapy (NAC) should be required for GC patients who have extensive and large-volume LNM (10). Tsuburaya et al. (11) showed that NAC followed by radical gastrectomy could improve the survival rate of patients with AGC with a large number of metastatic lymph nodes along the major perigastric vessels and/or aorta. Sasako et al. (12) found that radical gastrectomy for GC patients with para-aortic LNM did not improve patient survival times. Moreover, the benefits of NAC have been confirmed in some randomized controlled trials, and most oncologists recommend that NAC should be given to patients with AGC, especially those with LNM, to reduce preoperative TNM staging, thereby improving the rate of radical resection (13, 14). Therefore, accurate preoperative prediction of lymph node status is crucial for the selection of a therapeutic regimen for patients with GC.

Currently, multidetector computed tomography (MDCT), endoscopic ultrasonography (EUS), magnetic resonance imaging (MRI), and positron emission tomography-computed tomography (PET-CT) are used to assess lymph node status in GC patients, but due to their sensitivity, specificity, and accuracy, they are controversial, especially in EGC, so they have limitations as clinical applications (15, 16). Previous studies have also identified novel molecular biomarkers that predict LNM (17, 18), but their high cost and complex techniques have limited their clinical application. Nomograms have been widely used to quantify risk factors for LNM in various cancers, including EGC (1921), but the existing nomograms combine post-operative characteristics, thus reducing their clinical value. In this study, we used only the indicators available before surgery to establish the decision rules and risk assessment model for LNM in GC, thus providing a specific reference value for the formulation of a therapeutic schedule and the evaluation of prognosis in GC patients.

Methods

This study was approved by the Ethics Committee of the Second Affiliated Hospital of Nanchang University and was conducted in accordance with the ethical standards of the Helsinki Declaration. All patients signed informed consent forms.

Study Patients

The clinical data of 554 patients who underwent gastrectomy with D2 lymphadenectomy were collected in our hospital from June 2017 to April 2020. Gastroscope and contrast-enhanced MDCT were used for routine preoperative examination in all patients. In our institute, 64-slice MDCT is used. Two experienced radiologists determined the LNM by looking at axial CT images with a cross-section thickness of 5 mm. Positive was defined as the presence of any identifiable lymph nodes on the CT image. According to the presence or absence of LNM in the surgical pathological specimens, the patients were divided into an LNM group and a without LNM group. There were 312 cases in the LNM group and 242 cases in the without LNM group. According to age, sex and BMI, 1:1 matching was performed, and they clinical data of 466 patients who obtained matching were analyzed. The serum tumor marker, albumin, fibrinogen, hemoglobin (Hb), neutrophil, platelet, and lymphocyte levels were extracted at the first admission. The clinicopathological characteristics of the patients, including age, sex, BMI, tumor size, tumor location, and histological grade, were recorded in our study. Tumor size was measured according to the maximum diameter of the tumor. Laparoscopic-assisted radical gastrectomy or open radical gastrectomy was performed. Distal gastrectomy or total gastrectomy was conducted depending on the location of the tumor. The range of lymph node dissection was D2 lymphadenectomy.

Inclusion and Exclusion Criteria

The inclusion criteria of this study were as follows: (1) patients who underwent gastrectomy with D2 lymphadenectomy; (2) no neoadjuvant radiotherapy or chemotherapy was used before surgery; and (3) there was a complete pathological report after surgery. We excluded patients with bleeding, hyperthyroidism or hypothyroidism, acute infection, no radical resection, gastric stump cancer, distant metastasis, combined with other tumors, chronic inflammation, autoimmune diseases, hematological diseases, and incomplete data.

Statistical Analysis

The single-sample Kolmogorov-Smirnov test was used to test the normality of the data. If the quantitative data had a normal distribution, they are described as the mean ± standard deviation; otherwise, data are described as the median and quartile interval. Categorical variables are expressed as rates with 95% confidence intervals (CIs). Propensity score matching (PSM) was used to balance the covariates and number of cases between the groups. Univariate analysis between the two groups was performed using the Mann-Whitney U-test. Significant indicators identified in the univariate analysis were analyzed by multivariate conditional logistic regression. The random forest algorithm was used to extract the important risk factors for LNM in GC, and the mean decrease Gini (MDG) was used to rank the important indicators. The MDG quantifies which indicator contributes the most to classification accuracy (22). The cut-off value for important risk factors was determined by the receiver operating characteristic (ROC) curve analysis. The decision rules and nomogram model for LNM were constructed by classification tree and the “rms” package in R software, respectively. A classification tree is a non-linear discriminant method that uses a set of independent variables to gradually decompose the sample into smaller subgroups. This procedure iterates on each branch of the tree, and selects the independent variable with the strongest correlation with the dependent variable according to specific criteria (23). The decision rules provide specific information about risk factors based on rule induction. The ROC curve was used to evaluate the accuracy, sensitivity, and specificity of the model to predict LNM of GC. A two-sided P < 0.05 was considered statistically significant. Data were analyzed using SPSS 22.0 for Windows (SPSS Inc., Chicago, IL, USA) and R (version x64 3.6.1), including non-random, rpart, rpart.plot, randomForest, and rms packages.

Results

Clinical Characteristics of GC Patients

There were 233 cases in the LNM group, including 160 men and 73 women. Their median age was 63 years, their median BMI was 21.63 kg/m2, and their median tumor size was 4.5 cm. The positive CT findings were 136 cases and negative CT findings were 97 cases. There were 233 cases in the without LNM group, including 174 men and 59 women. Their median age was 62 years, their median BMI was 21.52 kg/m2, and their median tumor size was 3.0 cm. The positive CT findings were 54 cases and negative CT findings were 179 cases. There was no statistically significant difference in age (P = 0.44), sex (P = 0.15), BMI (P = 0.332), or tumor location (P = 0.146) between the two groups (Table 1).

Table 1

FactorsLNM (n = 233)Without LNM (n = 233)P-value
Age (years)63 (54, 69)62 (54, 68)0.44
Sex (n)0.15
Male160174
Female7359
BMI (kg/m2)21.63 (19.15, 23.36)21.51 (19.81, 23.51)0.332
Tumor size (cm)4.5 (3.5, 6.5)3.0 (1.8, 4.0)0.000
Tumor location (n)0.146
Fundus of stomach3933
Body of stomach6757
Antrum of stomach127143
Histological grade (n)0.000
Low12288
Low-medium6670
Medium4569
Medium-high06
CT findings (n)0.000
Node positive13654
Node negative97179
Neutrophil (109/L)3.56 (2.77, 4.71)3.55 (2.80, 4.50)0.862
Lymphocyte (109/L)1.48 (1.13, 1.83)1.53 (1.16, 1.84)0.536
Hemoglobin (g/L)122 (107, 134)131 (116.5, 143.5)0.000
Platelet (109/L)234 (183, 288.5)214 (172, 266.5)0.038
Albumin (g/L)39.03 ± 3.9140.43 ± 3.570.000
Fibrinogen (g/L)3.10 (2.50, 3.83)2.80 (2.41, 3.44)0.014
CEA (ng/ml)2.81 (1.70, 5.53)2.55 (1.70, 3.92)0.029
CA19-9 (U/ml)14.57 (8.88, 29.49)12.09 (8.24, 18.14)0.000
CA125 (U/ml)10.03 (6.80, 14.75)8.80 (6.00, 12.98)0.003
CA72-4 (IU/ml)3.28 (1.44, 5.57)2.06 (1.14, 4.63)0.008

General characteristics of patients with GC and comparison of the relevant factors between the two groups.

Univariate and Multivariate Analysis

Univariate analysis found that the factors associated with LNM in GC were tumor size (P = 0.000), histological grade (P = 0.000), CT findings (P = 0.000), Hb (P = 0.001), platelets (P = 0.038), albumin (P = 0.000), fibrinogen (P = 0.014), CEA (P = 0.029), CA19-9 (P = 0.000), CA125 (P = 0.003), and CA72-4 (P = 0.008) (Table 1). Multivariate analysis showed that the independent risk factors for LNM in GC were tumor size (OR: 2.058; P = 0.000), CT findings (OR: 1.969; P = 0.000), grade (OR: 0.479; P = 0.000), Hb (OR: 1.211; P = 0.005), CEA (OR: 1.111; P = 0.017), and CA19-9 (OR: 1.040; P = 0.033) (Table 2).

Table 2

Risk factorsBSEWalsP-valueOR (95%CI)
Tumor size1.1640.22626.6160.0002.058 (3.203, 4.985)
CT findings1.1200.22624.6480.0001.969 (3.064, 4.768)
Grade−0.4780.13113.2670.0000.479 (0.620, 0.802)
Hemoglobin0.6280.2237.9500.0051.211 (1.874, 2.900)
CEA0.5810.2435.7270.0171.111 (1.788, 2.879)
CA19-90.4990.2354.5280.0331.040 (1.648, 2.610)

Risk factors of LNM in GC for multivariate conditional logistic regression analysis.

ROC Curve

Figure 1 shows the correlations between LNM of GC and tumor size, CT findings, grade, Hb, CEA, and CA19-9. According to the ROC curve evaluation, the optimal cut-off values for tumor size, Hb, CEA, and CA19-9 were 3.1 cm, 122.5 g/L, 4.285 ng/ml, and 19.19 U/ml, respectively. The area under the curve (AUC) for tumor size was 0.745, the 95% CI was 0.701–0.789, the sensitivity was 78.5%, and the specificity was 57.9%. The AUC for CT findings was 0.676, the 95% CI was 0.627–0.725, the sensitivity was 58.4%, and the specificity was 76.8%. The AUC for grade was 0.592, the 95% CI was 0.541–0.643, the sensitivity was 62.2%, and the specificity was 52.4%. The AUC for CEA was 0.558, the 95% CI was 0.506–0.611, the sensitivity was 36.9%, and the specificity was 78.1%. The AUC for CA19-9 was 0.593, the 95% CI was 0.541–0.645, the sensitivity was 40.8%, and the specificity was 78.1% (Table 3). These results show that the accuracy and sensitivity of individual indicators for determining LNM were relatively low.

Figure 1

Table 3

FactorsAUC95% CISensitivity (%)Specificity (%)Cut-off valueP-value
Tumor size0.745(0.701, 0.789)78.557.93.10.000
CT finding0.676(0.627, 0.725)58.476.80.000
Grade0.592(0.541, 0.643)62.252.40.001
Hemoglobin0.616(0.565, 0.667)68.252.4122.50.000
CEA0.558(0.506, 0.611)36.978.14.2850.029
CA19-90.593(0.541, 0.645)40.878.119.190.001

Independent risk factors for LNM in GC of ROC curve.

Random Forest Algorithm to Extract Important Risk Factors for LNM in GC

Factors related to LNM in GC, including tumor size, grade, CT findings, Hb, platelet, albumin, fibrinogen, CEA, CA19-9, CA125, and CA72-4, were analyzed by random forest algorithm, and the importance of variables was ranked (Figure 2). A larger mean decrease in Gini indicated the variable was more important. Tumor size ranked first and was followed by CT findings, grade, Hb, CEA, and CA19-9.

Figure 2

Classification Tree Analysis to Obtain Decision Rules for the Preoperative Diagnosis of LNM in GC

The independent risk factors for LNM in GC, including tumor size, CT findings, grade, Hb, CEA, and CA19-9, were analyzed by classification tree, and the decision rules affecting LNM in GC were obtained (Figure 3). Through the classification tree procedure, all variables were involved in the construction of decision rules for LNM in GC. Tumor size was the most important determinant because it was the first-level split of the two initial branches of the classification tree. CT findings were the most important determinant in the second-level split. The accuracy, sensitivity, specificity, positive predictive value and negative predictive value of the decision rules for diagnosing LNM in GC were 75.6, 85.7, 73.9, 73.5, and 79.3%, respectively, indicating that the decision rules were effective in the preoperative diagnosis of LNM in GC.

Figure 3

Nomogram Model for Risk Assessment of Preoperative Diagnosis of LNM in GC

A logistic regression model that combined tumor size, CT findings, grade, Hb, CEA, and CA19-9 was constructed using the “rms” package in R language, and the C statistic of its evaluation was 0.790, indicating that the predictive model had high accuracy. Next, the plotting function was constructed, and the nomogram was plotted (Figure 4). The score for a tumor size ≥3.1 cm was 88 points, the score for a tumor size <3.1 cm was 0 points; the score for positive CT findings was 85 points; the score for negative CT findings was 0 points; the score for low differentiation was 100 points; the score for low-medium differentiation was 67 points; the score for medium differentiation was 33 points; the score for medium-high differentiation was 0 points; the score for hemoglobin ≥122.5 g/L was 0 points, the score for hemoglobin <122.5 g/L was 41 points; the score for CEA ≥4.285 ng/ml was 39 points, the score for CEA <4.285 ng/ml was 0 points; the score for CA19-9 ≥19.19 U/ml was 31 points, and the score for CA19-9 <19.19 U/ml was 0 points. The total score was 384, indicating that the probability of LNM in preoperative GC was 90–95%, indicating the risk of LNM in preoperative GC can be predicted based on the total points (Table 4). The AUC of the combined factors was 0.793, and the sensitivity and specificity were 80.3 and 79.4%, respectively (Figure 5), suggesting that this predictive model effectively predicted the risk of LNM in preoperative GC.

Figure 4

Table 4

Total pointsRisk of LNM
<32<10%
32–9311–20%
94–13421–30%
135–16831–40%
169–19841–50%
199–22951–60%
230–26261–70%
263–30371–80%
304–36481–90%
365–42191–95%
>421>95%

Relationship between total points and risk of LNM in GC.

Figure 5

Discussion

LNM is an important indicator when evaluating the prognosis of GC (8, 24, 25). Radical surgery and adjuvant therapy are standard treatments for GC (4, 26). Post-operative prognosis is mainly assessed based on the TNM staging system of the American Joint Committee on Cancer (AJCC) (8, 9). However, this system only reflects the characteristics of the cancer itself, and it is difficult to obtain accurate data before surgery. In addition, accurate judgment of preoperative lymph node status is crucial for selecting treatment options for GC. Therefore, an accurate preoperative diagnosis of LNM is of great significance for the formulation of treatment plans and the evaluation of prognosis of patients with GC. After a series of analyses, we only used the indicators available before surgery, including tumor size, CT findings, grade, Hb, CEA, and CA19-9, to establish the decision rules and risk assessment model for LNM in GC.

The NCCN guidelines recommend the use of CT scan, EUS, PET/CT, and MRI to diagnose LNM in patients with GC before surgery (9, 27). With the continuous development of technology, MDCT has become the most commonly used technique for preoperative staging of GC. It provides specific value in the evaluation of infiltration depth, ascites, and distant metastasis but has limitations in the diagnosis of LNM (28, 29). Although improved CT analysis techniques, including those that take into account size and/or imaging patterns (enhancement, necrosis, shape, and fat content), were used to assess LNM, the overall accuracy was also approximately 60–80% (30, 31). One reason for this result is that small lymph nodes may contain metastatic tumors, while large lymph nodes may be caused by inflammation (32). The previous MDCT diagnostic criteria for LNM in GC were as follows (31, 3335): (1) short axis >8 mm; (2) short axis (perigastric) >6 mm and short axis (extraperitoneal) >8 mm; (3) significantly enhanced long axis >8 mm; and (4) central necrosis or aggregation (three nodes or more), regardless of size. Moreover, some studies have suggested that any identifiable lymph nodes were positive (15, 36). The criteria used in this study to judge LNM were consistent with the latter. The results of LNM differed according to different criteria. Ohashi et al. found that the accuracy, sensitivity, and specificity of CE-MDCT in the diagnosis of LNM were 67.9, 79.1, and 50.0%, respectively (37). Hasegawa et al. (31) showed that the specificity of CT in the diagnosis of LNM was 96.8%, while the sensitivity was only 46.2%. A clinical trial conducted in recent years indicated that the sensitivity and positive predictive value of CT scans in the diagnosis of LNM were 62.5 and 77.7%, respectively (38). In this study, the accuracy, sensitivity, and specificity of CE-MDCT in the diagnosis of LNM were 67.6, 58.4, and 76.8%, respectively. Meanwhile, Lee et al. (39) showed that MDCT had some limitations in the determination of GC restaging after short-term NAC, and proposed that CT volumetry for primary gastric tumor can accurately predict the pathological reaction after NAC in AGC. Lundsgaard et al. (40) indicated that CT perfusion assessment of gastroesophageal junction carcinoma and GC showed moderate sensitivity and specificity in preoperative chemotherapy response, so it was not recommended for clinical decision purposes. Park et al. (41) demonstrated that after NAC for locally AGC, the accuracy of CT diagnosis of T and N staging were 57 and 37%, respectively. A randomized phase II study found that CT restaging after NAC was considered inaccurate and unreliable for GC (42). Recent studies (43) showed that CT radiomics—the high-throughput extraction of quantitative imaging features characterizing the spatial relationships and consistency of signal intensities—may make it possible to predict the adverse reactions after NAC for locally AGC.

In addition, the accuracy of MDCT in diagnosing LNM can also be influenced by the clinical experience of the radiologists. Therefore, there is a clinical need for objective indicators that predict LNM. Tumor size can, when based on the maximum diameter of the tumor, be measured using imaging techniques or endoscopy. Choi et al. (44) found that the tumor size measured by gastroscopy was very consistent with the tumor size determined by pathology, and in 90% of the patients, the absolute measurement difference was <0.6 cm. Zhao et al. (45) showed that there was no statistically significant difference between MSCT measurement of the maximum diameter and post-operative measurement results in GC, indicating that CT measurement of tumor maximum diameter can be used as a reliable method. Vilgrain et al. (46) demonstrated that the tumor size measured by sonography was correlated well with findings at surgery, and tumor length as seen on CT correlated with the macroscopic appearance of the lesion. In order to obtain the most accurate tumor size, it was measured from post-operative specimens in this study. Histological grade can be obtained by biopsy under a gastroscope before surgery. Tumor markers and hemoglobin are routine preoperative test items. In this study, multivariate analysis showed that tumor size, CT findings, grade, Hb, CEA, and CA19-9 were independent risk factors for LNM in GC, and tumor size was found to be the most important factor for the evaluation of LNM of GC through a random forest algorithm and classification tree. Previous studies have shown that tumor size is significantly associated with LNM in GC (4749). A larger tumor will have a higher risk of LNM (50, 51). Habermann et al. (52) and Isomoto et al. (53) proposed that tumor size, depth of invasion, and differentiation degree are important clinicopathological factors that should be considered to accurately determine the risk of LNM in GC. In EGC, tumor size, depth of invasion, lymphatic infiltration, gross type, and histologic type were related to LNM (5456). Nakagawa et al. (57) retrospectively analyzed the clinical data of 1,042 EGC patients undergoing gastrectomy and lymph node dissection and found that age, tumor size, the depth of invasion, the presence of ulcers, and positive CT findings were high-risk predictors of LNM. Lin et al. (58) retrospectively analyzed the clinical data of 1,460 EGC patients in multiple centers and found that women, tumors larger than 20 mm, submucosal invasion, and undifferentiated tumor histological types were independent risk factors for LNM. Xu et al. (59) believed that combining MDCT, preoperative histological type and tumor size was an effective method to predict LNM in GC.

Tumor markers are also related to LNM in tumors. Preoperative elevated serum CEA and CA-153 levels are risk factors for axillary LNM in patients with breast cancer (60). CA724, CA199, and CEA are significantly correlated with LNM in GC and can be used as reliable biomarkers for predicting LNM in GC (6163). Duraker and Celik (62) showed that CEA has value in the diagnosis of LNM in GC. Previous studies demonstrated that elevated levels of CA125 are associated with GC peritoneal dissemination and a poor prognosis (64, 65). In this study, univariate analysis found that CEA, CA19-9, CA125, and CA72-4 were associated with LNM in GC, and multivariate analysis showed that CEA and CA19-9 were independent risk factors for LNM in GC. Hb is an indicator of anemia. Hypoxia caused by anemia may accelerate tumor angiogenesis to promote tumor invasion and progression (66, 67). In addition, hypoxia also increases the expression of hypoxia-inducing factor 1 (HIF-1), which regulates gene products that can promote tumor invasion and metastasis (68). Lysyl oxidase (LOX) is also a hypoxic gene that is involved in the migration and invasion of cancer cells (69). Hypoxia may increase the potential for proliferation and metastasis by inducing proteomic and genomic changes (66). Previous studies found that anemia was related to a poor prognosis in patients with GC (70, 71). Shen et al. (72) showed that preoperative anemia was significantly correlated with large tumors, deep infiltration, and high tumor stage. In this study, we found that low Hb levels were a high-risk predictor of LNM in GC. Previous studies have also identified new molecular biomarkers for predicting LNM and successfully identified several markers for predicting LNM through proteomics and histological analysis, such as miR-1207-5p, LYVE-1 antibody, 14-3-3β, profilin-1 protein, inhibitors of cytokine signaling 3, p53 gene, Ki67, EGFR, and miR-1207-5p (17, 18, 7376). However, due to their high cost, complicated technology, and difficult preoperative detection, their clinical applications are limited.

A phase II trial demonstrated that NAC can improve the R0 resection rates in stage II or III GC patients with LNM, and showed that the 3-year relapse-free survival was 44.9%, and the 3-year overall survival rate was 48.0% (77). Tsuburaya et al. (11) indicated that for GC with widespread LNM, NAC followed by surgery can improve patient survival time. Sym et al. (78) found that for GC with para-aortic LNM, NAC can significantly improve the surgical resection rate. Meanwhile, a randomized controlled trial showed that for operable GC patients, including 93 patients with LNM, NAC can reduce tumor size and stage, and significantly improve progression-free survival and overall survival (13). Inoue et al. (79) found that in the seven GC patients with T4N1M0 tumor who underwent R0 resection after NAC, no patient had local relapse during the long-term follow-up. Therefore, they believed that NAC can improve the prognosis of GC patients with T4N1M0 tumor.

Nomograms have been widely used to quantify the risk factors for LNM in various cancers (19, 20). Recently, nomograms were also developed to predict LNM in GC (21, 80). However, some of the factors they used were available only after surgery, reducing their clinical value. In this study, we obtained six preoperative factors through analyses of routine examinations and developed a decision rules and risk assessment model for LNM in GC. The accuracy, sensitivity, and specificity of the decision rules in diagnosing preoperative LNM in GC were 75.6, 85.7, and 73.9%, respectively. The accuracy, sensitivity, and specificity of the risk assessment model in predicting preoperative LNM in GC were 79.3, 80.3, and 79.4%, respectively. These results indicate that these factors can effectively predict the incidence of LNM in preoperative GC. This study has some limitations. First, this was a single-center retrospective study. Second, the sample size was not large. Third, the established decision rules and nomogram models have clinical value in the diagnosis of LNM in GC but are limited in the ability to accurately predict the number of LNMs. Finally, tumor size was measured by post-operative specimens. Therefore, multicenter large-scale prospective randomized controlled trials are necessary.

Conclusion

In this study, we used only the indicators available before surgery to establish decision rules and a risk assessment model for LNM in GC and found that tumor size, CT findings, grade, Hb, CEA, and CA19-9 were independent risk factors for LNM in GC. Further analysis showed that tumor size was the most important factor in evaluating LNM in GC. The decision rules and nomogram model constructed to use tumor size, CT findings, grade, Hb, CEA, and CA19-9 cam effectively predict the incidence of LNM in preoperative GC and thereby provide a reference value for the formulation of treatment plans and the evaluation of prognosis in patients with GC.

Statements

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Second Affiliated Hospital of Nanchang University. The patients/participants provided their written informed consent to participate in this study.

Author contributions

CHua designed the study, analyzed the data, and wrote the manuscript with contributions from all authors. CHua, CHu, JZ, and WZ collected the clinical data. JH and ZZ provided critical comments for this paper. All authors read and approved the final version of the paper.

Funding

This present study was supported by the National Natural Science Foundation of China (Grant No. 81560389) and Key Research and Development Program of Jiangxi Province (Grant No. 20181BBG70015).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  • 1.

    BrayFFerlayJSoerjomataramISiegelRLTorreLAJemalA. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. (2018) 68:394424. 10.3322/caac.21492

  • 2.

    DickenBJBigamDLCassCMackeyJRJoyAAHamiltonSM. Gastric adenocarcinoma: review and considerations for future directions. Ann Surg. (2005) 241:2739. 10.1097/01.sla.0000149300.28588.23

  • 3.

    SaitoHFukumotoYOsakiTFukudaKTatebeSTsujitaniSet al. Prognostic significance of level and number of lymph node metastases in patients with gastric cancer. Ann Surg Oncol. (2007) 14:168893. 10.1245/s10434-006-9314-3

  • 4.

    SongunIPutterHKranenbargEMSasakoMvande Velde CJ. Surgical treatment of gastric cancer: 15-year follow-up results of the randomised nationwide Dutch D1D2 trial. Lancet Oncol. (2010) 11:43949. 10.1016/s1470-2045(10)70070-x

  • 5.

    SaragoniLMorgagniPGardiniAMarfisiCVittimbergaGGarceaDet al. Early gastric cancer: diagnosis, staging, and clinical impact. Evaluation of 530 patients. New elements for an updated definition and classification. Gastric Cancer. (2013) 16:54954. 10.1007/s10120-013-0233-2

  • 6.

    BaptistaVSinghAWassefW. Early gastric cancer: an update on endoscopic management. Curr Opin Gastroenterol. (2012) 28:62935. 10.1097/MOG.0b013e328358e5b5

  • 7.

    LeeHHYooHMSongKYJeonHMParkCH. Risk of limited lymph node dissection in patients with clinically early gastric cancer: indications of extended lymph node dissection for early gastric cancer. Ann Surg Oncol. (2013) 20:353440. 10.1245/s10434-013-3124-1

  • 8.

    Japanese Gastric Cancer Association. Japanese Gastric Cancer Treatment Guidelines 2014 (ver. 4). Gastric Cancer. (2017) 20:1-19. 10.1007/s10120-016-0622-4

  • 9.

    AjaniJAD'AmicoTAAlmhannaKBentremDJChaoJDasPet al. Gastric Cancer, version 3.2016, NCCN clinical practice guidelines in oncology. J Natl Compr Cancer Netw. (2016) 14:1286312. 10.6004/jnccn.2016.0137

  • 10.

    Japanese Gastric Cancer Association. Japanese gastric cancer treatment guidelines 2018 (5th edition). Gastric Cancer. (2020). 10.1007/s10120-020-01042-y. [Epub ahead of print].

  • 11.

    TsuburayaAMizusawaJTanakaYFukushimaNNashimotoASasakoM. Neoadjuvant chemotherapy with S-1 and cisplatin followed by D2 gastrectomy with para-aortic lymph node dissection for gastric cancer with extensive lymph node metastasis. Br J Surg. (2014) 101:65360. 10.1002/bjs.9484

  • 12.

    SasakoMSanoTYamamotoSKurokawaYNashimotoAKuritaAet al. D2 lymphadenectomy alone or with para-aortic nodal dissection for gastric cancer. New Engl J Med. (2008) 359:45362. 10.1056/NEJMoa0707035

  • 13.

    CunninghamDAllumWHStenningSPThompsonJNVande Velde CJNicolsonMet al. Perioperative chemotherapy versus surgery alone for resectable gastroesophageal cancer. New Engl J Med. (2006) 355:1120. 10.1056/NEJMoa055531

  • 14.

    YchouMBoigeVPignonJPConroyTBouchéOLebretonGet al. Perioperative chemotherapy compared with surgery alone for resectable gastroesophageal adenocarcinoma: an FNCLCC and FFCD multicenter phase III trial. J Clin Oncol. (2011) 29:171521. 10.1200/jco.2010.33.0597

  • 15.

    KweeRMKweeTC. Imaging in assessing lymph node status in gastric cancer. Gastric Cancer. (2009) 12:622. 10.1007/s10120-008-0492-5

  • 16.

    SeevaratnamRCardosoRMcGregorCLourencoLMaharASutradharRet al. How useful is preoperative imaging for tumor, node, metastasis (TNM) staging of gastric cancer? A meta-analysis. Gastric Cancer. (2012) 15(Suppl. 1):S318. 10.1007/s10120-011-0069-6

  • 17.

    HuangKHLanYTFangWLChenJHLoSSLiAFet al. The correlation between miRNA and lymph node metastasis in gastric cancer. Biomed Res Int. (2015) 2015:543163. 10.1155/2015/543163

  • 18.

    LiWYeFWangDSunXTongWLianGet al. Protein predictive signatures for lymph node metastasis of gastric cancer. Int J Cancer. (2013) 132:18519. 10.1002/ijc.27864

  • 19.

    DingemansSAde RooijPDvan der Vuurst de VriesRMBudelLMContantC. M, van der Pool AE. Validation of six nomograms for predicting non-sentinel lymph node metastases in a dutch breast cancer population. Ann Surg Oncol. (2016) 23:47781. 10.1245/s10434-015-4858-8

  • 20.

    BrigantiALarcherAAbdollahFCapitanioUGallinaASuardiNet al. Updated nomogram predicting lymph node invasion in patients with prostate cancer undergoing extended pelvic lymph node dissection: the essential importance of percentage of positive cores. Eur Urol. (2012) 61:4807. 10.1016/j.eururo.2011.10.044

  • 21.

    ZhengZZhangYZhangLLiZWuXLiuYet al. A nomogram for predicting the likelihood of lymph node metastasis in early gastric patients. BMC Cancer. (2016) 16:92. 10.1186/s12885-016-2132-5

  • 22.

    BreimanL. Random forests. Mach Learn. (2001) 45:532. 10.1023/A:1010933404324

  • 23.

    TüreMKurtIKürümT. Analysis of intervariable relationships between major risk factors in the development of coronary artery disease: a classification tree approach. Anadolu Kardiyol Derg. (2007) 7:1405.

  • 24.

    PyoJHShinCMLeeHMinBHLeeJHKimSMet al. A risk-prediction model based on lymph-node metastasis for incorporation into a treatment algorithm for signet ring cell-type intramucosal gastric cancer. Ann Surg. (2016) 264:103843. 10.1097/sla.0000000000001602

  • 25.

    DattaJMcMillanMTEckerBLKarakousisGCMamtaniRPlastarasJPet al. Implications of lymph node staging on selection of adjuvant therapy for gastric cancer in the united states: a propensity score-matched analysis. Ann Surg. (2016) 263:298305. 10.1097/sla.0000000000001360

  • 26.

    Van CutsemESagaertXTopalBHaustermansKPrenenH. Gastric cancer. The Lancet. (2016) 388:265464. 10.1016/S0140-6736(16)30354-3

  • 27.

    KweeRMKweeTC. Imaging in local staging of gastric cancer: a systematic review. J Clin Oncol. (2007) 25:210716. 10.1200/jco.2006.09.5224

  • 28.

    LeeSLLeeHHKuYMJeonHM. Usefulness of two-dimensional values measured using preoperative multidetector computed tomography in predicting lymph node metastasis of gastric cancer. Ann Surg Oncol. (2015) 22(Suppl. 3):S78693. 10.1245/s10434-015-4621-1

  • 29.

    MakinoTFujiwaraYTakiguchiSTsuboyamaTKimTNushijimaYet al. Preoperative T staging of gastric cancer by multi-detector row computed tomography. Surgery. (2011) 149:6729. 10.1016/j.surg.2010.12.003

  • 30.

    KawaguchiTKomatsuSIchikawaDOkamotoKShiozakiAFujiwaraHet al. Nodal counts on MDCT as a surrogate marker for surgical curability in gastric cancer. Ann Surg Oncol. (2012) 19:246570. 10.1245/s10434-012-2283-9

  • 31.

    HasegawaSYoshikawaTShiraiJFujikawaHChoHDoiuchiTet al. A prospective validation study to diagnose serosal invasion nodal metastases of gastric cancer by multidetector-row CT. Ann Surg Oncol. (2013) 20:201622. 10.1245/s10434-012-2817-1

  • 32.

    TatsumiYTanigawaNNishimuraHNomuraEMabuchiHMatsukiMet al. Preoperative diagnosis of lymph node metastases in gastric cancer by magnetic resonance imaging with ferumoxtran-10. Gastric Cancer. (2006) 9:1208. 10.1007/s10120-006-0365-8

  • 33.

    KimHJKimAYOhSTKimJSKimKWKimPNet al. Gastric cancer staging at multi-detector row CT gastrography: comparison of transverse and volumetric CT scanning. Radiology. (2005) 236:87985. 10.1148/radiol.2363041101

  • 34.

    ChenCYHsuJSWuDCKangWYHsiehJSJawTSet al. Gastric cancer: preoperative local staging with 3D multi-detector row CT–correlation with surgical and histopathologic results. Radiology. (2007) 242:47282. 10.1148/radiol.2422051557

  • 35.

    YanCZhuZGYanMZhangHPanZLChenJet al. Value of multidetector-row computed tomography in the preoperative T and N staging of gastric carcinoma: a large-scale Chinese study. J Surg Oncol. (2009) 100:20514. 10.1002/jso.21316

  • 36.

    YangDMKimHCJinWRyuCWKangJHParkCHet al. 64 multidetector-row computed tomography for preoperative evaluation of gastric cancer: histological correlation. J Comput Assist Tomogr. (2007) 31:98103. 10.1097/01.rct.0000234072.16209.ab

  • 37.

    OhashiMMoritaSFukagawaTWadaTKushimaROnayaHet al. Evaluation of 64-channel contrast-enhanced multi-detector row computed tomography for preoperative N staging in cT2-4 gastric carcinoma. World J Surg. (2016) 40:16571. 10.1007/s00268-015-3318-8

  • 38.

    FukagawaTKataiHMizusawaJNakamuraKSanoTTerashimaMet al. A prospective multi-institutional validity study to evaluate the accuracy of clinical diagnosis of pathological stage III gastric cancer (JCOG1302A). Gastric Cancer. (2018) 21:6873. 10.1007/s10120-017-0701-1

  • 39.

    LeeSMKimSHLeeJMImSABangYJKimWHet al. Usefulness of CT volumetry for primary gastric lesions in predicting pathologic response to neoadjuvant chemotherapy in advanced gastric cancer. Abdom Imaging. (2009) 34:43040. 10.1007/s00261-008-9420-8

  • 40.

    Lundsgaard HansenMFallentinELauridsenCLawIFederspielBBæksgaardLet al. Computed tomography (CT) perfusion as an early predictive marker for treatment response to neoadjuvant chemotherapy in gastroesophageal junction cancer and gastric cancer–a prospective study. PLoS ONE. (2014) 9:e97605. 10.1371/journal.pone.0097605

  • 41.

    ParkSRLeeJSKimCGKimHKKookMCKimYWet al. Endoscopic ultrasound and computed tomography in restaging and predicting prognosis after neoadjuvant chemotherapy in patients with locally advanced gastric cancer. Cancer. (2008) 112:236876. 10.1002/cncr.23483

  • 42.

    YoshikawaTTanabeKNishikawaKItoYMatsuiTKimuraYet al. Accuracy of CT staging of locally advanced gastric cancer after neoadjuvant chemotherapy: cohort evaluation within a randomized phase II study. Ann Surg Oncol. (2014) 21(Suppl. 3):S3859. 10.1245/s10434-014-3615-8

  • 43.

    LiZZhangDDaiYDongJWuLLiYet al. Computed tomography-based radiomics for prediction of neoadjuvant chemotherapy outcomes in locally advanced gastric cancer: a pilot study. Chin J Cancer Res. (2018) 30:40614. 10.21147/j.issn.1000-9604.2018.04.03

  • 44.

    ChoiJKimSGImJPKimJSJungHC. Endoscopic estimation of tumor size in early gastric cancer. Digest Dis Sci. (2013) 58:232936. 10.1007/s10620-013-2644-7

  • 45.

    ZhaoQLiYHuZTanBYangPTianY. Value of the preoperative TNM staging and the longest tumor diameter measurement of gastric cancer evaluated by MSCT. Zhonghua Wei Chang Wai Ke Za Zhi. (2015) 18:22731.

  • 46.

    VilgrainVMompointDPalazzoLMenuYGayetBOllierPet al. Staging of esophageal carcinoma: comparison of results with endoscopic sonography CT. AJR A J Roentgenol. (1990) 155:27781. 10.2214/ajr.155.2.2115251

  • 47.

    WangXWanFPanJYuGZChenYWangJJ. Tumor size: a non-neglectable independent prognostic factor for gastric cancer. J Surg Oncol. (2008) 97:23640. 10.1002/jso.20951

  • 48.

    AdachiYOshiroTMoriMMaeharaYSugimachiK. Tumor size as a simple prognostic indicator for gastric carcinoma. Ann Surg Oncol. (1997) 4:13740. 10.1007/bf02303796

  • 49.

    LiuXXuYLongZZhuHWangY. Prognostic significance of tumor size in T3 gastric cancer. Ann Surg Oncol. (2009) 16:187582. 10.1245/s10434-009-0449-x

  • 50.

    NovotnyARSchuhmacherC. Predicting lymph node metastases in early gastric cancer: radical resection or organ-sparing therapy?Gastric Cancer. (2008) 11:1313. 10.1007/s10120-008-0479-2

  • 51.

    OhashiSOkamuraSUranoFMaedaM. Clinicopathological variables associated with lymph node metastasis in submucosal invasive gastric cancer. Gastric Cancer. (2007) 10:24150. 10.1007/s10120-007-0442-7

  • 52.

    HabermannCRWeissFRieckenRHonarpishehHBohnackerSStaedtlerCet al. Preoperative staging of gastric adenocarcinoma: comparison of helical CT endoscopic US. Radiology. (2004) 230:46571. 10.1148/radiol.2302020828

  • 53.

    IsomotoHShikuwaSYamaguchiNFukudaEIkedaKNishiyamaHet al. Endoscopic submucosal dissection for early gastric cancer: a large-scale feasibility study. Gut. (2009) 58:3316. 10.1136/gut.2008.165381

  • 54.

    AnJYBaikYHChoiMGNohJHSohnTSKimS. Predictive factors for lymph node metastasis in early gastric cancer with submucosal invasion: analysis of a single institutional experience. Ann Surg. (2007) 246:74953. 10.1097/SLA.0b013e31811f3fb7

  • 55.

    KunisakiCTakahashiMNagahoriYFukushimaTMakinoHTakagawaRet al. Risk factors for lymph node metastasis in histologically poorly differentiated type early gastric cancer. Endoscopy. (2009) 41:498503. 10.1055/s-0029-1214758

  • 56.

    AdachiYYasudaKInomataMShiraishiNKitanoSSugimachiK. Clinicopathologic study of early-stage mucinous gastric carcinoma. Cancer. (2001) 91:698703. 10.1002/1097-0142(20010215)91:4<698::aid-cncr1054>3.0.co;2-o

  • 57.

    NakagawaMChoiYYAnJYChungHSeoSHShinHBet al. Difficulty of predicting the presence of lymph node metastases in patients with clinical early stage gastric cancer: a case control study. BMC Cancer. (2015) 15:943. 10.1186/s12885-015-1940-3

  • 58.

    LinJXWangZKWangWDesiderioJXieJWWangJBet al. Risk factors of lymph node metastasis or lymphovascular invasion for early gastric cancer: a practical and effective predictive model based on international multicenter data. BMC Cancer. (2019) 19:1048. 10.1186/s12885-019-6147-6

  • 59.

    XuJCaoHYangJYSuhYSKongSHKimSHet al. Is preoperative staging enough to guide lymph node dissection in clinically early gastric cancer?Gastric Cancer. (2016) 19:56878. 10.1007/s10120-015-0512-1

  • 60.

    WuSGHeZYRenHYYangLCSunJYLiFYet al. Use of CEA and CA15-3 to predict axillary lymph node metastasis in patients with breast cancer. J Cancer. (2016) 7:3741. 10.7150/jca.13090

  • 61.

    LiFLiSWeiLLiangXZhangHLiuJ. The correlation between pre-operative serum tumor markers and lymph node metastasis in gastric cancer patients undergoing curative treatment. Biomarkers. (2013) 18:6327. 10.3109/1354750x.2013.840800

  • 62.

    DurakerNCelikAN. The prognostic significance of preoperative serum CA 19-9 in patients with resectable gastric carcinoma: comparison with CEA. J Surg Oncol. (2001) 76:26671. 10.1002/jso.1044

  • 63.

    ChenXZZhangWKYangKWangLLLiuJWangLet al. Correlation between serum CA724 and gastric cancer: multiple analyses based on Chinese population. Mol Biol Rep. (2012) 39:90319. 10.1007/s11033-012-1774-x

  • 64.

    HuangCLiuZXiaoLXiaYHuangJLuoHet al. Clinical significance of serum CA125, CA19-9, CA72-4, and fibrinogen-to-lymphocyte ratio in gastric cancer with peritoneal dissemination. Front Oncol. (2019) 9:1159. 10.3389/fonc.2019.01159

  • 65.

    FujimuraTKinamiSNinomiyaIKitagawaHFushidaSNishimuraGet al. Diagnostic laparoscopy, serum CA125, and peritoneal metastasis in gastric cancer. Endoscopy. (2002) 34:56974. 10.1055/s-2002-33228

  • 66.

    VaupelP. The role of hypoxia-induced factors in tumor progression. Oncologist. (2004) 9(Suppl. 5):107. 10.2147/HP.S93413

  • 67.

    VaupelPMayerAHöckelM. Impact of hemoglobin levels on tumor oxygenation: the higher, the better?Strahlenther Onkol. (2006) 182:6371. 10.1007/s00066-006-1543-7

  • 68.

    SemenzaGL. Hypoxia, clonal selection, and the role of HIF-1 in tumor progression. Crit Rev Biochem Mol Biol. (2000) 35:71103. 10.1080/10409230091169186

  • 69.

    ErlerJTBennewithKLNicolauMDornhöferNKongCLeQTet al. Lysyl oxidase is essential for hypoxia-induced metastasis. Nature. (2006) 440:12226. 10.1038/nature04695

  • 70.

    FylesAMilosevicMHedleyDPintilieMLevinWManchulLet al. Tumor hypoxia has independent predictor impact only in patients with node-negative cervix cancer. J Clin Oncol. (2002) 20:6807. 10.1200/jco.2002.20.3.680

  • 71.

    ObermairAHandisuryaAKaiderASeveldaPKölblHGitschG. The relationship of pretreatment serum hemoglobin level to the survival of epithelial ovarian carcinoma patients: a prospective review. Cancer. (1998) 83:72631.

  • 72.

    ShenJGCheongJHHyungWJKimJChoiSHNohSH. Pretreatment anemia is associated with poorer survival in patients with stage I and II gastric cancer. J Surg Oncol. (2005) 91:12630. 10.1002/jso.20272

  • 73.

    MaYLiYFWangTPangRXueYWZhaoSP. Identification of proteins associated with lymph node metastasis of gastric cancer. J Cancer Res Clin Oncol. (2014) 140:173949. 10.1007/s00432-014-1679-2

  • 74.

    DengJJiaoXLiuHWuLZhangRWangBet al. Lymph node metastasis is mediated by suppressor of cytokine signaling-3 in gastric cancer. Tumour Biol. (2013) 34:362736. 10.1007/s13277-013-0944-3

  • 75.

    WangYWZhuMLWangRFXueWJZhuXRWangLFet al. Predictable factors for lymph node metastasis in early gastric cancer analysis of clinicopathologic factors and biological markers. Tumour Biol. (2016) 37:856778. 10.1007/s13277-015-4721-3

  • 76.

    LietoEFerraraccioFOrdituraMCastellanoPMuraALPintoMet al. Expression of vascular endothelial growth factor (VEGF) and epidermal growth factor receptor (EGFR) is an independent prognostic indicator of worse outcome in gastric cancer patients. Ann Surg Oncol. (2008) 15:6979. 10.1245/s10434-007-9596-0

  • 77.

    KochiMFujiiMKanamoriNMiharaYFunadaTTamegaiHet al. Phase II study of neoadjuvant chemotherapy with S-1 and CDDP in patients with lymph node metastatic stage II or III gastric cancer. Am J Clin Oncol. (2017) 40:1721. 10.1097/coc.0000000000000058

  • 78.

    SymSJChangHMRyuMHLeeJLKimTWYookJHet al. Neoadjuvant docetaxel, capecitabine and cisplatin (DXP) in patients with unresectable locally advanced or metastatic gastric cancer. Ann Surg Oncol. (2010) 17:102432. 10.1245/s10434-009-0838-1

  • 79.

    InoueTYachidaSUsukiHKimuraTHagiikeMOkanoKet al. Pilot feasibility study of neoadjuvant chemoradiotherapy with S-1 in patients with locally advanced gastric cancer featuring adjacent tissue invasion or JGCA bulky N2 lymph node metastases. Ann Surg Oncol. (2012) 19:293745. 10.1245/s10434-012-2332-4

  • 80.

    ZhaoLYYinYLiXZhuCJWangYGChenXLet al. A nomogram composed of clinicopathologic features and preoperative serum tumor markers to predict lymph node metastasis in early gastric cancer patients. Oncotarget. (2016) 7:596309. 10.18632/oncotarget.10732

Summary

Keywords

gastric cancer, lymph node metastasis, preoperative diagnosis, risk factors, decision rules, risk assessment model

Citation

Huang C, Hu C, Zhu J, Zhang W, Huang J and Zhu Z (2020) Establishment of Decision Rules and Risk Assessment Model for Preoperative Prediction of Lymph Node Metastasis in Gastric Cancer. Front. Oncol. 10:1638. doi: 10.3389/fonc.2020.01638

Received

08 June 2020

Accepted

27 July 2020

Published

02 September 2020

Volume

10 - 2020

Edited by

Jun Li, University Medical Center Hamburg-Eppendorf, Germany

Reviewed by

Ziv Radisavljevic, Brigham and Women's Hospital and Harvard Medical School, United States; Alfonso Recordare, Ospedale dell'Angelo, Italy

Updates

Copyright

*Correspondence: Zhengming Zhu

This article was submitted to Surgical Oncology, a section of the journal Frontiers in Oncology

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics