Magnetic Resonance Imaging Evaluation of the Accuracy of Various Lymph Node Staging Criteria in Rectal Cancer: A Systematic Review and Meta-Analysis

Background Magnetic resonance imaging (MRI)-based lymph node staging remains a significant challenge in the treatment of rectal cancer. Pretreatment evaluation of lymph node metastasis guides the formulation of treatment plans. This systematic review aimed to evaluate the diagnostic performance of MRI in lymph node staging using various morphological criteria. Methods A systematic search of the EMBASE, Medline, and Cochrane databases was performed. Original articles published between 2000 and January 2021 that used MRI for lymph node staging in rectal cancer were eligible. The included studies were assessed using the QUADAS-2 tool. A bivariate random-effects model was used to conduct a meta-analysis of diagnostic test accuracy. Results Thirty-seven studies were eligible for this meta-analysis. The pooled sensitivity, specificity, and diagnostic odds ratio of preoperative MRI for the lymph node stage were 0.73 (95% confidence interval [CI], 0.68–0.77), 0.74 (95% CI, 0.68–0.80), and 7.85 (95% CI, 5.78–10.66), respectively. Criteria for positive mesorectal lymph node metastasis included (A) a short-axis diameter of 5 mm, (B) morphological standard, including an irregular border and mixed-signal intensity within the lymph node, (C) a short-axis diameter of 5 mm with the morphological standard, (D) a short-axis diameter of 8 mm with the morphological standard, and (E) a short-axis diameter of 10 mm with the morphological standard. The pooled sensitivity/specificity for these criteria were 75%/64%, 81%/67%, 74%/79%, 72%/66%, and 62%/91%, respectively. There was no significant difference among the criteria in sensitivity/specificity. The area under the receiver operating characteristic (ROC) curve values of the fitted summary ROC indicated a diagnostic accuracy rate of 0.75–0.81. Conclusion MRI scans have minimal accuracy as a reference index for pretreatment staging of various lymph node staging criteria in rectal cancer. Multiple types of evidence should be used in clinical decision-making.


INTRODUCTION
Rectal cancer has become the leading cause of cancer-related deaths in China and worldwide. By 2030, it is estimated that there will be approximately 2.2 million cases (1,2). The determination of lymph node staging remains a significant challenge in rectal cancer treatment. Lymph nodes at a risk of metastasis in rectal cancer are mainly located in the mesentery and usually range in size from 1 to 10 mm. Lymph node status is the most important determinant of local recurrence and overall survival (3).
According to the National Comprehensive Cancer Network (NCCN) and American Joint Committee on Cancer (AJCC) staging standards (4,5), lymph node invasion should be evaluated before treatment to guide the formulation of treatment plans. Patients with lymph node involvement can benefit from preoperative neoadjuvant therapy, considerably reducing the local recurrence rate. However, over-treatment of the lymph node stage may lead to genitourinary system damage and other consequences (6,7). Therefore, accurate preoperative staging is essential for providing patients with the optimal treatment.
The diagnostic methods currently used for preoperative lymph node staging include magnetic resonance imaging (MRI), computed tomography (CT), and endoscopic ultrasound (EUS). MRI can accurately display the mesorectal fascia, the depth of tumor invasion, circumferential resection margin (CRM), and extramural venous invasion (EMVI), and it has now become the gold standard for preoperative staging and re-staging in local areas (8).
Unfortunately, the results of previous studies have shown that MRI has a poor performance in detecting metastatic lymph nodes (9,10). At present, there are various diagnostic criteria for metastatic lymph nodes, including size, shape, and boundaries, that have been widely discussed. However, there is no consensus on the accurate diagnosis of metastatic lymph nodes (11)(12)(13).
Four previous meta-analyses assessed the accuracy of MRI for lymph node staging of rectal cancer but did not differentiate the lymph nodes defined by different morphological standards (14)(15)(16)(17). Additionally, the included studies only used histological results to assess the lymph node status indirectly and did not directly assess lymph nodes on MRI scans. The studies did not perform a histological examination of each lymph node in the specimen so that the position of each lymph node was accurately matched with its corresponding MRI scan, allowing for the nodeby-node comparison of MRI scans and histological results to accurately analyze the status of each lymph node.
To the best of our knowledge, this study is the first systematic review and meta-analysis of the accuracy of various lymph node staging criteria in rectal cancer with MRI and includes the literature that contained the node-by-node correspondence between MRI scans and histopathologic results for analysis. To more accurately evaluate the accuracy of MRI in the pretreatment staging of rectal cancer lymph nodes, we hope to obtain more detailed results by synthesizing a large number of published studies.

Study Selection
Studies were included based on the following criteria: 1) original articles on the diagnostic performance of MRI in the staging of rectal cancer, 2) a phased-array MRI coil was used for imaging, 3) histopathologic findings were used as reference standards, 4) the reference criteria for assessing metastatic lymph nodes were clearly mentioned, and 5) sufficient data were available to calculate true-positive, false-positive, false-negative, and truenegative values.
The exclusion criteria were as follows: 1) inclusion of patients with non-rectal cancer, 2) research using other less common MRI types, 3) assessment of staging according to a non-Tumor-Node-Metastasis (TNM) staging system, 4) inclusion of patients who received preoperative chemoradiotherapy, 5) articles that were not original research articles, such as reviews, letters, or case reports, 6) repeated publications.
Titles and abstracts identified by the search strategy were independently reviewed by two reviewers. For all abstracts that met the inclusion criteria or were potentially eligible, full articles were retrieved and independently reviewed by two reviewers. Disagreements were resolved by consensus or by discussion with a third reviewer. All included studies followed the PICOS criteria.

Data Extraction and Quality Assessment
Two reviewers independently extracted the data. The following data was collected: (year of publication, sample size, country), study design (prospective or retrospective), MRI protocol (field strength and resolution parameters), reference criteria for assessing metastatic lymph nodes, and blinding procedure.
The diagnostic results were calculated on a lesion level for each outcome: Patients/lymph nodes with histologically confirmed lymph node metastasis are classified as nodepositive (pN+), regardless of the number of metastatic lymph nodes. Patients/lymph nodes without any metastatic lymph nodes are classified as node-negative (pN-).
The QUADAS-2 evaluation tool was used to evaluate the quality of all studies in the systematic review.
The threshold effect was evaluated using Spearman's correlation coefficient of the logit of sensitivity and logit of 1-specificity.
A bivariate random-effects model was used to summarize diagnostic statistics and displayed using summary receiver operating characteristics (SROC) plots.
Meta-regression and subgroup analyses were performed to detect heterogeneity. Additionally, a sensitivity analysis was conducted (20).
Publication bias was evaluated with an asymmetry test and a Deek's funnel plot assessment using Stata 16.0 (21).

Diagnostic Performance
The pooled sensitivity and specificity of MRI in the comprehensive diagnosis of metastatic lymph nodes were 0.73 (95% confidence interval [CI], 0.68-0.77) and 0.74 (95% CI, 0.68-0.80), respectively. The pooled sensitivity, specificity, diagnostic odds ratio, positive likelihood ratio, and negative likelihood ratio with corresponding 95% CIs are listed in Table 3. The area under the ROC curve (AUC) value of the fitted summary ROC was 0.7877 ( Figure 3).
Among the different morphological criteria, "a short-axis diameter of 5 mm with the morphological standard" revealed the highest sensitivity of 0.81 (95% CI, 0.74-0.87), and "a shortaxis diameter of 10 mm with the morphological standard" revealed the highest specificity of 0.91 (95% CI, 0.51-0.99) ( Table 3). The AUCs indicated a diagnostic accuracy rate of 0.75-0.81. The morphological standards with the highest accuracy were "a short-axis diameter of 5 mm with the morphological standard" and "a short-axis diameter of 10 mm with the morphological standard" (Figure 4).

Test of Heterogeneity and Metaregression Analysis
The heterogeneity tests showed that the Spearman's correlation coefficient was 0.446 (p = 0.004), indicating the presence of a threshold effect. This means that different evaluation criteria have led to a significant heterogeneity. Under different morphological standards, there is considerable heterogeneity among 1) the morphological standard, 2) a short-axis diameter of 8 mm with the morphological standard, and 3) a short-axis diameter of 10 mm with the morphological standard (all p < 0.05, i2 > 50%). Therefore, in addition to the threshold effect, there must be other factors that cause significant heterogeneity. A single-factor meta-regression analysis was performed on all the elements. The results showed that the blinding procedure had a particular impact on the heterogeneity of the research ( Table 4).

Subgroup Analysis
Subgroup analyses were performed for the different study characteristics. By comparing references with or without node-by-node correspondence, we found that a lower sensitivity of 0.55 (95% CI, 0.40-0.69) and higher specificity of 0.89 (95% CI, 0.79-0.95) were yielded. When considering different MRI types, both 3.0T and high-resolution MRI yielded a higher sensitivity and specificity. Through a subgroup analysis of the study design, read approach, and blinding procedure, studies that used double blinding yielded a higher sensitivity of 73% (95% CI, 0.67-0.78) and specificity of 78% (95% CI, 0.70-0.84), whereas prospective studies yielded a higher specificity of 77% (95% CI, 0.75-0.79). The results of the subgroup analysis are shown in Table 4.

Sensitivity Analysis
A sensitivity analysis of all the studies revealed that five original studies had a strong sensitivity ( Figure 5), whereas the other original studies did not strongly affect the calculation results. After excluding the literature mentioned above, the other 36 subdatasets still had threshold effects. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio were 0. 74

Publication Bias
For all studies, the p-value of the bias on the Deek's funnel plot asymmetry test was 0.55, indicating that these studies did not have significant publication bias ( Figure 6).

DISCUSSION
Lymph node status plays a vital role in selecting treatment strategies for colorectal cancer, with the presence or absence of regional lymph node metastasis being the key to treatment selection. The advantage of MRI is that it can identify the mesorectal fascia, enabling accurate preoperative identification of patients with lymph nodes that cannot be entirely surgically removed. Therefore, in the context of neoadjuvant therapy, preoperative MRI must provide an accurate diagnosis of regional lymph nodes, avoid overestimation and underestimation before treatment, and provide the optimal treatment decision for individual patients. In this study, we evaluated the ability of MRI to determine the lymph node stage of rectal cancer. The results showed that the value of MRI in diagnosing metastatic lymph nodes was low (57)(58)(59).
These findings are similar to those reported by Al-Sukhni et al. (15)(16)(17), who concluded that MRI only moderates the diagnostic ability for lymph node metastasis. It is worth noting that the previous meta-analysis found significant heterogeneity in the assessment of lymph node metastasis and speculated that  the threshold effect is the primary source of heterogeneity. Therefore, we corrected for some of the limitations recognized by previous studies by including more original articles and classifying lymph nodes for statistical analysis based on different morphological standards. Most MRI studies on colorectal cancer published have used lymph node size as a standard criterion for predicting lymph node involvement. However, previous studies demonstrated that using only the size of lymph nodes as a criterion does not improve the accuracy of lymph node staging of colorectal cancer (9)(10)(11), which is consistent with our results. We found that there was no significant difference in the accuracy of MRI diagnosis when using different standards. It is worth mentioning that under the same morphological standard, as the shorter diameter of the lymph node increases, the sensitivity gradually decreases and the specificity gradually increases ( Table 3). This may be because although malignant lymph nodes usually have a larger short-axis diameter than benign lymph nodes, there is a considerable size overlap between benign and malignant lymph nodes, with approximately 30% of metastatic lymph nodes having a diameter of ≤4 mm (12). In addition, benign lymph nodes may appear to increase in size with the development of fibrosis (60).
Compared with the size standard alone, different morphological features have been previously considered as good criteria for judging metastatic lymph nodes. Brown et al. first described the use of MRI to improve the correct diagnosis of lymph node involvement in rectal cancer when boundary contours and signal intensity features were used instead of size standards alone (9). Kim et al. demonstrated that in addition to size, new criteria, such as burr-like or inconspicuous borders and uneven appearance, can be used to predict regional lymph node involvement (11). Their results were better than our findings. We found that after adding morphological features, the pooled sensitivity and specificity of lymph node diagnosis improved. However, the diagnostic performance did not improve significantly ( Table 3), possibly because the morphological characteristics are more subjective among different observers. We found that both high-field strength (3.0 Tesla) and highresolution MRI yielded a higher sensitivity and specificity than low-field strength (1.5 Tesla) according to a subgroup analysis ( Table 4). Due to the retrospective design of the research, patient selection, and MRI plan, the diagnostic performance of prospectively designed research was slightly better in the subgroup analysis. In addition, double-blind studies had a higher specificity than single-blind studies (0.78, 95% CI 0.70-0.84). As with other diagnostic meta-analyses, heterogeneity is a vital limitation among studies, including study design, MRI protocols, blinding procedures, and reference standards. In the regression analysis, we found that the blinding procedure (singleblind/double-blind) helps assess heterogeneity, leading to differences among research conclusions.
In most previous studies, the assessment of the lymph node staging of patients mainly relied on the number of positive lymph nodes found in the mesorectum after the overall sampling of rectal specimens, which does not have a high accuracy and reliability. Thus far, few studies have reported that the individual lymph nodes seen on MRI scans match the exact pathological    correspondence after rectal resection. We included five references as subgroups. Our analysis found that the sensitivity of MRI for the diagnosis of a single lymph node decreased, the specificity significantly improved, and the accuracy of the assessment was lower than expected. The possible reasons for the inconsistent diagnostic accuracy could be due to small number of references, a lack of consistency in the threshold, and the difference in the realization of node-by-node correspondence.
Currently, new technologies are being explored to improve preoperative staging. The chemical shift effect is a reliable indicator for identifying benign and malignant lymph nodes (61), and Farshchian first proposed that it has the potential to diagnose benign lymph nodes (62). Grovik et al. showed that a low K trans of the primary tumor can predict the presence of nodal metastasis (63), which can be achieved by dynamic contrastenhanced MRI (DCE-MRI) (64,65). In addition to DCE-MRI, special diffusion-weighted MRI parameters are helpful in differentiating metastatic lymph nodes (45,66).
The use of lymphatic contrast agents is considered a method for improving the staging of lymph nodes. USPIO is the most widely used contrast agent (67,68). This technology allows for the differentiation of malignant and benign lymph nodes according to the contrast-enhanced pattern. Although MRI with USPIO has achieved some success in characterizing small lymph nodes, further research is needed regarding its clinical applicability (55,(69)(70)(71).
Radiomics is a rapidly developing discipline that uses computer algorithms to extract quantitative features from MRI scans (72)(73)(74). These algorithms capture the image texture and morphology of tumors based on their gray values. Since 2018, many reports on radiological methods for rectal cancer lymph node assessment have been published (75)(76)(77)(78). However, when analyzing imaging information and building predictive models, all these parameters require time-consuming calculations. In the future, artificial intelligence is expected to become the optimal option for determining lymph node staging and treatments options for patients with locally advanced rectal cancer.
Recently, the importance of lymph node metastasis in the process of tumor recurrence has begun to be questioned, i.e., the indications of neoadjuvant therapy are not based on clinical   (79)(80)(81). The MERCURY study showed that lymph node involvement is not an independent predictor of local recurrence, and using CRM was recommended for evaluating neoadjuvant therapy (82). In this case, clinical lymph node assessment for rectal cancer may only play a secondary role in guiding future treatment decisions (83,84). This study has some limitations. First, our meta-analysis included 37 studies and 2,875 patients. Although this is a comprehensive literature search, more studies may provide more accurate estimates and comparisons of results. Second, the content of some reports is insufficient, limiting our quality assessment and individual analysis of more subgroups. Finally, heterogeneity is still an essential issue in meta-analyses. In future studies, the definition of critical staging elements and MRI protocols should be standardized to reduce heterogeneity. Therefore, considering the limitations of diagnostic metaanalysis, the results should be interpreted prudently.

CONCLUSION
In summary, the performance of MRI in the detection of lymph node metastasis is inadequate, and either through using more morphological characteristics or shorter diameter, is not significantly improved. At present, when making preoperative neoadjuvant treatment decisions, evidence from a variety of imaging methods should be combined to determine the optimal treatment strategy.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.

AUTHOR CONTRIBUTIONS
ZZ contributed the most to this article. ZZ designed the project, developed the search strategy and wrote the manuscript. ZZ and YZ checked the search, and reviewed the manuscript. MW performed literature screening and data extraction, conduct the quality assessment of the included studies. XY carried out the data analysis. ZW reviewed the manuscript and finally approved the version to be published. All authors contributed to the article and approved the submitted version.