Prediction of axillary lymph node pathological complete response to neoadjuvant therapy using nomogram and machine learning methods

Abstract

Purpose:

To determine the feasibility of predicting the rate of an axillary lymph node pathological complete response (apCR) using nomogram and machine learning methods.

Methods:

A total of 247 patients with early breast cancer (eBC), who underwent neoadjuvant therapy (NAT) were included retrospectively. We compared pre- and post-NAT ultrasound information and calculated the maximum diameter change of the primary lesion (MDCPL): [(pre-NAT maximum diameter of primary lesion – post-NAT maximum diameter of preoperative primary lesion)/pre-NAT maximum diameter of primary lesion] and described the lymph node score (LNS) (1): unclear border (2), irregular morphology (3), absence of hilum (4), visible vascularity (5), cortical thickness, and (6) aspect ratio <2. Each description counted as 1 point. Logistic regression analyses were used to assess apCR independent predictors to create nomogram. The area under the curve (AUC) of the receiver operating characteristic curve as well as calibration curves were employed to assess the nomogram’s performance. In machine learning, data were trained and validated by random forest (RF) following Pycharm software and five-fold cross-validation analysis.

Results:

The mean age of enrolled patients was 50.4 ± 10.2 years. MDCPL (odds ratio [OR], 1.013; 95% confidence interval [CI], 1.002–1.024; p=0.018), LNS changes (pre-NAT LNS – post-NAT LNS; OR, 2.790; 95% CI, 1.190–6.544; p=0.018), N stage (OR, 0.496; 95% CI, 0.269–0.915; p=0.025), and HER2 status (OR, 2.244; 95% CI, 1.147–4.392; p=0.018) were independent predictors of apCR. The AUCs of the nomogram were 0.74 (95% CI, 0.68–0.81) and 0.76 (95% CI, 0.63–0.90) for training and validation sets, respectively. In RF model, the maximum diameter of the primary lesion, axillary lymph node, and LNS in each cycle, estrogen receptor status, progesterone receptor status, HER2, Ki67, and T and N stages were included in the training set. The final validation set had an AUC value of 0.85 (95% CI, 0.74–0.87).

Conclusion:

Both nomogram and machine learning methods can predict apCR well. Nomogram is simple and practical, and shows high operability. Machine learning makes better use of a patient’s clinicopathological information. These prediction models can assist surgeons in deciding on a reasonable strategy for axillary surgery.

Introduction

Neoadjuvant therapy (NAT) is a systemic treatment that precedes local surgery and not only monitors the response to systemic therapy, but also offers patients with early breast cancer (eBC) a higher rate of breast-conserving surgery and omission from axillary lymph node dissection (ALND) (1). While traditional anatomic pathological features are important in predicting the risk of recurrence and deciding on adjuvant treatment options for patients with eBC, the response of the primary breast lesion and lymph node post-NAT are also important for any subsequent adjuvant treatment regimen (2). At the same time, when patients achieve an axillary lymph node pathological complete response (apCR) post-NAT, an opportunity exists to omit ALND and avoid postoperative complications such as lymphedema, arm pain, and arm dyskinesia (3). According to the results of Z1071 (4), SENTINA (5), and SN FNAC (6) trials, although the proportion of patients who achieved apCR post-NAT was within acceptable range (40%–70%), the proportion of patients with a negative clinical evaluation of axillary lymph node post-NAT who obtained an ALND exemption through sentinel lymph node biopsy (SLNB) was small in the real world. Although the National Cancer Database shows that 42.2% of NAT patients are exempted from ALND by SLNB (7), many countries and regions show a far lower ALND rate. A cross-sectional survey of 110 large hospitals in mainland China showed that more than 50% of hospitals preferred to perform SLNB before NAT. If SLNB is positive at this time point, further ALND is required and the opportunity to perform SLNB post-NAT is lost (8). The reasons for this may be because of the high false negative rate (FNR) of 8.4–14.2% for SLNB. Although a subgroup analysis of the Z1071 trial found that the placement of marker clips in positive lymph nodes reduced the FNR of post-NAT sentinel lymph node biopsies to 6.8%, 20% of marker clips were not placed in sentinel lymph nodes and 17% were lost (9). If SLNB was only performed for patients with no suspicious lymph nodes on a post-NAT ultrasound, the FNR decreased to 9.8%. But of 138 patients with suspicious lymph nodes on ultrasound, no metastases were found in the postoperative lymph node pathology of 39 patients (10). However, the poor implementation of radioisotope and placement of marker clips that occurred in many medical institutions also influenced the performance of post-NAT SLNB. Therefore, the development of other practical tools is urgently required to screen for an appropriate population for post-NAT SLNB.

Previous studies have described such a prediction model, which were based on pre-and post-NAT clinicopathological and imaging information (11–14). One study included data on suspicious lymph nodes on post-NAT ultrasound in the prediction model to improve accuracy (15). Unfortunately, the dynamic changes in lymph node status post-NAT were not considered. A meta-analysis showed that the accuracy of ultrasound evaluation for lymph node status post-NAT was only 0.58 (16). In addition, machine learning to predict total pCR (tpCR) and long-term survival in patients with NAT eBC has been developed in several studies. A study based on machine learning to predict tpCR in patients with eBC had the highest AUC value of 0.87 but required additional genomic and transcriptomic profiles from patients (17). A report using multiparameter magnetic resonance imaging (MRI) combined with machine learning to predict tpCR and survival had the highest AUC value of 0.86; only imaging features of the primary lesion were extracted and the characteristics of lymph nodes were ignored (18). Meti et al. combined tumor size, histological grade, clinical stage, and molecular subtype to construct a prediction model, but did not include imaging features (19). Thus, these latest studies suggested an urgent need exists for developing a tool that can accurately predict the status of axillary lymph node post-NAT through routine examination and clinical information.

Therefore, the aim of our study was to predict the post-NAT lymph node status of patients with eBC using clinicopathological and ultrasound information during NAT by nomogram and random forest (RF) methods respectively, and to compare the advantages and disadvantages of the two methods. We expect that these models may provide clinicians with a simple, accurate, and easy-to-use method for predicting a candidate population for SLNB post-NAT, thus avoiding complex technical approaches.

Methods

Study population

Data on a total of 382 patients with eBC, who underwent NAT at the Second Hospital of Dalian Medical University and the First Affiliated Hospital of China Medical University between 2017 and 2021, were collected retrospectively. The eligibility criteria for the study were as follows (1): at least four cycles of NAT (2); surgery post-NAT (3); complete ultrasound examination in all treatment cycles (4); complete pathological information (5); treatment regimen containing anthracyclines and/or paclitaxel; and (6) a primary lesion detected on breast ultrasound.

Based on the medical history and imaging findings, we excluded (1): incomplete pathological information (2); incomplete ultrasound information (3); no surgery (4); distant metastasis (5); occult breast cancer (BC) (6); cN3; and (7) inflammatory BC (Figure 1). In total, 247 patients were included in the study.

Figure 1

Data collection

We collected clinical information including (1): age, and (2) pre-NAT clinical T and N stages. Pre- and post-NAT pathological information included (1): estrogen receptor (ER) status (2); progesterone receptor (PR) status (3); human epidermal growth factor receptor 2 (HER2) status; and (4) Ki-67 index. TNM staging was performed according to the 8th edition of the American Joint Committee on Cancer Staging Manual (20). ER and PR expression ≥ 1% was considered positive (21). ER or PR positive was considered hormone receptor (HR) positive (22). Immunohistochemistry (IHC) 3+ was considered HER2 positive. IHC 2+ was tested using fluorescence in situ hybridization (FISH): a HER2/CEP17 ratio ≥ 2.0 with a HER2 signal/cell ratio ≥ 4.0 or a HER2/CEP17 ratio < 2.0 with a HER2 signal/cell ≥ 6.0 were positive (23). Breast pCR (bpCR) was defined as no invasive disease in the breast and apCR was defined as no metastatic disease in an axillary lymph node (24) including isolated tumor cells and micrometastases (25). Total pCR (tpCR) was defined as ypTis/0N0M0.

Ultrasound

All patients ultimately included in the study underwent breast ultrasound during each cycle of NAT. The maximum diameter of the primary lesion (MDPL) and maximum diameter of a suspicious lymph node (MDSLN) were recorded for each cycle. The maximum diameter change of the primary lesion (MDCPL) was calculated: [(pre-NAT MDPL – post-NAT MDPL)/pre-NAT MDPL]. If multiple lesions were detected, information on the lesion with the largest diameter was selected. An ultrasound evaluation was made of the following suspicious axillary lymph node features (1): unclear border (Figure 2A) (2); irregular morphology (Figure 2B) (3); absence of hilum (Figure 2C) (4); visible vascularity (Figure 2D) (5); cortical thickness (Figure 2E); and (6) aspect ratio <2 (Figure 2F) (26–29). The lymph node score (LNS) was set up, with each abnormal description considered as a score of 1. For example, a NAT patient with lymph node ultrasound images showing an unclear border, irregular morphology, absence of hilum, and cortical thickness would have an LNS of 4 points (Figure 2G). Another patient with lymph node ultrasound images showing an unclearborder, irregular morphology and cortical thickness would have an LNSof 3 points (Figure 2H). The difference between pre- and post-NAT was calculated and divided into two groups, with 0 as the cutoff value (1): ≥0 (2) <0.

Figure 2

Machine learning

Machine learning was randomly divided into training and validation sets and stratified 7:3. The feature values of the training set sample data included patient pre-NAT T stage, N stage, MDPL and MDSLN for each cycle, LNS for each cycle, ER expression, PR expression, Ki67, and HER2 status.

The RF method is an integrated classification method (30). This model is a commonly used classifier in the intersectional field of medicine and artificial intelligence. It is composed of multiple decision trees and the output category is determined by majority voting according to the output category of each tree.

The software environment used in this study was Pycharm and the Python interpreter version was 3.8.12. First, the training and validation sets were stratified and randomly divided 7:3 to ensure a consistent proportion of apCR and non-apCR sample data. Second, the feature values of the sample data were standardized to eliminate differences between features. In view of the imbalance in the number of samples of various types in the data set, SMOTE synthesis method was used to oversampling the minority sample data (patients who reached apCR), so as to improve the recognition ability of the model for minority samples. When training the model, we try to use different class weight to adjust the influence degree of different types of training data on the loss function, in order to achieve the best classification effect. With the aim of the problem being that the total sample data was small, in the process of selecting the optimal parameters for the training model, this experiment used a cross-validation method to compare the advantages and disadvantages of the model with different parameters in the prediction result index on the validation set. Finally, after determining the parameters, all sample data of the training set was inputted to train the model, the final prediction results of the validation set on the model were saved, the confusion matrix and receiver operating characteristic (ROC) curve were drawn, and the performance index accuracy, sensitivity, specificity and area under the curve (AUC) of the ROC curve were calculated. For this experimental dataset, which was classified with unbalanced labels, the error could be well balanced. Python language was used in its programming part. In order to obtain a stable and reliable model, this experiment adopted a 5-fold cross-validation method to select parameters. Finally, the ROC was drawn based on predicted and true label values on the model. The AUC of the ROC curve was calculated as the performance index of the model.

Statistical analysis

Randomization into training and validation sets was performed at 8:2. We used logistic regression to obtain an odds ratio (OR) with 95% confidence interval (CI) for any association with the response. Univariable logistic regression was used to explore the clinicopathological and ultrasound factors associated with lymph node pCR. Variables with p < 0.05 in the univariable logistic regression were included in the multivariable logistic regression. Variables with p < 0.05 in the multivariable logistic regression were retained in the nomogram. The AUCs of the ROC and calibration curves were employed to assess the nomogram’s performance (31). Statistical analyses were carried out in SPSS 25 and R (version 4.2.1) software. In addition, “glm,” “rms,” “pROC,” “Calibration Curves” packages were used.

Results

Pre-NAT characteristics

In total, 247 patients were included in this retrospective study. The pre-NAT clinical and pathological characteristics of patients in the training and validation sets are shown in Table 1. The mean patient age ± standard deviation was 50.4 ± 10.2 years. The majority of patients with a T stage were T2 at 64% (158 of 247) of the population, followed by T3–T4 patients accounting for 27.5% (68 of 247), and T1 patients accounting for a minimum of 8.5% of the population (21 of 247). The highest percentage of patients with an N1 stage was 73.7% (182 of 247). 97.6% of patients were invasive ductal carcinoma (IDC). We classified molecular subtypes into (1): HR+/HER2- (2), HR+/HER2+ (3), HR-/HER2-, and (4) HR-/HER2+. The most frequent were HR+/HER2- at 49.8% (123 out of 247) of patients, and HR+/HER2+ at 23.5% (58 of 247) of patients. The rest of the types were HR-/HER2- at 12.5% (31 of 247) of patients and HR-/HER2+ at 14.2% (35 of 247) of patients, respectively. Anti-HER2 drugs were used in 81% (75 of 93) of HER2+ patients. The tpCR rate was 23.9%, and the rates for bpCR and apCR were 26.7% and 48.1%, respectively (Table 2).

Factors associated with apCR

In the training set (n=199), apCR was achieved in 48.7% of patients (Table 1). A total of 10 variables were included in the statistical analysis (Table 3). In a univariable logistic regression analysis, clinicopathological characteristics: ER expression (OR, 0.989; 95% CI, 0.982–0.996; p = 0.001), HER2 status (OR, 2.715; 95% CI, 1.511–4.876; p = 0.001), Ki67 index (OR, 6.560. 95% CI, 1.457–29.532; p = 0.014), and N stage (OR, 0.554; 95% CI, 0.319–0.962; p = 0.036) were associated with apCR. For ultrasound features, MDCPL (OR, 1.015; 95% CI, 1.005–1.025; p = 0.004) and LNS changes (OR, 3.650; 95% CI, 1.621–8.220; p = 0.002) were associated with apCR. The p values for T stage, PR expression, molecular subtype and age were less than 0.05 so they were not included in the multivariable logistic regression analysis (Table 3).

The above variables with p < 0.05 were included in a multivariable logistic regression analysis, which showed that the MDCPL (OR, 1.013; 95% CI, 1.002–1.024; p = 0.018), LNS changes (OR, 2.790; 95% CI, 1.190–6.544; p = 0.018), N stage (OR, 0.496; 95% CI, 0.269–0.915; p = 0.025), and HER2 status (OR, 2.244; 95% CI, 1.147–4.392; p = 0.018) were associated with apCR (Table 4).

Nomogram construction and validation

The ER was a predictor of apCR in previous studies (31–34). Although we did not see a statistically significant difference in ER expression in a multivariable logistic regression analysis (OR, 0.993; 95% CI, 0.985–1.001; p = 0.104), we still included ER expression in the nomogram. The total score of the nomogram was obtained by summing the respective scores of MDCPL, ER expression, LNS changes, HER2 status, and N stage. The probability of achieving apCR for an individual patient was obtained by applying the total score to the scale at the bottom of the nomogram (Figure 3).

Figure 3

Internal validation on a training set with a AUC value of 0.74 (95% CI, 0.68–0.81) and independent external validation on a validation set with a AUC value of 0.76 (95% CI, 0.63–0.90) indicated that the reformulated model had good predictive ability (Figures 4A, B). The calibration curve of the nomogram showed good agreement between actual observations and predicted outcomes in training and validation sets (Figures 4C, D). As the prediction curves were close to the standard curve (Y=X), the final model showed good performance and had high practicability.

Figure 4

Validation of machine learning (RF model construction)

In this study, we used features that were already available in the data; however, the values of individual features needed to be preprocessed. First, the values of ER, PR, and HER2 were binarized according to the critical values of negative and positive in medicine, and the remaining eigenvalues were retained. The processed training set data were sent to the RF model for training, and the appropriate model parameters were selected by comparing the mean of the 5-fold cross-validation result. Finally, pre-NAT T stage, N stage, MDPL and MDSLN for each cycle, LNS for each cycle, ER expression, PR expression, Ki67, and HER2 status were included in the model and the performance of the model was examined in the validation set. The indicators used in this experiment were AUC, accuracy, sensitivity, specificity, and ROC (Figure 5A), showing an AUC value of 0.85 (95% CI, 0.74–0.87). The confusion matrix in Figure 5B shows that the RF model has a prediction accuracy of 0.78, sensitivity of 0.74, and specificity of 0.83. These results indicated that this model was predictive.

Figure 5

Discussion

Due to the iterative update of new targeted drugs and the increasing refinement of molecular subtype, the proportion of patients with eBC who achieve surgical downstaging through NAT is increasing (34). Thus, more patients have conditions that lead to preservation of breast and axillary lymph nodes, especially those with triple-negative and HER2+ BC (35). Previous studies have shown better disease free survival (DFS) and overall survival (OS) rates could be obtained in patients who achieved pCR compared to those with residual lesions (22). An OS benefit is also seen in some subtypes with pCR in a single breast or axilla (36, 37). Also, several studies have reported that axillary lymph nodes appear to be more likely to reach pCR compared to the breast (2, 38, 39). A total of 48.1% of patients in our study achieved apCR, similar to previous studies (40–42). It is essential to perform an SLNB for patients who may then be exempted from ALND after NAT. How to select such feasible SLNB candidates by scientific methods is the key to constraining this problem.

The 2022 edition of the National Comprehensive Cancer Network breast cancer guidelines (43), and the 2021 edition of the Chinese breast cancer guidelines (CBCS, CSCO) (44, 45), generally recommend SLNB in selected cases when nodes are clinically negative after NAT. Conditions for SLNB were the pre-NAT placement of marker clips and their removal during surgery, use of dual tracers including radioisotopes, and removal of at least three sentinel lymph nodes. The rates for the utilization of radioisotope tracing in Z1071 (4), SENTINA (5), and SNFNAC (6) trials were 95.9%, 95%, and 100%, respectively. Unfortunately, the clinical prevalence of radioisotopes and marker clips has limited the widespread implementation of SLNB after NAT in many countries such as China. A cross-sectional study of 110 large hospitals in mainland China showed that only 14.5% of hospitals chose dual tracers containing a radioisotope (46). Another cross-sectional study showed that only 11% of hospitals chose to localize primary breast lesions and lymph nodes with marker clips (8).

Our study was designed to address this clinical challenge by using a scientific approach to help clinicians screen for patients who can truly avoid ALND through monitoring the axillary lymph node response of NAT dynamically. We also sought to develop an effective method for SLNB post-NAT that does need not be restricted by conditions, such as radioisotopes and marker clips. In this study, a prediction model based on nomogram and RF methods was developed to discriminate patients who could undergo SLNB post-NAT. The AUC values of nomogram and RF methods were 0.76 and 0.85, respectively, and both methods showed good predictive ability. The variables incorporated in the final nomogram by logistic regression analysis were MDCPL, LNS changes, ER expression percentage, HER2 status, and pre-NAT N stage. The eigenvalues incorporated in the RF model were MDPL by cycle, MDSLN by cycle, LNS by cycle, T stage, N stage, ER, PR, HER2, and Ki67. In our study, whether nomogram or machine learning, the included variables can be obtained during routine examination, which does not require additional examination and is more consistent with actual clinical operation.

ER, HER2, and other IHC indicators have been shown to have predictive value for apCR post-NAT in several studies (11–14). Ultrasound is a general screening tool, with the advantage of being low-cost and can simultaneously assist in lymph node biopsies. A previous study shows that ultrasound is more accurate than mammography and MRI in the evaluation of axillary lymph nodes (47). Thus, the predictive value of ultrasound for axillary lymph nodes is controversial. The essential problem is the homogeneity of different medical centers cannot be guaranteed due to strong subjective judgment (sensitivity of 87% and specificity of 53%–97%) (48). Ultrasound features of metastatic lymph nodes are generally as follows (1): unclear border (2), irregular morphology (3), absence of hilum (4), visible vascularity (5), cortical thickness (6), and an aspect ratio <2 (26–29). It is usually thought that if one of these descriptions is met, the lymph node is likely to be a metastatic lymph node. However, our clinical experience has found that post-NAT ultrasound features showing metastatic lymph nodes postoperatively usually correlate with the number of these features present. A study found that the post-NAT ultrasound detection of suspicious lymph nodes predicted a lower rate of apCR (15). We believe that the various descriptions associated with abnormal lymph nodes may have inconsistent weights for evaluating lymph node metastasis. But we do not know whether they are the independent factors. Therefore, we originally established LNS as an observation index, assigning scores to each abnormal description, and calculating the difference pre- and post-NAT. Interestingly, patients with an LNS difference ≥0 are more likely to achieve apCR. To our knowledge, this is the first study to considerthe “LNS” as an independent factor influencing the evaluation of lymph node metastasis post-NAT.

NAT is a process, and therefore, patients’ primary breast and lymph node metastatic lesions also show a dynamic process of change as treatment advances. Whether such a change process is reflected in imaging features that can be suggestive in predicting pCR is also a recent hot topic. A study that predicted tpCR, by assessing the change in depth and width of primary foci described by ultrasound pre- and post-NAT, showed that the greater the reduction in depth of primary foci in triple-negative BC, the easier it was to achieve tpCR (49). Li et al. comparing ultrasound changes during NAT to predict apCR, found that the clinical response of the primary focus and the percentage of lymph nodes showing a reduction in their short diameter after NAT predicted apCR (50). The results of the above study suggest that changes in imaging features before and after NAT do have an important predictive value for pCR. This also inspired us to choose the pre-NAT and post-NAT LNS difference when conducting this study, instead of selecting the LNS at a separate time point pre- or post-NAT.

Moreover, the response of primary and metastatic axillary lymph nodes to NAT is generally consistent, but occasionally discordant cases occur, sometimes not exactly with the same treatment effect (51). We therefore ventured to see if the inclusion of MDCPL in the model would improve its predictive ability. In our study, an interesting phenomenon was observed was that some patients showed a trend of enlargement of the primary breast lesion while the axilla got pCR (6 of 27). Previous studies have shown that about 14.5% of patients show different post-NAT outcomes in breast and axillary lymph nodes (52). We therefore retained this group of patients and included them in the study.

The nomogram is simple to construct and easy to apply. The apCR probability can be derived by calculating the score from post-NAT clinical characteristics; however, some variables were excluded after statistical analysis that resulted in data compression. The RF model is more complex to construct than a nomogram, and its AUC results are greater than those of the nomogram. Machine learning can use all available clinicopathological information and prediction results are more reliable. However, when using this algorithm, the training time is relatively long. In addition, the process of data pre-processing and finding the optimal parameters of the model requires some knowledge of machine learning and practical experience in programming. This makes the whole process more tedious in general.

Our study has several limitations. First, this was a retrospective study with a limited number of enrolled patients, but efforts were made to collect data from different study centers in order to minimize sample selection bias. Second, it is generally accepted that MRI is more accurate than ultrasound for primary breast lesion assessment (53), but ultrasound was used for primary lesion assessment because of excessive missing MRI data. This is also in line with the Chinese context and we believe that it would be more in line with Chinese clinical practice if we could indeed find a way to use ultrasound to achieve increased accuracy in being predictive. Third, not all suspicious axillary lymph nodes by clinical evaluation pre-NAT in our study underwent biopsy, which may influence the results to a greater or lesser extent. We also plan to increase the numbers of observed samples and design a prospective study in future.

In conclusion, our study developed a practical prediction model to help clinicians decide on an optimal surgical approach for axillary lymph node post-NAT based on nomogram and RF algorithm methods. Both prediction models can lead to the accurate prediction of apCR and guide the mode of surgical intervention of axillary lymph nodes in order to avoid non-essential ALND and minimize injury.

Funding

This study was supported by the “1+X” program cross-disciplinary innovation project (No. 2022JCXKYB07)

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  UntchMKonecnyGEPaepke Svon MinckwitzG. Current and future role of neoadjuvant therapy for breast cancer. Breast (2014) 23(5):526–37. doi: 10.1016/j.breast.2014.06.004

  • CrossRef
  • Google Scholar

• 2

  RouzierRExtraJMKlijanienkoJFalcouMCAsselainBVincent-SalomonAet al. Incidence and prognostic significance of complete axillary downstaging after primary chemotherapy in breast cancer patients with T1 to T3 tumors and cytologically proven axillary metastatic lymph nodes. J Clin Oncol (2002) 20(5):1304–10. doi: 10.1200/jco.2002.20.5.1304

  • CrossRef
  • Google Scholar

• 3

  NoguchiMInokuchiMNoguchiMMoriokaEOhno YKuritaT. Axillary surgery for breast cancer: past, present, and future. Breast Cancer (2021) 28(1):9–15. doi: 10.1007/s12282-020-01120-0

  • CrossRef
  • Google Scholar

• 4

  BougheyJCSumanVJMittendorfEAAhrendtGMWilkeLGTabackBet al. Sentinel lymph node surgery after neoadjuvant chemotherapy in patients with node-positive breast cancer: the ACOSOG Z1071 (Alliance) clinical trial. JAMA (2013) 310(14):1455–61. doi: 10.1001/jama.2013.278932

  • CrossRef
  • Google Scholar

• 5

  KuehnTBauerfeindIFehmTFleigeBHausschildMHelmsGet al. Sentinel-lymph-node biopsy in patients with breast cancer before and after neoadjuvant chemotherapy (SENTINA): a prospective, multicentre cohort study. Lancet Oncol (2013) 14(7):609–18. doi: 10.1016/s1470-2045(13)70166-9

  • CrossRef
  • Google Scholar

• 6

  BoileauJFPoirierBBasikMHollowayCMGabouryLSiderisLet al. Sentinel node biopsy after neoadjuvant chemotherapy in biopsy-proven node-positive breast cancer: the SN FNAC study. J Clin Oncol (2015) 33(3):258–64. doi: 10.1200/JCO.2014.55.7827

  • CrossRef
  • Google Scholar

• 7

  WongSMWeissAMittendorfEAKing T AGolshanM. Surgical management of the axilla in clinically node-positive patients receiving neoadjuvant chemotherapy: A national cancer database analysis. Ann Surg Oncol (2019) 26(11):3517–25. doi: 10.1245/s10434-019-07583-6

  • CrossRef
  • Google Scholar

• 8

  WangJXiuBQGuoRYangBLZhangQSuYHet al. [Current trend of breast cancer neoadjuvant treatment in China: a cross-sectional study]. Zhonghua Zhong Liu Za Zhi (2020) 42(11):931–6. doi: 10.3760/cma.j.cn112152-20190924-00623

  • CrossRef
  • Google Scholar

• 9

  BougheyJCBallmanKVLe-PetrossHTMcCallLMMittendorfEAAhrendtGMet al. Identification and resection of clipped node decreases the false-negative rate of sentinel lymph node surgery in patients presenting with node-positive breast cancer (T0-T4, N1-N2) who receive neoadjuvant chemotherapy: Results from ACOSOG Z1071 (Alliance). Ann Surg (2016) 263(4):802–7. doi: 10.1097/SLA.0000000000001375

  • CrossRef
  • Google Scholar

• 10

  BougheyJCBallmanKVHuntKKMcCallLMMittendorfEAAhrendtGMet al. Axillary ultrasound after neoadjuvant chemotherapy and its impact on sentinel lymph node surgery: Results from the American college of surgeons oncology group Z1071 trial (Alliance). J Clin Oncol (2015) 33(30):3386–93. doi: 10.1200/JCO.2014.57.8401

  • CrossRef
  • Google Scholar

• 11

  HaSMChaJHKimHHShinHJChae E YChoiWJ. Diagnostic performance of breast ultrasonography and MRI in the prediction of lymph node status after neoadjuvant chemotherapy for breast cancer. Acta Radiol (2017) 58(10):1198–205. doi: 10.1177/0284185117690421

  • CrossRef
  • Google Scholar

• 12

  Al-HattaliSVinnicombeSJGowdhNMEvansAArmstrongSAdamsonDet al. Breast MRI and tumour biology predict axillary lymph node response to neoadjuvant chemotherapy for breast cancer. Cancer Imaging (2019) 19(1):91. doi: 10.1186/s40644-019-0279-4

  • CrossRef
  • Google Scholar

• 13

  BaeMSShinSUSongSERyuHSHan WMoonWK. Association between US features of primary tumor and axillary lymph node metastasis in patients with clinical T1-T2N0 breast cancer. Acta Radiol (2018) 59(4):402–8. doi: 10.1177/0284185117723039

  • CrossRef
  • Google Scholar

• 14

  WeberJJJochelsonMSEatonAZaborECBarrioAVGemignaniMLet al. MRI And prediction of pathologic complete response in the breast and axilla after neoadjuvant chemotherapy for breast cancer. J Am Coll Surg (2017) 225(6):740–6. doi: 10.1016/j.jamcollsurg.2017.08.027

  • CrossRef
  • Google Scholar

• 15

  KimRChangJMLeeHBLeeSHKimSYKimESet al. Predicting axillary response to neoadjuvant chemotherapy: Breast MRI and US in patients with node-positive breast cancer. Radiology (2019) 293(1):49–57. doi: 10.1148/radiol.2019190014

  • CrossRef
  • Google Scholar

• 16

  SamieiSde MooijCMLobbesMBIKeymeulenKvan NijnattenTJASmidtML. Diagnostic performance of noninvasive imaging for assessment of axillary response after neoadjuvant systemic therapy in clinically node-positive breast cancer: A systematic review and meta-analysis. Ann Surg (2021) 273(4):694–700. doi: 10.1097/SLA.0000000000004356

  • CrossRef
  • Google Scholar

• 17

  SammutSJCrispin-OrtuzarMChinSFProvenzanoEBardwellHAMaWet al. Multi-omic machine learning predictor of breast cancer therapy response. Nature (2022) 601(7894):623–9. doi: 10.1038/s41586-021-04278-5

  • CrossRef
  • Google Scholar

• 18

  TahmassebiAWengertGJHelbichTHBago-HorvathZAlaeiSBartschRet al. Impact of machine learning with multiparametric magnetic resonance imaging of the breast for early prediction of response to neoadjuvant chemotherapy and survival outcomes in breast cancer patients. Invest Radiol (2019) 54(2):110–7. doi: 10.1097/RLI.0000000000000518

  • CrossRef
  • Google Scholar

• 19

  MetiNSaedniaKLagreeATabbarahSMohebpourMKissAet al. Machine learning frameworks to predict neoadjuvant chemotherapy response in breast cancer using clinical and pathological features. JCO Clin Cancer Inform (2021) 5:66–80. doi: 10.1200/cci.20.00078

  • CrossRef
  • Google Scholar

• 20

  TeichgraeberDCGuirguis M SWhitmanGJ. Breast cancer staging: Updates in the AJCC cancer staging manual, 8th edition, and current challenges for radiologists, from the AJR special series on cancer staging. AJR Am J Roentgenol (2021) 217(2):278–90. doi: 10.2214/ajr.20.25223

  • CrossRef
  • Google Scholar

• 21

  HammondMEHayesDFWolffACMangu P BTeminS. American Society of clinical oncology/college of american pathologists guideline recommendations for immunohistochemical testing of estrogen and progesterone receptors in breast cancer. J Oncol Pract (2010) 6(4):195–7. doi: 10.1200/jop.777003

  • CrossRef
  • Google Scholar

• 22

  CortazarPZhangLUntchMMehtaKCostantinoJPWolmarkNet al. Pathological complete response and long-term clinical benefit in breast cancer: the CTNeoBC pooled analysis. Lancet (2014) 384(9938):164–72. doi: 10.1016/S0140-6736(13)62422-8

  • CrossRef
  • Google Scholar

• 23

  WolffACHammondMEHAllisonKHHarveyBEManguPBBartlettJMSet al. Human epidermal growth factor receptor 2 testing in breast cancer: American society of clinical Oncology/College of American pathologists clinical practice guideline focused update. Arch Pathol Lab Med (2018) 142(11):1364–82. doi: 10.5858/arpa.2018-0902-SA

  • CrossRef
  • Google Scholar

• 24

  FumagalliDBedardPLNahlehZMichielsSSotiriouCLoiSet al. A common language in neoadjuvant breast cancer clinical trials: proposals for standard definitions and endpoints. Lancet Oncol (2012) 13(6):e240–8. doi: 10.1016/s1470-2045(11)70378-3

  • CrossRef
  • Google Scholar

• 25

  ProvenzanoEBossuytVVialeGCameronDBadveSDenkertCet al. Standardization of pathologic evaluation and reporting of postneoadjuvant specimens in clinical trials of breast cancer: recommendations from an international working group. Mod Pathol (2015) 28(9):1185–201. doi: 10.1038/modpathol.2015.74

  • CrossRef
  • Google Scholar

• 26

  ChoNMoonWKHanWParkIACho JNohDY. Preoperative sonographic classification of axillary lymph nodes in patients with breast cancer: node-to-node correlation with surgical histology and sentinel node biopsy results. AJR Am J Roentgenol (2009) 193(6):1731–7. doi: 10.2214/AJR.09.3122

  • CrossRef
  • Google Scholar

• 27

  YangWTChang JMetreweliC. Patients with breast cancer: differences in color Doppler flow and gray-scale US features of benign and malignant axillary lymph nodes. Radiology (2000) 215(2):568–73. doi: 10.1148/radiology.215.2.r00ap20568

  • CrossRef
  • Google Scholar

• 28

  MaoYHedgire SHarisinghaniM. Radiologic assessment of lymph nodes in oncologic patients. Curr Radiol Rep (2013) 2(2):36. doi: 10.1007/s40134-013-0036-6

  • CrossRef
  • Google Scholar

• 29

  Le-PetrossHTMcCallLMHuntKKMittendorfEAAhrendtGMWilkeLGet al. Axillary ultrasound identifies residual nodal disease after chemotherapy: Results from the American college of surgeons oncology group Z1071 trial (Alliance). AJR Am J Roentgenol (2018) 210(3):669–76. doi: 10.2214/AJR.17.18295

  • CrossRef
  • Google Scholar

• 30

  KalafiEYNorNAMTaibNAGanggayahMDTown CDhillonSK. Machine learning and deep learning approaches in breast cancer survival prediction using clinical data. Folia Biol (Praha) (2019) 65(5-6):212–20.

  • Google Scholar

• 31

  GuoRSuYSiJXueJYangBZhangQet al. A nomogram for predicting axillary pathologic complete response in hormone receptor-positive breast cancer with cytologically proven axillary lymph node metastases. Cancer (2020) 126 Suppl 16:3819–29. doi: 10.1002/cncr.32830

  • CrossRef
  • Google Scholar

• 32

  VilaJMittendorfEAFaranteGBassettRLVeronesiPGalimbertiVet al. Nomograms for predicting axillary response to neoadjuvant chemotherapy in clinically node-positive patients with breast cancer. Ann Surg Oncol (2016) 23(11):3501–9. doi: 10.1245/s10434-016-5277-1

  • CrossRef
  • Google Scholar

• 33

  KimJYParkHSKimSRyuJPark SKimSI. Prognostic nomogram for prediction of axillary pathologic complete response after neoadjuvant chemotherapy in cytologically proven node-positive breast cancer. Med (Baltimore) (2015) 94(43):e1720. doi: 10.1097/MD.0000000000001720

  • CrossRef
  • Google Scholar

• 34

  SchipperRJMoossdorffMNelemansPJNieuwenhuijzenGAde VriesBStrobbeLJet al. A model to predict pathologic complete response of axillary lymph nodes to neoadjuvant chemo(immuno)therapy in patients with clinically node-positive breast cancer. Clin Breast Cancer (2014) 14(5):315–22. doi: 10.1016/j.clbc.2013.12.015

  • CrossRef
  • Google Scholar

• 35

  LoiblSPoortmansPMorrowMDenkert CCuriglianoG. Breast cancer. Lancet (2021) 397(10286):1750–69. doi: 10.1016/s0140-6736(20)32381-3

  • CrossRef
  • Google Scholar

• 36

  MougalianSSHernandezMLeiXLynchSKuererHMSymmansWFet al. Ten-year outcomes of patients with breast cancer with cytologically confirmed axillary lymph node metastases and pathologic complete response after primary systemic chemotherapy. JAMA Oncol (2016) 2(4):508–16. doi: 10.1001/jamaoncol.2015.4935

  • CrossRef
  • Google Scholar

• 37

  FayanjuOMRenYThomasSMGreenupRAPlichtaJKRosenbergerLHet al. The clinical significance of breast-only and node-only pathologic complete response (pCR) after neoadjuvant chemotherapy (NACT): A review of 20,000 breast cancer patients in the national cancer data base (NCDB). Ann Surg (2018) 268(4):591–601. doi: 10.1097/SLA.0000000000002953

  • CrossRef
  • Google Scholar

• 38

  GajdosCTartterPIEstabrookAGistrakMAJaffer SBleiweissIJ. Relationship of clinical and pathologic response to neoadjuvant chemotherapy and outcome of locally advanced breast cancer. J Surg Oncol (2002) 80(1):4–11. doi: 10.1002/jso.10090

  • CrossRef
  • Google Scholar

• 39

  GradisharWJWedamSBJahanzebMErbanJLimentaniSATsaiKTet al. Neoadjuvant docetaxel followed by adjuvant doxorubicin and cyclophosphamide in patients with stage III breast cancer. Ann Oncol (2005) 16(8):1297–304. doi: 10.1093/annonc/mdi254

  • CrossRef
  • Google Scholar

• 40

  KimWHKimHJParkHYParkJYChaeYSLeeSMet al. Axillary pathologic complete response to neoadjuvant chemotherapy in clinically node-positive breast cancer patients: A predictive model integrating the imaging characteristics of ultrasound restaging with known clinicopathologic characteristics. Ultrasound Med Biol (2019) 45(3):702–9. doi: 10.1016/j.ultrasmedbio.2018.10.026

  • CrossRef
  • Google Scholar

• 41

  EunNLSonEJGweonHMKim J AYoukJH. Prediction of axillary response by monitoring with ultrasound and MRI during and after neoadjuvant chemotherapy in breast cancer patients. Eur Radiol (2020) 30(3):1460–9. doi: 10.1007/s00330-019-06539-4

  • CrossRef
  • Google Scholar

• 42

  ChoiHJRyuJMKimINamSJKimSWYuJet al. Prediction of axillary pathologic response with breast pathologic complete response after neoadjuvant chemotherapy. Breast Cancer Res Treat (2019) 176(3):591–6. doi: 10.1007/s10549-019-05214-y

  • CrossRef
  • Google Scholar

• 43

  GradisharWJMoranMSAbrahamJAftRAgneseDAllisonKHet al. Breast cancer, version 3.2022, NCCN clinical practice guidelines in oncology. J Natl Compr Canc Netw (2022) 20(6):691–722. doi: 10.6004/jnccn.2022.0030

  • CrossRef
  • Google Scholar

• 44

  HeJLiJChengYFanJGuoJJiangZet al. Chinese Society of clinical oncology guidelines committee. guidelines of Chinese society of clinical oncology (CSCO) for breast cancer diagnosis and treatment 2021. Peking: People's Medical Publishing House (2021).

  • Google Scholar

• 45

  The Society of Breast Cancer of China Anti-Cancer Association. Guidelines and specifications of China anti-cancer association for diagnosis and treatment of breast cancer. China Oncol (2021) 31(10):954–1040. doi: 10.19401/j.cnki.1007-3639.2021.10.013

  • CrossRef
  • Google Scholar

• 46

  YangBRenGSongEPanDZhangJWangYet al. Current status and factors influencing surgical options for breast cancer in China: A nationwide cross-sectional survey of 110 hospitals. Oncologist (2020) 25(10):e1473–80. doi: 10.1634/theoncologist.2020-0001

  • CrossRef
  • Google Scholar

• 47

  ZhengMHuangYPengJXiaYCuiYHanXet al. Optimal selection of imaging examination for lymph node detection of breast cancer with different molecular subtypes. Front Oncol (2022) 12:762906. doi: 10.3389/fonc.2022.762906

  • CrossRef
  • Google Scholar

• 48

  MarinoMAAvendanoDZapataPRiedl C CPinkerK. Lymph node imaging in patients with primary breast cancer: Concurrent diagnostic tools. Oncologist (2020) 25(2):e231–42. doi: 10.1634/theoncologist.2019-0427

  • CrossRef
  • Google Scholar

• 49

  OchiTTsunodaHMatsudaNNozakiFSuzukiKTakeiHet al. Accuracy of morphologic change measurements by ultrasound in predicting pathological response to neoadjuvant chemotherapy in triple-negative and HER2-positive breast cancer. Breast Cancer (2021) 28(4):838–47. doi: 10.1007/s12282-021-01220-5

  • CrossRef
  • Google Scholar

• 50

  LiZTongYChen XShenK. Accuracy of ultrasonographic changes during neoadjuvant chemotherapy to predict axillary lymph node response in clinical node-positive breast cancer patients. Front Oncol (2022) 12:845823. doi: 10.3389/fonc.2022.845823

  • CrossRef
  • Google Scholar

• 51

  HyltonNMBlumeJDBernreuterWKPisanoEDRosenMAMorrisEAet al. Locally advanced breast cancer: MR imaging for prediction of response to neoadjuvant chemotherapy–results from ACRIN 6657/I-SPY TRIAL. Radiology (2012) 263(3):663–72. doi: 10.1148/radiol.12110748

  • CrossRef
  • Google Scholar

• 52

  MorganCStringfellowTDRolphRKovacsTKothariAPinderSEet al. Neoadjuvant chemotherapy in patients with breast cancer: Does response in the breast predict axillary node response? Eur J Surg Oncol (2020) 46(4 Pt A):522–6. doi: 10.1016/j.ejso.2019.11.498

  • CrossRef
  • Google Scholar

• 53

  MannRMHooleyRBarr R GMoyL. Novel approaches to screening for breast cancer. Radiology (2020) 297(2):266–85. doi: 10.1148/radiol.2020200172

  • CrossRef
  • Google Scholar