Study protocol was approved by the institutional review boards of Sun Yat-Sen University Cancer Center (SYSUCC). Consecutive patients histologically diagnosed as breast cancer between January 2000 and March 2017 in SYSUCC were retrospectively reviewed. Inclusion criteria were as follows: (1) received mastectomy and IMLN surgery; (2) female; (3) pathological diagnosed as invasive ductal carcinoma (IDC) or invasive lobular carcinoma (ILC). Patients without enough data could be extracted. Every enrolled patient was randomly allocated as “training” or “validation” at the ratio of 2:1 and 75% of participants were selected as the training cohort. The remaining patients were grouped as the validation cohort. All enrolled patients had mastectomy and we conducted the IMLN surgery according to patients' pre-operative examination IMLNs dissection were conducted by video-assisted thoracoscopic surgery (VATS) for most patients. Of course, primary tumors in the inner quadrant are more likely to undergo IMLNs dissection, however, our case data were collected retrospectively and continuously.

Clinical characteristics collected for subsequent analysis included age, tumor location, pathological tumor stage, number of positive axillary lymph nodes, estrogen receptor (PR), progesterone receptor (PR), and human epidermal growth factor receptor-2 (HER2) status, Neoadjuvant chemotherapy (NAC) received or not, and imaging-reported IMLN status. The clinical stages were classified according to the AJCC TNM staging system (7th edition). ER and PR positivity were defined by the presence of more than 1% positive cells based on immunohistochemistry results, while HER2 positive was defined as “3+” in immunohistochemical test or “positive” in HER2 fluorescence in situ hybridization test. In our study, positive axillary lymph nodes (ALN) were defined after ALN dissection or sentinel node with complementary ALN dissection pathologically. Moreover, the detection methods for imaging-reported IMLN involvement status included computed tomography (CT) and/or Magnetic resonance examination (MR). Imaging scans were reviewed by two radiologists with >10 years of experience, who were blinded to clinical characteristics and post-operative pathological findings. Patients with IMLN of >1 cm and/or clusters of ≥3 lymph nodes were identified as clinically LN-positive, and patients without enlarged or clustered lymph nodes were regarded as clinically LN-negative. Any disagreement was resolved by consultation. Patients were censored from follow-up for survival at June 30, 2018, and from followup for relapse at the latest known recurrence-free date before or at June 30, 2018.

The primary outcome for this study was the likelihood of positive lymph nodes in internal mammary area following IMLN surgery. Multivariate logistic regression analysis was used to test the association between clinicopathological variables and the likelihood of IMLN metastasis. Coefficients for each variable and the constant in the equation were generated based on multivariate analysis. A nomogram was constructed to be a graphic representation of the prediction model with the R software.

Model performance was quantified in both the modeling group and the validation group with respect to discrimination and calibration. Discrimination was assessed by calculating the concordance index (c-index). Calibration was studied graphically after grouping patients into decile with respect to their predicted probabilities and plotting the mean predicted probabilities against the mean observed probabilities. Bootstrapping was applied to calculate 95% confidence intervals. Overall fit of the model was evaluated using the Hosmere-Lemeshow goodness of fit test for logistic regression. Reported P-values are two-sided with alpha 5%. Decision curve analysis for the nomogram was also evaluated. After obtaining the risk scores from the nomogram, we defined an optimal risk score cutoff value and patients were then classified into low- or high-risk groups accordingly. In low risk nomogram patients, the consistency between two breast oncology medical reviewers' recommendations and actual therapy were calculated in both the false negative (FN) and true negative (TN) groups separately, then we compared the clinical-pathological factors in the two groups.

Statistical analyses were performed using the statistical packages SPSS (SPSS for Windows, version 22.0, SPSS Inc., Chicago, IL) and R software (version 3.4.1; https://www.r-project.org/). The packages of R used in this study are as follows: “rms,” “Hmisc,” and “Dca.R.” The conventional 2-sided tests, and a significance level of 0.05 were used in all analyses. We compared the two groups using the χ² test or Fisher exact test for categorical variables.

A total of 152 female patients with primary invasive breast cancer (102 in the training cohort and 50 in the validation cohort) fulfill the inclusion criteria and were enrolled to develop and validate our predictive nomogram model (Figure 1). Among the 152 patients, only 4 patients are diagnosed as ILC, while others are IDC. All the enrolled patients underwent IMLN operation. Patient and tumor characteristics are shown in Table 1. No significant difference was observed between the training cohort and validation cohort regarding the clinicopathological factors analyzed. There were nine patients received NAC. The candidate positive rate of imaging-reported IMLN was only 17.1% (26/152), while the actual positive detection rate for IMLN was 28.9% (44/152). Among the ALN negative patients, 9 patients were IMLN positive pathologically. Similarly, among the imaging-reported IMLN negative patients, 35 patients were IMLN positive pathologically. A total of 489 IMLNs were examined and the average number of removed IMLNs per patients were 3. The average examined IMLN did not differ in the two cohorts (Table 1).

In univariate analysis of the training cohort (Table 2), IMLN metastases were significantly correlated with tumor location, tumor stage, LVI, and pathological axillary lymph node stage (pALN stage). In multivariable analysis of the training cohort (Table 2), IMLN metastasis were significantly correlated with tumor location, LVI and pALN stage. On the basis of the multivariable logistic regression of the training cohort, a nomogram incorporating the significant risk factors was set up to predict the involvement probability of IMLN (Figure 2A). A total score was calculated using tumor location, LVI and pALN stage. Each value of these risk factors was allocated a score on the point scale axis. For example, tumor location at lower outer quadrant (LOQ) was 0 point and upper inner quadrant (UIQ) was 79 points. Interestingly, the allocated scores of pathological ALN stage showed that patients with pALN3 and pALN2 might be more likely to be IMLN involvement than patients with pALN0 and pALN1. A total score could be easily calculated by adding each single score and located this sum on the total point scale axis. Then draw a vertical line downwards from this point and identify the IMLN metastasis risk probability after mastectomy.

The calibration curves for IMLN metastasis showed good calibration (Figures 2B,C) and predicted well in both the training cohort and the validation cohort (c-index: 0.86 for training cohort and 0.83 for validation cohort; Figures 2D,E). In addition, we conducted the decision curve analysis for the constructed nomogram model (Figures 3A,B). The plot showed that, for predicted probability thresholds between 0 and 84%, model-based decision showed a more net benefit than either the treat-none-patients scheme or the treat-all scheme.

After the enrolled patients were classified into low- or high- risk group by the nomogram model, we found 15 patients in low risk group are IMLN positive pathologically, while 15 patients in high risk group are identified to be IMLN negative (Figure 1). To further assess the potential adverse impact of false negative (FN) in low risk group classified by nomogram (FN-LR-nomogram) on adjuvant treatment strategy decision making, two breast oncologists performed a blinded review of clinical and pathological data (not include pIMLN status) from FN-LR-nomogram patients and matched ture negative (TN) patients in low risk group classified by nomogram (FN-LR-nomogram). The detailed data are shown in Supplementary Table 1. In the FN-LR-nomogram and TN-LR-nomogram groups, all risk factors were well-matched. Based on the data provided, the two reviewers made blinded adjuvant treatment recommendations for all LR-nomogram patients on the assumption that the patient's IMLN was negative pathologically. The treatments actually received were then compared with the recommendations of the two reviewers. There were no significant difference between the recommended treatment strategies and the treatment actually received in both the FN-LR-nomogram and TN-LR-nomogram groups (Table 3). Consistency between the reviewer #1 recommendation and the actual treatment strategy was 66.7% in the FN-LR-nomogram group and 80% in the TN-LR-nomogram group. Agreement between the reviewer #2 recommendation and the actual treatment strategy was 80% in the FN-LR-nomogram group and 77.3% in the TN-LR-nomogram group. Agreement between reviewer #1 and reviewer #2 was 80% in the FN-LR-nomogram group and 66.7% in the FN-LR-nomogram group. Details of the actual treatments received, and the treatments recommended in the blinded review are presented in Supplementary Figure 1.

Moreover, to evaluate the clinical impact of the FN and TN in an alternative way, we performed survival analyses comparing recurrence-free survival (RFS) based on nomogram and pathology results (Figures 4A,B). The RFS between FN-LR-nomogram and TN-LR-nomogram patients was equivalent (P = 0.47 or P = 0.053). We performed the cox regression analysis and found that only adjuvant radiotherapy can affect the patients' survival (Supplementary Table 2), however, in low-risk or high-risk patients (stratified by nomogram model), adjuvant radiotherapy cannot affect patients' prognosis (Supplementary Table 3). For low risk patients in nomogram, the similar survival status may be attributed to the similar treatment decision actually making in this two different subgroups, though the pathologically IMLN status is not the same. For high risk patients in nomogram, one possible reason for this result is that FN patients usually have adverse risk factors which can reduce the effectiveness of adjuvant treatment. Then, in this study, false classification by nomogram does not significantly affect treatment strategy and patient prognosis.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.