Lung cancer is the most common malignancy in the world and is the highest cause for cancer mortality (1). With a very poor prognosis, the 5-year survival rate for lung cancer is only 19.7% (2, 3) despite recent improvements (4). According to the eighth edition of the TNM staging of lung cancer published by the International Association for the Study of Lung Cancer, 80% of patients with stage IA non–small-cell lung cancer (NSCLC) are alive for ≥5 years after diagnosis. However, this proportion drops to <10% in patients with stage IV disease (5). The poor survival of patients with lung cancer may primarily be due to the fact that the majority of patients are diagnosed at an advanced stage (6).

Based on the findings of the National Lung Screening Trial (7), computed tomography (CT) or low-dose CT (LDCT) has been recommended as an effective tool for lung cancer screening in many countries or regions (8–11). Although CT or LDCT helps detect lung cancer at an early stage, the majority of pulmonary nodules (PNs) detected by CT are benign (7). Identifying malignant PNs from benign ones has become a challenge for clinicians, and follow-up examinations (e.g., follow-up scans and invasive biopsies) may lead to additional costs or harm the patient (12). In recent decades, several lung cancer risk prediction models based on radiological characteristics and clinical information have been developed to assist clinicians in managing patients with pulmonary nodules (13–18). These models have demonstrated a high value in discriminating independent cohorts. Moreover, some of them were recommended by guidelines for the classification of high- and low-risk pulmonary nodules (11).

However, most of these models were built on initial CT plain or LDCT scans and were used at the baseline. However, the diagnostic performance of models may be inaccurate within dissimilar populations. Clinicians rarely recommend performing an invasive procedure in patients with PNs after their initial scan. Consequently, a period of observation for PNs often exists before they make a decision. Having a tool accurate enough to assist clinicians in judging would be clinically useful to help avoid overdiagnosis and facilitate early diagnosis before deciding on an invasive procedure. Different from the previously reported models, the subjects of this study were those with PNs that were highly suspected by clinicians to be lung cancer (all of these patients underwent surgery).

Early detection and accurate diagnosis are effective ways to lower lung cancer mortality. Given the occult onset, CT screening may be currently the preferred test for early diagnosis and management of clinically significant lung nodules. However, the optimal target PNs and the timing of biopsy remain uncertain (23). The American College of Chest Physicians (CHEST) guidelines for lung cancer screening (version 2021) summarized the results of 17 clinical trials and revealed that 22.0% of surgeries were performed for benign diseases (ranged from 8% to 39%) (24). How to reduce benign resection without delaying the diagnosis of lung cancer has become a research hotspot. This evidence-based, retrospective project established a malignancy risk prediction model to reassess the PNs that clinicians considered need to be biopsied. This study reviewed data from 763 subjects diagnosed with lung nodules that were clinically considered to be highly malignant who underwent surgical resection in between 2017 and 2020. Except for a few confirmed benign diseases, most nodules were considered to be malignant preoperatively. Despite the received observation and intervention recommended in the guidelines (11, 25, 26) before surgery, nearly one in three nodules remained benign. The current initial M1, built with all predictors, showed excellent predictive accuracy (with an AUC of 0.876 in an external validation cohort) and calibration ( Figure 1 ). M2 was built because of the difference in the distributions of benign and malignant lesions in three nodule densities. However, M2 did not perform better in classifying solid nodules in the validation cohort (AUC, 0.904 vs. 0.896) than M1. Serum tumor markers did not prove to be a strong predictor as anticipated in the multivariate analyses. Thus, the M1b model was built to exclude tumor markers. In the validation data, which tend to be lower tumor markers levels even when malignant, M1 did not perform better than M1b. Even if CEA levels show differences between benign and malignant nodules, the effectiveness of tumor markers in the classification of PNs needs further verification.

Smoking is a risk factor for lung cancer (13, 14, 27). The smoking rate of malignant cohort in this study was 37% which was much lower than that in other studies, especially screen-based studies (15, 28, 29). Moreover, smoking history was an independent predictor for lung cancer in the current final multivariable model although no difference was demonstrated in the groups. The smoking prevalence in the current study may be lower greatly because of the varying smoking habits in the male and female populations (30). Females had a lower smoking prevalence than males in this study (10.2% vs. 68.2%, p < 0.001). Moreover, females were significantly associated with malignant PNs, which agrees with previous studies (15, 16, 31). Emphysema or COPD had been noticed to increase the risk of lung cancer (32), but it was not observed in this study. An intranodular vascularity was found to strongly correlate with lung cancer risk, which is consistent with the theory of tumor angiogenesis (33). Malignancy proportion was more frequent in subsolid nodules than in solid nodules because most subsolid nodules resected in this study were monitored until change in follow-up CT features. However, this process may exclude some benign lesions. Changes in CT image of subsolid PNs suggest malignancy (34, 35). Although the largest in diameter did not mean the highest probability of malignancy (15), similar to previous studies (13, 14, 18), malignancies were more often found in bigger nodules in our study (17 mm vs. 14 mm, p = 0.001). Other risk factors for earlier lung cancer differential diagnosis (e.g., nodules with spiculation, lobulation, calcification, or pleural indentation) were also significantly associated with lung cancer in this study (36–38).

Unlike previous models (13–18), the current model was determined following the preoperative contrast-enhanced CT scan and serum tumor markers. In the external validation set, the AUC for the current models was 0.876 compared with 0.644 and 0.683 for the Mayo and Brock models ( Figure 2 ). These models were also compared using the decision curve ( Figure 3 ), which showed that the current model had higher discriminatory power for malignancy than the Mayo or the Brock model. The density distribution of the predicted probability score of these models on the validation set was plotted to figure out whether these differences would be helpful in the clinical management of patients with PNs with a risk that is high enough to have an invasive procedure ( Figure 4 ). The current model classified 79% and 2% of malignant nodules at a probability threshold of ≥0.75 and ≤0.25, respectively. In comparison, the Mayo and Brock models have skewed score distributions for all PNs. Although the current model gave values for discrimination that outperforms the Mayo or the Brock model, they cannot be directly compared because accuracy can considerably vary within populations (39). The malignancy proportion of the Mayo (23.2%) and Brock (5.5%) models is much lower than that of the patients whose PNs were suspected to be malignant after observation recommended by guidelines. The models derived from the populations with a low prevalence of malignancy may underestimate the risk when used in the high-prevalence populations. Therefore, we suggest that medical centers could develop models according to their local populations to help with the clinical management of PNs, instead of directly applying some screening models. The current model is more suitable for reassessment for patients who were admitted for planned surgery or biopsy. The proportion of malignant and benign nodules in the density distribution of the predicted probability of the current model may be helpful in clinical decision-making given the pros and cons of observation, biopsy, or surgery ( Figure 4 ).

This study has several limitations. First, the history of previous imaging follow-up of the patient cohort was incomplete as ours was a tertiary referral center. Therefore, this study was unable to evaluate the effect of temporal nodule evolution. Moreover, there was a lack of uniform criteria for suspicion of malignancy, and they were determined based on the subjective judgment of thoracic surgeons. Furthermore, the time point to split the data into study and validation cohorts was used to limit the effect of overfitting. The current model may not perform as well in other study populations. Second, this study failed to build a model exclusively for subsolid nodules. The proportion of benign lesions was only 1 in 10 for subsolid nodules in this study and was too low to perform a multivariate logistic regression. The most likely explanation is that the subsolid nodules included in this study were all observed until they change in follow-up CT features. The changes were suspected to demonstrate usefulness in discriminating benign from malignant nodules. Unfortunately, however, we failed to sum up the period. Lastly, this study was not able to examine nodule classification models that incorporated other factors associated with lung cancer risk [i.e., positron emission tomography-CT (40) and nodule volume (16, 41)] due to the lack of such data.

This study developed and externally validated a risk model for estimating the probability of lung cancer in PNs that were recommended to have invasive interventions. The model could be considered before more invasive treatments to justify the necessity. Established by using readily available clinical information, this model provides valuable data for clinicians in decision-making. However, the application of the current model in identifying nodules in other populations, such as a screening population, needs further study.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the institutional ethics committee board of Tianjin Medical University General Hospital (No. IRB2019-KY-153), and individual consent for this retrospective analysis was waived.

Study design: CX, ML, and JC. Data acquisition: XiL, DW, YH, and DR. Quality control of data: CX, JC, XiL, MD, and HZ. Data analysis and interpretation: CX, JC, ML, HL, and XuL. Statistical analysis: CX, ML, JC, HZ. Manuscript preparation: CX, JC, ML, XiL, and HL. Manuscript editing and reviewing: all authors. All authors contributed to the article and approved the submitted version.

This study was supported by grants from the National Natural Science Foundation of China (82072595, 81773207, and 61973232), Natural Science Foundation of Tianjin (18PTZWHZ00240, 19YFZCSY00040, and 19JCYBJC27000), Shihezi University Oasis Scholars Research Startup Project (LX202002) and Special Support Program for the High Tech Leader and Team of Tianjin (TJTZJH-GCCCXCYTD-2-6), Tianjin Municipal Education Commission Natural Science Foundation (2019KJ202, 2020KJ151), and Tianjin Medical University General Hospital Incubation Fund (ZYYFY2017034). The funding sources had no role in the study design, data collection, and analysis; the decision to publish; or the preparation of the manuscript.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.