Inferring Tumor Immune Microenvironment (TIME)-Related Risk States from Pretreatment H&E Pathomics and Clinical Biomarkers to Predict Checkpoint Inhibitor Pneumonitis in Advanced NSCLC: A Multicenter Multimodal Study

Lei Yuan¹Qi Wang¹Fei Sun²Wenlong Yang³Jie Lei³Juan Liu⁴Yiwei Fan³Yibo Shan¹Yi Lu¹Yaojing Zhang¹Yilun Wang¹Jianwei Zhu¹Guo Lintao¹Wenxuan Chen¹Shichun Lu³Hongcan Shi^1,3*

• ¹Medical College, Yangzhou University, Yangzhou, China
• ²The Affiliated Taizhou People's Hospital of Nanjing Medical University, Taizhou, China
• ³Northern Jiangsu People's Hospital, Yangzhou, China
• ⁴The Fourth People's Hospital of Lianyungang, Affiliated with Kangda College of Nanjing Medical University, Lianyungang, China

Background: Checkpoint inhibitor pneumonitis (CIP) is a rare but potentially fatal immune-related adverse event (irAE) that can interrupt immune checkpoint blockade in NSCLC. With no validated pretreatment biomarkers and a diagnosis largely made by exclusion, upfront risk stratification is required. Recent advances in AI-driven pathomics have made it feasible to infer TIME-relevant risk states in patients with NSCLC. Accordingly, we leveraged hematoxylin and eosin(H&E)-based digital pathomics combined with clinical variables to interrogate the TIME in patients who developed CIP and to enable pretreatment and early prediction of CIP. Methods: In this retrospective study, 346 eligible patients from three hospitals were screened consecutively between January 2022 and January 2025. Patients were divided into CIP and non-CIP groups. We first developed a pathomics model that employed convolutional neural networks combined with multi-instance learning to generate predictions at both the patch and whole slide image levels on H&E-stained slides. Separately, we constructed a clinical model using logistic regression to process the structured clinical data accompanying each case. Subsequently, pathological and clinical information were integrated, where modeling was advanced from modality-specific feature learning to cross-modal representation learning, and final predictive modeling was completed. The predictive performance of different models was evaluated using the area under the ROC curve and benchmarked against unimodal models and standard ensemble methods. Results: When the models were evaluated across both internal validation and external test datasets, the pathomics model demonstrated noticeably stronger performance than the clinical approach, achieving area under the curve scores of 0.916, 0.875(test 1), and 0.843(test 2), respectively, while the clinical model posted more modest results of 0.880, 0.569(test 1), and 0.594(test 2). The most significant outcome, however, emerged from the multimodal fusion model, which produced the strongest results of all, with performance metrics of 0.930, 0.919(test 1), and 0.905(test 2) in the validation and test phases, respectively. Conclusion: Pretreatment H&E-derived pathomics, integrated with baseline clinical biomarkers, enable accurate prediction of CIP risk in locally advanced or metastatic NSCLC. This framework supports proactive surveillance and individualized ICI strategies and provides a scalable route to decode TIME-relevant states from routine pathology.