Multimodal Deep Learning Model for Enhanced Early Detection of Aortic Stenosis Integrating ECG and Chest X-ray with Cooperative Learning

Shun Nagai¹Makoto Nishimori^2*Masakzu Shinohara²Hidekazu Tanaka¹Hiromasa Otake¹

• ¹Division of Cardiovascular Medicine, Department of Internal Medicine, Kobe University Graduate School of Medicine, Kobe, Japan, kobe, Japan
• ²Division of Molecular Epidemiology, Kobe University Graduate School of Medicine, Kobe, Japan, kobe, Japan

Background: Aortic stenosis (AS) is diagnosed by echocardiography, the current gold standard, but examinations are often performed only after symptoms emerge, highlighting the need for earlier detection. Recently, artificial intelligence (AI)–based screening using non-invasive and widely available modalities such as electrocardiography (ECG) and chest X-ray(CXR) has gained increasing attention for valvular heart disease. However, single-modality approaches have inherent limitations, and in clinical practice, multimodality assessment is common. In this study, we developed a multimodal AI model integrating ECG and CXR within a cooperative learning framework to evaluate its utility for earlier detection of AS. Methods: We retrospectively analyzed 23,886 patient records from 7,483 patients who underwent ECG, CXR, and echocardiography. A multimodal model was developed by combining a 1D ResNet50– Transformer architecture for ECG data with an EfficientNet-based architecture for CXR. Cooperative learning was implemented using a loss function that allowed the ECG and CXR models to refine each other's predictions. We split the dataset into training, validation, and test sets, and performed 1,000 bootstrap iterations to assess model stability. AS was defined echocardiographically as peak velocity ≥2.5 m/s, mean pressure gradient ≥20 mmHg, or aortic valve area ≤1.5 cm². Results: Among 7,483 patients, 608 (8.1%) were diagnosed with AS. The multimodal model achieved a test AUROC of 0.812 (95% CI: 0.792–0.832), outperforming the ECG model (0.775, 95% CI: 0.753– 0.796) and the CXR model (0.755, 95% CI: 0.732–0.777). Visualization techniques (Grad-CAM, Transformer attention) highlighted distinct yet complementary features in AS patients. Conclusions: The multimodal AI model via cooperative learning outperformed single-modality methods in AS detection and may aid earlier diagnosis and reduce clinical burden.