Automated Multi-View Multi-Modal Assessment of COVID-19 Patients Using Reciprocal Attention and Biomedical Transform - Li, Yanhan; Zhao, Hongyun; Gan, Tian; Liu, Yang; Zou, Lian; Xu, Ting; Chen, Xuan; Fan, Cien; Wu, Meng (Frontiers in Public Health, 2022)
Automated severity assessment of COVID-19 patients can help rationally allocate medical resources and improve patients' survival rates. The existing methods conduct severity assessment tasks mainly on unitary modal and single view, which is apt to omit potential interactive information. To tackle the problem, in this paper, we propose a multi-view multi-modal model to automatically assess the severity of COVID-19 patients based on deep learning. The proposed model receives multi-view ultrasound images and biomedical indices of patients and generates comprehensive features for assessment tasks. Also, we propose Reciprocal Attention module to acquire the underlying interactions between multi-view ultrasound data. Moreover, we propose Biomedical Transform module to integrate biomedical data with ultrasound data to produce multi-modal features. The proposed model is trained and tested on compound datasets and it yields the best performance compared to other state-of-the-art methods. Further ablation experiments and discussions conformably indicate the feasibility and advancement of the proposed model.