AUTHOR=Li Yanhan , Zhao Hongyun , Gan Tian , Liu Yang , Zou Lian , Xu Ting , Chen Xuan , Fan Cien , Wu Meng 

TITLE=Automated Multi-View Multi-Modal Assessment of COVID-19 Patients Using Reciprocal Attention and Biomedical Transform

JOURNAL=Frontiers in Public Health

VOLUME=Volume 10 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/public-health/articles/10.3389/fpubh.2022.886958

DOI=10.3389/fpubh.2022.886958

ISSN=2296-2565

ABSTRACT=Automated severity assessment of COVID-19 patients can help rationally allocate medical resources and improve patients' survival rates. The existing methods conduct severity assessment tasks mainly on unitary modal and single view, which is apt to omit potential interactive information. To tackle the problem, in this paper, we propose a multi-view multi-modal model to automatically assess the severity of COVID-19 patients based on deep learning. The proposed model receives multi-view ultrasound images and biomedical indices of patients and generates comprehensive features for assessment tasks. Also, we propose Reciprocal Attention module to acquire the underlying interactions between multi-view ultrasound data. Moreover, we propose Biomedical Transform module to integrate biomedical data with ultrasound data to produce multi-modal features. The proposed model is trained and tested on compound datasets and it yields the best performance compared to other state-of-the-art methods. Further ablation experiments and discussions conformably indicate the feasibility and advancement of the proposed model.