AUTHOR=Yi Sanli , Zhang Gang , Qian Chaoxu , Lu YunQing , Zhong Hua , He Jianfeng 

TITLE=A Multimodal Classification Architecture for the Severity Diagnosis of Glaucoma Based on Deep Learning

JOURNAL=Frontiers in Neuroscience

VOLUME=Volume 16 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.939472

DOI=10.3389/fnins.2022.939472

ISSN=1662-453X

ABSTRACT=Objective: Glaucoma is an optic nephropathy leads to characteristic visual field defects, for which the treatment in the early stage is essential. The goal of our study was achieve the severity diagnosis of glaucoma through multimodal fusion technology based on deep learning.
Method: We constructed a multimodal classification architecture based deep learning for severity diagnosis of glaucoma. In this architecture: firstly, we used multimodal fusion technology to integrate fundus image and grayscale image of visual field as the input of the architecture; secondly, the inherit limitation of Convolutional Neural Networks(CNNs) was offset by replacing its original classifier with ours. Furthermore, the grayscale images of visual field were reconstructed with higher resolution, and more subtle feature information was provided for glaucoma diagnosis. 
Result: This architecture is trained and tested on the datasets provided by Yunnan Cancer Hospital, the results show that the proposed architecture obtained the superior performance to assess the glaucoma diagnosis.
Conclusion: The study showed proposed multimodal classification architecture was feasible to predict the severity of glaucoma. The number of datasets restricted the evolution of proposed technology, it was necessary for accumulation of more clinical data and verification of its superiority.