AUTHOR=Huang Yechong , Xu Jiahang , Zhou Yuncheng , Tong Tong , Zhuang Xiahai , the Alzheimer’s Disease Neuroimaging Initiative (ADNI) TITLE=Diagnosis of Alzheimer’s Disease via Multi-Modality 3D Convolutional Neural Network JOURNAL=Frontiers in Neuroscience VOLUME=Volume 13 - 2019 YEAR=2019 URL=https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2019.00509 DOI=10.3389/fnins.2019.00509 ISSN=1662-453X ABSTRACT=Abstract Alzheimer’s Disease (AD) is one of the most concerned neurodegenerative diseases. In the last decade, studies on AD diagnosis attached great significance to artificial intelligence (AI)-based diagnostic algorithms. Among the diverse modality imaging data, T1-weighted MRI and 18F-FDG-PET are widely researched for this task. In this paper, we propose a novel convolutional neural network (CNN) to fuse the multi-modality information including T1-MRI and FDG-PDT images around the hippocampal area for the diagnosis of AD. Different from the traditional machine learning algorithms, this method does not require manually extracted features, and utilizes the state-of-art 3D image-processing CNNs to learn features for the diagnosis and prognosis of AD. To validate the performance of the proposed network, we trained the classifier with paired T1-MRI and FDG-PET images using the ADNI datasets, including 731 Normal (NL) subjects, 647 AD subjects, 441 stable MCI (sMCI) subjects and 326 progressive MCI (pMCI) subjects. We obtained the maximal accuracies of 90.10% for NL/AD task, 87.46% for NL/pMCI task, and 76.90% for sMCI/pMCI task. The proposed framework yields comparative results against state-of-the-art approaches. Moreover, the experimental results have demonstrated that (1) segmentation is not a prerequisite by using CNN, (2) the hippocampal area provides enough information to give a reference to AD diagnosis.