AUTHOR=Xia Tian , Sanchez Pedro , Qin Chen , Tsaftaris Sotirios A. 

TITLE=Adversarial counterfactual augmentation: application in Alzheimer’s disease classification

JOURNAL=Frontiers in Radiology

VOLUME=Volume 2 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/radiology/articles/10.3389/fradi.2022.1039160

DOI=10.3389/fradi.2022.1039160

ISSN=2673-8740

ABSTRACT=Deep learning has become an increasingly important approach in the field of medical image analysis in the past decade. However, due to the limited availability of medical data, deep-learning based algorithms could suffer from over-fitting, i.e. a phenomenon where deep learning models cannot generalise well to unseen data. Data augmentation has been widely used in deep learning to reduce over-fitting and improve the robustness of models. However, traditional data augmentation techniques, e.g., rotation, cropping, flipping, etc., do not consider \textit{semantic} transformations, i.e., changing specific attributes of the input image. Previous works tried to achieve semantic augmentation by generating counterfactuals, e.g. synthesised data by deep generative models, but they focused on how to train deep generative models and created counterfactuals with the generative models indiscriminately inefficiently  without considering which counterfactuals are most effective for improving downstream training of the target task, e.g. classification. Different from these approaches,  we propose a novel adversarial counterfactual augmentation scheme that aims to find the most effective counterfactuals to improve downstream tasks given a pre-trained generative model. Specifically, we construct an adversarial game where  we update the input conditional factor of the generator and the downstream classifier with gradient backpropagation alternatively and iteratively.  The key idea is to find conditional factors that can result in hard counterfactuals for the classifier. This can be viewed as finding the `weakness' of the classifier and purposely forcing it to overcome its weakness via the generative model. To demonstrate the effectiveness of the proposed approach, we validate the method with the classification of Alzheimer's Disease (AD) as a downstream task based on a pre-trained model with synthesises brain images of desired age. Extensive experiments and ablation studies have been performed to show that the proposed approach improves classification performance and has potential to alleviate spurious correlations and catastrophic forgetting.  Code will be released upon acceptance.