AUTHOR=Bart Evgeniy , Hegdé Jay 

TITLE=Deep Synthesis of Realistic Medical Images: A Novel Tool in Clinical Research and Training

JOURNAL=Frontiers in Neuroinformatics

VOLUME=Volume 12 - 2018

YEAR=2018

URL=https://www.frontiersin.org/journals/neuroinformatics/articles/10.3389/fninf.2018.00082

DOI=10.3389/fninf.2018.00082

ISSN=1662-5196

ABSTRACT=Decision-making based on medical images is fundamentally a statistical decision-making exercise, in which the decision-maker must distinguish between image features that are clinically diagnostic from non-diagnostic variations (i.e., distinguishing signal from noise). To perform this task, the decision-maker must have learned the underlying distributions, i.e., identifying signal from noise. The same is true for machine learning algorithms that perform a given diagnostic task. In order to train and test human experts, or expert machine systems, in any diagnostic or analytical task, it is advisable to use large image sets, so as to capture the underlying statistical distributions adequately. Large numbers of images are also useful in clinical and scientific research in regards to the underlying diagnostic process, which remain poorly understood. Unfortunately, it is often difficult to obtain medical images of given specific descriptions in sufficiently large numbers. This represents a significant barrier to progress in the arenas of clinical care, education, and research. Here we describe a novel methodology that helps overcome this barrier. This method leverages the burgeoning technologies of deep learning (DL) and deep synthesis (DS) to synthesize images de novo. We provide a proof-of-principle of this approach using mammograms as an illustrative case. During the initial, prerequisite DL phase of the study, we trained a publicly available deep learning neural network (DNN), using open-sourced, radiologically vetted mammograms as labeled examples. During the subsequent DS phase of the study, the fully trained DNN was made to synthesize, de novo, images that capture the image statistics of a given input image. The resulting images indicated that our DNN was able to faithfully capture the image statistics of visually diverse sets of mammograms. We also briefly outline rigorous psychophysical testing methods to measure the extent to which synthesized images were sufficiently alike their original counterparts. We also present some results that are meant to illustrate the underlying testing and analytical methodologies, and are not meant to establish any empirical results per se. Altogether, this methodological approach has the potential to be impactful in all fields in which medical images play a key role, most notably in radiology and pathology.