Transfer learning with class activation maps in compositions driving carotid ultrasound plaque classification

Georgia D Liapi¹Christos P Loizou¹Constantinos S Pattichis²Andrew Nicolaidis³Efthyvoulos Kyriacou^1*

• ¹Cyprus University of Technology, Limassol, Cyprus
• ²University of Cyprus, Nicosia, Nicosia, Cyprus
• ³Vascular Screening and Diagnosis Center, Nicosia, Cyprus

Carotid B-mode ultrasound (U/S) imaging provides more than the degree of stenosis in stroke risk assessment. Plaque morphology and texture have been extensively investigated in U/S images, revealing plaque compositions, such as juxtaluminal black areas close to lumen (JBAs), whose size is linearly related to the risk of stroke. Convolutional neural networks (CNNs) have joined the battle for the identification of high-risk plaques, although the ways they perceive asymptomatic (ASY), and symptomatic (SY) plaque features need further investigation. In this study, the objective was to assess if class activations maps (CAMs) can reveal which U/S grayscale-(GS)-based plaque compositions (lipid cores, fibrous content, collagen, and/or calcified areas) influence the model's understanding of the ASY and SY cases. We used Xception via Transfer Learning, as a base for feature extraction (all layers frozen), whose output we fed into a new Dense layer, followed by a new classification layer, which we trained with standardized B-mode U/S longitudinal plaque images. From a total of 236 images (118 ASY and 118 SY), we used 168 in training (84 ASY and 84 SY), 22 in internal validation (11 ASY and 11 SY), and 46 in testing (23 ASY and 23 SY). In testing, the model reached an Accuracy, Sensitivity, Specificity and Area under the curve at 80.4%, 82.6%, 78.3% and 0.80 respectively. Precision and the F1-score were found at 81.8% and 80.2%, and 79.2% and 80.9%, for the ASY and SY cases, respectively. We used faster-Score-CAM to produce a heatmap for each tested image, quantifying each plaque composition area overlapping with the heatmap to find compositions areas related to ASY and SY cases. Dark areas (GS≤25) or JBAs (whose presence was verified priorly, by an experienced vascular surgeon) were found influential for the understanding of both the ASY and SY plaques. Calcified areas, fibrous content, and lipid cores, together, were more related to ASY plaques. These findings indicate the need for further investigation on how the GS≤25 plaque areas affect the CNN models' learning, and they will be further validated.