AUTHOR=Roche Stephanie D. , Ekwunife Obinna I. , Mendonca Rouella , Kwach Benn , Omollo Victor , Zhang Shengruo , Ongwen Patricia , Hattery David , Smedinghoff Sam , Morris Sarah , Were Daniel , Rech Dino , Bukusi Elizabeth A. , Ortblad Katrina F. 

TITLE=Measuring the performance of computer vision artificial intelligence to interpret images of HIV self-testing results

JOURNAL=Frontiers in Public Health

VOLUME=Volume 12 - 2024

YEAR=2024

URL=https://www.frontiersin.org/journals/public-health/articles/10.3389/fpubh.2024.1334881

DOI=10.3389/fpubh.2024.1334881

ISSN=2296-2565

ABSTRACT=HIV self-testing (HIVST) remains underutilized as a diagnostic tool in community-based, differentiated HIV service delivery models, possibly due to concerns about result misinterpretation. Ensuring that HIVST results are accurately interpreted for correct clinical decisions will be critical to maximizing HIVST's potential. We sought to understand how well AI technology performed at interpreting HIVST results. At 20 private pharmacies in Kisumu, Kenya, we offered free blood-based HIVST to clients ≥18 years purchasing products indicative of sexual activity. Each self-test was interpreted independently by the (1) client and (2) pharmacy provider, with the HIVST images subsequently interpreted by (3) an AI algorithm (trained on lab-captured images of HIVST results) and (4) an expert panel of three HIVST readers, used as the ground truth result. We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the AI algorithm as well as for pharmacy clients and providers, for comparison. From March to June 2022, we screened 1691 pharmacy clients and enrolled 1500 in the study. All clients completed HIVST. Among 854 clients whose HIVST images were of sufficient quality to be interpretable by the AI algorithm, 63% (540/854) were female, median age was 26 years ( interquartile range: 22-31), and 39% (335/855) reported casual sexual partners. The expert panel identified 94.9% (808/854) of HIVST images as HIV-negative, 5.1% (44/854) as HIV-positive, and 0.2% (2/854) as indeterminant. The AI algorithm demonstrated perfect sensitivity (100%), perfect NPV (100%), and 98.8% specificity, and 81.5% PPV (81.5%) due to seven false-positive results. By comparison, pharmacy clients and providers demonstrated lower sensitivity (93.2% and 97.7% respectively) and NPV (99.6% and 99.9% respectively) but perfect specificity (100%) and perfect PPV (100%).Conclusions: AI computer vision technology shows promise as a tool for providing additional quality assurance of HIV testing, particularly for catching Type II error (false-negative test interpretations) committed by human end-users. We discuss possible use cases for this technology to support differentiated HIV service delivery and identify areas for future research that is needed to assess the potential impacts-both positive and negative-of deploying this technology in real-world HIV service delivery settings.