AUTHOR=Phatak Sanat , Chakraborty Somashree , Goel Pranay 

TITLE=Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort

JOURNAL=Frontiers in Medicine

VOLUME=Volume 10 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/medicine/articles/10.3389/fmed.2023.1280462

DOI=10.3389/fmed.2023.1280462

ISSN=2296-858X

ABSTRACT=Introduction: Computer vision extracts meaning from pixelated images and holds promise in automating clinical tasks. Convolutional neural networks (CNN), deep learning networks used therein, have shown promise in X-ray images as well as joint photographs. We studied the performance of a CNN on standardized smartphone photographs in detecting inflammation in three hand joints, comparing it to a rheumatologist diagnosis.
Methods: We enrolled 100 consecutive patients with inflammatory arthritis of less than two years duration and excluded those with deformities. Each patient was examined by a rheumatologist and the presence of synovitis in each joint was recorded. Hand photographs were taken in a standardized manner and anonymized. Images were cropped to include joints of interest. A ResNet-101 backbone modified for two class outputs (inflamed or not) was used for training. We also tested a hue augmented dataset. We report accuracy, sensitivity and specificity for three joints: wrist, index finger proximal interphalangeal (IFPIP), middle finger interphalangeal (MFPIP) taking rheumatologist opinion as gold standard.
 
Results:
The cohort (n=100, 22 males) had a mean age of 49.7(SD12.9) years; most had rheumatoid arthritis(n=68, 68%). The wrist (125/200, 62.5%), MFPIP (94/200, 47%) and IFPIP (83/200, 41.5%) were the three most commonly inflamed joints. The CNN achieved the highest accuracy, sensitivity, specificity in being able to detect synovitis in the MFPIP (83%, 77%, 88% respectively) followed by the IFPIP (74%, 74%, 75%) and the wrist (62%, 90%, 21%). 
Discussion: We show that computer vision was able to detect inflammation in three joints of the hand with reasonable accuracy on standardized photographs despite a small dataset. Feature engineering was not required, and the CNN worked despite a diversity in clinical diagnosis. Larger datasets are likely to improve accuracy and help explain the basis of classification. These data suggest a potential use of computer vision in screening and follow-up of inflammatory arthritis.