AUTHOR=Li Huanyu , Zhang Peng , Wei Zikun , Qian Tian , Tang Yiqi , Hu Kun , Huang Xianqiong , Xia Xinxin , Zhang Yishuang , Cheng Haixing , Yu Fubing , Zhang Wenjia , Dan Kena , Liu Xuan , Ye Shujun , He Guangqiao , Jiang Xia , Liu Liwei , Fan Yukun , Song Tingting , Zhou Guomin , Wang Ziyi , Zhang Daojun , Lv Junwei 

TITLE=Deep skin diseases diagnostic system with Dual-channel Image and Extracted Text

JOURNAL=Frontiers in Artificial Intelligence

VOLUME=Volume 6 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2023.1213620

DOI=10.3389/frai.2023.1213620

ISSN=2624-8212

ABSTRACT=Background:Due to less reliant on laboratory tests, skin diseases are more suitable for diagnosis with AI models. There are limited AI dermatology diagnostic models combining images and text, few for Asian populations, and few covering the most common types of diseases.:Leveraging a dataset sourced from Asia, comprising over 200,000 images and 220,000 medical records, we explored a deep learning-based system for Dual-channel image and extracted text for the diagnosis of skin diseases model-DIET-AI to diagnose 31 skin diseases, covering the majority of common skin diseases. Ranging from 1st September to 1st December 2021, we prospectively collected 6043 cases images and medical records from 15 hospitals in 7 provinces in China. Then the performance of the DIET-AI with 6 doctors of different seniority in the clinical dataset was compared. Results:The average performance of the DIET-AI in 31 diseases is no less than that of all different seniority doctors. By comparing the area under the curve, sensitivity and specificity, we demonstrate that DIET-AI model is effective under the clinical scenario. In addition, medical records affect the performance of DIET-AI and physicians to varying degrees. Conclusion:This is the largest Dermatological dataset for the Chinese ethnic group. For the first time, we built a Dual-channel image classification model on non-cancer Dermatitis dataset with both images and medical records, achieved comparable diagnostic performance to senior doctors on common skin diseases. It provides references for exploring the feasibility and performance evaluation of DIET-AI in clinical use afterward.