AUTHOR=Zhang Heyu , He Yan , Wu Xiaomin , Huang Peixiang , Qin Wenkang , Wang Fan , Ye Juxiang , Huang Xirui , Liao Yanfang , Chen Hang , Guo Limei , Shi Xueying , Luo Lin 

TITLE=PathNarratives: Data annotation for pathological human-AI collaborative diagnosis

JOURNAL=Frontiers in Medicine

VOLUME=Volume 9 - 2022

YEAR=2023

URL=https://www.frontiersin.org/journals/medicine/articles/10.3389/fmed.2022.1070072

DOI=10.3389/fmed.2022.1070072

ISSN=2296-858X

ABSTRACT=Pathology is the gold standard of clinical diagnosis, especially for tumor diagnosis and treatment. Artificial intelligence (AI) in pathology has become a new trend, but its clinical use is still not widely accepted. One important challenge is that AI-suggested diagnosis results lack the explanations for pathologists to understand the rationale behind. By and large, clinic-compliant explanations besides diagnostic decision of pathological images are essential for AI model training to provide diagnostic suggestions assisting pathologists’ practice. In this paper we propose a new annotation form PathNarratives that can collect both decision labels and diagnostic logic reasoning for AI in pathology to better collaborate with human pathologists. PathNarratives includes a hierarchical decision-to-reason data structure, a narrative annotation process, and a multimodal interactive annotation tool. It allows annotators to work in a flexible and multimodal way by using clinical tags, voice, and pencil moving to not only mark the lesions but also point out the relative decisive features. Meanwhile the field-of-view moving and pausing behaviors are recorded simultaneously to together form the hierarchical annotation structure. Following PathNarratives, we recruited 8 pathologist annotators to build a colorectal pathological dataset CR-PathNarratives containing 174 whole-slide images (WSIs) with hierarchical decision-to-reason annotations and checked for consistency. We further experiment upon the dataset with classification and captioning tasks to explore the clinical scenarios of human-AI-collaborative pathological diagnosis. Results show that the classification and captioning tasks achieve better results with reason labels and provides explainable clues for doctors to understand and make the final decision, and thus can support better experience of human-AI collaboration in pathological diagnosis.