An intelligent diagnostic method for porcine gastrointestinal infectious diseases based on multimodal AI and large language model

Haiyan Wen¹Hongtao Shi²Jiashang Yu³Zhaobin Fan³Haicheng Dai⁴Lili Jiang³Qinye Song^1*

• ¹Hebei Agricultural University, Baoding, China
• ²Qingdao Agricultural University, Qingdao, China
• ³Heze University, Heze, China
• ⁴Rizhao Jiacheng Animal Health Products Co., Ltd, Rizhao, China

The swine farming industry, a key pillar of Chinese animal husbandry, faces significant challenges due to frequent outbreaks of porcine gastrointestinal infectious diseases (PGID). Traditional diagnostic methods reliant on human expertise suffer from low efficiency, high subjectivity, and poor accuracy. To address these issues, this paper proposes a multimodal artificial intelligence (AI) and large language model (LLM)-based diagnostic method for 6 common types of PGID. In this method, ChatGPT and image augmentation techniques were used to expand the dataset; Multi-scale TextCNN (MS-TextCNN) model was employed to capture multi-granularity semantic features from text; Improved Mask R-CNN model was used to segment small intestine lesion regions, and then 7 Convolutional Neural Network (CNN) models were employed to classify segmented images; Lastly, 5 machine learning models were used for multimodal classification diagnosis. Experimental results indicate that multimodal diagnostic model can accurately identify 6 common types of PGID.