Evaluating AI Performance in Infectious Disease Education: A Comparative Analysis of ChatGPT, Google Bard, Perplexity AI, Microsoft Copilot, and Meta AI

Abdulaziz I Alzarea¹Azfar Ishaqui^2*Muhammad Bilal Maqsood³Abdullah Salah Alanazi¹Aseel Alsaidan⁴Tauqeer Hussain Mallhi⁵Narendar Kumar⁶Muhammad Imran⁷Sultan M. Alshahrani¹Hassan Alhassan⁸Sami Alzarea⁹Omar Alsaidan¹⁰

• ¹Department of Clinical Pharmacy, College of Pharmacy, Jouf University, sakaka, Saudi Arabia
• ²Department of Clinical Pharmacy, College of Pharmacy, King Khalid University, Abha, Saudi Arabia
• ³Eastern Health Cluster, Dammam, Saudi Arabia
• ⁴Department of family and community medicine, college of medicine, Jouf University, Sakaka, Saudi Arabia
• ⁵School of Pharmacy, Faculty of Health and Medical sciences, Taylors University, Selangor, Malaysia
• ⁶Department of Pharmacy Practice, Faculty of Pharmacy, Sindh University, Jamshoro, Pakistan
• ⁷Iqra University - North Campus, Karachi, Pakistan
• ⁸Clinical Laboratory Science, Medical Applied College. Jouf University, Sakaka, Saudi Arabia
• ⁹Department of Pharmacology, College of Pharmacy, Jouf University, Sakaka, Saudi Arabia
• ¹⁰Department of Pharmaceutics, college of pharmacy, Jouf University, Sakaka, Saudi Arabia

Background This study systematically evaluates and compares the performance of ChatGPT 3.5, Google Bard (Gemini), Perplexity AI, Microsoft Copilot, and Meta AI in responding to infectious disease-related multiple-choice questions (MCQs). Methods A systematic comparative study was conducted using 20 infectious disease case studies sourced from Infectious Diseases: A Case Study Approach by Jonathan C. Cho. Each case study included 7–10 MCQs, resulting in a total of 160 questions. AI platforms were provided with standardized prompts containing the case study text and MCQs without additional context. Their responses were evaluated against a reference answer key from the textbook. Accuracy was measured by the percentage of correct responses, and consistency was assessed by submitting identical prompts 24 hours apart. Results ChatGPT 3.5 achieved the highest numerical accuracy (65.6%), followed by Perplexity AI (63.2%), Microsoft Copilot (60.9%), Meta AI (60.8%), and Google Bard (58.8%). AI models performed best in symptom identification (76.5%) and worst in therapy-related questions (57.1%). ChatGPT 3.5 demonstrated strong diagnostic accuracy (79.1%) but had a significant drop in antimicrobial treatment recommendations (56.6%). Google Bard performed inconsistently in microorganism identification (61.9%) and preventive therapy (62.5%). Microsoft Copilot exhibited the most stable responses across repeated testing, while ChatGPT 3.5 showed a 7.5% accuracy decline. Perplexity AI and Meta AI struggled with individualized treatment recommendations, showing variability in drug selection and dosing adjustments. AI-generated responses were found to change over time, with some models giving different antimicrobial recommendations for the same case scenario upon repeated testing. Conclusion AI platforms offer potential in infectious disease education but demonstrate limitations in pharmacotherapy decision-making, particularly in antimicrobial selection and dosing accuracy. ChatGPT 3.5 performed best but lacked response stability, while Microsoft Copilot showed greater consistency but lacked nuanced therapeutic reasoning. Further research is needed to improve AI-driven decision support systems for medical education and clinical applications through clinical trials, evaluation of real-world patient data, and assessment of long-term stability.