AUTHOR=Huang Mingxue , Wang Xiaoyan , Zhou Shiqi , Cui Xinyu , Zhang Zilin , Xu Yanwu , Yang Weihua , Chi Wei 

TITLE=Comparative performance of large language models for patient-initiated ophthalmology consultations

JOURNAL=Frontiers in Public Health

VOLUME=Volume 13 - 2025

YEAR=2025

URL=https://www.frontiersin.org/journals/public-health/articles/10.3389/fpubh.2025.1673045

DOI=10.3389/fpubh.2025.1673045

ISSN=2296-2565

ABSTRACT=BackgroundLarge language models (LLMs) are increasingly accessed by lay users for medical advice. This study aims to conduct a comprehensive evaluation of the responses generated by five large language models.MethodsWe identified 31 ophthalmology-related questions most frequently raised by patients during routine consultations and subsequently elicited responses from five large language models: ChatGPT-4o, DeepSeek-V3, Doubao, Wenxin Yiyan 4.0 Turbo, and Qwen. A five-point likert scale was employed to assess each model across five domains: accuracy, logical consistency, coherence, safety, and content accessibility. Additionally, textual characteristics, including character, word, and sentence counts, were quantitatively analyzed.ResultsChatGPT-4o and DeepSeek-V3 achieved the highest overall performance, with statistically superior accuracy and logical consistency (p < 0.05). Existing safety evaluations indicate that both Doubao and Wenxin Yiyan 4.0 Turbo exhibit significant security deficiencies. Conversely, Qwen generated significantly longer outputs, as evidenced by greater character, word, and sentence counts.ConclusionChatGPT-4o and DeepSeek-V3 demonstrated the highest overall performance and are best suited for laypersons seeking ophthalmic information. Doubao and Qwen, with their richer clinical terminology, better serve users with medical training, whereas Wenxin Yiyan 4.0 Turbo most effectively supports patients’ pre-procedural understanding of diagnostic procedures. Prospective randomized controlled trials are required to determine whether integrating the top-performing model into pre-consultation triage improves patient comprehension.