Evaluating Artificial Intelligence Bias in Nephrology: The Role of Diversity, Equity, and Inclusion in AI-Driven Decision-Making and Ethical Regulation

Suryanarayanan Balakrishnan¹Charat Thongprayoon¹Wannasit Wathanavasin¹Jing Miao¹Michael A Mao²Iasmina M Craici¹Wisit Cheungpasitporn^1*

• ¹Division of Nephrology and Hypertension, Mayo Clinic, Rochester, United States
• ²Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Jacksonville, FL, United States

Background: The integration of Artificial Intelligence (AI) in nephrology raises concerns about bias, fairness, and ethical decision-making, particularly regarding Diversity, Equity, and Inclusion (DEI). AI-driven models, including Large Language Models (LLMs) like ChatGPT, may unintentionally reinforce disparities in patient care and workforce recruitment. This study evaluates how ChatGPT 3.5 and 4.0 handle DEI-related ethical considerations in nephrology, emphasizing the need for improved regulatory oversight to ensure equitable AI deployment.Methods: Conducted in March 2024, this study used ChatGPT 3.5 and 4.0 to evaluate ethical AI decision-making in nephrology. Eighty simulated cases assessed ChatGPT's responses across diverse nephrology topics. Factors considered included age, sex, gender identity, race, ethnicity, religion, cultural beliefs, socioeconomic status, education, family structure, employment, insurance, geographic location, disability, mental health, language proficiency, and technology access.Results: ChatGPT 3.5 responded to all scenarios without refusal, contradicting DEI principles by not rejecting potentially discriminatory criteria. In contrast, ChatGPT 4.0 declined to make decisions based on discriminatory factors in 13 (16.3%) scenarios initially and in 5 (6.3%) during the second round.Conclusion: While ChatGPT 4.0 shows progress in ethical AI decision-making, its limited recognition of bias and DEI considerations underscores the need for robust AI regulation in nephrology. AI governance must incorporate structured DEI guidelines, ongoing bias detection, and ethical oversight to prevent disparities. This study highlights the importance of transparency, fairness, and inclusivity in AI development, calling for collaboration among AI developers, nephrologists, policymakers, and patient communities to ensure AI serves as an equitable tool in nephrology.