Same child, different risk: demographic bias in childhood obesity attribution by large language models

Abstract

Background:

Large language models (LLMs) are increasingly consulted for pediatric health information, yet their demographic biases remain unsystematically evaluated in pediatric contexts.

Objectives:

To assess bias and variability in childhood obesity risk attribution across seven LLMs (ChatGPT, Claude, DeepSeek, Gemini, GLM, Grok, and Qwen), spanning both Western and Chinese-origin developers; all prompts, including those submitted to the Chinese-origin models, were in English only.

Methods:

A structured prompt-based experimental design was employed across six clinical domains (general obesity risk, dietary pattern, physical activity, sleep, mental health, and genetic predisposition) and six demographic comparison dimensions (sex, three race/ethnicity pairings, socioeconomic status, and urban-rural residence). Seventy-eight unique prompts were submitted to each model in triplicate, yielding 1,638 outputs. Neutral prompts were scored on a five-dimension binary rubric (accuracy, representation, stigmatizing/harmful language, social determinants, cultural fit); comparative prompts were coded for directional risk attribution.

Results:

Claude achieved the highest neutral prompt composite score (mean 3.00 ± 0.91) and GLM the lowest (1.44 ± 0.51); between-model differences were statistically significant (Kruskal–Wallis H = 46.21, p < 0.001). All models achieved a 100% Stigmatizing/Harmful Language pass rate, yet representation and cultural fit were universally weak. Socioeconomic status produced the most consistent attribution pattern (low-income attribution in 40/42 decisions; decision change rate 19.0%). Most models attributed higher obesity risk to Black and Hispanic/Latino children across the majority of domains. Urban–rural attribution showed the greatest cross-model directional inconsistency (decision change rate 52.4%), with Western-origin models favoring rural attribution and Chinese-origin models favoring urban attribution.

Conclusions:

Publicly accessible English-language web-interface outputs from current LLMs showed systematic demographic patterns in pediatric obesity risk attribution, supporting the need for pre-deployment and post-deployment bias auditing before clinical or consumer health use.

1 Introduction

1.1 The global burden of childhood obesity

Childhood obesity is a well-recognized major global public health crisis. Data from the Non-Communicable Disease (NCD) Risk Factor Collaboration show that, among children and adolescents aged 5–19 years, the global prevalence of obesity increased from 1.7 to 6.9% in girls and from 2.1 to 9.3% in boys between 1990 and 2022, corresponding to an estimated 159 million school-aged children living with obesity worldwide (1). Evidence from the Global Burden of Disease Study 2021 further suggests that, in the absence of effective policy intervention, these trends are likely to intensify over the coming decade, with a particularly heavy burden falling on low- and middle-income countries (2). Consistent with this pattern, most countries have already failed to meet the WHO target of halting the rise in childhood overweight between 2010 and 2025 (3).

The causes of childhood obesity are complex and involve the interplay of genetic susceptibility, diet, physical activity, sleep, mental health, and broader social and structural conditions (4, 5). Its burden is also unevenly distributed across populations. In the United States, for example, non-Hispanic Black and Hispanic children have consistently shown higher obesity prevalence than non-Hispanic White and Asian children (6, 7). Socioeconomic status remains one of the most robust social determinants of childhood obesity across settings, while urban–rural differences arise through partly distinct environmental constraints, including unhealthy food environments and sedentary infrastructure in urban areas, as well as food insecurity, limited service availability, and reduced access to healthcare in rural communities (8). Taken together, these patterns point to structural inequities that shape differential exposure to obesogenic environments and unequal access to health-promoting resources (9).

1.2 Large language models in healthcare: promise and risk

Large language models (LLMs) are artificial intelligence (AI) systems trained on vast corpora of text to understand and generate natural language. These models have entered clinical and consumer health contexts with remarkable speed. Since the public release of ChatGPT in late 2022, LLMs have been adopted across a wide range of health-related applications, including patient education, symptom checking, and clinical decision support (10, 11). Survey evidence from 2024 demonstrates that lay users now perceive LLMs as broadly comparable to traditional search engines in ease of use and trustworthiness for health queries, suggesting a rapid normalization of AI-mediated health information-seeking among the general public (12). In pediatric contexts specifically, LLMs have been explored as explanatory tools for chronic conditions and as resources for caregivers of children with serious illness (13, 14).

However, the clinical usefulness of large language models (LLMs) is limited by well-documented issues with accuracy, adherence to guidelines, and, most relevant to this study, systematic demographic biases (15). An increasing body of evidence suggests that leading commercial LLMs not only reproduce these biases but, in some cases, amplify them. For example, Omiye et al. (16) showed that four major commercial LLMs propagated harmful, debunked race-based medical content, such as incorrect claims about racial differences in lung capacity and pain thresholds, in a significant portion of their responses. A large-scale study by Omar et al. (17, 18), analyzing over 1.7 million LLM outputs from nine models, found that cases labeled as Black, unhoused, or identifying as LGBTQIA+ were more frequently directed toward urgent care or invasive interventions in ways not supported by clinical reasoning, suggesting model-driven bias with the potential to exacerbate health disparities. Systematic reviews have further confirmed that gender bias is present in 93.7% of studies evaluating demographic disparities in medical LLMs, and racial or ethnic bias in 90.9% (18). Cross et al. (19) situate these findings within a broader concern: that biased medical AI, if left unaddressed, will not merely reflect existing healthcare disparities but actively perpetuate and deepen them.

1.3 Bias in LLM clinical risk attribution

How LLMs generate biased clinical outputs remains insufficiently understood. These models are trained on large and heterogeneous corpora, including medical textbooks, online health forums, and published clinical literature, all of which may carry forward outdated medical assumptions, culturally specific interpretations of disease risk, and clinical frameworks shaped primarily by Western healthcare systems (16). Although reinforcement learning from human feedback (RLHF) is commonly used by LLMs to align commercial models after pre-training, its capacity to mitigate such bias is inherently limited. In particular, it cannot reliably separate socially conditioned patterns of risk attribution from those grounded in clinical evidence, especially when the underlying training data and alignment procedures remain opaque (19). Under these conditions, demographic variables associated with disease prevalence at the population level, including race/ethnicity, socioeconomic status, and sex, may become embedded in model outputs in ways that blur the distinction between structural determinants of health and intrinsic biological susceptibility.

The possibility that LLM-assisted clinical reasoning may reproduce familiar cognitive biases has drawn growing attention in recent literature. Schmidgall et al. (20), for example, examined cognitive bias in medical language models and reported that heuristics such as anchoring and representativeness could impair diagnostic performance. Particularly relevant here is attribution bias, defined as the systematic assignment of outcomes or risks to particular demographic groups in ways that exceed or distort the available evidence. In clinical settings, such bias may affect risk communication directly. If an LLM repeatedly assigns greater disease risk to children from racial and ethnic minority groups, lower-SES households, or rural communities, its outputs may reproduce and legitimize existing inequitable narratives when used by parents, caregivers, or clinicians (17, 18).

1.4 The need for multi-model evaluation in pediatric contexts

Despite the proliferation of LLM bias research in healthcare, the pediatric domain has received comparatively little dedicated attention. Prior work has focused predominantly on adult clinical contexts—cardiovascular disease, oncology, emergency medicine, psychiatry, and has typically evaluated one or two models, limiting the generalisability of findings across the diverse commercial LLM ecosystem now accessible to the public (16, 17). The emergence of non-Western LLMs, including models developed by Chinese technology companies (DeepSeek, GLM, Qwen), introduces a further dimension of cross-cultural training diversity that has not been systematically evaluated in the context of clinical risk bias. These models are trained on corpora that reflect different epidemiological literatures, clinical guidelines, and cultural framings of health and disease, which may produce meaningfully different bias profiles compared to their Western counterparts.

Childhood obesity is a particularly instructive domain in which to evaluate LLM bias for several reasons. First, it is a condition with well-established, guideline-supported risk factors that are distributed unequally across demographic groups, creating a tension between epidemiologically accurate risk attribution and the risk of reinforcing structural inequity narratives (21, 22). Second, parents and primary care clinicians are plausible real-world users of LLMs for obesity-related queries, given the high public demand for accessible health information about child weight management (12). Third, the condition's aetiological complexity, spanning dietary, physical activity, sleep, mental health, and genetic domains, enables evaluation of whether LLM bias profiles are consistent across clinical contexts or domain-specific.

Chan and Kwek (23) provided a foundational framework for examining LLM risk attribution bias using structured comparative prompts in the cardiovascular domain. Their pilot study, conducted with a single LLM (ChatGPT-4o mini), demonstrated consistent race-based attribution of higher cardiovascular risk to Black patients across most clinical domains, with more variable sex-based patterns depending on comorbidity context. The present study builds on and extends this framework in three principal ways: by shifting the domain to childhood obesity, by expanding the evaluation from one to seven LLMs spanning Western and non-Western developers, and by incorporating six demographic comparison dimensions, including sex, SES, urban-rural residence, and three racial/ethnic comparator pairs, to provide a more comprehensive assessment of demographic bias in LLM pediatric risk attribution.

1.5 Study aims and objectives

The present study aimed to evaluate bias and variability in childhood obesity risk attribution across seven LLMs using a structured prompt-based experimental design. Specifically, the study addressed the following objectives:

• (1) To assess the quality of LLM responses to neutral pediatric obesity risk prompts using a multidimensional rubric encompassing accuracy, representation, stigmatizing/harmful language, social determinants of health, and cultural fit;

• (2) To examine whether LLMs attribute differential childhood obesity risk across demographic groups, defined by sex, race/ethnicity, SES, and urban–rural residence—in the absence of clinical risk factors (Comp-A condition) and in the presence of domain-specific risk factors (Comp-B condition);

• (3) To determine whether the introduction of a domain-specific clinical risk factor changes LLM risk attribution decisions (decision change analysis), and whether decision change rates differ systematically across demographic comparison dimensions;

• (4) To characterize cross-model variability in risk attribution patterns and intra-model run-to-run consistency, with particular attention to whether Western and non-Western LLMs exhibit systematically different bias profiles.

By conducting this evaluation across seven models and six demographic comparison dimensions, the study aims to provide the most comprehensive characterization to date of demographic bias in LLM pediatric health risk attribution, and to identify specific models, domains, and demographic comparison dimensions that warrant particular scrutiny prior to clinical or consumer deployment.

2 Methods

2.1 Study design

This study employed a structured prompt-based experimental design to evaluate bias and variability in childhood obesity risk attribution across seven large language models (LLMs). It extends the single-model cardiovascular risk attribution framework of Chan and Kwek to a pediatric obesity context by evaluating seven models and six demographic comparison dimensions, as described in Section 1.4.

All prompts were submitted in English, including for the three Chinese-origin models (DeepSeek, GLM, Qwen); no prompts were submitted in Chinese or any other language. Each prompt was submitted independently to each model in a new conversation, with no prior conversation history, to eliminate carry-over effects. All models were accessed via their respective commercial web interfaces using the Google Chrome browser, rather than through programmatic Application Programming Interface (API) calls, which would have allowed hyperparameters such as temperature and top_p to be fixed and recorded precisely. Web-interface access was chosen deliberately, rather than as a matter of convenience, to preserve the ecological validity of the evaluation: parents, caregivers, and primary care clinicians—the plausible real-world users motivating this study (Section 1.4)—overwhelmingly interact with commercial LLMs through consumer-facing web or app interfaces rather than developer API endpoints, so the biases measured here reflect the outputs a lay user would actually encounter. This choice carries a replicability cost, since web-interface default parameters and inference-time routing are not publicly documented and may change without notice; this trade-off is acknowledged explicitly as a limitation in Section 4.8. Every prompt was submitted in triplicate (three independent runs) to assess intra-model consistency and derive a majority decision for comparative prompts.

2.2 LLM selection and access

Seven LLMs were selected to represent a range of developers, training corpora, and alignment approaches. Selection criteria included: (1) free public accessibility via commercial web interfaces at the time of data collection; (2) established use in consumer-facing health information contexts; and (3) representation of both Western and non-Western (Chinese-origin) models to introduce cross-cultural training diversity. Table 1 summaries the models, versions, and access methods.

All prompts were submitted manually via each model's commercial web interface using the Google Chrome browser. Each prompt was entered as a new conversation without any preceding context to eliminate carry-over effects. No system-level instructions, user-specific personalisation, or memory features were enabled. Responses were manually copied and stored in txt files for subsequent scoring.

2.3 Prompt domain design

Six clinical and social domains relevant to childhood obesity risk were selected based on the existing pediatric obesity literature and current clinical guidelines [American Academy of Pediatrics (AAP) 2023 (21); World Health Organization (WHO) 2017 (24); National Institute for Health and Care Excellence (NICE) 2025 (25); Canadian Medical Association Journal (CMAJ) 2025 (26); Society of Pediatrics, Chinese Medical Association (CMA) 2022 (27)]. Each domain corresponds to a distinct and well-evidenced risk pathway for childhood obesity. The domains and their rationales are presented in Table 2.

2.4 Demographic dimensions

Four demographic categories were operationalised into six pairwise comparison dimensions within the comparative prompt structure. These comparison dimensions were: sex, White vs. Black race/ethnicity, White vs. Asian race/ethnicity, White vs. Hispanic/Latino race/ethnicity, socioeconomic status, and urban-rural residence. All children in every prompt were specified as aged 10 years to control for developmental stage effects. Table 3 presents the categories, comparison groups, and epidemiological rationale.

Race/ethnicity was operationalised as three separate White-vs.-minority pairwise comparisons rather than a single multi-group comparison, in order to permit clear directional attribution coding (Group A/Group B/Equal) for each pair. Each comparison pair constituted an independent prompt item; no prompt simultaneously compared more than two demographic groups.

All comparison groups were constructed such that the only variable differing between the two children in a given prompt was the specified demographic characteristic. The prompt language explicitly stated “all other conditions are equal” to prevent confounding by implied socioeconomic or environmental differences. For race/ethnicity comparisons, children were identified by racial/ethnic group labels only (e.g., “a 10-year-old White child and a 10-year-old Black child”), without specification of nationality, immigration status, or cultural practices, to isolate the model's default racial attribution.

2.5 Prompt structure and types

For each of the six domains, three prompt types were constructed, yielding a structured prompt set with distinct analytical functions. Table 4 describes the three types, their coding, wording templates, and purposes. Please refer to the Supplementary material S1 for detailed prompt words.

Prompt wording was standardized in natural language to simulate the style in which a parent or primary care clinician might query an LLM during a routine clinical encounter or health consultation. Technical jargon, leading language, and explicit reference to epidemiological statistics were avoided to prevent priming the model toward specific responses. Each prompt specified the child's age as 10 years and framed the clinical question in terms of obesity risk assessment and management or prevention recommendations.

All prompts were drafted by one pediatric nursing expert (P.L.) and independently reviewed by a second pediatric nursing expert (H.W.) for clarity, consistency with the domain framework, and absence of leading language prior to submission. Any disagreements were resolved by consensus.

2.5.1 Total prompt count and experimental runs

Table 5 presents the breakdown of unique prompts by type and dimension, and the resulting total number of experimental runs.

The total of 78 unique prompts, each submitted to seven models in triplicate, yielded 1,638 individual LLM outputs. Neutral prompts generated 126 scored responses (6 domains × 7 models × 3 runs). Comparative prompts generated 1,512 responses (72 prompts × 7 models × 3 runs), from which majority decisions (across 3 runs) were derived for the 504 comparative prompt items (72 × 7 models).

2.6 Scoring procedures

2.6.1 Neutral prompt scoring

Neutral prompt responses were evaluated using a five-dimension binary rubric, each dimension scored 0 (fail) or 1 (pass), yielding a maximum composite score of 5 per response. This rubric structure was adapted from the validated comparative-prompt scoring framework of Chan and Kwek, with the addition of a new Cultural Fit dimension for the present study, as described in Section 2.6.2. Binary (0/1) scoring was adopted for transparency and inter-rater reproducibility; we acknowledge that this approach may not fully capture partially correct or partially adequate responses, a limitation discussed further in Section 4.8. Table 6 presents the full rubric with pass/fail criteria and notes on rationale.

2.6.2 Rationale for the cultural fit dimension

Given the global diversity of the populations addressed in the present study, we considered it essential to assess whether LLMs could produce culturally appropriate dietary and lifestyle guidance beyond Western-centric frameworks. A response scored 1 on Cultural Fit if it spontaneously acknowledged dietary or lifestyle diversity, offered culturally adaptable recommendations, or explicitly noted that recommendations should be tailored to the patient's cultural context. A response scored 0 if all lifestyle and dietary recommendations were framed exclusively within a Western dietary paradigm (e.g., referring only to whole-grain bread, the Mediterranean diet, or Western portion-size guidelines) without acknowledgment of cultural variation.

All 18 neutral responses per model (6 domains × 3 runs) were scored independently by two trained pediatric clinical physicians (Y.J and T.Z), both of whom were blinded to the model identity of each response during scoring. Researchers scored the rubric dimensions based solely on the text of each response.

Inter-rater reliability was assessed using Fleiss' Kappa for each rubric dimension across all 126 scored items (7 models × 18 responses). Table 7 presents the Kappa coefficients and observed agreement rates for each dimension. For the Stigmatizing/Harmful Language dimension, Kappa was not applicable because all items were rated as pass by both raters, leaving no category variance. For Representation, Social Determinants, and Cultural Fit, both pass and fail ratings were present across items, and the two raters showed complete agreement; therefore, κ = 1.000, observed agreement = 1.000, and disagreements = 0 were reported. This pattern reflects the objective extractability of these criteria: representation was scored by checking whether any explicit mention of demographic variability appeared in the response; Social Determinants was scored by checking whether any named structural factor appeared; and Cultural Fit was scored by identifying the presence or absence of culturally adaptable, non-Western dietary or lifestyle guidance. Inter-rater agreement for Accuracy was “almost perfect” [κ = 0.880, 95% CI (0.786, 0.974)], using the thresholds of Landis and Koch (34). The only dimension requiring adjudication was Accuracy, where borderline cases arose from ambiguous application of age-specific BMI percentile cut-offs. In total, six of 126 Accuracy items (4.8%) were referred for team adjudication. Discrepancies were resolved through a consensus adjudication process in which the two pediatric clinical physicians reviewed the disputed items jointly and reached a final decision by consensus.

2.6.3 Comparative prompt coding

For each comparative prompt (Comp-A and Comp-B), the majority decision across the three replicate runs was coded as one of three categories:

• Group A: the LLM attributed higher obesity risk to Group A (e.g., male child, White child, low-income child, urban child).

• Group B: the LLM attributed higher obesity risk to Group B (e.g., female child, minority child, high-income child, rural child).

• Equal: the LLM judged risk to be equivalent across groups, or explicitly declined to differentiate.

Majority decision was defined as the decision appearing in at least two of three runs. In cases where all three runs produced different decisions, the run with the highest response length (most detailed response) was used as the tiebreaker. Decision coding was performed by Z.L. and independently verified by C.W. for a 20% random subsample; any discrepancies were resolved through adjudication.

A decision change was defined as a shift in majority decision between the Comp-A condition (no risk factor present) and the Comp-B condition (risk factor present) for the same domain–dimension pair. A decision change was coded as Yes if the majority decision differed between Comp-A and Comp-B, and No if it remained identical.

2.7 Statistical analysis

Statistical analyses were conducted using Python version 3.11 (Python Software Foundation, Wilmington, DE, USA) with the scipy.stats library. All tests were two-tailed. The significance threshold was set at α = 0.05 after correction where applicable. Table 8 summarizes the statistical methods used for each analysis objective.

Composite scores for neutral prompts were treated as ordinal-scale data given the binary nature of the underlying rubric dimensions (max = 5, integer values only). The Kruskal–Wallis H test was selected as the primary between-model comparison because the ordinal data structure and small sample sizes per cell (n = 18 per model) preclude parametric assumptions. Post-hoc pairwise comparisons used Mann–Whitney U tests with Bonferroni correction applied across all 21 model pairs.

Intra-model run-to-run consistency for neutral prompt composite scores was assessed using a two-way random-effects intraclass correlation coefficient (ICC), specifically the absolute agreement model (Shrout and Fleiss ICC (2, 1). This model treats both domains and runs as random effects and assesses whether responses are reproducible in absolute terms across the three replicate runs within each model.

For comparative prompt analyses, decision distributions (proportions of Group A, Group B, and Equal decisions) were reported descriptively. Cross-model agreement was operationalised as the proportion of the 36 Comp-B decision items on which all seven models produced the same majority decision. Decision change rates were reported as frequency counts and proportions by model and by demographic comparison dimension. To test whether decision change rates differed significantly across the six demographic comparison dimensions, an omnibus Pearson chi-square test of independence was applied to a 2 × 6 contingency table (changed vs. unchanged x dimension), with each of the 42 items per dimension (6 domains × 7 models) treated as independent observations. Post-hoc pairwise comparisons used Fisher's exact tests with Bonferroni correction applied across all 15 dimension pairs. Assumption verification confirmed that all expected cell frequencies exceeded 5 (minimum expected frequency = 13.5), satisfying the requirements for chi-square inference.

2.8 Ethical considerations

This study did not involve human participants, patient data, or personally identifiable information. All data were generated through standardized queries to publicly accessible LLM services. No ethical approval was required under the institutional guidelines applicable to this type of LLM evaluation research.

All prompt texts were designed to avoid eliciting harmful or dangerous content from the models. Prompts described hypothetical clinical scenarios involving children and were framed in terms of risk assessment and clinical management, consistent with the primary intended use cases of LLMs in pediatric health consultation. No sensitive personal data were collected, stored, or transmitted during the study. Model terms of service for research use were reviewed and complied with for all seven platforms.

3 Results

A total of 78 unique prompts were submitted to each of the seven LLMs (ChatGPT-5.3 Instant, Claude 4.6 Sonnet, DeepSeek-V3.2, Gemini 3 Thinking, GLM-4.7, Grok-4.1 Auto, and Qwen-3.5 Plus Thinking), each run in triplicate, yielding 1,638 individual outputs (78 prompts × 7 models × 3 runs). Results are organized into three sections: (1) neutral prompt quality scoring; (2) comparative prompt decision patterns; and (3) cross-model variability and consistency.

3.1 Neutral prompt quality scores

Neutral prompts (n = 6 per model × 3 runs = 18 scored responses per model) were evaluated on a five-dimension binary rubric (Accuracy, Representation, Stigmatizing/Harmful Language, Social Determinants, Cultural Fit; maximum composite score = 5 per response). Table 9 presents mean composite scores per domain and overall, ranked by overall performance (Figure 1).

Figure 1

Claude achieved the highest overall composite score (mean = 3.00 ± 0.91), followed by Qwen (2.50 ± 0.79), Grok (2.39 ± 0.85), ChatGPT (2.11 ± 0.32), and DeepSeek (2.06 ± 0.54). Gemini (1.67 ± 0.59) and GLM (1.44 ± 0.51) performed below the group median. Across all models, the Mental Health domain (D5) and General Obesity Risk domain (D1) tended to receive relatively higher composite scores, while the Physical Activity domain (D3) received the lowest scores in five of seven models.

A Kruskal–Wallis test revealed a statistically significant difference in composite scores across the seven models (H = 46.21, p < 0.001). Post-hoc pairwise Mann–Whitney U tests with Bonferroni correction (21 comparisons) identified nine significant pairs. Claude was significantly superior to ChatGPT (p = 0.006), Gemini (p = 0.001), and GLM (p < 0.001), and significantly better than DeepSeek (p = 0.005). GLM was significantly below ChatGPT (p = 0.004), DeepSeek (p = 0.024), Grok (p = 0.019), and Qwen (p = 0.003). Qwen was significantly better than Gemini (p = 0.039). All other pairwise comparisons were non-significant after correction.

3.1.1 Within-model domain variation: Friedman test

Friedman tests were conducted within each model to assess whether composite scores varied systematically across the six prompt domains. Table 10 presents the test statistics and p-values for all seven models.

No model showed a statistically significant within-model domain effect (all p > 0.05). Gemini produced the largest test statistic ( = 10.217, p = 0.069), reflecting a non-significant trend driven by the markedly lower scores in D3 (Physical Activity, mean = 1.00) compared to D1 and D4 (mean = 2.33 each). DeepSeek produced a Friedman statistic of 0 (p = 1.000), consistent with near-uniform scores across all six domains (all domain means = 2.00, except D1 = 2.33). These results indicate that within each model, composite score variation across domains did not reach statistical significance, and domain-specific effects cannot be distinguished from sampling variation given the current sample size (k = 3 runs per domain).

3.1.2 Dimension-level analysis

Table 11 presents pass rates (percentage of runs scoring 1) for each rubric dimension across all 18 neutral prompt runs per model (Figure 2).

Figure 2

Stigmatizing/Harmful Language was the only dimension on which all seven models achieved a perfect pass rate (100%), indicating that none produced overtly stereotyped or moralizing language in neutral prompt responses. Accuracy pass rates varied considerably: ChatGPT (100%), DeepSeek (94%), Claude and Qwen (89% each), Gemini (61%), Grok (56%), and GLM (44%). Representation—spontaneous acknowledgment of demographic variability in obesity risk—was the most consistently weak dimension, with pass rates of 0% for ChatGPT, Gemini, and GLM, and no model exceeding 22% (Grok). Social Determinants pass rates were similarly low, with Qwen highest (39%), followed by Claude (28%) and Grok (22%); DeepSeek, Gemini, and GLM all scored 6% or below. Cultural Fit showed a stark bimodal divide: Claude (72%) and Grok (39%) were the only models to achieve meaningful Cultural Fit pass rates; all remaining models scored 6% or below, and ChatGPT, DeepSeek, Gemini, and GLM scored 0%.

3.2 Comparative prompt decision patterns

Each model evaluated 36 comparative prompt pairs (6 domains × 6 demographic comparison dimensions), each with a Comp-A (no risk factor) and Comp-B (risk factor present) condition, yielding majority decisions (across 3 runs) coded as Group A, Group B, or Equal.

3.2.1 Decision change rates by model and domain

Table 12 reports how frequently each model changed its risk attribution when a domain-specific clinical risk factor was introduced (Comp-A → Comp-B). A decision change is defined as a shift in the majority decision between the two conditions for a given domain–dimension pair (Figure 3).

Figure 3

Cell values = number of demographic comparison dimensions changed out of six per domain. Right column = total changes per model out of 36 pairs. Color encodes instability: white/green = stable; amber/red = high change rate. D1 = General Obesity Risk; D2 = Dietary Pattern; D3 = Physical Activity; D4 = Sleep Duration; D5 = Mental Health (GAD); D6 = Genetic Predisposition.

ChatGPT and Qwen had the highest overall decision change rates (44% each; 16/36 pairs), followed by Claude and GLM (39% each; 14/36). DeepSeek was the most stable model (14%; 5/36 changes). The Genetic Predisposition domain (D6) showed the most pronounced instability: GLM changed all six dimensional decisions upon introduction of the risk factor (6/6), Claude and Qwen changed four (4/6), and Grok three (3/6). By contrast, DeepSeek changed only one D6 decision (1/6), and the Dietary Pattern domain (D2) was the most stable across models (median 2/6 changes).

3.2.2 Decision change rates by demographic dimension

Table 13 presents decision change rates aggregated across all models and domains for each demographic dimension, along with the results of the omnibus chi-square test and post-hoc pairwise Fisher exact tests (Bonferroni-corrected, 15 comparisons; Figure 4).

Figure 4

An omnibus chi-square test revealed a statistically significant difference in decision change rates across the six demographic comparison dimensions ( = 21.778, p < 0.001). Urban vs. Rural showed the highest change rate (52.4%; 22/42), and post-hoc comparisons confirmed it was significantly higher than both SES (19.0%; p_Bonferroni = 0.042) and Race: White vs. Latino (16.7%; p_Bonferroni = 0.017). Sex had the second-highest change rate (47.6%; 20/42) but did not reach significance against any individual dimension after correction. The three race/ethnicity dimensions showed intermediate change rates (16.7%−33.3%), with Race: White vs. Asian highest (33.3%) and Race: White vs. Latino lowest (16.7%). SES showed the lowest change rate (19.0%; 8/42), consistent with near-universal model consensus on low-income children carrying higher risk in both Comp-A and Comp-B conditions (see Section 3.2.4).

3.2.3 SES attribution: near-universal consensus

The SES dimension was the most consistent across all models and all domains. Across 42 SES Comp-B decisions (6 domains × 7 models), 40 attributed higher obesity risk to the low-income child (Group A), with only two exceptions: ChatGPT returned Equal in D4 (Sleep Duration), and GLM returned Equal in D3 (Physical Activity). No model attributed higher risk to the high-income child in any domain under any condition. This pattern was stable regardless of whether a risk factor was present (Comp-A) or absent (Comp-B), confirming near-universal model consensus that socioeconomic disadvantage amplifies childhood obesity risk independent of the specific clinical domain (Figure 5).

Figure 5

3.2.4 Race/ethnicity attribution

Table 14 summarizes the directional bias in race/ethnicity attributions and selected other dimensions across the six domains under Comp-B conditions.

White vs. Black. Six of seven models attributed higher obesity risk to the Black child in the majority of domains under Comp-B conditions. DeepSeek and Gemini attributed higher risk to the Black child in all six domains (6/6), while GLM did so in five (5/6), and Claude, Grok, and Qwen in four (4/6). ChatGPT was the sole exception, returning Equal across all six domains—the only model to decline to differentiate risk by Black vs. White race in any domain under any condition (Figure 6).

Figure 6

White vs. Asian. The White vs. Asian comparison showed the greatest cross-model disagreement. Gemini attributed higher risk to the Asian child in 4/6 domains; DeepSeek and Claude each showed an even 3/6 split between White-higher and Asian-higher attributions; ChatGPT attributed higher risk to the White child in 2/6 domains and Equal in the remainder. No model produced a fully directional pattern favoring Asian children across all domains. This pattern reflects the epidemiological complexity of obesity risk in Asian populations, including lower absolute BMI thresholds alongside higher metabolic risk at any given BMI (Figure 6).

White vs. Hispanic/Latino. Five of seven models attributed higher risk to the Hispanic/Latino child in five or six of six domains (DeepSeek, Gemini, Qwen: 6/6; Grok: 5/6; GLM: 5/6). Claude attributed higher risk to Hispanic/Latino in 4/6 domains. ChatGPT defaulted to Equal in 5/6 domains. In the Genetic Predisposition domain (D6), all models except ChatGPT attributed higher risk to the minority child across all three racial comparisons, suggesting that genetic framing may amplify race-based risk attribution specifically within this domain (Figure 6).

3.2.5 Urban vs. rural: directional inconsistency

Urban vs. Rural was the only demographic dimension showing systematic directional inconsistency across models, a pattern confirmed by the chi-square analysis as the dimension most susceptible to decision change (52.4%; Section 3.2.2). Claude, Grok, Qwen, and Gemini (partially) attributed higher risk to the rural child in the majority of domains, consistent with evidence on food deserts and limited healthcare access in rural settings. In contrast, DeepSeek and GLM attributed higher risk to the urban child in 4/6 domains, consistent with an urban obesogenic environment framing (food swamps, sedentary infrastructure). This cross-model divergence was most pronounced in D1 (General Risk) and D2 (Dietary Pattern), where no consensus direction emerged across the seven models (Figure 7).

Figure 7

3.3 Cross-model variability and intra-model consistency

3.3.1 Cross-model agreement on comparative decisions

Full cross-model agreement, defined as all seven models returning the same majority decision in the same direction, was observed in only eight of 36 Comp-B decision items (22%). These eight items of consensus were: SES × D1, SES × D2, SES × D3, SES × D5, and SES × D6 (all attributing higher risk to the low-income child), plus Race: White vs. Latino × D2 and Race: White vs. Latino × D4 (all attributing higher risk to the Hispanic/Latino child), and Race: White vs. Black × D1 (six of seven models attributed higher risk to the Black child, with ChatGPT returning Equal). The remaining 78% of comparative decisions showed at least one model diverging from the others, indicating substantial inter-model variability in risk attribution patterns (Figure 8).

Figure 8

Percentage of 36 decisions (6 domains × 6 demographic comparison dimensions) per model attributed to Group A (blue), Equal (gray), or Group B (red). Group A = male/White/urban child; Group B = female/minority/rural child.

In terms of overall Comp-B decision distributions, DeepSeek and Gemini never returned an Equal decision (0% Equal across 36 items), reflecting a consistent tendency toward directional attribution. ChatGPT showed the opposite pattern, returning Equal in 64% of decisions, markedly higher than all other models. Claude (17% Equal), Qwen (19%), and GLM (22%) showed intermediate hedging. The proportion of decisions attributed to Group A (typically male, White, or urban child) ranged from 22% (ChatGPT) to 39% (Grok); Group B attributions (typically female, minority, low-income, or rural child) ranged from 14% (ChatGPT) to 67% (Gemini).

3.3.2 Intra-model run-to-run consistency: ICC

Intra-model consistency across the three replicate runs was assessed using a two-way random-effects ICC (absolute agreement model, ICC(2, 1)). Table 15 presents ICC values for all seven models with interpretations.

ICC values ranged from −0.400 for DeepSeek to 0.429 for Gemini. In five of the seven models, ICC estimates were negative or close to zero, suggesting that variability across repeated runs within the same domain was greater than variability between domains. Put differently, outputs generated from three replicate runs within a given domain were not consistently more alike than outputs generated across different domains. Gemini was the only model to show a level of agreement approaching moderate reliability (ICC = 0.429), a pattern likely related to its wider spread in mean composite scores across domains, with D3 averaging 1.00 compared with 2.33 in both D1 and D4. By contrast, DeepSeek showed the lowest ICC (−0.400), which is consistent with its near-uniform composite scores across domains; five of the six domain means were 2.00, leaving little between-domain variation and making the ICC estimate unstable. Claude also yielded a negative ICC (−0.319), largely attributable to a ceiling-level outlier in D5 Run 1 (composite score 5/5), which increased within-domain variance for that domain. Taken together, these findings indicate that intra-model reproducibility of composite scores was limited overall and support the use of triplicate sampling in the present study.

3.4 Summary of key findings

Neutral prompt quality: Claude achieved the highest overall composite score (3.00 ± 0.91); GLM and Gemini were lowest (1.44 ± 0.51 and 1.67 ± 0.59, respectively). Between-model differences were statistically significant (Kruskal–Wallis H = 46.21, p < 0.001; 9 of 21 Bonferroni-corrected pairwise comparisons significant). Within-model domain variation did not reach significance in any model (all Friedman p > 0.05). Stigmatizing/Harmful Language was the only dimension achieving a perfect pass rate across all models (100%); Representation and Cultural Fit were the weakest dimensions system-wide.

SES consensus: near-universal attribution of higher obesity risk to low-income children across all models and all domains (40/42 decisions; lowest decision change rate of any dimension, 19.0%).

Race/ethnicity bias: Black and Hispanic/Latino children were attributed higher obesity risk in the majority of domains by most models; White vs. Asian comparisons showed the greatest cross-model inconsistency. ChatGPT uniquely withheld directional race-based judgments, defaulting to Equal across all race comparisons.

Urban vs. Rural divergence: the highest decision change rate of any demographic dimension (52.4%; significantly higher than SES and Race: White vs. Latino after Bonferroni correction). Cross-model directional disagreement: some models attributed higher risk to rural children, others to urban children.

Decision instability: D6 (Genetic Predisposition) exhibited the highest within-domain decision change rates across models. DeepSeek was the most stable model overall (14% change rate); ChatGPT and Qwen the most susceptible to decision shifts (44%).

Cross-model agreement and reproducibility: only 22% of Comp-B decision items achieved full cross-model consensus. Intra-model ICC values were predominantly negative or near-zero, confirming that triplicate sampling captures meaningful stochastic variation and single-run scores should not be treated as stable estimates.

4 Discussion

This study employed a structured prompt-based experimental design to evaluate bias and variability in childhood obesity risk attribution across seven LLMs. The findings reveal a complex and internally differentiated bias landscape: one in which demographic risk attribution is neither random nor uniform, but follows patterns that are in part epidemiologically defensible, in part reflecting post-training alignment choices, and in part potentially inequitable. The following sections interpret these findings in light of their underlying mechanisms, their correspondence with the existing literature on LLM bias in medicine, and their implications for the deployment of LLMs in pediatric health contexts.

4.1 Quality of neutral prompt responses: a ceiling obscured by selective competence

The statistically significant between-model differences in composite rubric scores (Kruskal–Wallis H = 46.21, p < 0.001) confirm that LLMs are not interchangeable tools in the pediatric clinical advisory context. Claude's leading performance (mean = 3.00 ± 0.91) and GLM's lowest scores (1.44 ± 0.51) reflect a performance gap that cannot be attributed to sampling variation alone. However, interpreting these scores requires an important disaggregation: the composite score conflates qualitatively distinct capabilities, and the dimension-level analysis reveals that the apparent performance hierarchy conceals a consistent and concerning failure mode shared across all seven models.

The finding that all seven models achieved a 100% Stigmatizing/Harmful Language pass rate, meaning none produced overtly stereotyped or moralizing language in neutral prompt responses, might, at first glance, suggest that contemporary LLMs have successfully internalized non-stigmatizing clinical communication norms. This interpretation, however, demands caution. The Stigmatizing/Harmful Language rubric assessed overt surface-level language rather than the more subtle forms of bias examined in the comparative prompts. The contrast between universal Stigmatizing/Harmful Language pass rates in neutral prompts and the systematic racial and SES-based directional attributions in comparative prompts (Sections 3.2.3 and 3.2.4) reveals a decoupling between explicit language norms and implicit risk attribution logic, a distinction that echoes Bai et al.'s (28) finding that LLMs can generate explicitly unbiased language while simultaneously forming and acting on biased associations. Surface linguistic compliance, in other words, does not preclude deep structural bias in the model's probabilistic reasoning.

Of greater concern is the near-universal failure on the Representation and Cultural Fit dimensions. No model exceeded a 22% pass rate for Representation, the spontaneous acknowledgment of demographic variability in obesity risk, and five of seven models scored 0% on Cultural Fit. These failures are not incidental: they indicate that when left unprompted, LLMs frame childhood obesity as a condition with a generic, demographically undifferentiated presentation, recommending interventions calibrated to a culturally Western, implicitly middle-class normative patient. This pattern directly replicates the finding of Chan and Kwek that models defaulted to Mediterranean diet recommendations regardless of the described patient population, and extends it to demonstrate that this Western-centric framing is near-universal across diverse model architectures, including Chinese-origin models that might have been expected to encode different dietary and lifestyle frameworks (23). The practical implication is that even the best-performing model (Claude, Cultural Fit 72%) would fail to provide culturally appropriate dietary guidance in more than a quarter of encounters, while four models, including two of the most widely used globally (ChatGPT and Gemini) would fail in every case.

The low Social Determinants pass rates (maximum 39% for Qwen) further indicate that LLMs predominantly frame childhood obesity through an individual behavioral lens rather than as a condition embedded in structural conditions. This framing is not merely an academic concern: it has direct implications for health equity. As the pediatric obesity literature clearly demonstrates, interventions that target individual behavior without addressing structural determinants—food access, housing quality, neighborhood safety, parental working conditions, are substantially less effective for families in high-disadvantage contexts (9, 29). An LLM that consistently omits structural factors from its advisory framing may inadvertently reinforce the false attribution of obesity risk to individual failure, a narrative that has been associated with weight stigma and reduced help-seeking among low-income and minority families (29).

4.2 SES attribution: when epidemiological accuracy and equity risk converge

The near-universal consensus attributing higher childhood obesity risk to low-income children (40/42 Comp-B decisions; 19.0% decision change rate, the lowest of any dimension) is, from an epidemiological standpoint, defensible. The relationship between socioeconomic disadvantage and childhood obesity is among the most robust and consistently replicated findings in pediatric public health, supported by evidence across multiple countries, income levels, and measurement approaches (8, 9). One would therefore expect a well-calibrated clinical advisory system to encode this association, and the fact that all seven models did so, including models with otherwise heterogeneous bias profiles, might appear to validate their factual grounding.

Yet this consensus warrants a more nuanced interpretation than simple endorsement. The key issue is not whether LLMs correctly identify that socioeconomic disadvantage is associated with higher obesity risk at the population level, it clearly is, but whether this association is being encoded as an immutable characteristic of low-income children rather than as the product of modifiable structural conditions. In the prompt design used in this study, both the low-income and high-income children in SES comparative prompts were described as otherwise equivalent, with the instruction that “all other conditions are equal.” A response attributing higher risk to the low-income child, while epidemiologically consistent with population-level data, implicitly treats SES as a risk factor intrinsic to the child rather than as a marker of exposure to risk-conferring environments. This subtle framing difference has clinical significance: it determines whether the model's advisory output positions families as passive recipients of structurally determined risk or as active agents who can benefit from targeted environmental and policy interventions.

The two exceptional Equal decisions—ChatGPT in D4 (Sleep) and GLM in D3 (Physical Activity), are insufficient to constitute meaningful departures from the SES consensus, but they may reflect domain-specific variability in how different models weight the risk factor introduced in Comp-B against the SES prior. The stability of SES attribution across all six clinical domains and all seven models, combined with its lowest decision change rate, suggests that SES has become the most deeply and consistently encoded demographic risk heuristic in these models, arguably the demographic dimension most reliably “baked in” to LLM clinical reasoning about childhood obesity regardless of clinical context.

4.3 Racial attribution patterns: epidemiological encoding vs. structural conflation

The racial attribution patterns revealed in this study represent the most ethically complex dimension of the findings. The majority attribution of higher obesity risk to Black and Hispanic/Latino children across most models is, at the level of population prevalence data, consistent with epidemiological evidence: CDC NHANES surveillance data consistently document higher childhood obesity prevalence among non-Hispanic Black and Hispanic children compared to non-Hispanic White children in the United States (7, 8). In this sense, the LLMs' attributions are not factually baseless. However, the critical question is what causal model underlies these attributions, and this is a question the prompt design, purposely, is unable to resolve from the output text alone, but which the pattern of results allows us to interrogate indirectly.

The persistence of racial risk attribution even in Comp-A prompts where no clinical risk factor was introduced is particularly informative. If a model attributes higher obesity risk to a Black child vs. a White child in the complete absence of any clinical, dietary, activity, or genetic information, this attribution cannot reflect genuine clinical reasoning based on presented evidence. It must reflect a prior encoded in the model's training distribution: one that maps racial group membership directly onto disease risk independent of any mechanistic pathway. This is precisely the pattern of race-based clinical attribution that Omiye et al. (16) identified as propagating potentially harmful, debunked race-based medicine, the encoding of epidemiological correlations as if they were biological properties of racial groups rather than consequences of differential exposure to structural risk factors. The broader concern, documented across multiple medical AI studies, is that such attributions operationalise race as a proxy for underlying structural inequity while framing it as an intrinsic clinical characteristic—a subtle but consequential distortion (17, 18).

The White vs. Asian comparison is the most instructive case for understanding the limits of simple epidemiological encoding as an explanatory account of LLM race-based attribution. The greatest cross-model disagreement in the entire dataset occurred in this dimension: Gemini attributed higher risk to the Asian child in 4/6 domains, while ChatGPT, Claude, and DeepSeek showed mixed or equal patterns across domains. This heterogeneity likely reflects the genuine epidemiological complexity of obesity risk in Asian populations where lower absolute BMI thresholds, higher visceral fat accumulation at lower BMI values, and different metabolic risk profiles require careful contextualization (7). The fact that different models encoded this complexity in fundamentally incompatible directions suggests that no single model has a well-calibrated representation of Asian childhood obesity risk, and that this particular demographic intersection remains underrepresented or inconsistently represented in the training corpora of current commercial LLMs. This finding has direct clinical relevance, as inappropriate BMI-based risk thresholds for Asian children, a longstanding challenge in pediatric clinical practice, may be further compounded by inconsistent LLM guidance.

ChatGPT's unique pattern of returning Equal across all race comparisons warrants specific discussion. On one interpretation, this represents a more equitable output, declining to attribute differential risk on the basis of race in the absence of presented clinical evidence. On another interpretation, it reflects an alignment-induced avoidance of racially sensitive outputs, a form of what might be termed “demographic hedging” in which the model has learned to abstain from racial risk differentiation regardless of the epidemiological evidence, potentially at the cost of suppressing clinically relevant population-level information about disparate risk exposures. This tension between demographic neutrality and clinical accuracy has been identified as a central challenge in the design of fair medical AI systems: algorithms that are constrained from acknowledging population-level disparities may fail to provide care recommendations tailored to high-risk groups, while those that freely encode disparities may perpetuate structural stigma (30). The present findings cannot resolve this tension, but they illustrate it in sharp relief: the seven models in this study sit at different points along the spectrum from unhedged racial attribution to complete demographic abstention, with no model demonstrably occupying the clinically optimal middle ground.

4.4 Urban–rural divergence: a cartography of model-specific framings

The Urban vs. Rural dimension produced the most striking finding of the comparative analyses: a 52.4% decision change rate, the highest of any demographic dimension, and significantly higher than SES (p = 0.042) and Race: White vs. Latino (p = 0.017) after Bonferroni correction, combined with systematic directional inconsistency across models. Claude, Grok, Qwen, and Gemini (partially) attributed higher risk to rural children, while DeepSeek and GLM attributed higher risk to urban children in 4/6 domains. This bidirectional split cannot be explained by one model group being more accurate than the other: the epidemiological evidence on urban–rural obesity risk differentials is itself context-dependent and directionally ambiguous, with rural disadvantage predominating in high-income countries (food deserts, limited recreational infrastructure, reduced healthcare access) and urban disadvantage more prominent in many low- and middle-income countries (food swamps, sedentary infrastructure, rapid nutritional transition) (1).

The directional divergence observed in this study therefore likely reflects the geographic and socioeconomic composition of each model's training corpus rather than a simple factual disagreement. Models trained predominantly on North American and Western European medical and policy literature—in which rural food deserts and healthcare access barriers are the dominant framing of geographic health disadvantage, are likely to attribute higher obesity risk to rural children, as Claude, Grok, and Qwen did. Models whose training corpora include substantial representation from East Asian or Global South contexts, where rapidly urbanizing settings are more often associated with obesogenic food environments, may instead weight the urban pathway more heavily, as DeepSeek and GLM appeared to do. While this interpretation is necessarily inferential given the opacity of LLM training data, it suggests that the urban–rural attribution pattern may function as an indirect indicator of the geographic and cultural provenance of a model's primary training corpus, a finding with significant implications for the deployment of globally diverse LLMs in geographically heterogeneous clinical settings.

The high decision change rate for Urban vs. Rural also indicates that this dimension is particularly sensitive to the introduction of specific clinical risk factors—that is, the clinical domain context (diet, sleep, physical activity, etc.) exerts stronger influence on urban–rural attribution than on SES attribution, for which the prior is so robust that it persists almost unchanged regardless of clinical context. This domain-sensitivity suggests that models' representations of urban and rural health risk are less consolidated and more amenable to contextual modulation than their SES priors, a characteristic that could potentially be leveraged in prompt engineering strategies designed to elicit more nuanced geographic risk assessments.

4.5 Decision instability in the genetic predisposition domain: encoding complexity or amplifying stereotypes?

The Genetic Predisposition domain (D6) consistently produced the highest within-domain decision change rates across models, with GLM changing all six demographic dimension decisions upon introduction of the parental obesity risk factor (6/6) and Claude and Qwen changing four (4/6). This pattern warrants particular interpretive scrutiny because the mechanism driving these changes differs qualitatively from the decision changes observed in other domains. In D6, the risk factor introduced in Comp-B (both parents with BMI >30) is explicitly framed as a genetic and familial loading, a framing that, in the existing LLM bias literature, has been associated with the amplification of race-based risk attribution through the pathway of conflating ancestry with genetically mediated biological risk (16).

The observation that all models except ChatGPT attributed higher risk to minority children (Black, Hispanic/Latino, and in some cases Asian) in D6 across all or most comparative pairs suggests that the introduction of a “genetic” framing may trigger a distinct attribution logic in which population-level genetic epidemiology data, including the well-established higher prevalence of obesity-associated gene variants in certain populations, is applied to individual children in a way that conflates statistical ancestry with biological determinism. This is precisely the form of race-based clinical reasoning that the medical genetics and bioethics communities have extensively critiqued: the attribution of population-level genetic statistics to individuals on the basis of their racial or ethnic group membership, without accounting for the within-group genetic heterogeneity that renders such attributions clinically uninformative for any given patient. That this pattern is most pronounced in D6 and that it drives decision changes from Equal or ambiguous attributions in Comp-A to directional minority-attributions in Comp-B, suggests that genetic framing functions as an amplifier of race-based attribution heuristics in these models, a finding with significant implications for the use of LLMs in contexts involving genetic counseling or precision medicine consultations.

The extreme instability observed in the Genetic Predisposition domain (D6) suggests that the probability density for risk attribution in this area is highly diffused. When a clinical domain involves multi-factorial and controversial evidence (like the intersection of genetics and race), the model's output probability for “Group A” vs. “Group B” may be nearly equal (e.g., 49 vs. 51%). In such high-entropy states, the stochastic sampling mechanism is more likely to produce divergent majority decisions across replicate runs, further undermining the reliability of LLMs for sensitive genetic risk communication.

4.6 Intra-model reproducibility: the implications of near-zero ICC values

The predominantly negative or near-zero ICC values observed across most models can be attributed to the inherent stochastic nature (i.e., the model's outputs are generated by random sampling from a probability distribution rather than by a fixed rule, so identical input can yield different output on different occasions) of the Transformer architecture. Unlike deterministic clinical algorithms, LLMs function as probabilistic engines performing next-token prediction.

In each run, the model generates a probability distribution for the next potential word based on the input prompt. The selection process, governed by a temperature (T) parameter, introduces variability by sampling from this distribution:

Even at low temperature settings, minor variations in the calculated logits (due to hardware-level floating-point non-determinism or routing in Mixture-of-Experts architectures) can lead the model to tip between two closely weighted clinical interpretations. This technical characteristic explains why the models exhibited “decision flipping” even when the clinical evidence provided in the prompts remained identical.

The predominantly negative or near-zero ICC values for neutral prompt composite scores (range: −0.400 to 0.429) require careful methodological interpretation. A negative ICC, counterintuitively, does not indicate random output: it indicates that within-domain variance exceeds between-domain variance, such that the three replicate runs for a given clinical domain are not systematically more similar to each other than runs drawn from different domains within the same model. This pattern is consistent with a model that produces stochastic variation in composite score within a domain rather than domain-dependent systematic variation, a property that has practical implications for clinical deployment.

For clinical advisory applications, low intra-model reproducibility means that the quality of a single LLM response to a pediatric obesity query cannot be taken as representative of the model's general performance on that clinical domain. A caregiver who receives a high-quality, culturally appropriate, and structurally aware response from Claude on one occasion may receive a qualitatively different response on a subsequent query about the same clinical domain, a characteristic that undermines the reliability of LLMs as consistent clinical information sources. This finding aligns with the broader literature on LLM stochasticity in clinical settings, which has identified run-to-run variability as a significant challenge to the safe deployment of LLMs in clinical decision support roles (15). The fact that Gemini, despite its low overall composite score, achieved the highest ICC (0.429) is particularly noteworthy: it suggests that consistency and quality are not correlated, a model can be consistently poor, just as it can be variably good.

The negative ICC for Claude (−0.319), the overall highest-performing model, is driven by a ceiling-effect outlier in D5 Run 1 (composite = 5/5), which inflates within-domain variance for the Mental Health domain. This ceiling effect itself carries interpretive significance: it demonstrates that Claude is capable of producing responses that satisfy all five rubric dimensions simultaneously, but that this level of performance is not reliably reproduced across replicate runs. Whether this variability reflects genuine stochastic noise in the model's output distribution at temperature = 0, or residual sensitivity to minor prompt-level variations not controlled by the standardized prompt design, cannot be determined from the current study design and warrants further investigation.

4.7 Western vs. non-Western models: cross-cultural training diversity and its limits

A central motivation for including Chinese-origin models (DeepSeek-V3, GLM-4, Qwen-Max) alongside Western models in this study was to test whether cross-cultural training diversity produces meaningfully different bias profiles. The results present a nuanced picture. On the SES dimension, all seven models converged on the same attribution direction, suggesting that the socioeconomic determinants of childhood obesity are sufficiently well-represented in both Western and Chinese-origin medical corpora to produce consistent model outputs. On racial attribution, Chinese-origin models did not differ systematically from Western models: DeepSeek (6/6 for both Black and Hispanic/Latino attribution), GLM (5/6 for both), and Qwen (6/6 for Hispanic/Latino) all exhibited directional race-based attribution patterns comparable to their Western counterparts, indicating that training on Chinese-language corpora does not insulate models from encoding race-based health risk associations derived from the English-language medical literature that forms a substantial component of all these models' training data.

The most meaningful difference between Western and non-Western models emerged in the Urban vs. Rural dimension, where DeepSeek and GLM consistently attributed higher risk to urban children, a pattern that, as discussed in Section 4.4, may reflect the different urban epidemiological contexts represented in their training corpora. This finding provides limited but suggestive evidence that training corpus composition can produce domain-specific differences in bias direction, even when overall bias magnitude is comparable. The Cultural Fit dimension might have been expected to reveal the clearest cross-cultural differences with Chinese-origin models more likely to offer non-Western dietary guidance, but in fact all three Chinese-origin models scored at or below 6% on Cultural Fit, indicating near-complete Western dietary framing in their pediatric obesity recommendations. This unexpected finding may reflect the heavy representation of English-language medical literature in the training data of Chinese-origin commercial LLMs, a phenomenon noted in the healthcare LLM literature as a driver of persistent Western-centric medical framing even in models intended for non-Western clinical contexts (31).

A further question raised by these findings is whether the Chinese-origin models' reproduction of Western race-based obesity risk associations under English-language prompting reflects data contamination or an alignment artifact. Two non-exclusive mechanisms are plausible. First, DeepSeek, GLM, and Qwen are trained on corpora with substantial representation of English-language biomedical literature, given the global dominance of English-language publishing in clinical research; under this account, English-language prompts would activate primarily English-trained associations regardless of the model's developer, a form of data contamination in which the language of the query, rather than the model's origin, determines which regional evidence base is retrieved. This interpretation is consistent with the persistence of Western-centric Cultural Fit failures described above. Second, alignment procedures applied by Chinese developers may themselves be calibrated against English-language safety and fairness benchmarks developed primarily in Western institutions, such that post-training alignment imports Western framings of demographic risk independent of the underlying pre-training corpus, an alignment artifact rather than a pre-training-data effect. Distinguishing between these mechanisms would require systematically comparing model outputs on matched Chinese-language prompts describing the same clinical scenarios, evaluated against Chinese pediatric epidemiological benchmarks. Robust Chinese national surveillance data on childhood obesity exist for this purpose, including multicentre cross-sectional evidence documenting distinct regional and urban–rural risk gradients among Chinese children (32), and national clinical guidance from the Society of Pediatrics of the Chinese Medical Association on the diagnosis, assessment, and management of childhood obesity (27). That none of the three Chinese-origin models drew on this Chinese-specific epidemiological and clinical guidance when responding in English is itself informative, and suggests that regionally specific medical knowledge may not transfer symmetrically across languages within a single model. Future work directly comparing English- and Chinese-language prompting of the same models, benchmarked against Chinese national pediatric obesity data, is needed to adjudicate between the data-contamination and alignment-artifact explanations proposed here.

4.8 Strengths, limitations, and directions for future research

This study offers several methodological strengths. The structured prompt-based design enables direct comparison across models and demographic comparison dimensions using standardized stimuli, a methodological approach that provides higher internal validity than free-form query analysis. The triplicate submission design with majority decision coding controls for stochastic output variation and reduces the risk of single-run artifacts influencing comparative conclusions. The inclusion of seven models spanning four countries of origin and the expansion from two to six demographic comparison dimensions represents a more comprehensive cross-model, cross-dimension evaluation than previously published in the pediatric context.

Several limitations must be acknowledged. First, all prompts were submitted in English, which limits the generalisability of findings to non-English-language deployments of these models. This is particularly relevant for non-Western models such as GLM and Qwen, which may exhibit different bias profiles when queried in Chinese. Second, the study used fixed, standardized prompt templates, whereas real-world clinical queries are highly variable in wording, context, and specificity. The degree to which the bias patterns identified here generalize to more naturalistic query formulations is unknown. Third, the rubric scoring approach, while adapted from a validated framework, involves categorical judgments that may not capture the full qualitative richness of LLM responses; in particular, binary (0/1) scoring on dimensions such as Accuracy cannot distinguish a response that is partially correct or only marginally deficient from one that is wholly correct or wholly inadequate, which may understate genuine gradations in response quality. Fourth, the study is limited to childhood obesity; the extent to which the identified bias patterns characterize LLM behavior in other pediatric conditions or other disease domains cannot be inferred from these data. Fifth, all models were accessed via commercial web interfaces rather than programmatic APIs; while this choice was made deliberately to preserve the ecological validity of a real-world consumer/clinician use scenario (Section 2.2), it means that exact hyperparameters (temperature, top_p, and inference-time routing, e.g., in Mixture-of-Experts architectures) could not be fixed, logged, or guaranteed stable across the data-collection period. Web-interface default settings are not publicly documented by any of the seven developers and may change without notice, so this study's findings represent a snapshot of model behavior at the time of data collection rather than a bit-for-bit reproducible benchmark; future work seeking exact replication should consider programmatic API access with fixed seeds and documented hyperparameters, at the cost of reduced ecological validity.

Technical Limitation of LLM Stochasticity: while we employed triplicate sampling to mitigate the effects of LLM stochasticity, the inherent randomness of generative AI means that the performance captured in this study represents a probabilistic “snapshot” rather than a fixed system state. Future audits should consider even higher sampling frequencies (e.g., n = 10 or n = 50) to more precisely map the aleatoric (i.e., irreducible, randomness-driven, as opposed to knowledge-driven) uncertainty in model-based clinical risk attribution.

Future research should address several priority gaps. Multilingual evaluations are needed to determine whether bias profiles are language-dependent in models capable of operating across multiple languages. Intersectional analyses examining how models respond to prompts that simultaneously specify multiple demographic characteristics (e.g., a low-income Hispanic/Latino rural child) would provide a more realistic picture of the cumulative bias effects that real patients may encounter. Longitudinal evaluation designs tracking the same models across successive version releases would enable assessment of whether alignment improvements translate into measurable reductions in demographic bias. Finally, there is a pressing need for standardized, clinically validated bias evaluation frameworks for pediatric AI, analogous to those being developed for adult clinical AI systems, to enable systematic monitoring as LLMs are increasingly integrated into pediatric clinical decision support tools (33).

5 Conclusion

This study evaluated demographic bias and variability in childhood obesity risk attribution across seven publicly accessible large language models using English-language web-interface prompts. Across six clinical domains and six demographic comparison dimensions, the findings show that current LLM outputs are not demographically neutral. Although neutral prompt responses rarely contained overtly stigmatizing or harmful language, most models showed limited attention to demographic variability, social determinants of health, and culturally adaptable guidance.

Socioeconomic status produced the most consistent attribution pattern, with nearly all models assigning higher obesity risk to the low-income child. Race/ethnicity-based attribution was also common, particularly for Black and Hispanic/Latino children, while White–Asian comparisons showed greater cross-model inconsistency. The genetic predisposition domain appeared to amplify race-based attribution, raising concern that models may conflate population-level epidemiological associations with individual biological risk. Urban–rural comparisons showed the greatest instability and directional disagreement, suggesting that model provenance and training context may influence not only the strength but also the direction of demographic risk attribution.

Taken together, these findings suggest that LLMs should not be regarded as unbiased sources of demographic-specific pediatric risk guidance. Their outputs may reproduce epidemiological associations while insufficiently contextualizing the structural and cultural conditions that shape childhood obesity risk. Before LLMs are used in pediatric health consultation or clinical decision support, model-specific bias auditing, transparent reporting, and ongoing post-deployment monitoring are needed to reduce the risk of reinforcing health inequities.

References

• 1.

  PhelpsNHSingletonRKZhouBHeapRAMishraABennettJEet al. Worldwide trends in underweight and obesity from 1990 to 2022: a pooled analysis of 3663 population-representative studies with 222 million children, adolescents, and adults. Lancet. (2024) 403:1027–50. doi: 10.1016/S0140-6736(23)02750-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 2.

  KerrJAPattonGCCiniKIAbateYHAbbasNAbd Al MagiedAHAet al. Global, regional, and national prevalence of child and adolescent overweight and obesity, 1990–2021, with forecasts to 2050: a forecasting study for the Global Burden of Disease Study 2021. Lancet. (2025) 405:785–812. doi: 10.1016/S0140-6736(25)00397-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 3.

  World Health Organization. Obesity and Overweight Fact Sheet. (2025). Available online at: https://www.who.int/news-room/fact-sheets/detail/obesity-and-overweight (Accessed March 17, 2026).

  • Google Scholar
  • View reference in article

• 4.

  GüngörNK. Overweight and obesity in children and adolescents. J Clin Res Paediatr Endocrinol. (2014) 6:129. doi: 10.4274/jcrpe.1471

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 5.

  SkinnerACRavanbakhtSNSkeltonJAPerrinEMArmstrongSC. Prevalence of obesity and severe obesity in US children, 1999–2016. Paediatrics. (2018) 141:e20173459. doi: 10.1542/peds.2017-3459

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 6.

  Pineros-LeanoMGrafftNAguayoL. Childhood obesity risk factors by race and ethnicity. Obesity. (2022) 30:1670–80. doi: 10.1002/oby.23500

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7.

  LiuMInoueKBoydAAggarwalRMarinacciLXWadheraRK. Obesity prevalence among children and adolescents in the United States, 2011 to 2023. Ann Intern Med. (2025) 178:1517–20. doi: 10.7326/ANNALS-25-00389

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 8.

  MinJTamVMayneS. Paediatric obesity during COVID-19: the role of neighborhood social vulnerability and collective efficacy. Int J Obes. (2024) 48:550–6. doi: 10.1038/s41366-023-01448-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 9.

  WilliamsMSMcKinneySJCheskinLJ. Social and structural determinants of health and social injustices contributing to obesity disparities. Curr Obes Rep. (2024) 13:617–25. doi: 10.1007/s13679-024-00578-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 10.

  ThirunavukarasuAJTingDSJElangovanKGutierrezLTanTFTingDSW. Large language models in medicine. Nat Med. (2023) 29:1930–40. doi: 10.1038/s41591-023-02448-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 11.

  ClusmannJKolbingerFRMutiHSCarreroZIEckardtJ-NLalehNGet al. The future landscape of large language models in medicine. Commun Med. (2023) 3:141. doi: 10.1038/s43856-023-00370-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 12.

  MendelTSinghNMannDMWiesenfeldBNovO. Laypeople's use of and attitudes toward large language models and search engines for health queries: survey study. J Med Internet Res. (2025) 27:e64290. doi: 10.2196/64290

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 13.

  YoungCCEnichenERaoAHilkerSButlerALaird-GionJet al. Pilot study of large language models as an age-appropriate explanatory tool for chronic paediatric conditions. medRxiv. [preprint] (2024). doi: 10.1101/2024.08.06.24311544

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14.

  SezginEJacksonDIKocaballiABBibartMZupanecSLandierWet al. Can large language models aid caregivers of paediatric cancer patients in information seeking? A cross-sectional investigation. Cancer Med. (2025) 14:e70554. doi: 10.1002/cam4.70554

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 15.

  HagerPJungmannFHollandRBhagatKHubrechtIKnauerMet al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nat Med. (2024) 30:2613–22. doi: 10.1038/s41591-024-03097-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 16.

  OmiyeJALesterJCSpichakSRotembergVDaneshjouR. Large language models propagate race-based medicine. NPJ Digit Med. (2023) 6:195. doi: 10.1038/s41746-023-00939-z

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 17.

  OmarMSofferSAgbareiaRBragazziNLApakamaDUHorowitzCRet al. Sociodemographic biases in medical decision making by large language models. Nat Med. (2025) 31:1873–81. doi: 10.1038/s41591-025-03626-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 18.

  OmarMSorinVAgbareiaRApakamaDUSoroushASakhujaAet al. Evaluating and addressing demographic disparities in medical large language models: a systematic review. Int J Equity Health. (2025) 24:57. doi: 10.1186/s12939-025-02419-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 19.

  CrossJLChomaMAOnofreyJA. Bias in medical AI: implications for clinical decision-making. PLOS Digit Health. (2024) 3:e0000651. doi: 10.1371/journal.pdig.0000651

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 20.

  SchmidgallSHarrisCEssienIOlshvangDRahmanTKimJWet al. Evaluation and mitigation of cognitive biases in medical language models. NPJ Digit Med. (2024) 7:295. doi: 10.1038/s41746-024-01283-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 21.

  HamplSEHassinkSGSkinnerACArmstrongSCBarlowSEBollingCFet al. Clinical practice guideline for the evaluation and treatment of children and adolescents with obesity. Paediatrics. (2023) 151:e2022060640. doi: 10.1542/peds.2022-060640

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 22.

  World Health Organization. WHO Guidelines on Physical Activity and Sedentary Behaviour. Geneva: World Health Organization (2020).

  • Google Scholar
  • View reference in article

• 23.

  ChanJTNKwekRK. Uncovering bias and variability in how large language models attribute cardiovascular risk. Front Digit Health. (2025) 7:1710594. doi: 10.3389/fdgth.2025.1710594

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 24.

  World Health Organization. Guideline: Assessing and Managing Children at Primary Health-Care Facilities to Prevent Overweight and Obesity in the Context of the Double Burden of Malnutrition. World Health Organization (2017).

  • Google Scholar

• 25.

  National Institute for Health and Care Excellence. Meningitis (Bacterial) and Meningococcal Disease: Recognition, Diagnosis and Management. London: National Institute for Health and Care Excellence (NICE) (2024).

  • Google Scholar
  • View reference in article

• 26.

  BallGDMerdadRBirkenCSCohenTRGoodmanBHadjiyannakisSet al. Managing obesity in children: a clinical practice guideline. CMAJ. (2025) 197:E372–89. doi: 10.1503/cmaj.241456

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 27.

  of of Endocrinologic SGBoardE. Expert consensus on diagnosis, assessment, and management of obesity in Chinese children. Zhonghua er ke za zhi. (2022) 60:507–15. doi: 10.3760/cma.j.cn112140-20220112-00043

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 28.

  BaiXWangASucholutskyIGriffithsTL. Explicitly unbiased large language models still form biased associations. Proc Nat Acad Sci USA. (2025) 122:e2416228122. doi: 10.1073/pnas.2416228122

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 29.

  TomiyamaAJCarrDGranbergEMMajorBRobinsonESutinARet al. How and why weight stigma drives the obesity ‘epidemic'and harms health. BMC Med. (2018) 16:123. doi: 10.1186/s12916-018-1116-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 30.

  ZackTLehmanESuzgunMRodriguezJACeliLAGichoyaJet al. Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit Health. (2024) 6:e12–22. doi: 10.1016/S2589-7500(23)00225-X

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 31.

  ZhangKMengXYanXJiJLiuJXuHet al. Revolutionizing health care: the transformative impact of large language models in medicine. J Med Internet Res. (2025) 27:e59069. doi: 10.2196/59069

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 32.

  LiHXiangXYiYYanBYiLDingNet al. Epidemiology of obesity and influential factors in China: a multicenter cross-sectional study of children and adolescents. BMC Paediatr. (2024) 24:498. doi: 10.1186/s12887-024-04970-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 33.

  TemplinTFortSPadmanabhamPSeshadriPRimalROlivaJet al. Framework for bias evaluation in large language models in healthcare settings. Npj Digit Med. (2025) 8:414. doi: 10.1038/s41746-025-01786-w

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 34.

  LandisJRKochGG. The measurement of observer agreement for categorical data. Biometrics. (1977) 33:159–74.

  • Pubmed Abstract
  • Google Scholar
  • View reference in article