Evaluating LLMs on Kazakhstan's Mathematics Exam for University Admission

Shirali Kadyrov^1,2Bolatbek Abdrasilov³Aslan Sabyrov³Nurseit Baizhanov^3*Alfira Makhmuotova¹Patrick Charles Kyllonen⁴

• ¹New Uzbekistan University, Tashkent, Uzbekistan
• ²Narxoz University, Almaty, Kazakhstan
• ³National Test Center, Astana, Kazakhstan
• ⁴ETS, Princeton, United States

The rapid advancement of large language models (LLMs) has prompted their exploration in educational contexts, particularly for high-stakes standardized tests like Kazakhstan's Unified National Testing (UNT) mathematics component, which is critical for university admission. While most existing benchmarks for mathematical reasoning focus on English, there is growing concern that LLMs may underperform in under-resourced or non-English languages. This study addresses that gap by evaluating LLM performance on a math test administered entirely in Russian. We assess six LLMs—Claude, Deepseek, Gemini, Llama, Qwen, and o1—on 139 UNT multiple-choice questions covering algebra, functions, geometry, inequalities, and trigonometry. The methodology includes three conditions: zero-shot performance, hybrid integration with SymPy for symbolic computation, and a role-specific simulated multi-agent refinement framework that builds on existing self-correction techniques with targeted feedback. Results show that Deepseek, Gemini, Qwen, and o1 achieved near-perfect or perfect accuracy (91.2–100%) in zero-shot settings across all difficulty levels and topics, while Claude and Llama lagged (43.5–76.5%). The hybrid approach significantly improved Claude and Llama's accuracy by 27.4% and 39.9%, respectively. As for the multi-agent approach, significant improvement was observed in Claude with accuracy reaching 97.8%, showing 58.1% improvement compared to the zero-shot. These findings provide important empirical evidence that LLMs can perform competitively on math tasks in non-English languages, challenging prior assumptions and underscoring their potential to support bilingual education and equitable access to higher education in under-resourced linguistic settings.