ORIGINAL RESEARCH article

Front. Educ., 05 August 2026

Sec. Assessment, Testing and Applied Measurement

Volume 11 - 2026 | https://doi.org/10.3389/feduc.2026.1884670

Assessing chemistry disciplinary competence in high-stakes examination forms: evidence from the 2023 Chinese Gaokao

  • 1. College of Chemistry and Chemical Engineering, Northwest Normal University, Lanzhou, China

  • 2. Chengdu Normal University, Chengdu, China

  • 3. School of Medicine, Tongji University, Shanghai, China

Abstract

Based on a model of chemistry disciplinary competence, this study compares the competence structures of five selected 2023 Gaokao chemistry examination forms: the National Paper B form, the New Curriculum Standards form, and the Beijing, Shanghai, and Jiangsu forms. Using directed content analysis, cognitive task units were coded within a framework of three first-level indicators—learning understanding, applied practice, and transfer and innovation—and nine second-level indicators. Coding reliability was examined through independent coding by multiple raters. Results show that all five forms covered the three first-level indicators, with an average weight ratio of 24:45:31 (learning understanding 24.08%, applied practice 44.71%, transfer and innovation 31.21%). This indicates a profile dominated by applied practice, followed by transfer and innovation and then learning understanding. Among the nine second-level indicators, inferential prediction (B2, 24.51%) and complex reasoning (C1, 18.28%) had the highest weights, whereas innovative thinking (C3, 1.20%) had the lowest. The five forms showed stronger convergence in applied practice and transfer-oriented reasoning than in learning-understanding indicators. The findings reveal the competence-oriented features and paper-and-pencil assessment constraints embedded in the selected 2023 Gaokao chemistry examination forms.

1 Introduction

1.1 Washback effects of high-stakes testing and the expression of curriculum values

Educational assessment is not an external add-on to schooling but a key mechanism for implementing curriculum goals, regulating instruction, and shaping educational values. In high-stakes contexts, examinations not only measure, certify, and select students but also influence teaching, learning, and curriculum enactment through assessment standards, task formats, competence demands, and score use (Au, 2007; ). How test scores are interpreted, used, and translated into educational consequences is therefore central to evaluating test quality (). Gaokao chemistry examination forms are thus not isolated examination texts; they embody curriculum ideas, assessment systems, and talent-development goals.

Figure 1

Figure 2

As the most authoritative assessment in Chinese basic education, the Gaokao selects students for higher education and exerts strong washback on senior high school teaching through its content, contexts, competence requirements, and scoring criteria (; ). This effect is especially salient in chemistry education (; ). Chemistry learning involves not only concepts, principles, and reaction facts but also the use of core chemical ideas to explain phenomena, analyze change, construct models, and solve authentic problems (). The competence structure of Gaokao chemistry examination forms therefore reflects both expected student performance and the forms of scientific thinking and knowledge use valued by the curriculum.

1.2 The gaokao chemistry examination: institutional context and examination forms

The National College Entrance Examination, commonly known as the Gaokao, is China's nationwide high-stakes examination used primarily for university admission. Rather than relying on a single uniform chemistry test form for all students, the Gaokao system uses different examination forms across provinces and reform contexts. Some forms are developed nationally, whereas others are developed locally by provincial or municipal examination authorities. In this article, the term “examination form” is used to refer to a specific chemistry test administered in a given national or regional context.

The chemistry examination is administered in paper-and-pencil format. Score comparison and admission decisions are normally made within provincial admission systems rather than through direct national comparison of raw chemistry scores across all forms. Therefore, this study compares different forms to examine how test developers operationalize chemistry disciplinary competence, rather than to evaluate score equivalence across provinces.

1.3 International research trends in chemistry education assessment

Chemistry education research also emphasizes the role of assessment in shaping disciplinary learning. Chemistry assessment should not focus only on whether students remember concepts, formulas, and reaction facts; it should also examine whether they can use chemical knowledge for reasoning, explanation, and transfer (; ). Stowe et al. further argue that what is repeatedly emphasized and rewarded in assessment sends students a clear signal about what counts as important chemistry learning (). Therefore, analyses of Gaokao chemistry forms should move beyond content coverage, item types, and score distribution to examine the forms of chemical knowledge use and cognitive activity expected by the tasks.

From the perspective of international science assessment reform, competence-oriented assessment has become a dominant paradigm. The PISA science framework identifies scientific reasoning and evidence evaluation, explaining scientific phenomena, and designing scientific inquiry as core assessment targets (). In chemistry education, Raker et al. proposed an item-complexity framework based on cognitive task analysis (2013), while Tekkumru-Kisa et al. developed an analytical tool for examining cognitive demand and practice integration in science tasks (2015). These frameworks converge on a central issue: how large-scale standardized tests can validly assess higher-order cognition. The reform of Chinese Gaokao chemistry assessment therefore provides a useful context for examining how competence-oriented science assessment is operationalized within a high-stakes paper-and-pencil examination system.

1.4 Institutional context of Gaokao chemistry assessment reform in China

Against the backdrop of China's new round of curriculum and Gaokao reform, chemistry test development has increasingly emphasized disciplinary competence, contextualized application, and integrated reasoning (). The General High School Chemistry Curriculum Standards identify five chemistry core competencies, including macroscopic identification and microscopic analysis, ideas of change and equilibrium, evidence-based reasoning and model cognition, scientific inquiry and innovation awareness, and scientific attitude and social responsibility (). The China Gaokao Evaluation Framework further emphasizes application, integration, and innovation as key principles of test design (). However, how these policy goals are translated into concrete competence structures in actual examination forms requires systematic empirical examination.

The coexistence of old and new curriculum and Gaokao systems in 2023 reflected the phased implementation of curriculum and examination reforms across Chinese provinces. Curriculum reform and Gaokao reform have not always proceeded simultaneously. Consequently, some regions retained examination arrangements associated with the old Gaokao structure, while their chemistry teaching and test content had already been influenced by the revised senior high school chemistry curriculum. This transitional situation produced multiple reform configurations, including old curriculum–old Gaokao, new curriculum–old Gaokao, and new curriculum–new Gaokao forms. Therefore, the five examination forms selected in this study were not intended to represent a single unified testing system; rather, they were selected to capture different institutional configurations during a period of reform transition.

Wang and Zhi's model of chemistry disciplinary competence provides a chemistry-specific framework for analyzing these examination forms. The model conceptualizes students’ cognitive activities in chemistry learning and problem solving through three first-level indicators: learning understanding, applied practice, and transfer and innovation. These are further divided into nine second-level indicators: recognition and recall, generalization and association, explanation and argumentation, analysis and explanation, inferential prediction, simple design, complex reasoning, systematic inquiry, and innovative thinking (). The model captures a progression from foundational understanding to contextualized application and then to transfer-oriented problem solving, making it suitable for analyzing how Gaokao chemistry forms operationalize disciplinary competence.

This framework is broadly aligned with China's chemistry core competencies. For example, macroscopic identification and microscopic analysis and ideas of change and equilibrium are reflected in tasks requiring conceptual association, analysis, prediction, and multilevel reasoning; evidence-based reasoning and model cognition is closely related to explanation, inferential prediction, complex reasoning, and systematic inquiry; scientific inquiry and innovation awareness corresponds especially to design, inquiry, and innovation-oriented indicators; and scientific attitude and social responsibility is reflected when chemistry tasks are situated in environmental, industrial, technological, or social contexts (). Internationally, the framework also resonates with PISA science competencies and NGSS science and engineering practices, including model use, data interpretation, evidence-based explanation, argumentation, and design of solutions (; ). Thus, Wang and Zhi's model provides a contextually grounded but internationally comparable framework for examining the competence structure of Gaokao chemistry examination forms.

1.5 Contributions and limitations of previous research

Existing studies on Gaokao chemistry forms have mainly followed three lines of inquiry. First, some have analyzed the features of individual forms based on curriculum standards and disciplinary core competencies (; ). Second, others have examined item content in terms of essential knowledge, key abilities, and problem contexts (). Third, some studies have conducted quantitative analyses of forms from specific regions or years using disciplinary competence models or assessment frameworks (). These studies provide an important basis for understanding reform trends in Gaokao chemistry assessment, but two gaps remain. First, most studies focus on a single forms or on forms from one region across years, with limited cross-sectional comparison of different forms types and regions within the same year. Second, many studies emphasize the distribution of knowledge points and competence indicators, while offering limited interpretation of the selection logic, curriculum value, and disciplinary educational orientation behind the competence structure. At the current stage, when the old and new Gaokao systems coexist, comparing different types of forms is particularly valuable. Such comparison can reveal not only the internal logic of test design but also the depth and variation of assessment reform implementation across regions.

1.6 Research purpose and research questions

Based on the above analysis, this study examines the 2023 National forms B, New Curriculum Standards forms, and the Beijing, Shanghai, and Jiangsu chemistry forms. Drawing on the model of chemistry disciplinary competence, it compares how different levels of competence are assessed across the five forms. The study addresses three research questions:.

  • RQ1: How are the nine second-level indicators of chemistry disciplinary competence represented in the five 2023 Gaokao chemistry examination forms?

  • RQ2: When the nine second-level indicators are aggregated, what first-level competence profiles emerge across the five examination forms?

  • RQ3: What patterns of convergence and differentiation can be identified across nationally developed and locally developed forms, and what do these patterns suggest about the competence-oriented logic of Gaokao chemistry assessment?

Through this analysis, the study seeks to move Gaokao item research beyond score distribution toward competence-structure interpretation, and beyond technical test analysis toward a value-oriented understanding of educational assessment. It also provides empirical evidence from China for international research on science education assessment.

2 Research design

2.1 Research objects

The year 2023 marked an important stage in which the old and new Gaokao systems coexisted and the core-competency orientation continued to deepen. Analyzing the competence structure of Gaokao chemistry forms from this period can help clarify the current reform direction and competence-oriented features of chemistry assessment ().

This study selected five 2023 Gaokao chemistry examination forms: the National forms B, the New Curriculum Standards forms, and the Beijing, Shanghai, and Jiangsu forms. These forms were selected because they represent both nationally and locally developed examinations under three distinct reform configurations—old curriculum–old Gaokao, new curriculum–old Gaokao, and new curriculum–new Gaokao—and were all officially released in 2023(see Table 1).

Table 1

FormsTest development typeReform contextTotal score usedRationale for selection
National forms BNationally developed formsUsed in old-Gaokao regionsCalculated out of 115 pointsRepresents the traditional national-forms format and includes elective items
New Curriculum Standards formsNationally developed formsNational forms under the new curriculum contextCalculated out of 100 pointsRepresents the reform direction of nationally developed forms
Beijing formsLocally developed formsSecond-batch new-Gaokao reform regionCalculated out of 100 pointsRepresents new-Gaokao test development in a municipality
Shanghai formsLocally developed formsFirst-batch new-Gaokao reform regionCalculated out of 100 pointsRepresents the assessment orientation of one of the earliest new-Gaokao reform regions
Jiangsu formsLocally developed formsThird-batch new-Gaokao reform regionCalculated out of 100 pointsRepresents locally developed new-Gaokao chemistry assessment in a leading education province

Research samples and rationale for selection.

The National forms B included a 15-point elective section. To preserve the full competence structure embedded in the form, the total score analyzed for this form was 115 points. The other four forms were analyzed on a 100-point scale.

2.2 Analytical framework

This study used directed content analysis to code the competence demands embedded in the cognitive tasks of Gaokao chemistry items (). The analytical framework was based on the model of chemistry disciplinary competence, which classifies students’ competence in chemistry learning and problem solving into three first-level indicators: learning understanding (A), applied practice (B), and transfer and innovation (C). These indicators are further divided into nine second-level indicators ().

Learning understanding (A) refers to students’ ability to recognize, connect, and explain chemical knowledge, activity experience, and basic methods. Applied practice (B) refers to students’ ability to use chemical knowledge in familiar or modified contexts to analyze problems, explain phenomena, predict outcomes, and design solutions. Transfer and innovation (C) refers to students’ ability to conduct integrated reasoning, systematic inquiry, and innovative problem solving in complex, unfamiliar, or uncertain contexts. This framework captures the progression from basic understanding to integrated application and from near transfer to far transfer in Gaokao chemistry tasks (see Table 2).The coding framework was derived from an established model of chemistry disciplinary competence and operationalized according to the cognitive features of Gaokao chemistry tasks. This provided theoretical grounding and content-validity support for linking competence indicators with item-level cognitive demands ().

Table 2

First-level indicatorSecond-level indicatorDescription
A. Learning UnderstandingA1. Recognition and RecallIdentifying and reproducing basic chemical facts, concepts, symbols, experimental phenomena, and prototypical activity experiences
A2. Generalization and AssociationEstablishing connections among concepts, principles, phenomena, operations, symbols, and models
A3. Explanation and ArgumentationExplaining and justifying prototypical schemes, experimental operations, reaction processes, or chemical conclusions
B. Applied PracticeB1. Analysis and ExplanationAnalyzing chemical processes, explaining experimental phenomena, or clarifying reaction principles in familiar or modified contexts
B2. Inferential PredictionPredicting material properties, reaction trends, variable relationships, or experimental results based on given information
B3. Simple DesignDesigning experimental procedures, operational pathways, or problem-solving schemes in relatively well-defined task contexts
C. Transfer and InnovationC1. Complex ReasoningIntegrating multiple sources of information in complex and unfamiliar contexts to conduct multi-step and multivariable reasoning
C2. Systematic InquirySystematically designing inquiry processes for complex problems, including hypothesis generation, evidence collection, and conclusion construction
C3. Innovative ThinkingProposing novel solutions, identifying new patterns, or solving cross-task problems in highly uncertain or open-ended contexts

Assessment framework for chemistry disciplinary competence.

In applying this framework to Gaokao chemistry examination forms, particular attention was given to the boundary among C1 complex reasoning, C2 systematic inquiry, and C3 innovative thinking. C1 was coded when a task primarily required students to integrate multiple sources of information, coordinate several variables, and complete multi-step reasoning within a relatively constrained problem structure. C2 was coded when the task emphasized the organization of an inquiry process, such as identifying a problem, controlling variables, collecting or evaluating evidence, and constructing a conclusion. C3 was coded only when the task required students to propose a relatively novel solution, identify a new pattern, transfer a strategy to a less familiar context, or evaluate multiple possible solutions in a relatively open-ended or uncertain situation. Therefore, C3 was not equated with general difficulty, unfamiliarity, or multi-step reasoning. It was reserved for tasks involving solution openness, uncertainty, or creative transfer within the constraints of a high-stakes paper-and-pencil examination.

2.3 Coding units and scoring rules

Complex chemistry items were decomposed into smaller cognitive task units for analysis (). A cognitive task unit was defined as the smallest meaningful segment of an item that required a distinct cognitive operation for successful problem solving. This unit-based approach was adopted because a single Gaokao chemistry item may contain several different competence demands, especially in constructed-response, experimental, industrial-process, and organic-inference tasks (; ).

For selected-response items, including multiple-choice and multi-select items, the coding unit was either the whole item or an individual option or judgment point. When all options followed the same reasoning logic, the whole item was coded according to its dominant cognitive demand. When different options required different reasoning bases, each option was treated as a separate cognitive task unit, and the item score was evenly distributed across the coded options. This score distribution was used only to estimate the relative weight of competence demands embedded in the item text, not to reproduce actual student scoring.

For constructed-response items, cognitive task units were identified according to sub-questions, scoring points, and key reasoning steps. Experimental design tasks were coded by distinguishing among experimental purpose, principle explanation, variable control, procedure design, evidence evaluation, error analysis, and conclusion construction. Industrial process tasks were coded by distinguishing among information extraction, process-step function, reaction explanation, condition optimization, product inference, and environmental or resource-related evaluation. Organic inference tasks were coded by distinguishing among structural identification, functional-group transformation, reaction-path inference, evidence integration, and representational conversion. Open-ended or semi-open-ended responses were coded according to the dominant cognitive activity required by the expected response; when more than one indicator was equally central, score splitting and dual coding were applied (see Table 3).

Table 3

Response formatItem/task subtypeCoding unitScore allocation and coding principle
Selected responseMultiple-choice (MC) itemWhole item or optionIf all options followed the same reasoning logic, the whole item was coded by the dominant indicator; if options required different cognitive operations, the score was evenly divided across options.
Multi-select/judgment itemOption, statement, or judgment pointScores were divided by option or judgment point when statements required different reasoning bases.
Constructed responseGeneral CR itemSub-question, scoring point, or reasoning stepScores were allocated according to sub-question scores or scoring points; coding focused on the dominant cognitive task in each response component.
Experimental/inquiry taskAim, principle, procedure, phenomenon, evidence, or conclusionCoding distinguished experimental explanation, scheme design, evidence evaluation, and inquiry reasoning.
Industrial process/organic inference taskProcess step, condition, product inference, structure, reaction path, or evidenceCoding focused on information integration, variable reasoning, multistep inference, and representation transformation.
Open-ended or semi-open-ended componentProposed scheme, solution element, or evaluation criterionC3 was coded only when the task involved solution openness, alternative schemes, creative transfer, or constrained multi-solution evaluation.

Coding rules for cognitive task units across paper-and-pencil response formats.

2.4 Coding examples

To increase transparency, one multiple-choice item and one constructed-response item were selected to illustrate the coding procedure.

Beijing forms, Item 5, was set in the context of recycling CO₂ and SO₂ from industrial waste gas and asked students to judge the correctness of four statements (3 points in total). The four options involved different cognitive tasks. Option A required students to recognize the factual link between SO₂ and acid rain and was coded as A1, recognition and recall. Option B required judgment of solution acidity based on the hydrolysis and ionization of HCO₃⁻ and was coded as A2, generalization and association. Option C required explanation of the function of the solution in the device and was coded as A3, explanation and argumentation. Option D required analysis of an electrochemical conversion process and identification of the overall reaction and was coded as B1, analysis and explanation. As the four options represented relatively independent cognitive tasks, the 3 points were evenly assigned, with 0.75 points for each option. This equal allocation was used only to estimate the relative weights of cognitive tasks in the item text, not to simulate actual scoring.

Shanghai forms, Item 21, focused on the equilibrium of methane reacting with steam and methanol synthesis from CO and H₂ (12 points in total). It involved molecular polarity judgment, reaction-rate calculation, equilibrium-state analysis, changes in conditions and equilibrium shift, and feed-ratio inference. Sub-question (1) was coded as A1, recognition and recall; sub-question (2) as A2, generalization and association; sub-questions (3) and (5) as B2, inferential prediction; and sub-question (4) as B3, simple design.

2.5 Inter-Rater reliability

To enhance coding reliability, three raters with backgrounds in chemistry education or senior high school chemistry teaching independently coded the data: one doctoral student in chemistry education, one master's student in chemistry education, and one senior high school chemistry teacher with a senior professional title. The author, as a member of the research team, did not participate in formal coding to ensure rater independence.

Before formal coding, the research team developed a coding manual based on the chemistry disciplinary competence framework and conducted pilot coding on selected multiple-choice and constructed-response items. Based on the pilot results, the boundaries among competence indicators, the division of cognitive task units, and the score-splitting rules were further clarified.

During formal coding, three raters independently coded each cognitive task unit into one of the nine second-level indicators. The simple percentage agreement was 82.3%, and Fleiss’ kappa was 0.66, indicating substantial agreement (). Disagreements were then resolved through consensus-based discussion rather than by simple majority vote. In each discrepant case, the raters revisited the item text, official answer, scoring points, and coding manual to identify the dominant cognitive activity required by the task unit. A final code was adopted only when all raters reached agreement on the most defensible indicator assignment.

2.6 Data analysis

Descriptive statistics were used to analyze the distribution of chemistry disciplinary competence indicators across the five forms. First, the scores for the nine second-level indicators and their proportions of the total score were calculated for each form. Second, the second-level indicators were aggregated into three first-level indicators—learning understanding, applied practice, and transfer and innovation—to compare competence structures across forms. Finally, mean weight, standard deviation, and coefficient of variation were used to describe common patterns and differences in competence demands.

Because the study analyzed five selected Gaokao chemistry forms based on item-text coding rather than random sampling, no inferential statistical tests were conducted. CV was used only as a descriptive indicator of relative dispersion. In this study, CV values below 0.15 were interpreted as relatively low dispersion, values from 0.15 to 0.30 as moderate dispersion, and values above 0.30 as relatively high dispersion. These thresholds were used only for descriptive interpretation and were not treated as inferential statistical criteria.

3 Results

Following the coding procedures described above, we calculated the score weight of each second-level indicator in the five 2023 Gaokao chemistry examination forms. Table 4 reports the exact percentage weights, and Figure 3 presents a five-panel visualization of the nine second-level indicators across the five forms. The values displayed in Figure 3 are rounded to the nearest whole percentage for readability.

Table 4

Level 1Level 2National B (%)NCS (%)Beijing (%)Shanghai (%)Jiangsu (%)Mean (%)SDCV
A. Learning UnderstandingA11.749.004.007.006.005.552.500.45
A210.436.0012.0016.009.0010.693.310.31
A35.22010.0012.0012.007.844.640.59
Subtotal17.3915.0026.0035.0027.0024.087.190.30
B. Applied PracticeB113.0415.0020.0011.0014.0014.613.000.21
B229.5733.0019.0020.0021.0024.515.670.23
B36.963.004.008.006.005.591.850.33
Subtotal49.5751.0043.0039.0041.0044.714.740.11
C. Transfer and InnovationC117.3921.0019.0014.0020.0018.282.450.13
C215.6513.0010.0012.008.0011.732.610.22
C3002.0004.001.201.601.33
Subtotal33.0434.0031.0026.0032.0031.212.790.09

Weights and dispersion of chemistry disciplinary competence indicators across five 2023 Gaokao chemistry examination forms.

A1, recognition and recall; A2, generalization and association; A3, explanation and argumentation; B1, analysis and explanation; B2, inferential prediction; B3, simple design; C1, complex reasoning; C2, systematic inquiry; C3, innovative thinking. NCS, new curriculum standards. The National B form was calculated out of 115 points because it included a 15-point elective section, whereas the other four forms were calculated out of 100 points. SD and CV were calculated descriptively across the five selected forms. CV, SD/Mean.

Figure 3

3.1 Distribution of the nine second-level indicators

The distribution of the nine second-level indicators shows that the five examination forms did not allocate competence demands evenly. Across the five forms, B2 inferential prediction had the highest mean weight (24.51%), followed by C1 complex reasoning (18.28%) and B1 analysis and explanation (14.61%). At the lower end, C3 innovative thinking had the lowest mean weight (1.20%), with A1 recognition and recall (5.55%) and B3 simple design (5.59%) also showing relatively low representation. This pattern indicates that the most frequently represented competence demands were not direct recall or routine design, but inference, explanation, and multi-step reasoning in contextualized chemistry problems.

At the form level, B2 inferential prediction was the most prominent indicator in the National forms B and the New Curriculum Standards forms, accounting for 29.57% and 33.00%, respectively. It also maintained a substantial proportion in the Beijing, Shanghai, and Jiangsu forms, with weights of 19.00%, 20.00%, and 21.00%, respectively. These tasks typically required students to predict material properties, reaction trends, variable relationships, or experimental results based on given information. They appeared frequently in reaction principles, chemical equilibrium, electrochemistry, experimental analysis, and industrial process contexts.

C1 complex reasoning was also consistently represented across the five forms. Its weight ranged from 14.00% in the Shanghai form to 21.00% in the New Curriculum Standards forms. C1 tasks generally required students to integrate textual, graphical, tabular, experimental, and symbolic information and to construct multi-step reasoning chains across multiple variables. This consistent representation of C1 across all five forms (range: 14.00%–21.00%) suggests that multi-step information integration was a shared competence demand regardless of form type or reform context.

Learning-understanding indicators showed more visible variation across forms. A2 generalization and association had the highest average weight within the learning-understanding group (10.69%), whereas A1 recognition and recall had a relatively low mean weight (5.55%). The Shanghai form showed the highest A2 weight (16.00%), suggesting stronger emphasis on connecting chemical concepts, principles, phenomena, representations, and data. A3 explanation and argumentation varied more noticeably: it was absent in the New Curriculum Standards forms but reached 10.00%, 12.00%, and 12.00% in the Beijing, Shanghai, and Jiangsu forms, respectively.

The least represented indicator was C3 innovative thinking. No clear C3 tasks were identified in the National forms B, the New Curriculum Standards forms, or the Shanghai form. Only the Beijing and Jiangsu forms included limited C3 tasks, with weights of 2.00% and 4.00%, respectively. These tasks involved relatively open-ended scheme evaluation, cross-context transfer, or the proposal of alternative solutions. The very low proportion of C3 indicates that highly open-ended innovation-oriented tasks were rare in the selected 2023 forms.

The dispersion statistics further show that C3 innovative thinking had the highest coefficient of variation (CV = 1.33), mainly because it appeared only in the Beijing and Jiangsu forms and was absent from the other three forms. A3 explanation and argumentation also showed relatively large variation (CV = 0.59), reflecting its absence from the New Curriculum Standards Paper and its higher representation in several locally developed forms. By contrast, C1 complex reasoning had the lowest coefficient of variation among the nine second-level indicators (CV = 0.13), indicating relatively stable representation across the selected forms.

3.2 Aggregated first-level competence profiles

When the nine second-level indicators were aggregated into three first-level indicators, all five forms covered learning understanding, applied practice, and transfer and innovation. However, the relative weights of these three first-level indicators differed.

Across the five forms, applied practice had the highest average weight (44.71%), followed by transfer and innovation (31.21%) and learning understanding (24.08%). This aggregated profile indicates that the selected forms placed greater emphasis on applying and transferring chemical knowledge than on basic recognition or recall, consistent with a competence-oriented assessment approach.

Applied practice was the largest first-level indicator in all five forms. It accounted for 49.57% in the National forms B, 51.00% in the New Curriculum Standards forms, 43.00% in the Beijing form, 39.00% in the Shanghai form, and 41.00% in the Jiangsu form. The dominance of applied practice was mainly driven by the high weight of B2 inferential prediction and, to a lesser extent, B1 analysis and explanation.

Transfer and innovation also accounted for a substantial proportion in each form. Its weight was 33.04% in the National forms B, 34.00% in the New Curriculum Standards forms, 31.00% in the Beijing form, 26.00% in the Shanghai form, and 32.00% in the Jiangsu form. However, this first-level indicator was mainly supported by C1 complex reasoning and C2 systematic inquiry rather than by C3 innovative thinking.

Learning understanding had the lowest average weight among the three first-level indicators, but its distribution varied considerably across forms. It was relatively low in the National forms B (17.39%) and the New Curriculum Standards forms (15.00%), but higher in the Beijing (26.00%), Shanghai (35.00%), and Jiangsu (27.00%) forms. This suggests that locally developed forms, especially the Shanghai form, assigned relatively more weight to conceptual association and explanation within the learning-understanding category.

3.3 Convergence and differentiation across examination forms

The five forms showed both shared patterns and form-specific emphases. A shared pattern was the relatively strong representation of B2 inferential prediction and C1 complex reasoning across all five forms. These two indicators together accounted for approximately 42.79% of the total weight on average. This suggests that inference from given information and multi-step reasoning in complex contexts were central competence demands in the selected 2023 Gaokao chemistry forms.

At the same time, the forms differed in their allocation of specific indicators. The National forms B and the New Curriculum Standards forms placed the strongest emphasis on B2 inferential prediction, indicating a pronounced focus on prediction based on chemical principles, experimental information, and variable relationships. The Beijing form showed the highest weight for B1 analysis and explanation, suggesting greater emphasis on explaining chemical processes, experimental phenomena, and device functions. The Shanghai form assigned the highest weight to B2 generalization and association and had the largest overall proportion of learning understanding. The Jiangsu form showed relatively high weights for A3 explanation and argumentation and C1 complex reasoning, and it included the highest proportion of C3 innovative thinking among the five forms.

The coefficients of variation further describe these patterns. According to the descriptive criteria used in this study, the CV for learning understanding (0.30) was at the upper boundary of moderate dispersion and was higher than those for applied practice (0.11) and transfer and innovation (0.09), both of which indicated relatively low dispersion. This suggests that the five forms differed more in the allocation of learning-understanding indicators than in the aggregated proportions of applied practice and transfer and innovation. Because CV was used descriptively and the study did not involve random sampling, these differences should not be interpreted as statistically significant.

Overall, the results show that the five selected 2023 Gaokao chemistry examination forms were characterized by a strong emphasis on applied practice, inferential prediction, and complex reasoning. They also reveal a clear structural limitation: innovative thinking was only marginally represented.

4 Discussion and teaching implications

This study examined how chemistry disciplinary competence was represented in five selected 2023 Gaokao chemistry examination forms. The results show a competence profile characterized by the dominance of applied practice, the strong representation of inferential prediction and complex reasoning, and the marginal presence of innovative thinking. Because the sample covered only five of the eighteen chemistry forms administered in 2023—including two nationally developed forms and three locally developed forms under different reform configurations—these findings should be understood as descriptive patterns within the selected forms rather than as statistically representative or longitudinally comparative conclusions.

4.1 Competence-oriented assessment and contextualized knowledge use

A major finding of this study is that applied practice accounted for the largest average proportion across the five forms, whereas learning understanding, especially A1 recognition and recall, received a relatively lower weight. This pattern suggests that the selected forms placed substantial emphasis on students’ ability to use chemical knowledge in contextualized problem situations. In these tasks, knowledge was not assessed merely as isolated facts, symbols, formulas, or reaction rules; rather, it functioned as a cognitive resource for explanation, prediction, analysis, and problem solving.

This finding is consistent with the broader international movement toward competence-oriented science assessment. The PISA 2025 science framework emphasizes students’ ability to explain phenomena scientifically, construct and evaluate designs for scientific enquiry, interpret scientific data and evidence critically, and use scientific information for decision making (). AP Chemistry similarly organizes assessment around both chemistry content and science practices, including model representation, question and method development, data analysis, mathematical routines, explanation, and argumentation (). IB Chemistry also emphasizes practical work, investigation, inquiry, and the development of both conceptual understanding and scientific thinking (). Although the Gaokao differs from PISA, AP Chemistry, and IB Chemistry in purpose, format, and stakes, the selected Gaokao chemistry forms similarly emphasized students’ use of chemistry knowledge for reasoning in context.

However, this comparison also reveals a distinctive feature of Gaokao chemistry assessment. PISA is a low-stakes international assessment designed to monitor scientific literacy at the system level, whereas the Gaokao is a high-stakes selection examination used for university admission. AP Chemistry and IB Chemistry include free-response, laboratory, or inquiry-related components, but the Gaokao chemistry examination relies primarily on time-limited paper-and-pencil tasks. Therefore, competence-oriented assessment in the Gaokao must operate under the institutional requirements of fairness, scoring reliability, time efficiency, and score comparability (; ). This institutional context helps explain why contextualized knowledge use and inferential reasoning were strongly represented, whereas fully open-ended innovation-oriented tasks remained limited.

4.2 Inferential prediction and Complex reasoning as central competence demands

Among the nine second-level indicators, B2 inferential prediction and C1 complex reasoning were the most prominent competence demands. Together, they accounted for a large proportion of the total score weight across the five forms.

The prominence of B2 inferential prediction is understandable from a disciplinary perspective. Chemistry reasoning frequently involves predicting properties from structure, inferring reaction outcomes from conditions, explaining trends from data, and using evidence to evaluate possible mechanisms. In high school chemistry assessment, such reasoning is commonly operationalized through tasks involving chemical equilibrium, electrochemistry, reaction rates, experimental results, industrial conditions, and material properties. These tasks require more than factual recall because students must use chemical principles and given information to make justified predictions.

The prominence of C1 complex reasoning is similarly rooted in the disciplinary nature of chemistry. Chemistry problems often require students to coordinate macroscopic phenomena, submicroscopic explanations, symbolic equations, graphical information, tabular data, and experimental evidence (; ; ). In the selected forms, C1 tasks typically required students to integrate multiple sources of information and reason across several variables. This aligns with research in chemistry education showing that high-quality assessment should evaluate not only whether students know chemical concepts, but also whether they can coordinate representations, evidence, and models to explain and predict chemical phenomena (; ; ; ).

The dominance of B2 and C1 can also be interpreted through the logic of high-stakes selection. These task types require more than factual recall, but they can still be constrained by given information, expected reasoning pathways, and relatively clear scoring criteria. They therefore provide a feasible way to assess higher-order chemical reasoning while maintaining fairness, scoring reliability, and score comparability.

Therefore, the prominence of B2 and C1 does not simply indicate that the selected forms included more difficult items. Rather, it suggests that these forms tended to assess forms of chemical reasoning that are both disciplinary meaningful and psychometrically manageable within a large-scale standardized examination.

4.3 Convergence and differentiation across examination forms

The five forms showed both convergence and differentiation. Convergence was most evident in the relatively strong representation of applied practice, inferential prediction, and complex reasoning. This suggests that different test developers shared a broadly competence-oriented assessment logic, even though the selected forms were developed under different curriculum and examination reform contexts. This convergence may reflect the normalizing influence of the senior high school chemistry curriculum standards and the China Gaokao Evaluation Framework, both of which emphasize application, integration, reasoning, and innovation as important directions for assessment reform (; ).

However, this convergence should be interpreted with caution. Because this study analyzed only five selected forms, the findings cannot be generalized to all Gaokao chemistry examinations in China. The difference between the Shanghai form and the New Curriculum Standards Paper deserves careful interpretation. The Shanghai form allocated a relatively higher proportion to learning understanding, especially A2 generalization and association, whereas the New Curriculum Standards Paper allocated a much lower proportion to learning understanding and a higher proportion to B2 inferential prediction. This contrast may be associated with differences in test-development traditions, regional examination philosophies, item-format structures, and reform implementation contexts. For example, the Shanghai form may have retained a stronger emphasis on relational understanding and explanation within a locally developed assessment tradition, whereas the New Curriculum Standards Paper may have placed greater emphasis on prediction and information-based reasoning under a nationally developed reform-oriented framework. However, this study cannot determine the causal source of these differences. Because the analysis is based on selected examination texts from one year, these explanations should be treated as interpretive possibilities rather than causal claims.

The absence of clearly coded A3 explanation and argumentation tasks in the New Curriculum Standards forms also requires cautious interpretation. This result does not necessarily mean that the form lacked explanatory reasoning. Rather, explanation may have been embedded within inferential prediction, calculation, or information-processing tasks and was therefore coded according to the dominant cognitive activity required by each task unit. In standardized chemistry examinations, explanation and argumentation are often compressed into constrained formats, such as selecting a correct reason, completing a reaction pathway, interpreting a data pattern, or providing a brief justification. Such formats may reduce the visibility of full claim–evidence–reasoning structures. Therefore, the absence of A3 in this form should be interpreted as a feature of item design and coding classification rather than as direct evidence that explanation and argumentation were entirely absent.

4.4 Structural constraints on assessing innovative thinking

The marginal representation of C3 innovative thinking is one of the most important findings of this study. C3 had the lowest average weight and appeared only in limited portions of the Beijing and Jiangsu forms. This finding should not be read as evidence that Gaokao chemistry reform ignores innovation. Rather, it reveals the structural difficulty of assessing innovative thinking within large-scale high-stakes standardized examinations.

The low weight of C3 is therefore closely related to the institutional logic of high-stakes testing. Large-scale selection examinations must prioritize fairness, objectivity, scoring consistency, and score comparability (; ). Open-ended innovation tasks often allow multiple legitimate solution paths and require complex rubrics, expert judgment, and extended response time. These features increase the risk of rater inconsistency and reduce the scalability of scoring. Research on creativity assessment has similarly shown that open-ended creative responses are difficult to judge reliably because they involve novelty, appropriateness, and task-specific interpretation (; ). Therefore, the marginal representation of C3 reflects a limitation of large-scale paper-and-pencil testing: innovation-oriented tasks require openness and solution diversity, whereas this assessment format requires concise responses, clearly bounded scoring criteria, and high score comparability.

This tension is not unique to the Gaokao. PISA can include more contextualized and inquiry-oriented tasks because it is a system-level assessment rather than an individual selection examination (). IB Chemistry can assess practical work and inquiry partly because it includes investigation-oriented learning and assessment components (). AP Chemistry includes free-response tasks and science practices, but these responses must still be sufficiently constrained to support standardized scoring (). Compared with these systems, the Gaokao has less room for extended open-ended tasks because of its exceptionally high stakes, large candidate population, and need for rapid, comparable scoring. Thus, the low proportion of C3 should be understood as a boundary condition of the assessment format rather than simply as a weakness of test design.

4.5 Washback and teaching implications for chemistry classrooms

The competence structure identified in this study has important washback implications for senior high school chemistry teaching. High-stakes examinations influence what teachers emphasize, what students practice, and what schools treat as valuable learning outcomes (; ; ). The strong representation of B2 inferential prediction and C1 complex reasoning may encourage instruction to move beyond direct recall and routine exercise practice. However, if this assessment signal is interpreted narrowly, it may also lead to repetitive training in examination-style reasoning rather than broader competence development. Therefore, teachers need to transform the assessment signal into deeper learning opportunities rather than merely increasing test-preparation drills.

First, industrial process problems should be used to cultivate integrated information processing and variable reasoning. These tasks typically involve flowcharts, reaction conditions, separation procedures, yield improvement, environmental treatment, and resource utilization. Teachers can guide students to identify the function of each process step, explain why a condition is controlled, predict how changes in temperature, concentration, pH, or reagent dosage may influence the outcome, and construct a complete evidence chain from raw material to target product. This can help students develop B2 inferential prediction, C1 complex reasoning, and C2 systematic inquiry in a quasi-authentic chemical production context.

Second, reaction-principle topics should be taught through model-based and evidence-based reasoning rather than formula substitution alone. In chemical equilibrium, reaction rate, thermodynamics, and electrochemistry, students should be asked to connect symbolic equations, graphs, experimental data, and submicroscopic explanations. For example, teachers can require students to explain why a graph changes under a specific condition, justify the direction of equilibrium shift using evidence, compare alternative explanations for an electrochemical phenomenon, or infer how a variable change affects reaction yield. Such instruction directly supports A2 generalization and association, B1 analysis and explanation, B2 inferential prediction, and C1 complex reasoning.

Third, experimental inquiry tasks should be redesigned to include variable control, evidence evaluation, error analysis, and scheme improvement. Because B3 simple design and C2 systematic inquiry were less prominent than B2 and C1, classroom teaching should provide additional opportunities for students to design and evaluate experimental procedures. Teachers can ask students to compare two possible experimental schemes, identify uncontrolled variables, justify the purpose of a reagent or apparatus, evaluate the reliability of evidence, and revise a procedure after unexpected results. These activities can strengthen inquiry processes that may be difficult to fully assess in time-limited standardized tests.

Fourth, organic inference tasks should be used to develop multi-step reasoning and representation transformation. Organic chemistry problems often require students to infer structures from reaction conditions, functional-group transformations, molecular formulas, and experimental evidence. Teachers can help students build “structure–property–reaction–evidence” maps, compare possible reaction pathways, and justify why one pathway is more plausible than another. This strategy can make students’ reasoning visible and support the development of C1 complex reasoning and A2 generalization and association.

Finally, because C3 innovative thinking was marginal in the selected forms, classroom teaching should deliberately create spaces for open-ended and design-oriented chemistry learning. This does not mean replacing examination preparation with unrestricted creativity tasks. Rather, teachers can use scaffolded open-response prompts, cross-context transfer tasks, and small-scale project-based activities that remain connected to core chemistry content. For example, students may design an improved wastewater treatment process, compare alternative battery materials, propose safer experimental procedures, or evaluate different routes for preparing a target compound. Such tasks can compensate for the limited representation of C3 in standardized testing and help students experience chemistry as a discipline of inquiry, design, and problem solving (; ).

In sum, the competence profile revealed by this study implies that effective chemistry teaching should go beyond preparation for the most frequent item types and focus on cultivating deeper reasoning, inquiry, and design-oriented thinking.

5 Limitations and future directions

This study has several limitations that should be considered when interpreting the findings.

First, the sample was limited in scope and structure. Although the five selected examination forms were informative for comparing different test-development and reform contexts, they represented only five of the eighteen chemistry examination forms administered in 2023. In addition, locally developed provincial or municipal forms constituted three of the five selected forms, whereas nationally developed forms constituted two. Therefore, the findings should be interpreted as evidence from selected representative forms rather than as a complete or statistically representative description of all Gaokao chemistry examinations in China. In particular, conclusions regarding regional variation, national–local differences, and transitional features between old and new curriculum and examination systems should be treated with caution. Future research should include more years, more provinces, and a broader range of nationally and locally developed forms to examine whether the competence structures identified in this study are stable across time and regions.

Second, the analysis was based on examination texts rather than student response data. The coding results therefore captured the intended competence demands embedded in the item design, not necessarily the actual cognitive processes students used when solving the tasks. Students may solve the same item through different reasoning paths, shortcuts, memorized procedures, or partial understandings. This distinction is important because the cognitive validity of assessment tasks cannot be fully established from item texts alone (; Smith, 2017). Future studies could combine item analysis with student response data, written-solution analysis, error-pattern analysis, think-aloud protocols, retrospective interviews, and teacher or item-writer interviews to examine how students actually interpret, reason through, and solve Gaokao chemistry tasks.

Third, this study did not systematically classify the contexts of the items. Many Gaokao chemistry tasks are embedded in specific contexts such as industrial processes, experimental inquiry, environmental governance, energy conversion, material preparation, reaction principles, or organic synthesis. These contexts may shape the competence demands of the tasks and may influence the relative representation of indicators such as inferential prediction, complex reasoning, systematic inquiry, and innovative thinking. Future research should develop a more fine-grained context-coding scheme and examine how different task contexts are associated with different competence indicators.

Fourth, the study did not conduct longitudinal comparison across years. As a result, the findings cannot demonstrate a historical shift in Gaokao chemistry assessment. The results only reveal the competence profiles of the selected 2023 forms.

Fifth, although the use of directed content analysis provided a clear theoretical framework, it also introduced certain methodological constraints. The coding framework was based on Wang and Zhi's chemistry disciplinary competence model, which allowed systematic comparison across forms but may also have limited sensitivity to task features not fully captured by the predefined indicators. In addition, assigning each cognitive task unit to a dominant indicator may compress tasks that involve multiple intertwined cognitive processes. This limitation is common in framework-driven content analysis (). Future studies could combine expert coding with computational text analysis, cognitive diagnostic modeling, or evidence-centered design to improve the precision and validity of competence-structure analysis.

Finally, the present study focused on the intended assessment structure of examination forms and did not examine how teachers and students respond to these structures in classroom practice. Therefore, the washback implications discussed above should be interpreted as theoretically grounded implications rather than directly observed classroom effects. Future research could investigate how Gaokao chemistry competence demands influence teaching design, review strategies, classroom discourse, student learning approaches, and school-level curriculum implementation. Interviews with item writers, teachers, and students would also help clarify how assessment intentions are translated into classroom practice.

6 Conclusions

Using Wang and Zhi's chemistry disciplinary competence framework, this study analyzed the competence structures embedded in five selected 2023 Gaokao chemistry examination forms.

Three main conclusions can be drawn. First, the selected forms showed a competence profile dominated by applied practice, followed by transfer and innovation and then learning understanding. This indicates that the examined forms placed greater emphasis on contextualized knowledge use, explanation, inference, and problem solving than on direct recognition or recall alone.

Second, at the second-level indicator level, B2 inferential prediction and C1 complex reasoning were the most prominent competence demands. Their strong representation suggests that the selected forms valued students’ ability to draw conclusions from given information, coordinate multiple variables, and construct multi-step reasoning chains in chemistry contexts such as reaction principles, industrial processes, experimental analysis, and information-rich problem situations.

Third, the five forms displayed both convergence and differentiation. Convergence was reflected in the shared emphasis on inferential prediction and complex reasoning, whereas differentiation appeared in the allocation of learning-understanding indicators and in the limited representation of innovative thinking. In particular, C3 innovative thinking remained marginal across the selected forms, highlighting a limitation of large-scale paper-and-pencil assessment in representing open-ended innovation-oriented competence.

Overall, the findings suggest that the five selected 2023 Gaokao chemistry examination forms embodied a competence-oriented assessment profile centered on applied practice, inferential prediction, and complex reasoning. However, these conclusions should be interpreted within the scope of the selected forms. Because the sample covered only five of the eighteen chemistry forms administered in 2023 and included a larger proportion of locally developed new-Gaokao forms, the results should not be generalized to all Gaokao chemistry examinations in China without further evidence from broader multi-year and multi-region datasets.

Statements

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding authors.

Author contributions

DR: Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Resources, Writing – original draft, Writing – review & editing. TH: Formal analysis, Methodology, Validation, Writing – review & editing. XD: Formal analysis, Methodology, Writing – review & editing. ZQ: Conceptualization, Formal analysis, Methodology, Writing – review & editing, Funding acquisition, Supervision. HY: Supervision, Validation, Writing – review & editing.

Funding

The author(s) declared that financial support was received for this work and/or its publication. Northwest Normal University's 2024 University-Level Graduate Student Research Funding Project.

Conflict of interest

The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Summary

Keywords

chemistry disciplinary competence, competence-oriented assessment, directed content analysis, Gaokao chemistry examination forms, washback effect

Citation

Ren D, Hu T, Dong X, Quan Z and Yao H (2026) Assessing chemistry disciplinary competence in high-stakes examination forms: evidence from the 2023 Chinese Gaokao. Front. Educ. 11:1884670. doi: 10.3389/feduc.2026.1884670

Received

18 May 2026

Revised

05 July 2026

Accepted

24 July 2026

Published

05 August 2026

Volume

11 - 2026

Edited by

José Luis Gómez Ramos, University of Extremadura, Spain

Reviewed by

Ji Zeng, Michigan Department of Education, United States

Hongxia Liu, Anshan Normal University, China

YanHua Fan, Henan University, China

Updates

Copyright

*Correspondence: Dong Ren Hu Tao Zhengjun Quan Hongkai Yao

† These authors have contributed equally to this work and share first authorship

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics