Assessing engineering thinking in the context of biology among senior high school students in mainland China: development and validation of a two-tier assessment scale and the relationship between self-efficacy and practical performance

This study designed and validated the Engineering Thinking Assessment Scale (ETAS) to assess the engineering thinking of senior high school students in mainland China and to explore the correlation between their self-efficacy in different engineering thinking elements and their practical performance. The ETAS, comprising 28 items, was designed through four phases: defining dimensions, designing, pilot testing, and final assessment. Each item includes two tiers: the emotional tendency tier, measuring self-efficacy, and the practical performance tier, assessing actual application in engineering projects. A study with 510 students (485 valid responses) confirmed the scale's strong reliability and validity. While most students demonstrated above-average engineering thinking, higher self-efficacy did not consistently translate to better performance. Notably, 10 engineering thinking elements showed weak correlations (0.2 < |r| < 0.3) between self-efficacy and performance, indicating confidence without effective application. This study offers a detailed evaluation of engineering thinking abilities, highlighting gaps between perceived and actual competency in specific engineering elements.

1 Introduction

To address global challenges such as climate change, resource scarcity, environmental pollution, and the energy crisis, it is imperative to cultivate citizens with a sense of global responsibility. In response, countries like the United States, China, and the United Kingdom have introduced various policy documents related to engineering education (Lead States, 2013; EngineeringUK., 2024; Ministry of Education of the People's Republic of China, 2018). Engineering education is implemented in various forms across different educational stages in these countries, with popular approaches including STEAM education, project-based learning, and problem-oriented learning (Palmer and Hall, 2011; Servant-Miklos and Kolmos, 2022; Leung et al., 2024). The primary goal of these engineering education initiatives is to foster and develop students' engineering thinking, enabling them to think independently and solve problems in their future careers. China's modernization requires a highly skilled engineering workforce, which underscores the necessity of cultivating engineering thinking from the early stages of education. However, the extent of engineering thinking among senior high school students in mainland China remains uncertain; thus, it requires systematic evaluation and thorough investigation.

This study aims to develop a scale for evaluating senior high school students' engineering thinking. This scale provides a more precise evaluation of students' engineering thinking, highlighting the relationship between their self-efficacy and practical performance. These insights help teachers refine instructional strategies to better cultivate students' engineering thinking.

2 Literature review

2.1 The connotation and definition of engineering thinking

Engineering thinking is rooted in multidisciplinary knowledge and engineering technology, forming the basis for practical applications and innovative problem-solving in engineering. Scholars suggest that engineering thinking represents a holistic and strategic approach, integrating knowledge from multiple disciplines to tackle complex engineering challenges (Yu et al., 2020). Multidisciplinary knowledge entails understanding fundamental principles across different fields, whereas engineering technology focuses on applying these principles in real-world contexts (Cross, 2001; Heywood, 2005). These elements are essential for engineers to conceptualize and implement effective solutions (Atman et al., 2007).

Additionally, engineering thinking requires design and implementation abilities, which are crucial for solving complex problems and innovating across disciplines (Aranda et al., 2020). Design abilities involve specific ways of thinking and doing, such as utilizing solution-focused strategies, abductive thinking, and non-verbal, graphic/spatial communication (Cross, 1995). This ability is unique, differentiating it from artistic and scientific forms of knowledge (Cross, 1995; Sutton and Williams, 2010). Implementation ability is also a key aspect of engineering thinking. However, research on this ability is limited. For example, Cadet evaluated students' implementation ability using the cost standards of engineering projects in his study (Cadet et al., 2004).

Engineering thinking also involves optimization awareness and value consciousness. Engineers must be adaptable and forward-thinking to navigate rapidly evolving technological landscapes (Varadarajan, 2020), continuously optimizing products to meet future consumer demands. Higuera Martinez and Fernandez-Samaca (2020) emphasize creativity as essential for the continuous improvement of products in engineering. New models have been developed to help engineers identify valid design problems and promote creativity in the engineering design process, effectively addressing the challenges of product updates (Obieke et al., 2021). Engineering thinking also involves recognizing the broader impacts of engineering work on societal, economic, ethical, and environmental aspects. Engineers must consider the value of these factors in their designs and decisions, making responsible choices that benefit both present and future generations (Fordyce, 1986; Papalambros and Wilde, 2017).

According to the National Academy of Engineering (2016), engineering thinking is goal-oriented and addresses problems and decisions within constraints by using available material and human resources. While this description outlines the basic framework, it lacks the detailed key elements necessary to capture its full depth and breadth. Therefore, this study posits that engineering thinking is a systematic process in which engineers use multidisciplinary knowledge and engineering technology to iteratively design and implement solutions for greater value.

2.2 The interplay between engineering thinking, self-efficacy, and engineering practice

Engineering thinking is a multifaceted cognitive process involving problem identification, analysis, systems thinking, design, and implementation skills. It also requires students to cultivate psychological resilience and maintain sustained motivation while navigating uncertainty and challenges (Kaur et al., 2019; Hu et al., 2021). As a result, engineering thinking is reflected not only in belief-driven intrinsic motivation for problem-solving but also in observable performance in engineering practice.

According to Bandura's self-efficacy theory, self-efficacy is a key psychological factor shaping an individual's behavior in specific contexts. High self-efficacy boosts students' confidence and motivation in tackling complex tasks and fosters perseverance and adaptability in engineering practice. This intrinsic motivation drives students to explore solutions and refine designs despite uncertainty and failure—key characteristics of engineering thinking. Therefore, self-efficacy is both a prerequisite for engaging in engineering practice and a catalyst for the ongoing development of engineering thinking.

Meanwhile, engineering practice performance, as an external manifestation of engineering thinking, is a measurable indicator of students' cognitive proficiency (Mahajan and Bansal, 2021). In practice, students translate abstract engineering thinking into concrete actions by analyzing key problem components, devising feasible design solutions, and refining existing approaches. Thus, engineering practice performance is a critical metric for evaluating students' engineering thinking skills.

Assessing engineering thinking through self-efficacy and engineering practice performance offers a comprehensive framework for understanding both students' cognitive processes and behavioral execution. Self-efficacy indicates students' confidence and motivational intensity in tasks, whereas engineering practice performance reveals the degree to which their thinking translates into action. Integrating these two measures allows for a dynamic assessment of students' development of engineering thinking across different stages while providing effective feedback for instructional improvement through longitudinal evaluations. Moreover, this assessment approach aligns with psychological theories of cognitive development, which propose that thinking patterns are reinforced and refined through continuous task engagement, a process that depends on high self-efficacy for sustained progress (Bandura, 1997; Standl and Schlomske-Bodenstein, 2021). Therefore, assessing both self-efficacy and practical performance not only provides a rigorous measure of students' engineering thinking but also promotes continuous cognitive development by reinforcing self-efficacy. Ultimately, this approach provides a valuable framework for informing instructional design in engineering education and fostering students' competency in real-world engineering contexts.

2.3 Studies on the assessment of engineering thinking

Based on the definition, engineering thinking is a systematic cognitive process encompassing multidisciplinary knowledge, technology, abilities, and emotions. Current research primarily focuses on assessing the dimensions of abilities, but it lacks a comprehensive evaluation of engineering thinking.

Some researchers have used the 14 competency standards from the Institution of Civil Engineers (ICE)'s 2010 Professional Development Framework to assess engineering thinking (Institution of Civil Engineers., 2010). Using these standards, they designed two 1-h individual problem-solving tasks that required students to apply engineering knowledge and skills to real-world problems. This approach compared the engineering thinking performance of students from humanities and engineering disciplines across various engineering projects. However, this evaluation method has several significant limitations. Although it emphasizes authentic contexts, its reliance on short-term tasks to assess long-term engineering thinking development may not accurately reflect students' actual competencies. Additionally, findings reveal that humanities students significantly improved their engineering thinking scores by the end of the semester, whereas engineering students showed a slight decline (Bell et al., 2019). This trend raises a critical question: does contemporary engineering education prioritize knowledge transmission at the expense of continuous engineering thinking development? The significant progress of humanities students may be attributed more to the novelty of adopting a new cognitive approach than to an actual improvement in engineering proficiency. Furthermore, competency assessment is only one aspect of engineering thinking. A thorough evaluation should integrate multiple interacting factors for a more comprehensive assessment. This study highlights critical gaps in assessment tools and educational approaches for fostering and measuring engineering thinking, underscoring the need for further research and refinement.

Becker and Mentzer (2015) identified design ability as a key element of engineering thinking and examined high school students‘ performance in design tasks using the think-aloud method, comparing their approaches with those of professional engineers. The findings revealed a notable trend: high school students struggled to consider problems from the client's perspective and often relied on a single-solution approach rather than conducting comparative analysis and optimization. However, this research approach has several significant limitations. First, the think-aloud protocol and problem-solving experiments involve small sample sizes, limiting the generalizability of the findings to broader populations. Additionally, the study exclusively examined design ability, neglecting the complexity and multidimensional aspects of engineering thinking. Consequently, relying solely on this research to assess high school students' overall engineering thinking abilities may lead to overly generalized conclusions.

Coleman et al. (2020) identified various dimensions of design ability and developed a quantitative scale for assessment. Their findings revealed significant disparities in design ability between first-year and senior engineering students, suggesting that education plays a crucial role in enhancing design competence. However, despite employing a quantitative scale, this study has several significant limitations. Although design ability is a key aspect of engineering thinking, it does not fully encompass its multidimensional nature. Consequently, while the study improves the understanding of design skill development, it does not offer a holistic evaluation of engineering thinking. As a result, it does not effectively track students' development in engineering thinking across different educational stages.

3 Purpose of the research

This study aims to achieve three objectives through a comprehensive literature review: (1) designing a psychometrically sound assessment scale for engineering thinking, (2) systematically assessing the current level of engineering thinking among senior high school students in mainland China, and (3) examining the relationship between students' self-efficacy in various aspects of engineering thinking and their practical performance.

4 Method

4.1 Development procedure of the engineering thinking assessment scale

This section outlines the development procedure of the Engineering Thinking Assessment Scale (ETAS), comprising four stages: dimension division, scale development, revision and pilot testing, and formal testing. The aim is to use scientific methods to create a high-quality assessment scale.

4.1.1 Stage 1—dimensions of engineering thinking

Engineering thinking is a key cognitive process for engineers in solving complex problems and designing effective solutions. According to the definition, engineering thinking is a systematic thought process that engineers use, based on diverse knowledge and engineering technology, to iteratively design and implement solutions for better value. This definition suggests that engineering thinking consists of four primary dimensions—multidisciplinary knowledge, engineering technology, abilities, and emotions—categorized as first-tier dimensions. Multidisciplinary knowledge integrates both scientific and non-scientific domains; engineering technology comprises fundamental and advanced technical skills; abilities pertain to design and implementation; and emotions include optimization awareness and value consciousness. These eight elements constitute the second-tier dimensions.

According to the disciplines outlined in the STEAM educational framework (U. S. Congress., 2014), the third-tier dimensions of scientific knowledge encompass physics, chemistry, biology, mathematics, and engineering. Non-scientific knowledge includes humanities and aesthetics. Based on High School Biology Elective Course 3: Biotechnology and Engineering (Zhu and Zhao, 2017), the third-tier dimensions of simple technical skills comprise aerobic and anaerobic fermentation techniques, whereas advanced technical skills involve microbial culture technique, plant tissue culture technique, DNA and protein extraction technique, and plant active ingredient extraction technique. Drawing from the basic model of science and engineering projects in the Next Generation Science Standards (Lead States, 2013), the third-level dimensions derived from design abilities include problem and requirement identification, model development and use, investigation planning and implementation, data analysis and interpretation, and decision-making abilities. According to the principles of project management (three controls, two managements, and one coordination; Tong et al., 2012) and the core competencies outlined by the Institution of Civil Engineers. (2010), the third-level dimensions derived from implementation abilities include schedule control, cost control, quality control, safety management, and organizational coordination abilities. Following product value orientations (Karababa and Kjeldgaard, 2013; Prados-Peña et al., 2023), the third-level dimensions derived from value consciousness include economic value, aesthetic value, social value, ecological value, and psychological value. As reported by the escalating demands of consumers and the economic orientation principles of producers (Lai et al., 2007; Cui et al., 2017), the third-level dimensions derived from optimization awareness include appearance optimization, functional optimization, and cost optimization (Table 1).

Table 1

Table 1. The dimension of engineering thinking.

The Delphi Method was used to validate the preliminary three-tier framework of engineering thinking through multiple rounds of anonymous surveys, incorporating both qualitative and quantitative analyses to enhance scientific rigor and validity. A panel of ten experienced experts was selected to ensure diverse and authoritative feedback. The panel consisted of two engineers specializing in biology, three secondary school STEM teachers with over 5 years of experience, two university professors in education, and three doctoral students in science education. In the first round, experts received a detailed framework description and provided feedback on its completeness, tier rationale, and potential modifications via an open-ended questionnaire. The first-round analysis indicated expert consensus that multidisciplinary knowledge and engineering technology should be integrated into a single knowledge dimension, as technology constitutes procedural knowledge. Experts also recommended combining aerobic fermentation technique and anaerobic fermentation technique into the broader category of fermentation technique. Based on this feedback, the framework was revised, and a second round of closed-ended surveys was conducted. A Likert scale (1–5) assessed the framework's scientific validity, applicability, and feasibility, while open-ended questions remained for further refinement. Kendall's W coefficient measured expert consensus. The results showed a high level of agreement (W = 0.76 > 0.70), indicating strong expert consensus. Given the strong consensus, a third consultation round was deemed unnecessary, and the revised framework was finalized. Table 1 displays the finalized three-tier framework of engineering thinking.

4.1.2 Stage 2—development of the scale

Leveraging the interrelationship among self-efficacy, practical performance, and engineering thinking and incorporating engineering projects from the High School Biology Selective Compulsory Course 3: Biotechnology and Engineering (Zhu and Zhao, 2017), a set of two-tiered items was developed for each of the 28 engineering thinking elements at the third-level dimension. The first tier is the emotional tendency tier, which evaluates students‘ self-efficacy in applying elements of engineering thinking. The second tier, the practical performance tier, evaluates students' practical performance regarding the engineering thinking elements in the engineering projects. Consequently, a comprehensive ETAS consisting of 28 items was created. The correspondence between these 28 items and the third-tier dimensions is presented in Table 1.

4.1.3 Stage 3—revision and pilot testing

Before the pilot test, the expert panel evaluated the content validity of the initial scale version. Their assessment considered readability, the meaningfulness of the requirements, and adherence to the biology textbooks and curriculum standards. Experts observed that the emotional tendency tier lacked subjective expressions like believe, causing statements to resemble objective factual assertions rather than reflections of belief levels. In response to expert feedback, the emotional tendency tier was revised to include believe, reinforcing its focus on personal subjective attitudes. Table 2 provides an example of a modified item about value consciousness. Subsequently, 50 high school students, who were not part of the formal test, were randomly selected from the participating schools for the pilot test. The results indicated that the reliability of the assessment scale is acceptable (Cronbach's α = 0.77 > 0.70).

Table 2

Table 2. One item from ETAS.

4.1.4 Stage 4—formal testing

The formal testing was conducted during evening study sessions with each class as a unit and using standardized instructions. A total of 510 Grade 10 and Grade 11 high school students participated in the 20-min assessment. To ensure data reliability for statistical analysis, invalid questionnaires were excluded based on the following criteria:

• Criterion 1: If there was an obvious pattern in the responses, such as the same answer being given for six or more consecutive items, the questionnaire was excluded.

• Criterion 2: If more than ten consecutive items were left unanswered, the questionnaire was discarded.

A total of 510 questionnaires were distributed, and 503 were returned, yielding a response rate of 98.6%. Of these, 485 were deemed valid, resulting in an effective rate of 95.1%.

4.2 Subjects

The two high schools selected for this study are situated in central Mainland China, where overall educational quality is moderate at the national level. This region contrasts with the developed eastern coastal areas, which possess abundant educational resources and superior school conditions, and the underdeveloped western regions, where resources remain relatively scarce. Therefore, these schools serve as representative cases of secondary education in regions with average educational standards. Furthermore, both schools rank slightly above the regional average in educational quality assessments. They are neither top-tier key schools nor low-performing institutions, making them representative of mainstream secondary education in the region. This enables a comprehensive analysis of how schools with moderate to high educational standards foster engineering thinking. Notably, both schools adhere to the national standardized high school curriculum and incorporate engineering-related courses and practical activities, aligning with broader trends in engineering education within China's general high school system. Therefore, students' engineering thinking development can be considered representative of the broader student population in similar schools. Additionally, the student populations in both schools exhibit a relatively balanced academic performance, including high-achieving and average-performing students. This diversity mitigates potential sample bias associated with selecting schools with either exceptionally high or low academic standards, thereby enhancing the generalizability of the study's findings. Nan Zheng High School offers 23 senior-year classes, whereas Han Tai High School has 17. To enhance representativeness in the sample selection, four classes were randomly chosen from each school, resulting in a total of eight classes and 510 participating students. These students had completed all biotechnology and engineering-related coursework in their curriculum and had participated in corresponding biological engineering projects.

4.3 Data processing

A four-point Likert scale was used for each tier of the two-tiered questions. The emotional tendency tier was rated from A to D, and the practical performance tier was rated from a to d, with options ranked from high to low. The A/a option was assigned a value of 3 points, B/b 2 points, C/c 1 point, and D/d 0 points. Each item could score a maximum of 6 points and a minimum of 0 points. Students choosing A/a or B/b were considered to have high emotional tendency and practical performance, while those choosing C/c or D/d were considered to have low emotional tendency and practical performance. The data were analyzed using SPSS 24.0 to assess reliability, validity, and descriptive statistics. For ease of analysis, the items in the scale were coded, for example, Item 1 was coded as I1, Item 2 as I2, and so on.

5 Results and discussion

5.1 Reliability and validity testing of the ETAS

To ensure that the ETAS is both psychometrically sound and theoretically robust, a series of reliability and validity tests were conducted. These analyses aimed to examine the internal consistency of the scale, evaluate its structural validity, and confirm whether the proposed six-dimension framework is supported by empirical data.

5.1.1 Reliability testing of the ETAS

SPSS 24.0 was used to perform a Cronbach's alpha reliability test on the total scores of 485 participants. The results indicate that the scale's reliability is acceptable (α = 0.849 > 0.800), meaning that at least 84.9% of the variance in students' total scores is due to true score differences (Table 3).

Table 3

Table 3. Reliability statistics.

5.1.2 Structural validity testing of the ETAS

The expert panel confirmed that the engineering thinking scale possesses good content validity. Based on this, SPSS 24.0 was employed to analyze the Pearson correlation between the six secondary dimensions and engineering thinking, as well as the Pearson correlations among the six elements (Table 4). The results show that engineering thinking is significantly and moderately to highly correlated with the six elements (0.60 < r < 0.80, p < 0.01), indicating alignment with the assessment of engineering thinking. The pairwise correlations between the six elements are significantly low to moderate (0.20 < r < 0.60, p < 0.01), suggesting that while the elements are consistent in their assessment direction, they maintain a certain degree of independence. These findings demonstrate that the ETAS has good structural validity.

Table 4

Table 4. Pearson correlation coefficients.

5.1.3 Confirmatory factor analysis

A confirmatory factor analysis was conducted on 28 items from 485 participants using SPSS 24.0. The suitability of the data for factor analysis was first examined. The results indicated a KMO value of 0.83, and Bartlett's test of sphericity yielded χ² = 2,560.4, df = 378, p < 0.001, suggesting that the correlation matrix significantly differed from the identity matrix and that the data were appropriate for factor analysis. Based on these results, factors were extracted using principal component analysis, and a Varimax orthogonal rotation was applied, constraining the solution to a six-factor structure. The rotated factor loading matrix showed that all 28 items had high loadings (above 0.65) on their corresponding factors and low loadings on others, indicating good discriminant capacity (Table 5). This result aligned with the hypothesized six second-level dimensions, namely multidisciplinary knowledge, engineering technology, design abilities, implementation abilities, value consciousness, and optimization awareness.

Table 5

Table 5. Rotated factor loadings of the 28 items across six factors.

5.2 The current situation of students' engineering thinking

The overall scores of students' engineering thinking were obtained through the assessment scale. Based on the range, mean, central tendency, and variance, the score intervals were set at 25 points each and divided into six categories. Figure 1 illustrates the distribution of students' engineering thinking scores across different intervals. The majority of students scored between 76 and 100 (n = 223, 45.98%) and 101–125 (n = 211, 43.51%), indicating a concentration of scores within the upper-middle range of the assessment scale. The intervals 51–75 and 126–150 represented a smaller proportion of the sample, with 6.39% (n = 31) and 3.92% (n = 19) of the students falling into these categories respectively. Notably, the extremes of the scoring range, ≤ 50 and 151–168, were the least represented within the population, with a single individual (0.21%) scoring at or below 50, and no students scoring within the 151–168 interval. These results indicate that the vast majority of students' engineering thinking is at an upper-middle level. This conclusion aligns with findings from previous research. Research on high school students' engineering thinking indicates that they generally demonstrate an upper-intermediate level of proficiency (Franske, 2009). Several factors may account for this consistency. First, the growing influence of information technology has contributed to greater standardization in global education, ensuring that students receive relatively uniform curricular training (Baghdoyan, 2016). Second, advancements in engineering thinking may be attributed to curriculum reforms in science and technology education. Many schools have progressively implemented engineering-focused STEAM courses or project-based learning approaches to strengthen students' problem-solving abilities (Zhang, 2024). The key contribution of this study is its assessment of students' engineering thinking across 28 distinct elements, offering a more comprehensive and reliable measurement than previous research.

Figure 1

Figure 1. Bar and line chart of total engineering thinking scores.

The data reveals that the average scores of participants in the second-level dimensions ranged from 11.48 for optimization awareness to 19.40 for implementation abilities. Variability in these scores, as indicated by the standard deviations, was moderately low to moderate, suggesting some degree of homogeneity across the sample. The skewness coefficients were all within the range of mild skewness (0 ≤ skewness coefficient ≤ 0.5), indicating that the skewness of the data would not significantly affect data analysis and suggesting a generally symmetrical distribution of scores across the domains (Table 6). The results show that students' performance in each of the second-level dimension was generally at an upper-middle level, consistent with the overall level of engineering thinking. Additionally, most students' performance on these indicators was relatively concentrated, with no extremely high or low scores. Calderón Saldierna (2015) found that although students exhibited strong proficiency in engineering technology and design skills, notable disparities persisted in value consciousness and optimization awareness. This discrepancy may result from variations in schools' curricular priorities and training approaches. For example, some institutions prioritize technical and design skill development but offer limited training in value consciousness and optimization awareness (Haney, 2024). Furthermore, some studies have identified weaker student performance in specific aspects of engineering thinking, such as implementation ability and optimization awareness, whereas this study did not yield similar results (ALZenki, 2023). This inconsistency may be attributed to variations in sample populations, assessment methodologies, or educational contexts. Prior research primarily examined students from schools with limited access to engineering practice, whereas participants in this study had greater exposure to hands-on engineering training, which likely contributed to their stronger performance in these competency areas (Hixson, 2023).

Table 6

Table 6. Descriptive statistics for second-level dimensions of engineering thinking.

5.3 Correlation analysis between self-efficacy and practical performance

A Pearson correlation analysis was performed using SPSS 24.0 to investigate the relationship between the emotional tendency tier and the practical performance tier across 28 elements of engineering thinking (Figure 2). The results suggest that high self-efficacy does not always translate into strong performance in engineering practice. Research indicates that although self-efficacy is generally linked to performance improvement, it does not always serve as a reliable predictor of actual ability, especially when moderating factors like supervision quality and contextual experience exert a significant influence (Tugendrajch, 2022). Among the 28 elements, 18 showed a moderate to high correlation between self-efficacy and practical performance (0.5 < |r| < 1.0). However, ten elements showed weak correlations (0.1 < |r| < 0.3). These elements included scientific knowledge, fermentation technique, microbial culture technique, plant tissue culture technique, DNA and protein extraction technique, plant active ingredient extraction technique, problem and requirement identification ability, model development and usage ability, data analysis and interpretation ability, and decision-making ability. These findings suggest that while students may be confident in utilizing these engineering elements, they often struggle to effectively apply them in practical contexts. The discrepancy between students' confidence and their actual ability to apply knowledge continues to be a challenge in engineering education. Research indicates that although students may demonstrate strong confidence in theoretical learning and conceptual understanding, they frequently encounter difficulties in applying this knowledge to real-world engineering scenarios. Bays-Muchmore and Chronopoulou (2018) found that while first-year engineering students generally perceived themselves to possess strong engineering competence, variations in their ability to apply these skills emerged due to factors such as gender and field of study, highlighting that confidence and practical performance do not always show a positive correlation. Similarly, Jiang (2022) investigated the effects of project-based learning (PBL) on STEM students and observed that while PBL enhanced students' self-efficacy, its effectiveness varied depending on individual backgrounds. While previous studies have broadly explored the correlation between self-efficacy and practical performance, the key contribution of this study is the identification of specific engineering thinking elements that show weak correlations between self-efficacy and practical application. To bridge this gap, engineering education must emphasize hands-on training and incorporate individualized learning approaches to support students' transition from theoretical understanding to practical application.

Figure 2

Figure 2. Pearson correlation between self-efficacy and practical performance.

The multidisciplinary knowledge dimension encompasses one element of engineering thinking: scientific knowledge. In the emotional tendency tier of I5, students generally perceived themselves as highly capable of applying scientific knowledge in engineering projects. However, only 46.39% successfully demonstrated the application of such knowledge in practical tasks, such as designing an ecological fish tank or growing bean sprouts. This discrepancy suggests that while students exhibit high confidence in their ability to apply scientific knowledge, their actual performance falls significantly short of their self-perception. Studies suggest that although scientific self-efficacy is generally positively associated with academic achievement, a high level of self-efficacy does not necessarily lead to superior performance in the practical application of scientific knowledge (Andrew, 1998). This discrepancy may result from a lack of deep conceptual understanding in science education or insufficient problem-solving experience in real-world contexts (Webb-Williams, 2018). Furthermore, there is a need for engineering education to prioritize the development of students' ability to effectively apply scientific knowledge rather than merely fostering their confidence (Daun et al., 2021; Sulandari et al., 2021).

The engineering technology dimension includes five elements of engineering thinking: fermentation technique, microbial culture technique, plant tissue culture technique, DNA and protein extraction technique, and the plant active ingredient extraction technique. While students generally exhibited high confidence in applying these techniques, their actual performance in practical applications was substantially lower. For example, in I8, when performing chrysanthemum tissue culture, only 26.80% of students correctly followed the sequence of shoot induction before root development. Some studies attribute this discrepancy to the inherent complexity of plant tissue culture, which demands precise aseptic techniques, carefully balanced nutrient compositions, and a thorough understanding of plant growth regulators. These factors make it particularly challenging for students to achieve mastery (Siritunga et al., 2012). Furthermore, research highlights the importance of hands-on experience in developing technical proficiency. However, high material costs and the need for specialized equipment often restrict students' opportunities for practical training (Patra et al., 2020). Similarly, in the emotional tendency tier, most students believed they had a strong grasp of the DNA and protein extraction technique. Yet, in practice, only 20.41% successfully followed the correct procedure of gradually adding distilled water to a beaker containing a high-concentration sodium chloride solution (2 mol/L) while gently stirring with a glass rod to obtain a viscous DNA-containing substance. A high school biology teacher noted that training in these techniques is rarely incorporated into routine instruction, as the current examination system prioritizes paper-based assessments over experimental practice. As a result, students often excel in “paper-and-pencil techniques” while lacking the practical experience needed to apply these methods effectively in real-world scenarios.

The design ability dimension includes four elements of engineering thinking: problem and requirement identification ability, model development and usage ability, data analysis and interpretation ability, and decision-making ability. While at least 80% of students in the emotional tendency tier expressed confidence in applying these skills, their practical performance fell significantly short. For example, in I11, students generally reported confidence in their problem and needs identification abilities. However, when presented with a scenario where overfeeding led to water pollution and fish mortality in a pond, only about half of the students correctly identified the cause of fish deaths. This finding aligns with previous research on high school students' problem-solving proficiency, which suggests that their ability to recognize problems and needs remains at a moderate level (Suroso et al., 2021). The implementation of STEM education has been shown to significantly enhance students' problem-solving skills, particularly their ability to identify and define real-world problems (Parno et al., 2020). Similarly, in I14, most students believed they had strong data analysis and interpretation ability. However, only 45.16% accurately analyzed and interpreted a population density-time relationship graph to understand interspecies interactions and survival conditions. Research suggests that in data mining and analysis, high self-efficacy is typically linked to increased learning motivation and engagement but does not necessarily lead to enhanced practical skills (Liu et al., 2023). Multiple factors may explain this phenomenon. First, students may gain a strong conceptual understanding in theoretical courses, fostering confidence in their abilities. However, when encountering complex datasets and real-world challenges, insufficient hands-on experience may result in underperformance (Menon and Sadler, 2016). Moreover, data analysis and interpretation ability necessitate continuous practice and structured guidance, as perceived competence alone may not bridge the gap in practical application (Webb-Williams, 2018).

6 Conclusion and recommendations

This study systematically evaluated the reliability and validity of the Engineering Thinking Assessment Scale (ETAS), with Cronbach's alpha and Pearson correlation coefficients confirming the instrument's internal consistency and structural validity. The findings indicate that most students exhibited an upper-middle level of engineering thinking, with their performance in secondary dimensions generally aligning with their overall proficiency in engineering thinking. However, the study also identified ten engineering thinking elements where students demonstrated high self-efficacy but struggled with practical application. These elements include scientific knowledge, fermentation technique, microbial culture technique, plant tissue culture technique, DNA and protein extraction technique, plant active ingredient extraction technique, problem and requirement identification ability, model development and usage ability, data analysis and interpretation ability, and decision-making ability. This discrepancy underscores the necessity of enhancing hands-on learning experiences within engineering education.

To bridge this gap, instructional strategies should prioritize practical engagement through laboratory experiments, project-based learning, and interdisciplinary education, fostering seamless integration of theoretical knowledge and practical skills (Quiles-Carrillo et al., 2019; Bolick et al., 2024). Additionally, incorporating problem-based instructional designs can equip students with the ability to identify and address real-world challenges, ultimately strengthening their engineering practice capabilities and enhancing the transferability of scientific knowledge (Chen et al., 2020). These strategies will not only advance students' engineering thinking proficiency but also provide valuable insights for improving engineering education practices.

7 Limitations

This study offers both theoretical and practical implications while also acknowledging certain limitations. Theoretically, the validated six-dimension structure refines the framework of engineering thinking by providing empirical evidence for its multidimensional nature. Practically, the developed ETAS serves as a diagnostic tool for educators to identify students' strengths and weaknesses in engineering thinking, thereby supporting targeted instructional strategies, and it may also guide policymakers and curriculum designers in systematically integrating engineering thinking into secondary education. However, the study also has limitations, as the sample was drawn from a limited number of regions in mainland China, which may restrict the generalizability of the results to other cultural or educational contexts. Furthermore, despite the adequate sample size, potential imbalances in sociodemographic characteristics such as gender, grade level, and school type may have influenced the outcomes. Future research should therefore adopt more diverse and representative sampling strategies to enhance external validity.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found at: The data that support the findings of this study are openly available in Science Data Bank, 2025 [2025-02-27]. https://doi.org/10.57760/sciencedb.20524.

Ethics statement

Informed consent was obtained from all students who took part in this study. The study was approved by the Nan Zheng High School Ethics Committee and Han Tai High School Ethics Committee. Approval Number: EDU-010506 and RA06020726. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants' legal guardians/next of kin.

Author contributions

HM: Writing – original draft, Writing – review & editing. WL: Writing – original draft, Writing – review & editing. BL: Writing – review & editing. YL: Writing – review & editing. GL: Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This work was funded by the Fundamental Research Funds for the Central Universities, “A Four-Tier Theory of Secondary School Students' Pre-Scientific Concepts in Biology and Its Application in Teacher Education” under Grant Number GK202303004.

Acknowledgments

The authors would like to thank all the students who voluntarily participated in this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Gen AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

ALZenki, F. S. (2023). Motivating Interior Design Students Toward Professional Creativity: The Role of Feedback. Ames, IA, United States: Iowa State University.

Google Scholar

Andrew, S. (1998). Self-efficacy as a predictor of academic performance in science. J. Adv. Nurs. 27, 596–603. doi: 10.1046/j.1365-2648.1998.00550.x

PubMed Abstract | Crossref Full Text | Google Scholar

Aranda, M. L., Lie, R., and Selcen Guzey, S. (2020). Productive thinking in middle school science students' design conversations in a design-based engineering challenge. Int. J. Technol. Design Educ. 30, 67–81. doi: 10.1007/s10798-019-09498-5

Crossref Full Text | Google Scholar

Atman, C. J., Adams, R. S., Cardella, M. E., Turns, J., Mosborg, S., Saleem, J., et al. (2007). Engineering design processes: a comparison of students and expert practitioners. J. Eng. Educ. 96, 359–379. doi: 10.1002/j.2168-9830.2007.tb00945.x

Crossref Full Text | Google Scholar

Baghdoyan, L. (2016). Interrelationships Between Creative Thinking, Academic Achievement, Anxiety and Peer Relations Among Lebanese Armenian Secondary School Students. Emphasis: Clinical Psychology at Haigazian University.

Google Scholar

Bandura, A. (1997). Self-Efficacy: The Exercise of Control. New York, NY, US: W. H. Freeman/Times Books/Henry Holt & Co.

Google Scholar

Bays-Muchmore, M. F., and Chronopoulou, A. (2018). “First-year engineering student's perceptions of engineering,” in 2018 ASEE Annual Conference & Exposition (Washington, DC: American Society for Engineering Education).

Google Scholar

Becker, K., and Mentzer, N. (2015). “Engineering design thinking: high school students' performance and knowledge,” in 2015 International Conference on Interactive Collaborative Learning (ICL) (Firenze), 5–12.

Google Scholar

Bell, S., Chilvers, A., Jones, L., and Badstuber, N. (2019). Evaluating engineering thinking in undergraduate engineering and liberal arts students. Eur. J. Eng. Educ. 44, 429–444. doi: 10.1080/03043797.2018.1552663

Crossref Full Text | Google Scholar

Bolick, M. A., Thomassen, M., Apland, J., Spencer, O., Nicole, F., Tran, S. K. N., et al. (2024). Project-based learning in interdisciplinary spaces: a case study in Norway and the United States. Educ. Sci. 14:866. doi: 10.3390/educsci14080866

Crossref Full Text | Google Scholar

Cadet, C., Béteau, J. F., and Carlos Hernandez, S. (2004). Multicriteria control strategy for cost/quality compromise in wastewater treatment plants. Control Eng. Pract. 12, 335–347. doi: 10.1016/S0967-0661(03)00134-5

Crossref Full Text | Google Scholar

Calderón Saldierna, M. L. (2015). A collection of resources for the study of educational reverse engineering activities in engineering design education (thesis). Universitat Politècnica de Catalunya, Barcelona, Spain.

Google Scholar

Chen, J., Kolmos, A., and Du, X. (2020). Forms of implementation and challenges of PBL in engineering education: a review of literature. Eur. J. Eng. Educ. 46, 90–115. doi: 10.1080/03043797.2020.1718615

Crossref Full Text | Google Scholar

Coleman, E., Shealy, T., Grohs, J., and Godwin, A. (2020). Design thinking among first-year and senior engineering students: a cross-sectional, national study measuring perceived ability. J. Eng. Educ. 109, 72–87. doi: 10.1002/jee.20298

Crossref Full Text | Google Scholar

Cross, N. (1995). Discovering Design Ability. Chicago, IL: University of Chicago Press.

Google Scholar

Cross, N. (2001). “Chapter 5 - design cognition: results from protocol and other empirical studies of design activity,” in Design Knowing and Learning: Cognition in Design Education, eds. C. M. Eastman, W. M. McCracken, and W. C. Newstetter (Oxford: Elsevier Science), 79–103.

Google Scholar

Cui, Y., Geng, Z., Zhu, Q., and Han, Y. (2017). Multi-objective optimization methods and application in energy saving. Energy 125, 681–704. doi: 10.1016/j.energy.2017.02.174

Crossref Full Text | Google Scholar

Daun, M., Brings, J., Obe, P. A., and Stenkova, V. (2021). Reliability of self-rated experience and confidence as predictors for students' performance in software engineering. Empir. Software Eng. 26:80. doi: 10.1007/s10664-021-09972-6

PubMed Abstract | Crossref Full Text | Google Scholar

EngineeringUK. (2024). Graduate Outcomes-Engineering and Technology. London: EngineeringUK.

Google Scholar

Fordyce, D. (1986). Engineering education: a total concept? Assess. Eval. High. Educ. 11, 240–256. doi: 10.1080/0260293860110308

Crossref Full Text | Google Scholar

Franske, B. J. (2009). Engineering Problem Finding in High School Students. Minneapolis, MN, United States: University of Minnesota.

Google Scholar

Haney, C. W. (2024). Hyper-elastic triply periodic minimal surfaces design: engineering mechanics and properties. Polymers 15:4475. doi: 10.3390/polym15234475

PubMed Abstract | Crossref Full Text | Google Scholar

Heywood, J. (2005). Engineering Education: Research and Development in Curriculum and Instruction. Hoboken, NJ: Wiley. doi: 10.1002/0471744697

Crossref Full Text | Google Scholar

Higuera Martinez, O., and Fernandez-Samaca, L. (2020). “Fostering creativity in engineering through PBL,” in Educate for the Future: PBL, Sustainability and Digitalisation 2020 (Aalborg: Aalborg Universitetsforlag).

PubMed Abstract | Google Scholar

Hixson, S. W. (2023). Apparel eduLARP for computer-aided design: design, implementation, and evaluation of a live-action roleplay for undergraduate students within an apparel program (thesis). Iowa State University, Ames, IA, United States.

Google Scholar

Hu, M., Shealy, T., and Milovanovic, J. (2021). Cognitive differences among first-year and senior engineering students when generating design solutions with and without additional dimensions of sustainability. Design Sci. 7:e1. doi: 10.1017/dsj.2021.3

Crossref Full Text | Google Scholar

Institution of Civil Engineers. (2010). Competency Framework for Professional Development. London: Institution of Civil Engineers.

Google Scholar

Jiang, W. (2022). Effects of extra-curricular project-based learning experiences on self-efficacy and interest in STEM fields in high school (dissertations). San José State University, San Jose, CA, United States.

Google Scholar

Karababa, E., and Kjeldgaard, D. (2013). Value in marketing: toward sociocultural perspectives. Mark. Theory 14, 119–127. doi: 10.1177/1470593113500385

Crossref Full Text | Google Scholar

Kaur, N., Patel, A., and Dasgupta, C. (2019). Collaborative Uncertainty Management While Solving an Engineering Design Problem. Lyon, France: International Society of the Learning Sciences.

PubMed Abstract | Google Scholar

Lai, X., Tan, K-. C., and Xie, M. (2007). Optimizing product design using quantitative quality function deployment: a case study. Qual. Reliab. Eng. Int. 23, 45–57. doi: 10.1002/qre.819

PubMed Abstract | Crossref Full Text | Google Scholar

Lead States, N. G. S. S. (2013). Next Generation Science Standards: For States, By States. Washington, DC: The National Academies Press.

Google Scholar

Leung, S., Wu, J., and Ho, T. H. (2024). Early Childhood visual arts education: teachers' content knowledge, pedagogical content knowledge, and challenges. Asia Pac. Educ. Res. 34, 351–363. doi: 10.1007/s40299-024-00859-w

PubMed Abstract | Crossref Full Text | Google Scholar

Liu, L., Ye, P., and Tan, J. (2023). Exploring college students' continuance learning intention in data analysis technology courses: the moderating role of self-efficacy. Front. Psychol. 14:1241693. doi: 10.3389/fpsyg.2023.1241693

PubMed Abstract | Crossref Full Text | Google Scholar

Mahajan, R., and Bansal, D. (2021). Designing performance metrics and rubrics to assess student outcome attainment in engineering project design course. J. Educ. 203, 459–467. doi: 10.1177/00220574211032587

Crossref Full Text | Google Scholar

Menon, D., and Sadler, T. D. (2016). Preservice elementary teachers' science self-efficacy beliefs and science content knowledge. J. Sci. Teach. Educ. 27, 649–673. doi: 10.1007/s10972-016-9479-y

Crossref Full Text | Google Scholar

Ministry of Education of the People's Republic of China. (2018). 普通高中生物课程标准 (2017 版) [Biology curriculum standards for senior high school]. Beijing: People's Education Press.

Google Scholar

National Academy of Engineering (2016). Infusing Ethics Into the Development of Engineers: Exemplary Education Activities and Programs. Washington, DC: The National Academies Press.

Google Scholar

Obieke, C. C., Milisavljevic-Syed, J., and Han, J. (2021). Data-driven creativity: computational problem-exploring in engineering design. Pro. Design Soc. 1, 831–840. doi: 10.1017/pds.2021.83

Crossref Full Text | Google Scholar

Palmer, S., and Hall, W. (2011). An evaluation of a project-based learning initiative in engineering education. Eur. J. Eng. Educ. 36, 357–365. doi: 10.1080/03043797.2011.593095

Crossref Full Text | Google Scholar

Papalambros, P. Y., and Wilde, D. J. (2017). Principles of Optimal Design: Modeling and Computation. Cambridge: Cambridge University Press. doi: 10.1017/9781316451038

Crossref Full Text | Google Scholar

Parno, Yuliati, L., Munfaridah, N., Ali, M., Rosyidah, F. U. N., and Indrasari, N. (2020). The effect of project based learning-STEM on problem solving skills for students in the topic of electromagnetic induction. J. Phys. Conf. Ser. 1521:22025. doi: 10.1088/1742-6596/1521/2/022025

Crossref Full Text | Google Scholar

Patra, J. K., Das, G., Das, S. K., and Thatoi, H. (2020). “Plant tissue culture techniques and nutrient analysis,” in A Practical Guide to Environmental Biotechnology, eds. J. K. Patra, G. Das, S. K. Das, and H. Thatoi. (Singapore: Springer Singapore), 135–164. doi: 10.1007/978-981-15-6252-5_5

Crossref Full Text | Google Scholar

Prados-Peña, M. B., Gálvez-Sánchez, F. J., and García-López, A. (2023). Moving toward sustainable development: social, economic and environmental value as antecedents of purchase intention in the sustainable crafts sector. Sustainable Dev. 31, 3024–3037. doi: 10.1002/sd.2567

Crossref Full Text | Google Scholar

Quiles-Carrillo, L., Montanes, N., Sánchez-Nácher, L., Gimeno, R. A. B., and Boronat, T. (2019). Results of Incorporation of Project-Based Learning and Cooperative Learning Into Laboratory Practices. Istanbul: OCERINT.

Google Scholar

Servant-Miklos, V. F. C., and Kolmos, A. (2022). Student conceptions of problem and project based learning in engineering education: a phenomenographic investigation. J. Eng. Educ. 111, 792–812. doi: 10.1002/jee.20478

Crossref Full Text | Google Scholar

Siritunga, D., Navas, V., and Diffoot-Carlo, N. (2012). Enhancing hispanic minority undergraduates' botany laboratory experiences: implementation of an inquiry-based plant tissue culture module exercise. Int. Educ. Stud. 5:2012. doi: 10.5539/ies.v5n5p14

Crossref Full Text | Google Scholar

Standl, B., and Schlomske-Bodenstein, N. (2021). “Exploring Indicators to Promote Pre-service Teachers' Self-Efficacy in Programming Tasks”, in Proceedings of the 21st Koli Calling International Conference on Computing Education Research (Joensuu, Finland: Association for Computing Machinery). doi: 10.1145/3488042.3489965

Crossref Full Text | Google Scholar

Sulandari, N., Lesmana, C., and Setyana, C. (2021). “Engineering education: measuring the relationship between knowledge and confidence to the student performance,” in Proceedings of the 1st International Conference on Emerging Issues in Technology, Engineering and Science (Setúbal: SciTePress). doi: 10.5220/0010748400003113

Crossref Full Text | Google Scholar

Suroso, J., Indrawati, S., Mudakir, I., and Chotib, M. (2021). Analysis of high school students' skills in solving science problems in the environment. AIP Conf. Proc. 2330:50004. doi: 10.1063/5.0044038

Crossref Full Text | Google Scholar

Sutton, K., and Williams, A. (2010). Implications of Spatial Abilities on Design Thinking. Montreal, QC, Canada: Design Research Society. doi: 10.21606/drs.2010.115

Crossref Full Text | Google Scholar

Tong, K. Q., Ruan, T., Fang, Z. M., and Chen, Y. S. (2012). Research on construction, development, and disease prevention supervision of small and medium-sized hydraulic engineering. J. Yangtze River Sci. Res. Inst. 29:122. doi: 10.3969/j.issn.1001-5485.2012.11.027

Crossref Full Text | Google Scholar

Tugendrajch, S. K. (2022). What Is the Evidence Base Supporting Best Practice Supervision Guidelines? A Systematic Review and Multilevel Meta-Analysis of the Association Between Supervision and Therapist Performance. Columbia, MO, United States: University of Missouri-Columbia.

Google Scholar

U.S. Congress (2014). “H.Res.51 - Expressing the sense of the House of Representatives that adding art and design into Federal programs that target the fields of science, technology, engineering, and mathematics (STEM) encourages innovation and economic growth in the United States,” in 113th Congress, 2nd Session (Washington, DC: U.S. Government Publishing Office).

Google Scholar

Varadarajan, S. (2020). “Measuring the value of systems thinking for design-centric engineering education,” in Proceedings of the Design Society: DESIGN Conference (Cambridge: Cambridge University Press), 1835–1842. doi: 10.1017/dsd.2020.72

Crossref Full Text | Google Scholar

Webb-Williams, J. (2018). Science self-efficacy in the primary classroom: using mixed methods to investigate sources of self-efficacy. Res. Sci. Educ. 48, 939–961. doi: 10.1007/s11165-016-9592-0

Crossref Full Text | Google Scholar

Yu, K-. C., Wu, P-. H., and Fan, S-. C. (2020). Structural relationships among high school students' scientific knowledge, critical thinking, engineering design process, and design product. Int. J. Sci. Math. Educ. 18, 1001–1022. doi: 10.1007/s10763-019-10007-2

Crossref Full Text | Google Scholar

Zhang, M. (2024). Traceable Calibration and Evaluation of Toroidal Current Transformers for High Impedance Fault Detection in Electricity Networks. Wellington: Te Herenga Waka-Victoria University of Wellington.

Google Scholar

Zhu, Z. W., and Zhao, Z. L. (2017). High School Biology Selective Compulsory Course 3: Biotechnology and Engineering. Beijing: People's Education Press.

Google Scholar