<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="research-article" dtd-version="2.3" xml:lang="EN">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Psychol.</journal-id>
<journal-title>Frontiers in Psychology</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Psychol.</abbrev-journal-title>
<issn pub-type="epub">1664-1078</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fpsyg.2022.774022</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Psychology</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Exploring the Integration of Engineering Design Practices in Tenth-Grade Chemistry Activities</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Aldahmash</surname>
<given-names>Abdulwali H.</given-names>
</name>
<xref rid="aff1" ref-type="aff"><sup>1</sup></xref>
<xref rid="c001" ref-type="corresp"><sup>&#x002A;</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/909138/overview"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Alfarraj</surname>
<given-names>Yousef F.</given-names>
</name>
<xref rid="aff2" ref-type="aff"><sup>2</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/1801102/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Curriculum and Instruction Department, College of Education, The Excellence Research Center of Science and Mathematics Education, King Saud University</institution>, <addr-line>Riyadh</addr-line>, <country>Saudi Arabia</country></aff>
<aff id="aff2"><sup>2</sup><institution>Curriculum and Instruction Department, College of Education, King Saud University</institution>, <addr-line>Riyadh</addr-line>, <country>Saudi Arabia</country></aff>
<author-notes>
<fn id="fn0001" fn-type="edited-by"><p>Edited by: Canan Nakibo&#x011F;lu, Bal&#x0131;kesir University, Turkey</p></fn>
<fn id="fn0002" fn-type="edited-by"><p>Reviewed by: Ana K. Houseal, University of Wyoming, United States; Zuzana Hal&#x00E1;kov&#x00E1;, Comenius University, Slovakia; Martin Rusek, Charles University, Czechia</p></fn>
<corresp id="c001">&#x002A;Correspondence: Abdulwali H. Aldahmash, <email>aaldahmash@ksu.edu.sa</email>; <email>wadialsail@gmail.com</email></corresp>
<fn id="fn0003" fn-type="other"><p>This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology</p></fn>
</author-notes>
<pub-date pub-type="epub">
<day>13</day>
<month>07</month>
<year>2022</year>
</pub-date>
<pub-date pub-type="collection">
<year>2022</year>
</pub-date>
<volume>13</volume>
<elocation-id>774022</elocation-id>
<history>
<date date-type="received">
<day>10</day>
<month>09</month>
<year>2021</year>
</date>
<date date-type="accepted">
<day>10</day>
<month>06</month>
<year>2022</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x00A9; 2022 Aldahmash and Alfarraj.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Aldahmash and Alfarraj</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract>
<p>Chemistry textbooks are the most popular teaching material in schools. They can contribute significantly to the attainment of scientific educational goals. Internationally, educational reforms in science subjects are adopting newer practices such as Engineering Design Processes (EDP) for addressing real-world requirements. This study, conducted in Saudi Arabia, employed a qualitative and quantitative content analysis method to evaluate the level of the EDP incorporated in the tenth grade chemistry textbook and accompanying student&#x2019;s experiments&#x2019; guidebook. The results found the inclusive mean for EDP was 1.05, which indicated incorporation was found to be at level one. This inclusion EDP in chemistry textbooks has been rated as novice or deficient, which indicates that it does not fulfill the majority of the requirements for inclusion as suggested by NGSS.</p>
</abstract>
<kwd-group>
<kwd>chemistry textbook</kwd>
<kwd>engineering designed practices</kwd>
<kwd>new generation science standards</kwd>
<kwd>textbooks analysis</kwd>
<kwd>science education activities</kwd>
</kwd-group>
<counts>
<fig-count count="0"/>
<table-count count="5"/>
<equation-count count="0"/>
<ref-count count="68"/>
<page-count count="11"/>
<word-count count="8518"/>
</counts>
</article-meta>
</front>
<body>
<sec id="sec1" sec-type="intro">
<title>Introduction</title>
<p>In the early 19th century, technology and engineering education attracted educators from around the world. For example, the National Academy of Engineering carried out a project called &#x201C;Engineering in K-12 Education,&#x201D; that was concerned with the connection between curricular materials related to science, technology, engineering, and mathematics education (STEM) subjects (<xref ref-type="bibr" rid="ref52">Pieper and Mentzer, 2013</xref>). Since then, engineering education has been integrated with k-12 science curricula in a number of countries (<xref ref-type="bibr" rid="ref53">Purzer et al., 2015</xref>; <xref ref-type="bibr" rid="ref54">Purzer and Quintana-Cifuentes, 2019</xref>). For example, the National Science Education Standards (NSES), [<xref ref-type="bibr" rid="ref41">National Research Council (NRC), 1996</xref>; <xref ref-type="bibr" rid="ref40">National Academy of Engineering and National Research Council (NAE and NRC), 2009</xref>, p. 4; <xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>; <xref ref-type="bibr" rid="ref17">Brand, 2020</xref>], adopted engineering design processes and recommended that it be included in science curricula. This integration of engineering design processes in K-12 science education led to reforms in pedagogical, epistemological, and methodological courses and programs for teacher preparation (<xref ref-type="bibr" rid="ref14">Benz, 2019</xref>; <xref ref-type="bibr" rid="ref54">Purzer and Quintana-Cifuentes, 2019</xref>). In addition, the NGSS raised engineering design to the same level as scientific inquiry, which was initiated by NSES, by pledging to integrate engineering design into all levels of science education.</p>
<p>To enhance students&#x2019; interest in scientific and engineering process, as well as to improve the teaching of science, the NRC further developed its NSES to establish the New Generation Scientific Standard (NGSS). The NGSS were designed to guarantee science education that is grounded in real-world experience (<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>). Students were expected to be able to investigate and apply important scientific concepts as one of three elements of the NGSS practices. One of the goals of NGSS was also to ensure that students graduated from high school with expertise in engineering design activities and inquiry methods. One way to achieve this would include developing, science textbooks and experiments&#x2019; guidebooks with interdisciplinary knowledge, particularly engineering knowledge that would allow students to gain experience rather than memorize concepts and ideas.</p>
<p>The <xref ref-type="bibr" rid="ref004">National Research Council (NRC) (2009)</xref> highlights the necessity for science training to reflect scientific investigative procedures when generating theories and models and engineering process when building solutions. <xref ref-type="bibr" rid="ref26">Ghalia et al. (2016)</xref> argues that integrating engineering design processes in K&#x2013;12 schools can improve student results. They state that incorporating engineering in science education could increase desire to study mathematics and scientific materials. Would also learn to contextualize open-ended issues (<xref ref-type="bibr" rid="ref57">Rubel and Mccloskey, 2021</xref>). An understanding of the engineering design process could also improve&#x2019; knowledge of mathematics and scientific content, and their interest in STEM careers (<xref ref-type="bibr" rid="ref16">Berland et al., 2014</xref>; <xref ref-type="bibr" rid="ref26">Ghalia et al., 2016</xref>). The NGSS [<xref ref-type="bibr" rid="ref44">National Science Foundation (NSF), 2010</xref>; <xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>] was established to reiterate the opinion that scientific courses that &#x201C;stimulate and create enthusiasm&#x201D; in STEM would boost the number of students choosing STEM careers. STEM education predominantly emphasizes engineering (<xref ref-type="bibr" rid="ref63">Ubuz, 2019</xref>). Consequently, there has been an increase in efforts to develop and implement programs like STEM for high schools and K-12 engineering outreach programs (<xref ref-type="bibr" rid="ref17">Brand, 2020</xref>; <xref ref-type="bibr" rid="ref11">Aysun and Ozlen, 2021</xref>). However, as <xref ref-type="bibr" rid="ref26">Ghalia et al. (2016)</xref> point out, although these activities have been successful in introducing the engineering design process to children, instructors remain the most important influence on student outcomes. In addition, <xref ref-type="bibr" rid="ref56">Ross et al. (2018)</xref> stress, professional development opportunities and knowledge of curriculum-integrated engineering applications are limited, leaving instructors with little assistance to modify their methods.</p>
<p>Educators, in recent years. Have adopted the Engineering Design Process (EDP; <xref ref-type="bibr" rid="ref30">Hynes, 2012</xref>) as a tool for STEM implementation in a variety of STEM-related topics (<xref ref-type="bibr" rid="ref50">Nurtanto et al., 2020</xref>). This importance of EDP can be verified by the number of research studies that have been conducted on the efficacy of EDP in STEM education. As a field of study, engineering design can be characterized in many ways. The engineering desirability steps include [<xref ref-type="bibr" rid="ref41">National Research Council (NRC), 1996</xref>, p. 192; <xref ref-type="bibr" rid="ref43">National Research Council (NRC), 2012</xref>]: (1) Define the problem, (2) Select a systematic solution, (3) Modeling and Analysis and (4) Repeat the design process. Students&#x2019; ability to apply science and mathematics to real-world problems can be considerably enhanced by the implementation of the phases of the EDP (<xref ref-type="bibr" rid="ref41">National Research Council (NRC), 1996</xref>, p. 192; <xref ref-type="bibr" rid="ref23">English and King, 2015</xref>; <xref ref-type="bibr" rid="ref28">Hafiz and Ayop, 2019</xref>). Later in 2012, the <xref ref-type="bibr" rid="ref43">National Research Council (NRC) (2012)</xref> further developed these phases and recommended eight EDP steps for the K-12 Framework for science education.</p>
<p>Because chemistry textbooks have the potential to influence how students learn as well as how chemistry and science teachers teach and achieve curriculum goals [<xref ref-type="bibr" rid="ref003">National Research Council (NRC), 2007</xref>, <xref ref-type="bibr" rid="ref43">2012</xref>; <xref ref-type="bibr" rid="ref49">Niaz and Maza, 2011</xref>; <xref ref-type="bibr" rid="ref15">Bergqvist and Chang Rundgren, 2017</xref>; <xref ref-type="bibr" rid="ref32">Kremen et al., 2022</xref>]. They influence not only what and how students learn in school, but also what and how science teachers teach science (<xref ref-type="bibr" rid="ref49">Niaz and Maza, 2011</xref>). Chemistry textbooks play a vital role in schooling. The course structure, as well as the framework for learners&#x2019; experience and evaluation, are primarily reliant on textbooks for science and chemistry teachers (<xref ref-type="bibr" rid="ref20">Devetak and Vogrinc, 2013</xref>; <xref ref-type="bibr" rid="ref5">Aldahmash et al., 2016</xref>; <xref ref-type="bibr" rid="ref50">Nurtanto et al., 2020</xref>). Our study is guided by the NGSS and its suggested eight EDP steps [<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>; <xref ref-type="bibr" rid="ref002">National Science Teachers Association (NSTA), 2014</xref>, <xref ref-type="bibr" rid="ref46">2018</xref>]:</p>
<list list-type="order">
<list-item>
<p>Asking questions (for science) and defining problems (for engineering)</p>
</list-item>
<list-item>
<p>Developing and using models</p>
</list-item>
<list-item>
<p>Planning and carrying out investigations</p>
</list-item>
<list-item>
<p>Analyzing and interpreting data</p>
</list-item>
<list-item>
<p>Using mathematics and computational thinking</p>
</list-item>
<list-item>
<p>Constructing explanations (for science) and designing solutions (for engineering)</p>
</list-item>
<list-item>
<p>Engaging in argument from evidence</p>
</list-item>
<list-item>
<p>Obtaining, evaluating, and communicating information</p>
</list-item>
</list>
<p>We also added another process (redesign) to our analysis rubric, based on the suggestions made by <xref ref-type="bibr" rid="ref29">Hirsch et al. (2017)</xref>. We are convinced that redesign is an important process for students&#x2019; mastery of science and chemistry and redesigning activities are essential components of any scientific processes.</p>
<sec id="sec2">
<title>The Rationale of the Study</title>
<p>In 1979, the American University of Beirut&#x2019;s scientific education experts worked on a reform initiative involving the construction of science curricula in Saudi Arabia. While this reform initiative was completed this year, the enterprise continues to create and execute novel curricular projects. Our qualitative study sought to better understand the impact of the reform initiative in public secondary schools of Saudi. Furthermore, the Saudi science community has grown more concerned that the present curriculum does not adequately reflect the current and future social, cultural, and economic demands of the Saudi society, nor does it adequately satisfy the needs of all students since that time (<xref ref-type="bibr" rid="ref8">Alhomairi, 2018</xref>). Schools in Saudi Arabia still rely on teacher-centered techniques, which scholars claim, have resulted in a decrease in students&#x2019; participation in inquiry-based learning (<xref ref-type="bibr" rid="ref7">Al-Ghanem, 1999</xref>; <xref ref-type="bibr" rid="ref9">Almazroa and Aloraini, 2012</xref>). A reform effort to improve the Saudi&#x2019;s mathematics and science curriculum involved modifications in the McGraw-Hill series textbooks used for primary and secondary education (<xref ref-type="bibr" rid="ref3">Al Zughaybi, 2011</xref>; <xref ref-type="bibr" rid="ref2">Al Zubaidi, 2017</xref>). This new curriculum was criticized for being solely inquiry-based, and not incompatible with the standards established by the new NGSS and the science American Association for the Advancement of Science [<xref ref-type="bibr" rid="ref10">American Association for the Advancement of Science (AAAS), 1993</xref>]. In addition, according to the Trends in International Mathematics and Science Study (TIMSS), this curriculum failed to enhance the achievement of Saudi Arabian students (<xref ref-type="bibr" rid="ref39">Mullis et al., 2020</xref>; <xref ref-type="bibr" rid="ref32">Kremen et al., 2022</xref>).</p>
<p>One of the goals of integrating EDP into science courses is to equip students with the knowledge, skills, and abilities to study and apply major scientific ideas in their everyday lives. According to the findings from the Theoretical background, secondary chemistry textbook and experiments&#x2019; guidebook that utilized EDP, enabled students to operate like engineers and scientists (<xref ref-type="bibr" rid="ref60">Syukri et al., 2018</xref>). The efficacy of these books to assist students in practicing EDP skills, on the other hand, has been called into question (<xref ref-type="bibr" rid="ref50">Nurtanto et al., 2020</xref>). Therefore, one of the goals of our study was to determine the EDP skills contained in science textbooks and the distribution of these skills. We also explored ways to improve the current science curriculum, and identify educational reforms that could significantly improve science and chemistry textbooks. While studies, like ours, is necessary for Saudi Arabian science education, we also believe that our findings will be applicable to education systems in other countries.</p>
<p>The findings of several studies (<xref ref-type="bibr" rid="ref61">Tamir, 1991</xref>; <xref ref-type="bibr" rid="ref21">Dreyfus, 1992</xref>; <xref ref-type="bibr" rid="ref62">Tamir and Pilar-Garcia, 1992</xref>; <xref ref-type="bibr" rid="ref27">Green et al., 2006</xref>; <xref ref-type="bibr" rid="ref29">Hirsch et al., 2017</xref>; <xref ref-type="bibr" rid="ref34">Lopez and Garlick, 2017</xref>) have emphasized the need for adhering to specific rules while examining the content of targeted textbooks. The attention placed on EDP skills to enhance student&#x2019;s comprehension of science and critical thinking abilities inspired us to delve deeper. As a result, we analyzed chemistry textbooks to understand the integration of EDP in them. One of our concerns was voiced by <xref ref-type="bibr" rid="ref21">Dreyfus (1992)</xref>, who suggested that examining textbooks with such a declared purpose raises a new question: does the textbook match the curriculum&#x2019;s intentions? This is why we used scientific EDP abilities.</p>
<p>The inclusion of EDP abilities in chemistry textbooks and guidebooks can help students in many ways (<xref ref-type="bibr" rid="ref59">Silk et al., 2009</xref>; <xref ref-type="bibr" rid="ref66">Wendell et al., 2017</xref>). First it may help them gain higher test scores in chemistry as well as other science subjects. Another study suggested that EDP would make science more relevant to students as well as increase their scientific reasoning skills. Based on these arguments, it is very important to assess how well school chemistry textbooks include EDP and how they would assist students develop EDP skills.</p>
<p>In our study, we examined the integration of EDP skills in tenth-grade Saudi chemistry textbooks. We selected tenth-grade chemistry textbook because they are taught to students who are in a transition stage, between the middle school and the high school. After graduating the tenth-grade students will have to choose either science or art as their future educational streams. We argue that the inclusion of EDP in these textbooks would increase students&#x2019; motivation to select science education.</p>
</sec>
<sec id="sec3">
<title>Purpose of the Study</title>
<p>The purpose of this study was to examine the tenth-grade chemistry textbook and experiment guidebook for the academic year 2020/2021 in order to identify EDP steps incorporated in their activities. The research questions and sub-questions that guided this study were:</p>
<sec id="sec4">
<title>The Main Question</title>
<p>To what extent is Engineering Design Process represented in the tenth-grade chemistry textbooks and experiments guidebooks?</p>
<sec id="sec5">
<title>Sub-questions</title>
<list list-type="bullet">
<list-item>
<p>To what extent is the EDP emphasized in the activities provided in both the student&#x2019;s chemistry textbooks and experiment guidebooks?</p>
</list-item>
<list-item>
<p>What types of activities could be most effective in promoting EDP among the tenth-grade students?</p>
</list-item>
<list-item>
<p>Do the EDP skill levels determined in current tenth-grade chemistry textbooks and experiment guidebooks align with the NGSS&#x2019;s recommendations?</p>
</list-item>
</list>
</sec>
</sec>
</sec>
</sec>
<sec id="sec6">
<title>Research Method</title>
<p>Chemistry textbooks and experiment guidebooks for tenth-grade students were examined using a mixed-method approach that included quantitative data analysis and qualitative content analysis. The units of the analysis were the activities or experiments provided in these books. There were 30 activities and experiments in both the chemistry textbooks and experiments&#x2019; guidebook for the tenth-grade students (19 in the textbook and 11 in the guidebook). A methodical approach to examine the targeted textbooks involves examining the activities included in the textbooks <xref ref-type="bibr" rid="ref12">Babbie (2001)</xref> and <xref ref-type="bibr" rid="ref67">White and Marsh (2006)</xref> was employed as a guideline in this study. In addition, we looked at the paragraphs, worked examples, figures, and tables to determine whether there were any connections between the practical and theoretical aspects of the document. The processes developed by <xref ref-type="bibr" rid="ref12">Babbie (2001)</xref> and <xref ref-type="bibr" rid="ref33">Krippendorf (2004)</xref> were employed to convert raw textual material into standardized codes. This allowed categorization of data for <italic>text-driven analyses, problem-driven analyses, and method-driven analyses</italic> (<xref ref-type="bibr" rid="ref48">Nguyen et al., 2020</xref>). This was further explained in Part II, titled &#x201C;Components of Content Analysis,&#x201D; where the author detailed the many techniques employed in content analyses, starting with their procedural logic and going organically from unitizing to sampling to recording and coding. Thus, in this paper the coding was accomplished by assigning meaning to science concepts in the document and interpreting them in accordance to the analysis cards or rubric. Qualitative data were collected for situations in which EDP abilities were not clearly mentioned in the text or activity but were implied by the phrases used. The coding approach was also used for qualitative data to search for implicit terms and then group them together based on their meaning.</p>
</sec>
<sec id="sec7">
<title>Materials Analyzed</title>
<p>Two books were analyzed; the first one is the students&#x2019; textbook, while the second one was the experiments guide. The activities and experiments, as well as the text and graphs that accompany them, were examined to determine the extent to which EDP was incorporated into the tenth-grade chemistry textbook and experiment guidebook used in Saudi Arabian schools. These books were chosen because they are taught in a grade that functions as a transition year. These books were also chosen because the student in the tenth grade take chemistry for the first time after completing middle school.</p>
<p>The tenth-grade chemistry textbook is 212 pages in length and is divided into five sections and includes 19 activities. The student textbook is divided into the following sections: safety signs, laboratory first aid, a general introduction, how to use the book effectively, general objectives, and a content list. Each chapter of the textbook contains a content list, a launch experiment, and the lessons. Each lesson contains objectives, concepts, new vocabulary, and assessment questions. Numerous lessons incorporate a variety of activities, such as experiments, data analysis laboratories, chemistry laboratories, and problem solving laboratories. Each chapter is concluded with a validated questionnaire.</p>
<p>On the other hand, the tenth-grade chemistry experiment guidebook or experiment guide is 46 pages in length and contains 11 activities named as experiments. It includes an introduction, general objectives, content list, lab safety, instrument, and five chapters. Each chapter contains an experiment, with the exception of chapter two, which contains a three-in-one experiment. Each experiment contains the following components: a conceptual introduction, a problem, objectives, list of tools and instruments required, safety precautions, prior to the experiment, working steps, hypothesis, cleaning and discarding residuals, a table for data and observation, analysis and deduction, error analysis, and real-world chemistry.</p>
<p>The NGSS (<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>), the <xref ref-type="bibr" rid="ref41">National Research Council (NRC) (1996</xref>, <xref ref-type="bibr" rid="ref42">2000</xref>, <xref ref-type="bibr" rid="ref43">2012</xref>), and constructivist theories (<xref ref-type="bibr" rid="ref55">Rintaningrum, 2008</xref>) all suggest that textbooks and experiments&#x2019; guidebooks should be linked, with experiments conducted first and the related content in the textbook discussed and organized based on the results of each activity. According to NGSS students should be treated as scientists when it comes to the design of activities in the students&#x2019; science textbook or experiments&#x2019; guidebook.</p>
<sec id="sec8">
<title>Instrument</title>
<p>The rubric was developed and the targeted textbook analysis was conducted using the conceptual framework as a guide. Numerous other conceptual frameworks served as the prototype and guide for our study (<xref ref-type="bibr" rid="ref18">Chiappetta and Fillman, 2007</xref>; <xref ref-type="bibr" rid="ref31">Kahveci, 2010</xref>; <xref ref-type="bibr" rid="ref001">Dunne et al., 2012</xref>; <xref ref-type="bibr" rid="ref64">Vesterinen et al., 2013</xref>; <xref ref-type="bibr" rid="ref5">Aldahmash et al., 2016</xref>; <xref ref-type="bibr" rid="ref29">Hirsch et al., 2017</xref>; <xref ref-type="bibr" rid="ref4">Aldahmash and Alamri, 2020</xref>; <xref ref-type="bibr" rid="ref6">Aldahmash and Omar, 2021</xref>). <xref ref-type="bibr" rid="ref18">Chiappetta and Fillman (2007)</xref>, for example, used alternative frameworks to outline the inclusion of subject matter content, analyze the difficulty or readability of textbooks or content, and investigate the texts&#x2019; epistemological orientation. The current study examined the chemistry textbook and experiment manual for the incorporation of EDP using modified rubrics based on the National Science Education Standards (NSES; <xref ref-type="bibr" rid="ref43">National Research Council (NRC), 2012</xref>; <xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>). The modifications were carried out using the same techniques as in related investigations (<xref ref-type="bibr" rid="ref18">Chiappetta and Fillman, 2007</xref>; <xref ref-type="bibr" rid="ref31">Kahveci, 2010</xref>; <xref ref-type="bibr" rid="ref001">Dunne et al., 2012</xref>; <xref ref-type="bibr" rid="ref35">Mangold and Robinson, 2013</xref>; <xref ref-type="bibr" rid="ref64">Vesterinen et al., 2013</xref>; <xref ref-type="bibr" rid="ref5">Aldahmash et al., 2016</xref>; <xref ref-type="bibr" rid="ref4">Aldahmash and Alamri, 2020</xref>; <xref ref-type="bibr" rid="ref6">Aldahmash and Omar, 2021</xref>). The rubric for this study was designed using the eight NGSS practices (<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>) and the additional redesign process (<xref ref-type="bibr" rid="ref29">Hirsch et al., 2017</xref>; see <xref rid="tab1" ref-type="table">Table 1</xref>) included by us (henceforth the term EDP will include all nine practices):</p>
<table-wrap position="float" id="tab1">
<label>Table 1</label>
<caption><p>The rubric for analyzing the inclusion of engineering design process and their levels.</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top">Step</th>
<th align="left" valign="top">Process</th>
<th align="left" valign="top">Level 3: (Advanced) or (proficient) meets all the criteria</th>
<th align="left" valign="top">Level 2: (Developing) meets most of the criteria</th>
<th align="left" valign="top">Level 1: (Novice) Or does not meet a majority of the criteria</th>
<th align="left" valign="top">Level 0: (No evidence) Not found</th>
</tr>
</thead>
<tbody>
<tr>
<td align="char" valign="top" char=".">1</td>
<td align="char" valign="top" char="&#x00B1;">Identify the problem</td>
<td align="char" valign="top" char="&#x00B1;">Clearly stated and worded, meets the criteria</td>
<td align="char" valign="top" char="&#x00B1;">Adequately stated and worded, meets most of the criteria</td>
<td align="char" valign="top" char="&#x00B1;">Poorly stated and worded, does not meet a majority of the criteria</td>
<td align="char" valign="top" char="&#x00B1;">Did not include problem statement</td>
</tr>
<tr>
<td align="char" valign="top" char=".">2</td>
<td align="char" valign="top" char="&#x00B1;">Framing a design brief</td>
<td align="char" valign="top" char="&#x00B1;">Clearly stated, meets all the criteria and specifications</td>
<td align="char" valign="top" char="&#x00B1;">Adequately stated, meets most of the criteria and specifications</td>
<td align="char" valign="top" char="&#x00B1;">Poorly stated, does not meet the majority of the criteria</td>
<td align="char" valign="top" char="&#x00B1;">Id not include the design brief</td>
</tr>
<tr>
<td align="char" valign="top" char=".">3</td>
<td align="char" valign="top" char="&#x00B1;">Research and investigation</td>
<td align="char" valign="top" char="&#x00B1;">The content enables students to do thorough research and investigation of various components of their design</td>
<td align="char" valign="top" char="&#x00B1;">The content just enables students to do adequate research on various components of their design</td>
<td align="char" valign="top" char="&#x00B1;">The content poorly guides students to do research on various components of their design</td>
<td align="char" valign="top" char="&#x00B1;">The content did not include research or investigations</td>
</tr>
<tr>
<td align="char" valign="top" char=".">4</td>
<td align="char" valign="top" char="&#x00B1;">Generation of alternative solutions</td>
<td align="char" valign="top" char="&#x00B1;">Generated 3&#x2009;+&#x2009;thorough sketches of possible design solutions</td>
<td align="char" valign="top" char="&#x00B1;">Generated 2 adequate sketches of possible design solutions</td>
<td align="char" valign="top" char="&#x00B1;">Generated 1 adequate/poor sketch of a possible design solution</td>
<td align="char" valign="top" char="&#x00B1;">Did not include sketches of possible design solutions</td>
</tr>
<tr>
<td align="char" valign="top" char=".">5</td>
<td align="char" valign="top" char="&#x00B1;">Choosing the best solution</td>
<td align="char" valign="top" char="&#x00B1;">Thoroughly explained how they objectively chose their solution</td>
<td align="char" valign="top" char="&#x00B1;">Adequately explained how they objectively chose their solution</td>
<td align="char" valign="top" char="&#x00B1;">Did not thoroughly explain solution choice or chose solution objectively</td>
<td align="char" valign="top" char="&#x00B1;">Did not include how they chose the best solution</td>
</tr>
<tr>
<td align="char" valign="top" char=".">6</td>
<td align="char" valign="top" char="&#x00B1;">Developmental work</td>
<td align="char" valign="top" char="&#x00B1;">Created thorough sketches, bill of materials, steps needed to create design</td>
<td align="char" valign="top" char="&#x00B1;">Created adequate sketches and bill of materials/steps to create their design</td>
<td align="char" valign="top" char="&#x00B1;">Poor sketch, did not include bill of materials or steps used to create design</td>
<td align="char" valign="top" char="&#x00B1;">Did not include developmental work</td>
</tr>
<tr>
<td align="char" valign="top" char=".">7</td>
<td align="char" valign="top" char="&#x00B1;">Modeling (prototyping)</td>
<td align="char" valign="top" char="&#x00B1;">Created a well-designed prototype, allows for testing, Works properly, looks good decent</td>
<td align="char" valign="top" char="&#x00B1;">Created a prototype that can be tested. Works relatively well</td>
<td align="char" valign="top" char="&#x00B1;">Created almost complete prototype, may be able to test, does not work well</td>
<td align="char" valign="top" char="&#x00B1;">Did not include the prototype</td>
</tr>
<tr>
<td align="char" valign="top" char=".">8</td>
<td align="char" valign="top" char="&#x00B1;">Testing and evaluating</td>
<td align="char" valign="top" char="&#x00B1;">Thoroughly explained how to test prototype, testing process makes sense stronger</td>
<td align="char" valign="top" char="&#x00B1;">Clearly explained how to test prototype, testing process could be</td>
<td align="char" valign="top" char="&#x00B1;">Did not clearly explain how to test prototype, process not clear</td>
<td align="char" valign="top" char="&#x00B1;">Did not explain how to test the prototype</td>
</tr>
<tr>
<td align="char" valign="top" char=".">9</td>
<td align="char" valign="top" char="&#x00B1;">Redesign</td>
<td align="char" valign="top" char="&#x00B1;">Made valid decisions for change/improvement based on test results</td>
<td align="char" valign="top" char="&#x00B1;">Decisions to change were loosely based on test results</td>
<td align="char" valign="top" char="&#x00B1;">Decisions to change were not based on test results</td>
<td align="char" valign="top" char="&#x00B1;">Did not make needed improvements</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>As shown in <xref rid="tab1" ref-type="table">Table 1</xref>, the rubric contains nine processes, each of which has four alternatives (level 3, meets all the criteria, level 2, meets most of the criteria, level 1, does not meet the majority of the criteria, and level 0, not found). The standards established by the <xref ref-type="bibr" rid="ref42">National Research Council (NRC) (2000)</xref> and NGSS (<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>) were used to indicate the level of EDP incorporated in the teaching strategy. The level of openness of a strategy, according to NGSS and NRC can range from level one (teacher-centered) to level four (student-centered), with a preference for levels 3 and 4. According to the <xref ref-type="bibr" rid="ref42">National Research Council (NRC) (2000)</xref> and NGSS (<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>) chemistry teaching should incorporate all types of openness levels.</p>
</sec>
<sec id="sec9">
<title>Reliability of Content Analysis</title>
<p>The authors investigated a sample of the activities in the targeted textbook and experiment guidebook, where two units were analyzed by each of the two raters. Then each rater coded the sampled actions to denote the EDP contained in the studied texts. The kappa formula (<xref ref-type="bibr" rid="ref19">Cohen, 1990</xref>) was utilized to determine the agreement value. Research (<xref ref-type="bibr" rid="ref58">Rubinstein and Brown, 1984</xref>) indicates that kappa values between 0.40 and 0.75 are appropriate. The values in <xref rid="tab2" ref-type="table">Table 2</xref> demonstrate strong inter-rater agreement for all eight EDP abilities contained in the chemistry textbook and experiment guidebook used in this investigation. According to <xref rid="tab2" ref-type="table">Table 2</xref>, the kappa values ranged between 0.877 and (0.559). These values are located in the range between moderate and perfect agreement, which indicates that adequate degree of agreements, has been reached. in this regard, Cohen suggested &#x201C;the kappa result be interpreted as follows: values &#x2264; 0 as indicating no agreement and 0.01&#x2013;0.20 as none to slight, 0.21&#x2013;0.40 as fair, 0.41&#x2013;0.60 as moderate, 0.61&#x2013;0.80 as substantial, and 0.81&#x2013;1.00 as almost perfect agreement&#x201D; (<xref ref-type="bibr" rid="ref38">McHugh, 2012</xref>). These values indicate that the list used in this analysis is adequate for this study and can be used in other similar studies.</p>
<table-wrap position="float" id="tab2">
<label>Table 2</label>
<caption><p>Inter-rater reliability of the analysis of EDP in tenth-grade student&#x2019;s chemistry textbooks and experiment guidebooks.</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top">Process</th>
<th align="center" valign="top">% Agreement</th>
<th align="center" valign="top">kappa</th>
<th align="left" valign="top">Strength</th>
</tr>
</thead>
<tbody>
<tr>
<td align="char" valign="top" char=".">Identify the problem</td>
<td align="char" valign="top" char="&#x00B1;">93.3</td>
<td align="char" valign="top" char="&#x00B1;">0.877</td>
<td align="char" valign="top" char="&#x00B1;">Almost prefect</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Framing a design brief</td>
<td align="char" valign="top" char="&#x00B1;">80.0</td>
<td align="char" valign="top" char="&#x00B1;">0.696</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Research and investigation</td>
<td align="char" valign="top" char="&#x00B1;">86.7</td>
<td align="char" valign="top" char="&#x00B1;">0.792</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Generation of alternative solutions</td>
<td align="char" valign="top" char="&#x00B1;">93.3</td>
<td align="char" valign="top" char="&#x00B1;">0.776</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Choosing the best solution</td>
<td align="char" valign="top" char="&#x00B1;">80.0</td>
<td align="char" valign="top" char="&#x00B1;">0.605</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Developmental work</td>
<td align="char" valign="top" char="&#x00B1;">80.0</td>
<td align="char" valign="top" char="&#x00B1;">0.679</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Modeling (prototyping)</td>
<td align="char" valign="top" char="&#x00B1;">80.0</td>
<td align="char" valign="top" char="&#x00B1;">0.559</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Testing and evaluating</td>
<td align="char" valign="top" char="&#x00B1;">86.7</td>
<td align="char" valign="top" char="&#x00B1;">0.800</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Redesign</td>
<td align="char" valign="top" char="&#x00B1;">86.7</td>
<td align="char" valign="top" char="&#x00B1;">0.797</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Total</td>
<td align="char" valign="top" char="&#x00B1;">85.19</td>
<td align="char" valign="top" char="&#x00B1;">0.731</td>
<td align="char" valign="top" char="&#x00B1;">Substantial</td>
</tr>
</tbody>
</table>
</table-wrap>
<sec id="sec10">
<title>Levels Assigned Through Calculated Means</title>
<p>To make sense of and draw inferences from the findings, the weighted means and percentages as a measure of central tendency were computed. The interval means were calculated as follows:</p>
<p>First, the range was calculated by subtracting the lowest value from the highest value, so the range is 3&#x2013;1&#x2009;=&#x2009;2. Second, the length of the interval was calculated by dividing the range by the highest value of the scale: (2 &#x00F7; 3&#x2009;=&#x2009;0.67).</p>
<p>Finally, the level for each category of the rubric was determined as follows: Not found means were set for the values from 0 to 0.67, Novice means were set from 0.68 to 1.34, Developing means were set from 1.35 to 2.01, and finally, means for the Advanced are set from 2.02 to 3.</p>
</sec>
</sec>
</sec>
<sec id="sec11">
<title>Findings</title>
<p>The results presented in <xref rid="tab3" ref-type="table">Table 3</xref> indicated that the first practice, &#x201C;<italic>Identify the problem,&#x201D;</italic> was the most included EDP. It was directly included in the experiments&#x2019; guidebook and indirectly in the activities of the textbook. The practice &#x201C;<italic>Generating Alternative solutions&#x201D;</italic> was the least included, all indirectly, with 23.3% of inclusion. The total of the EDP inclusion was found to be 62.58% at the first level, which indicates that 37.42 of the activities did not include EDP. Furthermore, all EDP skills were not properly included in the textbook and the experiment guidebook.</p>
<table-wrap position="float" id="tab3">
<label>Table 3</label>
<caption><p>Frequencies, percentages, means, and standard deviations on the extent of the inclusion of the 9 EDP in the tenth grade chemistry textbook and guidebook.</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top" rowspan="2">Process</th>
<th align="center" valign="top" colspan="2">Advance</th>
<th align="center" valign="top" colspan="2">Developing</th>
<th align="center" valign="top" colspan="2">Novice</th>
<th align="center" valign="top" colspan="2">Not found</th>
<th align="center" valign="top" colspan="2">Total of included EDP</th>
<th align="center" valign="top" rowspan="2">M</th>
<th align="center" valign="top" rowspan="2">SD</th>
<th align="center" valign="top" rowspan="2">level</th>
</tr>
<tr>
<th align="center" valign="top">f</th>
<th align="center" valign="top">%</th>
<th align="center" valign="top">f</th>
<th align="center" valign="top">%</th>
<th align="center" valign="top">f</th>
<th align="center" valign="top">%</th>
<th align="center" valign="top">f</th>
<th align="center" valign="top">%</th>
<th align="center" valign="top">f</th>
<th align="center" valign="top">%</th>
</tr>
</thead>
<tbody>
<tr>
<td align="char" valign="top" char=".">Identify the problem</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
<td align="char" valign="top" char="&#x00B1;">3.3</td>
<td align="char" valign="top" char="&#x00B1;">9</td>
<td align="char" valign="top" char="&#x00B1;">30.0</td>
<td align="char" valign="top" char="&#x00B1;">20</td>
<td align="char" valign="top" char="&#x00B1;">66.7</td>
<td align="char" valign="top" char="&#x00B1;">0</td>
<td align="char" valign="top" char="&#x00B1;">0</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">100</td>
<td align="char" valign="top" char="&#x00B1;">1.37</td>
<td align="char" valign="top" char="&#x00B1;">0.556</td>
<td align="char" valign="top" char="&#x00B1;">2</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Framing Design</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
<td align="char" valign="top" char="&#x00B1;">3.3</td>
<td align="char" valign="top" char="&#x00B1;">16</td>
<td align="char" valign="top" char="&#x00B1;">53.3</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">36.7</td>
<td align="char" valign="top" char="&#x00B1;">2</td>
<td align="char" valign="top" char="&#x00B1;">6.7</td>
<td align="char" valign="top" char="&#x00B1;">28</td>
<td align="char" valign="top" char="&#x00B1;">93.3</td>
<td align="char" valign="top" char="&#x00B1;">1.53</td>
<td align="char" valign="top" char="&#x00B1;">0.681</td>
<td align="char" valign="top" char="&#x00B1;">2</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Research investigation</td>
<td align="char" valign="top" char="&#x00B1;">3</td>
<td align="char" valign="top" char="&#x00B1;">10.0</td>
<td align="char" valign="top" char="&#x00B1;">10</td>
<td align="char" valign="top" char="&#x00B1;">33.3</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">36.7</td>
<td align="char" valign="top" char="&#x00B1;">6</td>
<td align="char" valign="top" char="&#x00B1;">20.0</td>
<td align="char" valign="top" char="&#x00B1;">24</td>
<td align="char" valign="top" char="&#x00B1;">80.0</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">0.922</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Generating Alternative solution</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
<td align="char" valign="top" char="&#x00B1;">3.3</td>
<td align="char" valign="top" char="&#x00B1;">3</td>
<td align="char" valign="top" char="&#x00B1;">10.0</td>
<td align="char" valign="top" char="&#x00B1;">3</td>
<td align="char" valign="top" char="&#x00B1;">10.0</td>
<td align="char" valign="top" char="&#x00B1;">23</td>
<td align="char" valign="top" char="&#x00B1;">76.7</td>
<td align="char" valign="top" char="&#x00B1;">7</td>
<td align="char" valign="top" char="&#x00B1;">23.3</td>
<td align="char" valign="top" char="&#x00B1;">0.40</td>
<td align="char" valign="top" char="&#x00B1;">0.814</td>
<td align="char" valign="top" char="&#x00B1;">0</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Choosing best solution</td>
<td align="char" valign="top" char="&#x00B1;">0</td>
<td align="char" valign="top" char="&#x00B1;">0</td>
<td align="char" valign="top" char="&#x00B1;">5</td>
<td align="char" valign="top" char="&#x00B1;">16.7</td>
<td align="char" valign="top" char="&#x00B1;">5</td>
<td align="char" valign="top" char="&#x00B1;">16.7</td>
<td align="char" valign="top" char="&#x00B1;">20</td>
<td align="char" valign="top" char="&#x00B1;">66.7</td>
<td align="char" valign="top" char="&#x00B1;">10</td>
<td align="char" valign="top" char="&#x00B1;">33.3</td>
<td align="char" valign="top" char="&#x00B1;">0.50</td>
<td align="char" valign="top" char="&#x00B1;">0.777</td>
<td align="char" valign="top" char="&#x00B1;">0</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Developmental work</td>
<td align="char" valign="top" char="&#x00B1;">0</td>
<td align="char" valign="top" char="&#x00B1;">0</td>
<td align="char" valign="top" char="&#x00B1;">12</td>
<td align="char" valign="top" char="&#x00B1;">40.0</td>
<td align="char" valign="top" char="&#x00B1;">4</td>
<td align="char" valign="top" char="&#x00B1;">13.3</td>
<td align="char" valign="top" char="&#x00B1;">14</td>
<td align="char" valign="top" char="&#x00B1;">46.7</td>
<td align="char" valign="top" char="&#x00B1;">16</td>
<td align="char" valign="top" char="&#x00B1;">53.3</td>
<td align="char" valign="top" char="&#x00B1;">0.93</td>
<td align="char" valign="top" char="&#x00B1;">0.944</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Modeling</td>
<td align="char" valign="top" char="&#x00B1;">2</td>
<td align="char" valign="top" char="&#x00B1;">6.7</td>
<td align="char" valign="top" char="&#x00B1;">6</td>
<td align="char" valign="top" char="&#x00B1;">20.0</td>
<td align="char" valign="top" char="&#x00B1;">4</td>
<td align="char" valign="top" char="&#x00B1;">13.3</td>
<td align="char" valign="top" char="&#x00B1;">18</td>
<td align="char" valign="top" char="&#x00B1;">60.0</td>
<td align="char" valign="top" char="&#x00B1;">12</td>
<td align="char" valign="top" char="&#x00B1;">40.0</td>
<td align="char" valign="top" char="&#x00B1;">0.73</td>
<td align="char" valign="top" char="&#x00B1;">1.015</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Testing and evaluating</td>
<td align="char" valign="top" char="&#x00B1;">3</td>
<td align="char" valign="top" char="&#x00B1;">10.0</td>
<td align="char" valign="top" char="&#x00B1;">12</td>
<td align="char" valign="top" char="&#x00B1;">40.0</td>
<td align="char" valign="top" char="&#x00B1;">6</td>
<td align="char" valign="top" char="&#x00B1;">20.0</td>
<td align="char" valign="top" char="&#x00B1;">9</td>
<td align="char" valign="top" char="&#x00B1;">30.0</td>
<td align="char" valign="top" char="&#x00B1;">21</td>
<td align="char" valign="top" char="&#x00B1;">70.0</td>
<td align="char" valign="top" char="&#x00B1;">1.30</td>
<td align="char" valign="top" char="&#x00B1;">1.022</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Redesign</td>
<td align="char" valign="top" char="&#x00B1;">4</td>
<td align="char" valign="top" char="&#x00B1;">13.3</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">36.7</td>
<td align="char" valign="top" char="&#x00B1;">6</td>
<td align="char" valign="top" char="&#x00B1;">20.0</td>
<td align="char" valign="top" char="&#x00B1;">9</td>
<td align="char" valign="top" char="&#x00B1;">30.0</td>
<td align="char" valign="top" char="&#x00B1;">21</td>
<td align="char" valign="top" char="&#x00B1;">70.0</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">1.061</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Total</td>
<td align="char" valign="top" char="&#x00B1;">15</td>
<td align="char" valign="top" char="&#x00B1;">5.5</td>
<td align="char" valign="top" char="&#x00B1;">84</td>
<td align="char" valign="top" char="&#x00B1;">31.1</td>
<td align="char" valign="top" char="&#x00B1;">70</td>
<td align="char" valign="top" char="&#x00B1;">25.9</td>
<td align="char" valign="top" char="&#x00B1;">101</td>
<td align="char" valign="top" char="&#x00B1;">37.4</td>
<td align="char" valign="top" char="&#x00B1;">169</td>
<td align="char" valign="top" char="&#x00B1;">62.6</td>
<td align="char" valign="top" char="&#x00B1;">1.05</td>
<td align="char" valign="top" char="&#x00B1;">0.48</td>
<td align="char" valign="top" char="&#x00B1;">1</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Regarding the level of inclusion, <xref rid="tab3" ref-type="table">Table 3</xref> indicates that the inclusion of all EDP skills varied between 1.53 and 0.40, where the practice <italic>Framing Design</italic> recorded the highest level (<italic>f</italic>&#x2009;=&#x2009;28, M&#x2009;=&#x2009;1.53) among all other EDP and practice <italic>Generating Alternative solution</italic> recorded the lowest level of inclusion (M&#x2009;=&#x2009;0.04). It should be noted that all EDP were not included directly or literally in the textbooks. Instead, the inclusion was deduced from the meaning of the sentences or the words. For example, the practice <italic>Identify the problem</italic> was not found in the student&#x2019;s textbook.</p>
<p>However, all activities included this practice as researchable questions. As an example, in the activity (Experiment: Development of Observation Skills) in the chemistry textbook, Chapter 1, Lesson 3, page 23, the EDP practice <italic>Identify the problem</italic>, was found as a question (<italic>Why are observational skills important in chemistry? Observations are often used to reach conclusions. The conclusion is an interpretation or clarification of the observation</italic>). This was the case in all the activities in the tenth-grade student&#x2019;s chemistry book. In another example, we used the meaning of the practice <italic>Modeling or Prototype</italic>, provided in the NGSS, to locate this practice in the chemistry textbook. This practice was not found in most of the activities in the targeted chemistry textbook, directly or indirectly.</p>
<p>In some activities, we deduced the inclusion of the practice <italic>Modeling</italic> only from the meaning. For example, in Chapter 4, Lesson 3, the activity (Experiment, observe a reaction that forms precipitate) in page 135, the practice <italic>Create a prototype that can be tested</italic> was included. It worked relatively well. It was written as: <italic>Write a symbolic chemical equation balanced for the reaction between NaOH and MgSO4. Note that most of the sulfate compounds are present as ions in aqueous solutions. The text asks students to: (1) write</italic> <italic>a balanced equation, which requires mathematical modeling.</italic> <italic>(2) Write the complete ionic equation for this reaction, and (3) Determine which ions are spectating, then write the final ionic equation for the reaction</italic>. The general mean of the inclusion of EDP in the activities of the tenth-grade chemistry book was 1.0481. This value indicates that the inclusion was at level one, which indicates that the inclusion did not meet a majority of the criteria. The level one of inclusion is categorized as novice or deficient.</p>
<p>According to <xref rid="tab4" ref-type="table">Table 4</xref>, the total mean of the students&#x2019; textbook was 1.0058, while the mean for the inclusion in the guidebook was 1.12. This may show that the inclusion of the EDP in the guide book is higher than in the student&#x2019;s book. However, the inclusion value for both the textbook and guidebook were at level one.</p>
<table-wrap position="float" id="tab4">
<label>Table 4</label>
<caption><p>Means and standard deviations on the extent of the inclusion of The 9 EDP in both the student&#x2019;s textbook and experiment guidebook.</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top" colspan="2">Book type</th>
<th align="center" valign="top">Identify the<break/>problem</th>
<th align="center" valign="top">Framing a<break/>design brief</th>
<th align="center" valign="top">Research and<break/>investigation</th>
<th align="center" valign="top">Generation of<break/>alternative solutions</th>
<th align="center" valign="top">Choosing<break/>the best solution</th>
<th align="center" valign="top">Developmental<break/>work</th>
<th align="center" valign="top">Modeling<break/>(prototyping)</th>
<th align="center" valign="top">Testing and evaluating</th>
<th align="center" valign="top">Redesign</th>
<th align="center" valign="top">Total</th>
</tr>
</thead>
<tbody>
<tr>
<td align="char" valign="top" char="." rowspan="3">Textbook</td>
<td align="char" valign="top" char="&#x00B1;">M</td>
<td align="char" valign="top" char="&#x00B1;">1.42</td>
<td align="char" valign="top" char="&#x00B1;">1.37</td>
<td align="char" valign="top" char="&#x00B1;">1.21</td>
<td align="char" valign="top" char="&#x00B1;">0.37</td>
<td align="char" valign="top" char="&#x00B1;">0.53</td>
<td align="char" valign="top" char="&#x00B1;">1.05</td>
<td align="char" valign="top" char="&#x00B1;">0.68</td>
<td align="char" valign="top" char="&#x00B1;">1.16</td>
<td align="char" valign="top" char="&#x00B1;">1.26</td>
<td align="char" valign="top" char="&#x00B1;">1.01</td>
</tr>
<tr>
<td align="char" valign="top" char="&#x00B1;">N</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
<td align="char" valign="top" char="&#x00B1;">19</td>
</tr>
<tr>
<td align="char" valign="top" char="&#x00B1;">SD</td>
<td align="char" valign="top" char="&#x00B1;">0.607</td>
<td align="char" valign="top" char="&#x00B1;">0.761</td>
<td align="char" valign="top" char="&#x00B1;">1.032</td>
<td align="char" valign="top" char="&#x00B1;">0.684</td>
<td align="char" valign="top" char="&#x00B1;">0.772</td>
<td align="char" valign="top" char="&#x00B1;">0.970</td>
<td align="char" valign="top" char="&#x00B1;">1.057</td>
<td align="char" valign="top" char="&#x00B1;">1.167</td>
<td align="char" valign="top" char="&#x00B1;">1.284</td>
<td align="char" valign="top" char="&#x00B1;">0.54240</td>
</tr>
<tr>
<td align="char" valign="top" char="." rowspan="3">Experiments&#x2019; guidebook</td>
<td align="char" valign="top" char="&#x00B1;">M</td>
<td align="char" valign="top" char="&#x00B1;">1.27</td>
<td align="char" valign="top" char="&#x00B1;">1.82</td>
<td align="char" valign="top" char="&#x00B1;">1.55</td>
<td align="char" valign="top" char="&#x00B1;">0.45</td>
<td align="char" valign="top" char="&#x00B1;">0.45</td>
<td align="char" valign="top" char="&#x00B1;">0.73</td>
<td align="char" valign="top" char="&#x00B1;">0.82</td>
<td align="char" valign="top" char="&#x00B1;">1.55</td>
<td align="char" valign="top" char="&#x00B1;">1.45</td>
<td align="char" valign="top" char="&#x00B1;">1.12</td>
</tr>
<tr>
<td align="char" valign="top" char="&#x00B1;">N</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
<td align="char" valign="top" char="&#x00B1;">11</td>
</tr>
<tr>
<td align="char" valign="top" char="&#x00B1;">SD</td>
<td align="char" valign="top" char="&#x00B1;">0.467</td>
<td align="char" valign="top" char="&#x00B1;">0.405</td>
<td align="char" valign="top" char="&#x00B1;">0.688</td>
<td align="char" valign="top" char="&#x00B1;">1.036</td>
<td align="char" valign="top" char="&#x00B1;">0.820</td>
<td align="char" valign="top" char="&#x00B1;">0.905</td>
<td align="char" valign="top" char="&#x00B1;">0.982</td>
<td align="char" valign="top" char="&#x00B1;">0.688</td>
<td align="char" valign="top" char="&#x00B1;">0.522</td>
<td align="char" valign="top" char="&#x00B1;">0.34589</td>
</tr>
<tr>
<td align="char" valign="top" char="." rowspan="3">Total</td>
<td align="char" valign="top" char="&#x00B1;">M</td>
<td align="char" valign="top" char="&#x00B1;">1.37</td>
<td align="char" valign="top" char="&#x00B1;">1.53</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">0.40</td>
<td align="char" valign="top" char="&#x00B1;">0.50</td>
<td align="char" valign="top" char="&#x00B1;">0.93</td>
<td align="char" valign="top" char="&#x00B1;">0.73</td>
<td align="char" valign="top" char="&#x00B1;">1.30</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">1.05</td>
</tr>
<tr>
<td align="char" valign="top" char="&#x00B1;">N</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
</tr>
<tr>
<td align="char" valign="top" char="&#x00B1;">SD</td>
<td align="char" valign="top" char="&#x00B1;">0.556</td>
<td align="char" valign="top" char="&#x00B1;">0.681</td>
<td align="char" valign="top" char="&#x00B1;">0.922</td>
<td align="char" valign="top" char="&#x00B1;">0.814</td>
<td align="char" valign="top" char="&#x00B1;">0.777</td>
<td align="char" valign="top" char="&#x00B1;">0.944</td>
<td align="char" valign="top" char="&#x00B1;">1.015</td>
<td align="char" valign="top" char="&#x00B1;">1.022</td>
<td align="char" valign="top" char="&#x00B1;">1.061</td>
<td align="char" valign="top" char="&#x00B1;">0.47651</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>In the experiments&#x2019; guidebook, the inclusion of some EDP was found to be direct, i.e., the word &#x201C;problem&#x201D; was found at the beginning of each experiment. However, the problems were stated as questions and not as statements as recommended by the NGSS. For example, in Experiment 4.1 &#x201C;<italic>single-replacement reaction</italic>,&#x201D; in page 33, the problem was stated in the form of two questions; &#x201C;<italic>The problem What elements replace others in simple substitution reactions? How can the results of these reactions be used to form a chain of chemical activity?</italic>&#x201C;. In another example, for the practice &#x201C;<italic>Redesign</italic>,&#x201D; <italic>in &#x201C;Experiment 5.2: Mole ratios&#x201D; page 43,</italic> the book asks students &#x201C;<italic>what you can do to improve the precision of the measurement of the mole ratio.&#x201D;</italic> In some cases, the book asked students to redesign the experiment or to design another experiment to achieve the same result.</p>
<sec id="sec12">
<title>Activity Types</title>
<p>As presented in <xref rid="tab5" ref-type="table">Table 5</xref>, there were five types of activities (the activities are labeled similar to the names used in the textbooks or the experiments&#x2019; guidebook) identified in the students&#x2019; textbook and only one type of activity (labeled experiment) in the experiment guidebook. The mean for all types of activities ranged between 2.00 and 0.00. The activities that represented most of the EDP skills were <italic>experiments</italic>, and the activities that least represented the EDP skills were <italic>problem labs</italic>. However, all the activities had level one EDP skills, which indicates that the inclusion of EDP skills in all activities was at the novice level.</p>
<table-wrap position="float" id="tab5">
<label>Table 5</label>
<caption><p>Means and standard deviations on the extent of the inclusion of the 9 EDP in all types of activities.</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top">Activity type</th>
<th align="center" valign="top">
<italic>N</italic></th>
<th align="center" valign="top">Identify the problem</th>
<th align="center" valign="top">Framing a design brief</th>
<th align="center" valign="top">Research and investigation</th>
<th align="center" valign="top">Generation of alternative solutions</th>
<th align="center" valign="top">Choosing the best solution</th>
<th align="center" valign="top">Developmental work</th>
<th align="center" valign="top">Modeling (prototyping)</th>
<th align="center" valign="top">Testing and evaluating</th>
<th align="center" valign="top">Redesign</th>
<th align="center" valign="top">Total</th>
</tr>
</thead>
<tbody>
<tr>
<td align="char" valign="top" char=".">Preliminary experiment:</td>
<td align="char" valign="top" char="&#x00B1;">5</td>
<td align="char" valign="top" char="&#x00B1;">1.80</td>
<td align="char" valign="top" char="&#x00B1;">1.40</td>
<td align="char" valign="top" char="&#x00B1;">1.40</td>
<td align="char" valign="top" char="&#x00B1;">1.00</td>
<td align="char" valign="top" char="&#x00B1;">1.00</td>
<td align="char" valign="top" char="&#x00B1;">1.80</td>
<td align="char" valign="top" char="&#x00B1;">1.00</td>
<td align="char" valign="top" char="&#x00B1;">1.40</td>
<td align="char" valign="top" char="&#x00B1;">2.00</td>
<td align="char" valign="top" char="&#x00B1;">1.42</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Experiment</td>
<td align="char" valign="top" char="&#x00B1;">16</td>
<td align="char" valign="top" char="&#x00B1;">1.19</td>
<td align="char" valign="top" char="&#x00B1;">1.62</td>
<td align="char" valign="top" char="&#x00B1;">1.44</td>
<td align="char" valign="top" char="&#x00B1;">0.31</td>
<td align="char" valign="top" char="&#x00B1;">0.38</td>
<td align="char" valign="top" char="&#x00B1;">0.63</td>
<td align="char" valign="top" char="&#x00B1;">0.63</td>
<td align="char" valign="top" char="&#x00B1;">1.38</td>
<td align="char" valign="top" char="&#x00B1;">1.13</td>
<td align="char" valign="top" char="&#x00B1;">0.97</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Chemistry analysis lab</td>
<td align="char" valign="top" char="&#x00B1;">6</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">1.50</td>
<td align="char" valign="top" char="&#x00B1;">0.83</td>
<td align="char" valign="top" char="&#x00B1;">0.33</td>
<td align="char" valign="top" char="&#x00B1;">0.67</td>
<td align="char" valign="top" char="&#x00B1;">1.17</td>
<td align="char" valign="top" char="&#x00B1;">1.00</td>
<td align="char" valign="top" char="&#x00B1;">1.17</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">1.04</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Problem lab</td>
<td align="char" valign="top" char="&#x00B1;">3</td>
<td align="char" valign="top" char="&#x00B1;">1.67</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">1.67</td>
<td align="char" valign="top" char="&#x00B1;">0.00</td>
<td align="char" valign="top" char="&#x00B1;">0.00</td>
<td align="char" valign="top" char="&#x00B1;">0.67</td>
<td align="char" valign="top" char="&#x00B1;">0.33</td>
<td align="char" valign="top" char="&#x00B1;">1.00</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">0.89</td>
</tr>
<tr>
<td align="char" valign="top" char=".">Total</td>
<td align="char" valign="top" char="&#x00B1;">30</td>
<td align="char" valign="top" char="&#x00B1;">1.47</td>
<td align="char" valign="top" char="&#x00B1;">1.48</td>
<td align="char" valign="top" char="&#x00B1;">1.33</td>
<td align="char" valign="top" char="&#x00B1;">0.41</td>
<td align="char" valign="top" char="&#x00B1;">0.51</td>
<td align="char" valign="top" char="&#x00B1;">1.04</td>
<td align="char" valign="top" char="&#x00B1;">0.74</td>
<td align="char" valign="top" char="&#x00B1;">1.25</td>
<td align="char" valign="top" char="&#x00B1;">1.42</td>
<td align="char" valign="top" char="&#x00B1;">1.07</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>The result shows that only the practice (<italic>Identify the problem</italic>) was explicitly included in the experiments&#x2019; guidebook, but not in the students&#x2019; textbook, even though this inclusion applies only to the title of the Process, not the specification. All other skills were implicitly included in either the students&#x2019; textbook or the experiments&#x2019; guidebooks. These findings suggest that there is a need to redesign the chemistry textbooks and guidebooks&#x2019; activities, in line with the NGSS standards and indicators (<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>). The activities should also be cognitively related to the rest of the text or the concepts, so that all the content support the development of EDP abilities in students.</p>
</sec>
</sec>
<sec id="sec13" sec-type="discussions">
<title>Discussion</title>
<p>The result revealed that the inclusion of all nine EDP abilities in the tenth-grade chemistry textbook and experiment guidebook was categorized as level one. The inclusion of the EDP abilities in the experiments&#x2019; guidebook was higher than that in the students&#x2019; book. Nonetheless, both books can be categorized as novice or deficient in the inclusion of EDP, which indicates that they did not meet a majority of the NGSS criteria. This result was expected since the chemistry textbook and experiment guidebook used in this study were translated McGraw-Hill series, which were prepared using the scientific inquiry approach, as suggested by NRC standards [<xref ref-type="bibr" rid="ref41">National Research Council (NRC), 1996</xref>, <xref ref-type="bibr" rid="ref42">2000</xref>, <xref ref-type="bibr" rid="ref43">2012</xref>], rather than the EDP suggested by the NGSS (<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>). The EDP is supposed to prepare students to cope with real life experiences after their graduation from general education (<xref ref-type="bibr" rid="ref4">Aldahmash and Alamri, 2020</xref>; <xref ref-type="bibr" rid="ref6">Aldahmash and Omar, 2021</xref>). The NGSS recommends using the textbooks to engage students in some form of scientific and engineering design process, such as those included in the analysis rubric. It should be noted that inquiry skills are also not well-represented in the current science (<xref ref-type="bibr" rid="ref5">Aldahmash et al., 2016</xref>). Analysis approaches for different science curricula have been made and revealed similar results (<xref ref-type="bibr" rid="ref24">Ferreira and Morais, 2014</xref>; <xref ref-type="bibr" rid="ref22">Elmas et al., 2020</xref>; <xref ref-type="bibr" rid="ref13">Bakken and Andersson-Bakken, 2021</xref>; <xref ref-type="bibr" rid="ref25">Ferreira and Saraiva, 2021</xref>; <xref ref-type="bibr" rid="ref65">Voj&#x00ED;&#x0159; and Rusek, 2022</xref>), mathematics (<xref ref-type="bibr" rid="ref4">Aldahmash and Alamri, 2020</xref>) or chemistry textbooks (<xref ref-type="bibr" rid="ref6">Aldahmash and Omar, 2021</xref>). Moreover, the inclusion of all inquiry skills was directed toward teacher-centered learning rather than student-centered learning, contrary to the NRC standards [<xref ref-type="bibr" rid="ref41">National Research Council (NRC), 1996</xref>, <xref ref-type="bibr" rid="ref42">2000</xref>, <xref ref-type="bibr" rid="ref43">2012</xref>].</p>
<p>Similarly, the results for the second sub-question (what types of activities could be most effective in promoting EDP among tenth-grade student?) asked in this study, revealed that none of the activities were effective in promoting EDP among the tenth-grade students. This was because all activities and experiments exhibited EDP at level one and all activities were novice in the inclusion of EDP skills. This implies that all activities in both the textbook and the experiments&#x2019; guidebook should be redesigned to provide students with opportunities to participate in real and authentic science activities. EDP enriched activities would allow students to learn science meaningfully and then acquire all the required skills. To further understand the connection between the activities and the texts the contents of the textbooks were also analyzed. The results indicated that there were some connections between the experiments in the experiments&#x2019; guidebook and the text in the student textbook. Unfortunately, there was no indication that the experiments were explorative or EDP enablers. This means that the experiments just serve as factors for confirming the concepts. This situation has to change for students to function as scientists and engineers who are able to cope with real life requirements after graduation. This necessity can be fulfilled by assuring that experiments and other activities used in science textbooks serve as EDP agents. This will also help students build their authentic science knowledge and their EDP abilities (<xref ref-type="bibr" rid="ref50">Nurtanto et al., 2020</xref>).</p>
<p>The result for the third sub-question (do the EDP skill levels determined in current tenth-grade chemistry textbooks and experiment guidebooks align with the NGSS&#x2019;s recommendations for engineering design), indicated that the inclusion of all eight EDP skills or abilities were not consistent with the NGSS that recommend constructing all activities to improve engineering designs abilities in students. The textbook and guidebook used in this study lacked EDP practices that could help students build skills required to be scientists or engineers (<xref ref-type="bibr" rid="ref60">Syukri et al., 2018</xref>; <xref ref-type="bibr" rid="ref36">Margot and Kettler, 2019</xref>; <xref ref-type="bibr" rid="ref50">Nurtanto et al., 2020</xref>).</p>
<p>The result also revealed that only the <italic>Identify the problem</italic> ability or talent was directly included in the experiment guide, but not in the students&#x2019; textbook, despite the fact that this inclusion only pertains to the skill&#x2019;s title and not the specification. The textbook for the students and the guidebook for the experiments did not expressly mention any of the other abilities. This indicates that the activities in chemistry textbooks and guidebooks must be modified to meet the NGSS performance expectations [<xref ref-type="bibr" rid="ref45">National Science Teachers Association (NSTA), 2016</xref>, <xref ref-type="bibr" rid="ref46">2018</xref>]. The activities should also be cognitively tied to the text or topics presented in the textbook, so that the entire book content can help students develop EDP skills. Therefore, redesigning the curriculum according to the K-12 science education standards provided by the NGSS is a necessary step to employ these engaging pedagogies. The EDP can be used by science educators to incorporate new projects and technologies into the classroom.</p>
</sec>
<sec id="sec14">
<title>Conclusion and Implementations</title>
<p>The findings revealed that the EDP skills were included at level one in a tenth-grade chemistry textbook. Level one of inclusion has been labeled as novice or deficient, indicating that the chemistry curriculum fails to meet the NGSS standards and indicators (<xref ref-type="bibr" rid="ref47">NGSS, Lead States, 2013</xref>). Several barriers prevent effective implementation of EDP in science education. Furthermore, the EDP is a new learning strategy, and many science educators are uninformed about its full potential. These issues, suggest a need for further review studies that explore the nature of the EDP integration in science education. The findings of these investigations can provide useful information that can be applied to improve learning and teaching materials used in STEM education. The EDP&#x2019;s strength is in the multiple benefits it provides to scientific education stakeholders, but the knowledge required to successfully implement it is still inadequate.</p>
<p>The overall results of the analysis confirmed that the inclusion of the EDP skills in chemistry textbooks does not fulfill current national and international science education standards, including NGSS. Thus, the study may lead to a better understanding of the complete and progressive character of the science curricula. It may pave the way for the improvement of the current science and chemistry curricula. Science instructors can use the EDP practices identified here to introduce new projects and technologies in their classroom. The study&#x2019;s findings imply that all activities in chemistry textbooks and the accompanying guidebook should be rewritten to encourage authentic science activities that address real-world concerns. Incorporating EDP abilities in chemistry textbooks and guides would also benefit students in a variety of ways. To begin, it may aid students in achieving greater test results in chemistry and other science disciplines (<xref ref-type="bibr" rid="ref66">Wendell et al., 2017</xref>). Another study suggested that EDP would increase students&#x2019; interest in science while also improving their scientific reasoning abilities (<xref ref-type="bibr" rid="ref59">Silk et al., 2009</xref>). The legitimacy of this process for curriculum creation stems from its long history of use in the EDP. For now, a curriculum that acknowledges the conceptual framework of EDP and is designed to the standards established by the NGSS is required to understand the best practices that can be employed to create engaging pedagogies in science education. Science instructors can use the EDP to introduce new projects and technologies into the classroom. The well-known design method related to such framework is also taught to engineering students as industrial design. As a result, additional study is required to determine the effects of incorporating the EDP into high school chemistry curricula to develop engineering design thinking and conceptual knowledge in high school students. Another future study that focuses on merging the EDP into STEM-based learning in order to strengthen the engineering design skills of chemistry and science instructors could be extremely beneficial.</p>
<sec id="sec15">
<title>Limitation of the Study</title>
<p>This analysis on targeted tenth grade chemistry textbook for the academic year 2021/2022 for The Engineering Designed Processes. However, for the future we would analyzed physics textbooks and activities or experiments guide books for the whole secondary stage.</p>
</sec>
</sec>
<sec id="sec16" sec-type="data-availability">
<title>Data Availability Statement</title>
<p>The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.</p>
</sec>
<sec id="sec17">
<title>Author Contributions</title>
<p>AA developed the theoretical formalism, performed the content analytic procedure, statistical analysis, and the numerical simulations, contributed to the final version of the manuscript, and supervised the project. YA proposed the idea, contributed to the development of the theoretical formalism and the final version of the manuscript, and supervised the project. All authors contributed to the article and approved the submitted version.</p>
</sec>
<sec id="sec18" sec-type="funding-information">
<title>Funding</title>
<p>This research was supported by the KSU Deanship for Scientific Research (Group #RG-1441-482).</p>
</sec>
<sec id="conf1" sec-type="COI-statement">
<title>Conflict of Interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec id="sec20" sec-type="disclaimer">
<title>Publisher&#x2019;s Note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
</body>
<back>
<ack>
<p>The Authors thank the Deanship for Scientific Research for their technical support.</p>
</ack>
<ref-list>
<title>References</title>
<ref id="ref4"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aldahmash</surname> <given-names>A. H.</given-names></name> <name><surname>Alamri</surname> <given-names>N. M.</given-names></name></person-group> (<year>2020</year>). <article-title>An analysis of the inclusion of mathematical discourse components in Arabic mathematical textbooks: the case of Saudi Arabia</article-title>. <source>Front. Psychol.</source> <volume>11</volume>:<fpage>534803</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fpsyg.2020.534803</pub-id>, PMID: <pub-id pub-id-type="pmid">33329173</pub-id></citation></ref>
<ref id="ref5"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aldahmash</surname> <given-names>A. H.</given-names></name> <name><surname>Mansour</surname> <given-names>N. S.</given-names></name> <name><surname>Alshamrani</surname> <given-names>S. M.</given-names></name> <name><surname>Almohi</surname> <given-names>S.</given-names></name></person-group> (<year>2016</year>). <article-title>An analysis of activities in Saudi Arabian middle school science textbooks and experiments&#x2019; guidebooks for the inclusion of essential features of inquiry</article-title>. <source>Res. Sci. Educ.</source> <volume>46</volume>, <fpage>879</fpage>&#x2013;<lpage>900</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11165-015-9485-7</pub-id></citation></ref>
<ref id="ref6"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aldahmash</surname> <given-names>A.</given-names></name> <name><surname>Omar</surname> <given-names>S.</given-names></name></person-group> (<year>2021</year>). <article-title>Analysis of activities included in Saudi Arabian chemistry textbooks for the inclusion of argumentation-driven inquiry skills</article-title>. <source>Stud. Educ. Eval.</source> <volume>68</volume>:<fpage>100968</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.stueduc.2020.100968</pub-id></citation></ref>
<ref id="ref7"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Al-Ghanem</surname> <given-names>G.</given-names></name></person-group> (<year>1999</year>). Reforming science education in saudi arabia: a conceptual framework for the design of coordinated and thematic science (CATS): module for grades 7&#x2013;9. PhD dissertation, West Virginia University.</citation></ref>
<ref id="ref8"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Alhomairi</surname> <given-names>A.</given-names></name></person-group> (<year>2018</year>). <article-title>A proposed perspective for developing science curriculum for the upper primary grades in accordance to Saudi Arabia's vision for 2030: an analytical and descriptive study according to Delphi method</article-title>. <source>Int. J. High. Educ.</source> <volume>7</volume>:<fpage>69</fpage>. doi: <pub-id pub-id-type="doi">10.5430/ijhe.v7n1p69</pub-id></citation></ref>
<ref id="ref9"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Almazroa</surname> <given-names>H.</given-names></name> <name><surname>Aloraini</surname> <given-names>A.</given-names></name></person-group> (<year>2012</year>). <source>Science and Math Teachers&#x2019; Perceptions of Professional Development Within the New Science Curriculum Implementation</source>. <publisher-name>A report for the Ministry of Education</publisher-name>, <publisher-loc>Saudi Arabia</publisher-loc>.</citation></ref>
<ref id="ref2"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Al Zubaidi</surname> <given-names>T. A.</given-names></name></person-group> (<year>2017</year>). Science Teaching: Between Reality and Hope. Arab Bureau of Education for the Gulf States. Available at: <ext-link xlink:href="http://www.abegs.org/" ext-link-type="uri">http://www.abegs.org/</ext-link> (Accessed December 6, 2021).</citation></ref>
<ref id="ref3"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Al Zughaybi</surname> <given-names>M.</given-names></name></person-group> (<year>2011</year>). &#x201C;The Reality of the Application of Mathematics and Science Project,&#x201D; in <italic>A Paper Presented to the Symposium of the Project of Developing Mathematics and Natural Sciences Curricula in Public Education between Theory and Practice</italic>. April 3, 2011; Faculty of Science, King Saud University.</citation></ref>
<ref id="ref10"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll1">American Association for the Advancement of Science (AAAS)</collab></person-group> (<year>1993</year>). <source>Project 2061: Benchmarks for Science Literacy</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Oxford University Press</publisher-name>.</citation></ref>
<ref id="ref11"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Aysun</surname> <given-names>A.</given-names></name> <name><surname>Ozlen</surname> <given-names>D.</given-names></name></person-group> (<year>2021</year>). <article-title>Supporting preschool Children&#x2019;s STEM learning with parent-involved early engineering education</article-title>. <source>Early Childhood Educ. J.</source> <volume>49</volume>, <fpage>607</fpage>&#x2013;<lpage>621</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10643-020-01100-1</pub-id></citation></ref>
<ref id="ref12"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Babbie</surname> <given-names>E.</given-names></name></person-group> (<year>2001</year>). <source>The Practice of Social Research</source> (<edition>9th Edition</edition>). <publisher-loc>Belmont, CA</publisher-loc>: <publisher-name>Wadsworth/Thomson Learning</publisher-name>.</citation></ref>
<ref id="ref13"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bakken</surname> <given-names>J.</given-names></name> <name><surname>Andersson-Bakken</surname> <given-names>E.</given-names></name></person-group> (<year>2021</year>). <article-title>The textbook task as a genre</article-title>. <source>J. Curric. Stud.</source> <volume>53</volume>, <fpage>729</fpage>&#x2013;<lpage>748</lpage>. doi: <pub-id pub-id-type="doi">10.1080/00220272.2021.1929499</pub-id></citation></ref>
<ref id="ref14"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Benz</surname> <given-names>R. W.</given-names></name></person-group> (<year>2019</year>). <article-title>Book review - science and engineering for grades 6-12: investigation and Design at the Center, 2019. National Academies of sciences, engineering, and medicine (US)</article-title>. <source>Ohio J. Sci.</source> <volume>119</volume>, <fpage>92</fpage>&#x2013;<lpage>93</lpage>. doi: <pub-id pub-id-type="doi">10.18061/ojs.v119i2.7326</pub-id></citation></ref>
<ref id="ref15"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bergqvist</surname> <given-names>A.</given-names></name> <name><surname>Chang Rundgren</surname> <given-names>S.</given-names></name></person-group> (<year>2017</year>). <article-title>The influence of textbooks on teachers&#x2019; knowledge of chemical bonding representations relative to students&#x2019; difficulties understanding</article-title>. <source>Res. Sci. Technol. Educ.</source> <volume>35</volume>, <fpage>215</fpage>&#x2013;<lpage>237</lpage>. doi: <pub-id pub-id-type="doi">10.1080/02635143.2017.1295934</pub-id></citation></ref>
<ref id="ref16"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Berland</surname> <given-names>L.</given-names></name> <name><surname>Steingut</surname> <given-names>R.</given-names></name> <name><surname>Ko</surname> <given-names>P.</given-names></name></person-group> (<year>2014</year>). <article-title>High school student perceptions of the utility of the engineering design process: creating opportunities to engage in engineering practices and apply math and science</article-title>. <source>J. Sci. Educ. Technol.</source> <volume>23</volume>, <fpage>705</fpage>&#x2013;<lpage>720</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10956-014-9498-4</pub-id></citation></ref>
<ref id="ref17"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Brand</surname> <given-names>B. R.</given-names></name></person-group> (<year>2020</year>). <article-title>Integrating science and engineering practices: outcomes form a collaborative professional development</article-title>. <source>Int. J. STEM Educ.</source> <volume>7</volume>:<fpage>13</fpage>. doi: <pub-id pub-id-type="doi">10.1186/s40594-020-00210-x</pub-id></citation></ref>
<ref id="ref18"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chiappetta</surname> <given-names>E. L.</given-names></name> <name><surname>Fillman</surname> <given-names>D. A.</given-names></name></person-group> (<year>2007</year>). <article-title>Analysis of five high school biology textbooks used in the United States for inclusion of the nature of science</article-title>. <source>Int. J. Sci. Educ.</source> <volume>29</volume>, <fpage>1847</fpage>&#x2013;<lpage>1868</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500690601159407</pub-id></citation></ref>
<ref id="ref19"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cohen</surname> <given-names>J.</given-names></name></person-group> (<year>1990</year>). <article-title>Things I have learned (so far)</article-title>. <source>Am. Psychol.</source> <volume>45</volume>, <fpage>1304</fpage>&#x2013;<lpage>1312</lpage>. doi: <pub-id pub-id-type="doi">10.1037/0003-066X.45.12.1304</pub-id></citation></ref>
<ref id="ref20"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Devetak</surname> <given-names>I.</given-names></name> <name><surname>Vogrinc</surname> <given-names>J.</given-names></name></person-group> (<year>2013</year>). &#x201C;<article-title>The criteria for evaluating the quality of the science textbooks</article-title>,&#x201D; in <source>Critical Analysis of Science Textbooks</source>. ed. <person-group person-group-type="editor"><name><surname>Khine</surname> <given-names>M.</given-names></name></person-group> (<publisher-loc>Springer, Dordrecht</publisher-loc>).</citation></ref>
<ref id="ref21"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dreyfus</surname> <given-names>A.</given-names></name></person-group> (<year>1992</year>). <article-title>Content analysis of school textbooks: the case of a technology-oriented curriculum</article-title>. <source>Int. J. Sci. Educ.</source> <volume>14</volume>, <fpage>3</fpage>&#x2013;<lpage>12</lpage>. doi: <pub-id pub-id-type="doi">10.1080/0950069920140102</pub-id></citation></ref>
<ref id="ref001"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dunne</surname> <given-names>J. P.</given-names></name> <name><surname>John</surname> <given-names>J. G.</given-names></name> <name><surname>Adcroft</surname> <given-names>A. J.</given-names></name> <name><surname>Griffies</surname> <given-names>S. M.</given-names></name> <name><surname>Hallberg</surname> <given-names>R. W.</given-names></name> <name><surname>Shevliakova</surname> <given-names>E.</given-names></name> <etal/></person-group>. (<year>2012</year>). <article-title>GFDL&#x2019;s ESM2global coupled climate-carbon Earth System Models Part I: physical formulation and baseline simulation characteristics</article-title>. <source>J. Clim.</source> <volume>25</volume>, <fpage>6646</fpage>&#x2013;<lpage>6665</lpage>.</citation></ref>
<ref id="ref22"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Elmas</surname> <given-names>R.</given-names></name> <name><surname>Rusek</surname> <given-names>M.</given-names></name> <name><surname>Lindell</surname> <given-names>A.</given-names></name> <name><surname>Nieminen</surname> <given-names>P.</given-names></name> <name><surname>Kasapoglu</surname> <given-names>K.</given-names></name> <name><surname>B&#x00ED;lek</surname> <given-names>M.</given-names></name></person-group> (<year>2020</year>). <article-title>The intellectual demands of the intended chemistry curriculum in Czechia, Finland, and Turkey: a comparative analysis based on the revised Bloom&#x2019;s taxonomy</article-title>. <source>Chem. Educ. Res. Pract.</source> <volume>21</volume>, <fpage>839</fpage>&#x2013;<lpage>851</lpage>. doi: <pub-id pub-id-type="doi">10.1039/D0RP00058B</pub-id></citation></ref>
<ref id="ref23"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>English</surname> <given-names>L. D.</given-names></name> <name><surname>King</surname> <given-names>D. T.</given-names></name></person-group> (<year>2015</year>). <article-title>STEM learning through engineering design: fourth-grade students&#x2019; investigations in aerospace</article-title>. <source>Int. J. STEM Educ.</source> <volume>2</volume>:<fpage>14</fpage>. doi: <pub-id pub-id-type="doi">10.1186/s40594-015-0027-7</pub-id></citation></ref>
<ref id="ref24"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ferreira</surname> <given-names>S.</given-names></name> <name><surname>Morais</surname> <given-names>A. M.</given-names></name></person-group> (<year>2014</year>). <article-title>Conceptual demand of practical work in science curricula: a methodological approach</article-title>. <source>Res. Sci. Educ.</source> <volume>44</volume>, <fpage>53</fpage>&#x2013;<lpage>80</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11165-013-9377-7</pub-id></citation></ref>
<ref id="ref25"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ferreira</surname> <given-names>S.</given-names></name> <name><surname>Saraiva</surname> <given-names>L.</given-names></name></person-group> (<year>2021</year>). <article-title>Complexity of practical work in Portuguese primary science textbooks</article-title>. <source>Investiga&#x00E7;&#x00F5;es em Ensino de Ci&#x00EA;ncias</source> <volume>26</volume>, <fpage>281</fpage>&#x2013;<lpage>297</lpage>. doi: <pub-id pub-id-type="doi">10.22600/1518-8795.ienci2021v26n3p281</pub-id></citation></ref>
<ref id="ref26"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ghalia</surname> <given-names>M. B.</given-names></name> <name><surname>Carlson</surname> <given-names>R.</given-names></name> <name><surname>Estrada</surname> <given-names>V.</given-names></name> <name><surname>Hug</surname> <given-names>H.</given-names></name> <name><surname>Ramos</surname> <given-names>J.</given-names></name></person-group> (<year>2016</year>). <article-title>Engaging K-12 teachers in engineering through a professional development program: implementation strategies, results and lessons learned</article-title>. <source>IEEE Front. Educ. Conf.</source> <volume>2016</volume>, <fpage>1</fpage>&#x2013;<lpage>7</lpage>. doi: <pub-id pub-id-type="doi">10.1109/FIE.2016.7757530</pub-id></citation></ref>
<ref id="ref27"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Green</surname> <given-names>B. N.</given-names></name> <name><surname>Johnson</surname> <given-names>C. D.</given-names></name> <name><surname>Adams</surname> <given-names>A.</given-names></name></person-group> (<year>2006</year>). <article-title>Writing narrative literature reviews for peer-reviewed journals: secrets of the trade</article-title>. <source>J. Chiropr. Med.</source> <volume>5</volume>, <fpage>101</fpage>&#x2013;<lpage>117</lpage>. doi: <pub-id pub-id-type="doi">10.1016/S0899-3467(07)60142-6</pub-id></citation></ref>
<ref id="ref28"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hafiz</surname> <given-names>N. R. M.</given-names></name> <name><surname>Ayop</surname> <given-names>S. K.</given-names></name></person-group> (<year>2019</year>). <article-title>Engineering design process in STEM education: a systematic review</article-title>. <source>Int. J. Acad. Res. Bus. Soc. Sci.</source> <volume>9</volume>, <fpage>676</fpage>&#x2013;<lpage>697</lpage>. </citation></ref>
<ref id="ref29"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hirsch</surname> <given-names>L. S.</given-names></name> <name><surname>Berliner-Heyman</surname> <given-names>S.</given-names></name> <name><surname>Cusack</surname> <given-names>J. L.</given-names></name></person-group> (<year>2017</year>). <article-title>Introducing middle school students to engineering principles and the engineering design process through an academic summer program</article-title>. <source>Int. J. Eng. Educ.</source> <volume>33</volume>, <fpage>398</fpage>&#x2013;<lpage>407</lpage>.</citation></ref>
<ref id="ref30"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hynes</surname> <given-names>M. M.</given-names></name></person-group> (<year>2012</year>). <article-title>Middle-school teachers&#x2019; understanding and teaching of the engineering design process: a look at subject matter and pedagogical content knowledge</article-title>. <source>Int. J. Technol. Des. Educ.</source> <volume>22</volume>, <fpage>345</fpage>&#x2013;<lpage>360</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10798-010-9142-4</pub-id></citation></ref>
<ref id="ref31"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kahveci</surname> <given-names>A.</given-names></name></person-group> (<year>2010</year>). <article-title>Quantitative analysis of science and chemistry textbooks for indicators of reform: a complementary perspective</article-title>. <source>Int. J. Sci. Educ.</source> <volume>32</volume>, <fpage>1495</fpage>&#x2013;<lpage>1519</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500690903127649</pub-id></citation></ref>
<ref id="ref32"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kremen</surname> <given-names>&#x0412;. &#x0413;.</given-names></name> <name><surname>Hrynevych</surname> <given-names>&#x041B;. &#x041C;.</given-names></name> <name><surname>Lugovyi</surname> <given-names>&#x0412;. &#x0406;.</given-names></name> <name><surname>Talanova</surname> <given-names>&#x0416;. &#x0412;.</given-names></name></person-group> (<year>2022</year>). <article-title>Quality of education and innovation-driven development: the new Ukrainian school reform in the context of global</article-title>. <source>Trends Sci. Innov.</source> <volume>18</volume>, <fpage>29</fpage>&#x2013;<lpage>43</lpage>. doi: <pub-id pub-id-type="doi">10.15407/scine18.01.029</pub-id></citation></ref>
<ref id="ref33"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Krippendorf</surname> <given-names>K.</given-names></name></person-group> (<year>2004</year>). <source>Content Analysis an Introduction to Its Methodology</source>. <edition>2nd Edn.</edition> <publisher-loc>Thousand Oaks, CA</publisher-loc>: <publisher-name>Sage Publications</publisher-name>.</citation></ref>
<ref id="ref34"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Lopez</surname> <given-names>A.</given-names></name> <name><surname>Garlick</surname> <given-names>J.</given-names></name></person-group> (<year>2017</year>). &#x201C;<article-title>Content analysis of middle school students&#x2019; argumentation Wilson in engineering [paper presentation].</article-title>&#x201D; in <italic>2017 ASEE Annual Conference &#x0026; Exposition Proceedings</italic>.</citation></ref>
<ref id="ref35"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Mangold</surname> <given-names>J.</given-names></name> <name><surname>Robinson</surname> <given-names>S. L.</given-names></name></person-group> (<year>2013</year>). The Engineering design process as a problem Solving and Learning tool in K-12 Classrooms.</citation></ref>
<ref id="ref36"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Margot</surname> <given-names>K. C.</given-names></name> <name><surname>Kettler</surname> <given-names>T.</given-names></name></person-group> (<year>2019</year>). <article-title>Teachers&#x2019; perception of STEM integration and education: a systematic literature review. International journal of STEM</article-title>. <source>Education</source> <volume>6</volume>, <fpage>1</fpage>&#x2013;<lpage>16</lpage>. doi: <pub-id pub-id-type="doi">10.1186/s40594-018-0151-2</pub-id></citation></ref>
<ref id="ref38"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>McHugh</surname> <given-names>M. L.</given-names></name></person-group> (<year>2012</year>). <article-title>Interrater reliability: the kappa statistic</article-title>. <source>Biochem. Med.</source> <volume>22</volume>, <fpage>276</fpage>&#x2013;<lpage>282</lpage>. doi: <pub-id pub-id-type="doi">10.11613/BM.2012.031</pub-id>, PMID: <pub-id pub-id-type="pmid">23092060</pub-id></citation></ref>
<ref id="ref39"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Mullis</surname> <given-names>I. V. S.</given-names></name> <name><surname>Martin</surname> <given-names>M. O.</given-names></name> <name><surname>Foy</surname> <given-names>P.</given-names></name> <name><surname>Kelly</surname> <given-names>D. L.</given-names></name> <name><surname>Fishbein</surname> <given-names>B.</given-names></name></person-group> (<year>2020</year>). TIMSS 2019 international results in mathematics and science. Available at: <ext-link xlink:href="https://timssandpirls.bc.edu/timss2019/international-results/" ext-link-type="uri">https://timssandpirls.bc.edu/timss2019/international-results/</ext-link> (Accessed January 20, 2021).</citation></ref>
<ref id="ref40"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll2">National Academy of Engineering and National Research Council (NAE and NRC)</collab></person-group> (<year>2009</year>). <source>Engineering in K-12 Education: Understanding the Status and Improving the Prospects</source>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>The National Academies Press</publisher-name>.</citation></ref>
<ref id="ref41"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll3">National Research Council (NRC)</collab></person-group> (<year>1996</year>). <source>National Science Education Standards</source>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>National Academy Press</publisher-name>.</citation></ref>
<ref id="ref42"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll4">National Research Council (NRC)</collab></person-group> (<year>2000</year>). <source>Inquiry and the National Science Education Standards: a Guide for Teaching and Learning</source>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>National Academy Press</publisher-name>.</citation></ref>
<ref id="ref003"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll50">National Research Council (NRC)</collab></person-group> (<year>2007</year>). <source>The Nutrient Requirements of Horses 6th Edn.</source> <publisher-loc>Washington DC</publisher-loc>: <publisher-name>National Academy Press</publisher-name>.</citation></ref>
<ref id="ref004"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll51">National Research Council (NRC)</collab></person-group> (<year>2009</year>). <source>Learning Science in Informal Environments: People, Places, and Pursuits.</source> <publisher-loc>Washington DC</publisher-loc>: <publisher-name>The National Academy Press</publisher-name>.</citation></ref>
<ref id="ref43"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll5">National Research Council (NRC)</collab></person-group> (<year>2012</year>). <source>A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and core Ideas</source>. <publisher-loc>Arlington, Virginia</publisher-loc>: <publisher-name>National Academy Press</publisher-name>.</citation></ref>
<ref id="ref44"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll6">National Science Foundation (NSF)</collab></person-group> (<year>2010</year>). <source>Preparing the Next Generation of STEM Innovators: Identifying and Developing Our Nation&#x2019;s Human Capital</source>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>NSF</publisher-name>.</citation></ref>
<ref id="ref002"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll70">National Science Teachers Association (NSTA)</collab></person-group> (<year>2014</year>). <source>Early Childhood Science Education</source>. <publisher-loc>Arlington, VA</publisher-loc>: <publisher-name>NSTA</publisher-name>.</citation></ref>
<ref id="ref45"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll7">National Science Teachers Association (NSTA)</collab></person-group> (<year>2016</year>). <source>The Next Generation Science Standards</source>. <publisher-loc>Arlington, VA</publisher-loc>: <publisher-name>NSTA</publisher-name>.</citation></ref>
<ref id="ref46"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll8">National Science Teachers Association (NSTA)</collab></person-group> (<year>2018</year>). <source>Transitioning From Scientific Inquiry to Three-Dimensional Teaching and Learning</source>. <publisher-loc>Arlington, VA</publisher-loc>: <publisher-name>NSTA</publisher-name>.</citation></ref>
<ref id="ref47"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll9">NGSS, Lead States</collab></person-group> (<year>2013</year>). <source>Next Generation Science Standards: For States, by States</source>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>National Academies Press</publisher-name>.</citation></ref>
<ref id="ref48"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nguyen</surname> <given-names>D.</given-names></name> <name><surname>Liakata</surname> <given-names>M.</given-names></name> <name><surname>Dedeo</surname> <given-names>S.</given-names></name> <name><surname>Eisenstein</surname> <given-names>J.</given-names></name> <name><surname>Mimno</surname> <given-names>D.</given-names></name> <name><surname>Tromble</surname> <given-names>R.</given-names></name> <etal/></person-group>. (<year>2020</year>). <article-title>How we do things with words: analyzing text as social and cultural data</article-title>. <source>Front. Artif. Intell.</source> <volume>3</volume>:<fpage>62</fpage>. doi: <pub-id pub-id-type="doi">10.3389/frai.2020.00062</pub-id></citation></ref>
<ref id="ref49"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Niaz</surname> <given-names>M.</given-names></name> <name><surname>Maza</surname> <given-names>A.</given-names></name></person-group> (<year>2011</year>). <source>Nature of Science in General Chemistry Textbooks</source>. <publisher-loc>Dordrecht, Heidelberg, London &#x0026; New York</publisher-loc>: <publisher-name>Springer</publisher-name>.</citation></ref>
<ref id="ref50"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nurtanto</surname> <given-names>M.</given-names></name> <name><surname>Pardjono</surname> <given-names>P.</given-names></name> <name><surname>Widarto</surname> <given-names>W.</given-names></name> <name><surname>Ramdani</surname> <given-names>S. D.</given-names></name></person-group> (<year>2020</year>). <article-title>The effect of STEM-EDP in professional learning on automotive engineering competence in vocational high school</article-title>. <source>J. Educ. Gift. Young Sci.</source> <volume>8</volume>, <fpage>633</fpage>&#x2013;<lpage>649</lpage>. doi: <pub-id pub-id-type="doi">10.17478/jegys.645047</pub-id></citation></ref>
<ref id="ref52"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pieper</surname> <given-names>J.</given-names></name> <name><surname>Mentzer</surname> <given-names>N.</given-names></name></person-group> (<year>2013</year>). <article-title>High school students&#x2019; use of paper-based and internet-based information sources in the engineering design process</article-title>. <source>J. Technol. Educ.</source> <volume>24</volume>, <fpage>78</fpage>&#x2013;<lpage>95</lpage>. doi: <pub-id pub-id-type="doi">10.21061/jte.v24i2.a.6</pub-id></citation></ref>
<ref id="ref53"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Purzer</surname> <given-names>S.</given-names></name> <name><surname>Goldstein</surname> <given-names>M.</given-names></name> <name><surname>Adams</surname> <given-names>R.</given-names></name> <name><surname>Xie</surname> <given-names>C.</given-names></name> <name><surname>Nourian</surname> <given-names>S.</given-names></name></person-group> (<year>2015</year>). <article-title>An exploratory study of informed engineering design behaviors associated with scientific explanations</article-title>. <source>Int. J. STEM Educ.</source> <volume>2</volume>, <fpage>1</fpage>&#x2013;<lpage>12</lpage>. doi: <pub-id pub-id-type="doi">10.1186/s40594-015-0019-7</pub-id></citation></ref>
<ref id="ref54"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Purzer</surname> <given-names>S.</given-names></name> <name><surname>Quintana-Cifuentes</surname> <given-names>J. P.</given-names></name></person-group> (<year>2019</year>). <article-title>Integrating engineering in K-12 science education: spelling out the pedagogical, epistemological, and methodological arguments</article-title>. <source>Discip. Interdscip. Sci. Educ. Res.</source> <volume>1</volume>:<fpage>13</fpage>. doi: <pub-id pub-id-type="doi">10.1186/s43031-019-0010-0</pub-id></citation></ref>
<ref id="ref55"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rintaningrum</surname> <given-names>R.</given-names></name></person-group> (<year>2008</year>). <article-title>Constructivist theory and a teaching and learning cycle in english</article-title>. <source>J. Soc. Human.</source> <volume>1</volume>, <fpage>1</fpage>&#x2013;<lpage>12</lpage>. doi: <pub-id pub-id-type="doi">10.12962/j24433527.v1i1.682</pub-id></citation></ref>
<ref id="ref56"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ross</surname> <given-names>J. M.</given-names></name> <name><surname>Peterman</surname> <given-names>K.</given-names></name> <name><surname>Daugherty</surname> <given-names>J. L.</given-names></name> <name><surname>Custer</surname> <given-names>R. L.</given-names></name></person-group> (<year>2018</year>). <article-title>An engineering innovation tool: providing science educators a picture of engineering in their classroom</article-title>. <source>J. STEM Educ. Innov. Res.</source> <volume>19</volume>, <fpage>13</fpage>&#x2013;<lpage>18</lpage>.</citation></ref>
<ref id="ref57"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rubel</surname> <given-names>L.</given-names></name> <name><surname>Mccloskey</surname> <given-names>A.</given-names></name></person-group> (<year>2021</year>). <article-title>Contextualization of mathematics: which and whose world?</article-title> <source>Educ. Stud. Math.</source> <volume>107</volume>, <fpage>383</fpage>&#x2013;<lpage>404</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10649-021-10041-4</pub-id></citation></ref>
<ref id="ref58"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rubinstein</surname> <given-names>R. A.</given-names></name> <name><surname>Brown</surname> <given-names>R. T.</given-names></name></person-group> (<year>1984</year>). <article-title>An evaluation of the validity of the diagnostic category of attention deficit disorder</article-title>. <source>Am. J. Orthopsychiatry</source> <volume>54</volume>, <fpage>398</fpage>&#x2013;<lpage>414</lpage>. doi: <pub-id pub-id-type="doi">10.1111/j.1939-0025.1984.tb01506.x</pub-id>, PMID: <pub-id pub-id-type="pmid">6465292</pub-id></citation></ref>
<ref id="ref59"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Silk</surname> <given-names>E. M.</given-names></name> <name><surname>Schunn</surname> <given-names>C. D.</given-names></name> <name><surname>Strand Cary</surname> <given-names>M.</given-names></name></person-group> (<year>2009</year>). <article-title>The impact of an engineering design curriculum engineering design curriculum on science reasoning science reasoning in an urban setting urban setting</article-title>. <source>J. Sci. Educ. Technol.</source> <volume>18</volume>, <fpage>209</fpage>&#x2013;<lpage>223</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10956-009-9144-8</pub-id></citation></ref>
<ref id="ref60"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Syukri</surname> <given-names>M.</given-names></name> <name><surname>Soewarno</surname> <given-names>S.</given-names></name> <name><surname>Halim</surname> <given-names>L.</given-names></name> <name><surname>Mohtar</surname> <given-names>L.</given-names></name></person-group> (<year>2018</year>). <article-title>The impact of engineering design process in teaching and learning to enhance students&#x2019; science problem-solving skills</article-title>. <source>J. Pendidik. IPA Indon.</source> <volume>7</volume>, <fpage>66</fpage>&#x2013;<lpage>75</lpage>. doi: <pub-id pub-id-type="doi">10.15294/jpii.v7i1.12297</pub-id></citation></ref>
<ref id="ref61"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Tamir</surname> <given-names>P.</given-names></name></person-group> (<year>1991</year>). &#x201C;<article-title>Practical work in school science: an analysis of current practice</article-title>,&#x201D; in <source>Practical Science</source>. ed. <person-group person-group-type="editor"><name><surname>Woolnough</surname> <given-names>B.</given-names></name></person-group> (<publisher-loc>Milton Keynes</publisher-loc>: <publisher-name>Open University</publisher-name>), <fpage>13</fpage>&#x2013;<lpage>20</lpage>.</citation></ref>
<ref id="ref62"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tamir</surname> <given-names>P.</given-names></name> <name><surname>Pilar-Garcia</surname> <given-names>M.</given-names></name></person-group> (<year>1992</year>). <article-title>Characteristics of laboratory exercises included in science textbooks in Catalonia (Spain)</article-title>. <source>Int. J. Sci. Educ.</source> <volume>14</volume>, <fpage>381</fpage>&#x2013;<lpage>392</lpage>. doi: <pub-id pub-id-type="doi">10.1080/0950069920140402</pub-id></citation></ref>
<ref id="ref63"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Ubuz</surname> <given-names>B.</given-names></name></person-group> (<year>2019</year>). &#x201C;<article-title>Opportunities to Engage in STEM Practices: Technology and design course</article-title>.&#x201D; in <source>Eleventh Congress of the European Society for Research in Mathematics Education</source>. <publisher-loc>Utrecht, Netherlands</publisher-loc>: <publisher-name>Utrecht University</publisher-name>.</citation></ref>
<ref id="ref64"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vesterinen</surname> <given-names>V.</given-names></name> <name><surname>Aksela</surname> <given-names>M.</given-names></name> <name><surname>Lavonen</surname> <given-names>J.</given-names></name></person-group> (<year>2013</year>). <article-title>Quantitative analysis of representations of nature of science in Nordic upper secondary school textbooks using framework upper secondary school textbooks using framework of analysis based on philosophy of chemistry</article-title>. <source>Sci. Educ.</source> <volume>22</volume>, <fpage>1839</fpage>&#x2013;<lpage>1855</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11191-011-9400-1</pub-id></citation></ref>
<ref id="ref65"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Voj&#x00ED;&#x0159;</surname> <given-names>K.</given-names></name> <name><surname>Rusek</surname> <given-names>M.</given-names></name></person-group> (<year>2022</year>). <article-title>Opportunities for learning: analysis of Czech lower-secondary chemistry textbook tasks</article-title>. <source>Acta Chimica Slovenica</source> <volume>69</volume>, <fpage>359</fpage>&#x2013;<lpage>370</lpage>.</citation></ref>
<ref id="ref66"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wendell</surname> <given-names>K.</given-names></name> <name><surname>Wright</surname> <given-names>C.</given-names></name> <name><surname>Paugh</surname> <given-names>P.</given-names></name></person-group> (<year>2017</year>). <article-title>Reflective decision-making in elementary Students' engineering design</article-title>. <source>J. Eng. Educ.</source> <volume>106</volume>, <fpage>356</fpage>&#x2013;<lpage>397</lpage>. doi: <pub-id pub-id-type="doi">10.1002/jee.20173</pub-id></citation></ref>
<ref id="ref67"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>White</surname> <given-names>M. D.</given-names></name> <name><surname>Marsh</surname> <given-names>E. E.</given-names></name></person-group> (<year>2006</year>). <article-title>Content analysis: a flexible methodology</article-title>. <source>Libr. Trends</source> <volume>55</volume>, <fpage>22</fpage>&#x2013;<lpage>45</lpage>. doi: <pub-id pub-id-type="doi">10.1353/lib.2006.0053</pub-id></citation></ref>
</ref-list>
</back>
</article>