<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="research-article" dtd-version="2.3" xml:lang="EN">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Educ.</journal-id>
<journal-title>Frontiers in Education</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Educ.</abbrev-journal-title>
<issn pub-type="epub">2504-284X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/feduc.2023.1187237</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Education</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Students&#x2019; systems thinking while modeling a dynamic ecological system</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Lankers</surname>
<given-names>Annika</given-names>
</name>
<xref rid="c001" ref-type="corresp"><sup>&#x002A;</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2021631/overview"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Timm</surname>
<given-names>Justin</given-names>
</name>
<uri xlink:href="https://loop.frontiersin.org/people/2283250/overview"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Schmiemann</surname>
<given-names>Philipp</given-names>
</name>
<uri xlink:href="https://loop.frontiersin.org/people/2251847/overview"/>
</contrib>
</contrib-group>
<aff><institution>Biology Education Research and Learning Lab, University of Duisburg-Essen</institution>, <addr-line>Essen</addr-line>, <country>Germany</country></aff>
<author-notes>
<fn id="fn0001" fn-type="edited-by"><p>Edited by: Moritz Krell, University of Kiel, Germany</p></fn>
<fn id="fn0002" fn-type="edited-by"><p>Reviewed by: Elon Langbeheim, Ben-Gurion University of the Negev, Israel; Anna Beniermann, Humboldt University of Berlin, Germany</p></fn>
<corresp id="c001">&#x002A;Correspondence: Annika Lankers, <email>annika.lankers@uni-due.de</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>26</day>
<month>07</month>
<year>2023</year>
</pub-date>
<pub-date pub-type="collection">
<year>2023</year>
</pub-date>
<volume>8</volume>
<elocation-id>1187237</elocation-id>
<history>
<date date-type="received">
<day>15</day>
<month>03</month>
<year>2023</year>
</date>
<date date-type="accepted">
<day>30</day>
<month>05</month>
<year>2023</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x00A9; 2023 Lankers, Timm and Schmiemann.</copyright-statement>
<copyright-year>2023</copyright-year>
<copyright-holder>Lankers, Timm and Schmiemann</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract>
<p>The world is facing global ecological changes, making it essential to prepare the future generation with the necessary understanding to effectively navigate and address complex tasks. Previous research has shown that both systems thinking and scientific modeling are particularly relevant in investigating the comprehensive understanding of such complex phenomena. However, there has been little research on the interrelation between systems thinking and scientific modeling. To address this research gap, we conducted a thinking-aloud study with nine high school students by confronting them with a simulation of a dynamic ecological system. Our qualitative content analysis of the students&#x2019; statements indicates an interrelation between systems thinking and scientific modeling. The students infrequently show systems thinking during the exploration, whereas when developing a graphical model, the students are involved in identifying the system organization and analyzing the system behavior. When predicting future system states, students engage in modeling the system evolution. Furthermore, during verbalizing analogies and experiences, students refer to the system organization and behavior, whereas in mental modeling, students additionally model the system evolution. These results illustrate a central difference between the two perspectives. Thus, scientific modeling focuses on students&#x2019; activities during their understanding process, while systems thinking addresses students&#x2019; analysis of systems and their properties. While the phenomenon exploration may not require systems thinking, pattern recognition and model development are frequently associated with identifying the system organization and analyzing the system behavior. Systems thinking must also be applied when deriving possible future system states by modeling the system evolution, an activity that is closely related to the prediction phase of scientific modeling. Interestingly, in our study, the students also demonstrated the modeling of system evolution in their mental modeling. In conclusion, a complementary consideration of systems thinking and scientific modeling affords a deeper understanding of students&#x2019; cognitive processes in dealing with complex phenomena.</p>
</abstract>
<kwd-group>
<kwd>complex phenomena</kwd>
<kwd>systems thinking</kwd>
<kwd>scientific modeling</kwd>
<kwd>dynamic ecological system</kwd>
<kwd>alien species</kwd>
<kwd>science education</kwd>
<kwd>secondary students</kwd>
<kwd>qualitative content analysis</kwd>
</kwd-group>
<counts>
<fig-count count="2"/>
<table-count count="3"/>
<equation-count count="0"/>
<ref-count count="105"/>
<page-count count="14"/>
<word-count count="11936"/>
</counts>
<custom-meta-wrap>
<custom-meta>
<meta-name>section-at-acceptance</meta-name>
<meta-value>STEM Education</meta-value>
</custom-meta>
</custom-meta-wrap>
</article-meta>
</front>
<body>
<sec id="sec1" sec-type="intro">
<label>1.</label>
<title>Introduction</title>
<p>Attention to global changes and its consequences have increased immensely over the last decades. The world is facing global social and environmental issues. One of them is biotic homogenization including several losing and a few winning species (<xref ref-type="bibr" rid="ref62">McKinney and Lockwood, 1999</xref>). Of particular concern are invasive alien species, as they are a significant threat not only to biodiversity (<xref ref-type="bibr" rid="ref20">Early et al., 2016</xref>). Invasions negatively affect ecosystem stability and have a direct impact on habitat design, ecosystem performance, agricultural success, the spread of diseases, and human well-being (<xref ref-type="bibr" rid="ref41">IPBES, 2019</xref>). To deal with such global challenges, a comprehensive understanding of biodiversity and ecosystem functioning is critical for constructive policy and management (<xref ref-type="bibr" rid="ref20">Early et al., 2016</xref>). Future generations must be prepared for these tasks by developing a holistic understanding of global changes facing the world community (<xref ref-type="bibr" rid="ref70">NGSS Lead States, 2013</xref>; <xref ref-type="bibr" rid="ref54">Ladrera et al., 2020</xref>). This issue persists, as the findings of the Programme for International Student Assessment (PISA) reveal that students with little knowledge and skills have na&#x00EF;ve and unrealistic ideas about the self-regulation of the world, whereas scientifically educated students are able to make realistic assessments based on ethical reasoning (<xref ref-type="bibr" rid="ref71">OECD, 2022</xref>). Hence, science education aims to provide students with a set of essential skills and subject-specific concepts that can be used as a basis for dealing with decision-making and problem solving in issues of global concern as well as everyday life challenges (<xref ref-type="bibr" rid="ref68">National Research Council, 2012</xref>; <xref ref-type="bibr" rid="ref70">NGSS Lead States, 2013</xref>). Systems thinking and scientific modeling are cognitive activities discussed in the context of a deeper understanding of such complex phenomena in dealing with personal, social, and global challenges (<xref ref-type="bibr" rid="ref42">Jackson et al., 1991</xref>; <xref ref-type="bibr" rid="ref38">Hogan, 2000</xref>; <xref ref-type="bibr" rid="ref31">Gotwals and Songer, 2010</xref>; <xref ref-type="bibr" rid="ref70">NGSS Lead States, 2013</xref>; <xref ref-type="bibr" rid="ref7">Bielik et al., 2021</xref>). Both are directly related to understanding complex phenomena, but from different perspectives: Scientific modeling focuses on the process of accessing complex phenomena and primarily addresses how students proceed, whereas systems thinking primarily addresses how students analyze complex systems. Even though it seems plausible that there might be a close relationship between systems thinking and scientific modeling, the connection between the two perspectives has not yet been investigated systematically. This study investigates how students integrate systems thinking and scientific modeling when instructed to develop a model that explains the observed population dynamics of an ecological system invaded by an alien species.</p>
<sec id="sec2">
<label>1.1.</label>
<title>Systems thinking</title>
<p>Systems thinking is a cognitive skill that provides the theoretical concepts for explaining and predicting natural phenomena (<xref ref-type="bibr" rid="ref96">Verhoeff et al., 2018</xref>). In recent years, systems thinking has been examined for a variety of systems in manifold contexts, such as ecology (<xref ref-type="bibr" rid="ref79">Riess and Mischo, 2010</xref>; <xref ref-type="bibr" rid="ref23">Eliam and Reisfeld, 2017</xref>; <xref ref-type="bibr" rid="ref19">Dor-Haim et al., 2022</xref>), human physiology (<xref ref-type="bibr" rid="ref57">Liu and Hmelo-Silver, 2009</xref>; <xref ref-type="bibr" rid="ref92">Tripto et al., 2018</xref>; <xref ref-type="bibr" rid="ref46">Kiesewetter and Schmiemann, 2022</xref>), cell biology (<xref ref-type="bibr" rid="ref97">Verhoeff et al., 2008</xref>), and biogeochemistry (<xref ref-type="bibr" rid="ref4">Ben Zvi Assaraf and Orion, 2005</xref>; <xref ref-type="bibr" rid="ref56">Lee et al., 2019</xref>; <xref ref-type="bibr" rid="ref90">Torkar and Korfiatis, 2022</xref>). All (biological) systems share universal characteristics. For example, systems are composed of system elements that together form a complex whole. Ecosystems encompass the entirety of abiotic and biotic factors within a specific habitat. Despite their interconnectedness, (eco-)systems are open to their environment, allowing for the exchange of matter, energy, and information. Generally, the openness of systems is closely related to the dynamics of systems. Systems with constant composition of elements may exhibit temporary behavioral stability. However, over the long term, system dynamics can be significantly affected by internal factors or external disturbances, such as the presence of alien species as a destabilizing element (<xref ref-type="bibr" rid="ref1">Andrade et al., 2015</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>). Understanding the complexity of system dynamics can be challenging (<xref ref-type="bibr" rid="ref98">Wellmanns and Schmiemann, 2022</xref>). This especially applies to ecological systems undergoing degradation, which results in significant modifications to the ecosystem, leading to noticeable changes in species composition and overall dynamics (<xref ref-type="bibr" rid="ref105">Zimmerman and Cuddington, 2007</xref>).</p>
<p>Several frameworks of systems thinking exist alongside each other (<xref ref-type="bibr" rid="ref4">Ben Zvi Assaraf and Orion, 2005</xref>, <xref ref-type="bibr" rid="ref5">2010</xref>; <xref ref-type="bibr" rid="ref37">Hmelo-Silver et al., 2007</xref>; <xref ref-type="bibr" rid="ref97">Verhoeff et al., 2008</xref>; <xref ref-type="bibr" rid="ref79">Riess and Mischo, 2010</xref>; <xref ref-type="bibr" rid="ref87">Sommer and L&#x00FC;cken, 2010</xref>; <xref ref-type="bibr" rid="ref9">Boersma et al., 2011</xref>; <xref ref-type="bibr" rid="ref40">Hokayem and Gotwals, 2016</xref>; <xref ref-type="bibr" rid="ref35">Hmelo-Silver et al., 2017</xref>; <xref ref-type="bibr" rid="ref85">Snapir et al., 2017</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>; <xref ref-type="bibr" rid="ref28">Gilissen et al., 2020</xref>; <xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>; <xref ref-type="bibr" rid="ref67">Momsen et al., 2022</xref>). However, many of these frameworks incorporate similar cognitive skills, even if individual frameworks assume different relationships between these skills or define additional skills (e.g., <xref ref-type="bibr" rid="ref4">Ben Zvi Assaraf and Orion, 2005</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>). Most systems thinking frameworks consider in some way the following three skills: identifying the system organization, analyzing the system behavior, and modeling the system evolution. The first skill, identifying the system organization, covers recognizing the structure and boundaries of the system as well as identifying the elements and their relationships (<xref ref-type="bibr" rid="ref4">Ben Zvi Assaraf and Orion, 2005</xref>; <xref ref-type="bibr" rid="ref25">Evagorou et al., 2009</xref>; <xref ref-type="bibr" rid="ref79">Riess and Mischo, 2010</xref>; <xref ref-type="bibr" rid="ref87">Sommer and L&#x00FC;cken, 2010</xref>; <xref ref-type="bibr" rid="ref36">Hmelo-Silver et al., 2015</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>; <xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>). The second skill includes analyzing the system behavior by capturing system interactions and dynamics as well as emerging patterns (<xref ref-type="bibr" rid="ref4">Ben Zvi Assaraf and Orion, 2005</xref>; <xref ref-type="bibr" rid="ref37">Hmelo-Silver et al., 2007</xref>; <xref ref-type="bibr" rid="ref87">Sommer and L&#x00FC;cken, 2010</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>; <xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>). The third skill incorporates modeling the system evolution and thus the development of predictions (<xref ref-type="bibr" rid="ref4">Ben Zvi Assaraf and Orion, 2005</xref>; <xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>), possibly to derive regulatory measures (<xref ref-type="bibr" rid="ref79">Riess and Mischo, 2010</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>). Overall, the naming of the individual skills is consistent with the terminology of <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref>, who conducted research in the field of ecology, and thus based on the work of <xref ref-type="bibr" rid="ref63">Mehren et al. (2018)</xref>, who originally derived the three listed skills theoretically and empirically tested their model in the context of geography.</p>
<p>In addition to describing individual skills that constitute systems thinking, it is possible and reasonable to consider the complexity of systems thinking (<xref ref-type="bibr" rid="ref40">Hokayem and Gotwals, 2016</xref>; <xref ref-type="bibr" rid="ref56">Lee et al., 2019</xref>). Systems can be big or small, and relationships and behaviors may be simple or complex. Notably, there is a considerable difference between the degree of linkage in relationships (<xref ref-type="bibr" rid="ref43">Jin et al., 2019</xref>; <xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>, <xref ref-type="bibr" rid="ref60">2022b</xref>). Simple relations immediately connect two elements. Whereas relationships that involve a minimum of three elements form either a linear chain or a complex linkage through a central node. This can be well exemplified with a food web and the representation thereof. A statement mentioning rabbits eating grass would name a simple relationship (<xref ref-type="bibr" rid="ref60">Mambrey et al., 2022b</xref>). This direct predator&#x2013;prey relationship would be graphically represented by an arrow from prey to predator within a food web. In addition, a typical linear linked relationship is a food chain between a carnivore and a producer. The populations of grass, rabbit, and predator form a food chain as the elements are linked by two arrows in an unbranched manner. In contrast, we define all relationships as complex linked that do not represent mere chains but contain at least one branch (<xref ref-type="bibr" rid="ref58">Mambrey et al., 2022a</xref>). The most simple example of a complex relationship is food competition between two herbivores such as rabbit and goose for grass, as the system is branched starting from the grass as central node. Previous research in the context of ecology shows that students often prefer to refer to low linked relationships including a small number of elements as an increased number of elements in a system strongly affects the difficulty of comprehension (<xref ref-type="bibr" rid="ref89">Strogatz, 2001</xref>; <xref ref-type="bibr" rid="ref31">Gotwals and Songer, 2010</xref>; <xref ref-type="bibr" rid="ref39">Hokayem, 2016</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>; <xref ref-type="bibr" rid="ref60">Mambrey et al., 2022b</xref>).</p>
</sec>
<sec id="sec3">
<label>1.2.</label>
<title>Modeling in systems thinking research</title>
<p>Various authors imply that systems thinking is connected in some way with models or modeling (<xref ref-type="bibr" rid="ref3">Ben Zvi Assaraf and Knippels, 2022</xref>; <xref ref-type="bibr" rid="ref6">Bielik et al., 2023</xref>). However, very different aspects of modeling have been emphasized in the systems thinking literature: Various frameworks explicitly include modeling as a systems thinking skill, which, however, can describe different skills. For example, modeling can involve inferring statements about future system states but also understanding and/or creating system models (e.g., <xref ref-type="bibr" rid="ref87">Sommer and L&#x00FC;cken, 2010</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>; <xref ref-type="bibr" rid="ref28">Gilissen et al., 2020</xref>; <xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>; <xref ref-type="bibr" rid="ref88">Streiling et al., 2021</xref>). From a different perspective, models are viewed as representations of complex systems (<xref ref-type="bibr" rid="ref88">Streiling et al., 2021</xref>; <xref ref-type="bibr" rid="ref46">Kiesewetter and Schmiemann, 2022</xref>; <xref ref-type="bibr" rid="ref60">Mambrey et al., 2022b</xref>). In addition, they are the starting point for developing predictions (<xref ref-type="bibr" rid="ref28">Gilissen et al., 2020</xref>). External representations of students&#x2019; mental models are evaluated as expressions of their systems thinking and thus serve as an indicator of skill (<xref ref-type="bibr" rid="ref4">Ben Zvi Assaraf and Orion, 2005</xref>; <xref ref-type="bibr" rid="ref11">Brandst&#x00E4;dter et al., 2012</xref>; <xref ref-type="bibr" rid="ref91">Tripto et al., 2013</xref>). From yet another perspective, modeling can be used as an activity and strategy to improve systems thinking (<xref ref-type="bibr" rid="ref97">Verhoeff et al., 2008</xref>; <xref ref-type="bibr" rid="ref101">Wilson et al., 2020</xref>). Modeling thus can have a variety of meanings in the context of systems thinking; in this study we focus on the cognitive activity of scientific modeling.</p>
</sec>
<sec id="sec4">
<label>1.3.</label>
<title>Scientific modeling</title>
<p>Scientific modeling is a practice aimed at examining and explaining complex phenomena or systems (<xref ref-type="bibr" rid="ref70">NGSS Lead States, 2013</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>; <xref ref-type="bibr" rid="ref83">Schwarz et al., 2022</xref>). It describes the process of gaining access to phenomena that are not completely accessible without further examination (<xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>). Accordingly, the emerging models are not just products but useful tools for investigating phenomena (<xref ref-type="bibr" rid="ref32">Gouvea and Passmore, 2017</xref>). The process of scientific modeling entails activities of model development, evaluation, and revision (<xref ref-type="bibr" rid="ref84">Schwarz and White, 2005</xref>), which can occur individually or continuously and in cycles (<xref ref-type="bibr" rid="ref72">Oh and Oh, 2011</xref>; <xref ref-type="bibr" rid="ref13">Campbell et al., 2013</xref>; <xref ref-type="bibr" rid="ref27">Gilbert and Justi, 2016</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>). The frameworks of <xref ref-type="bibr" rid="ref17">Clement (1989)</xref>, <xref ref-type="bibr" rid="ref49">Krell and Kr&#x00FC;ger (2016)</xref> and <xref ref-type="bibr" rid="ref45">Justi and Gilbert (2002)</xref> provide a process-oriented conceptualization of modeling activities. Usually, the process of scientific modeling starts with the perception of a real-world phenomenon (<xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>), based on which and with the activation of analogies and experiences, a mental model arises (<xref ref-type="bibr" rid="ref93">Upmeier zu Belzen et al., 2019</xref>), which may be labeled mental (<xref ref-type="bibr" rid="ref69">Nersessian, 2008</xref>), initial (<xref ref-type="bibr" rid="ref17">Clement, 1989</xref>), or proto-model (<xref ref-type="bibr" rid="ref27">Gilbert and Justi, 2016</xref>). The mental model is expressed or externalized in any kind of representation (<xref ref-type="bibr" rid="ref45">Justi and Gilbert, 2002</xref>; <xref ref-type="bibr" rid="ref27">Gilbert and Justi, 2016</xref>; <xref ref-type="bibr" rid="ref93">Upmeier zu Belzen et al., 2019</xref>), which is subsequently evaluated for consistency (<xref ref-type="bibr" rid="ref17">Clement, 1989</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>). Once this evaluation is completed, an epistemological change of perspective may take place: A shift toward the epistemic usage of the model can be accomplished by working beyond the developmental perspective of scientific modeling (<xref ref-type="bibr" rid="ref32">Gouvea and Passmore, 2017</xref>). This perspective of model utilization is characterized by the performance of empirical tests of model-based predictions (<xref ref-type="bibr" rid="ref17">Clement, 1989</xref>; <xref ref-type="bibr" rid="ref45">Justi and Gilbert, 2002</xref>; <xref ref-type="bibr" rid="ref26">Giere et al., 2006</xref>; <xref ref-type="bibr" rid="ref27">Gilbert and Justi, 2016</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>; <xref ref-type="bibr" rid="ref93">Upmeier zu Belzen et al., 2019</xref>). Therefore, comparing obtained data and predictions provides information on model fit (<xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>). If the predictions turn out to be true, the model has proven its validity, at least within the scope of the predictions, and can fulfill its epistemic aim (<xref ref-type="bibr" rid="ref45">Justi and Gilbert, 2002</xref>; <xref ref-type="bibr" rid="ref32">Gouvea and Passmore, 2017</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>). If the data do not bear out the prediction, the model needs to be revised or rejected (<xref ref-type="bibr" rid="ref17">Clement, 1989</xref>; <xref ref-type="bibr" rid="ref45">Justi and Gilbert, 2002</xref>). This leads back to the process of model development, thereby revealing the cyclic nature of scientific modeling (<xref ref-type="bibr" rid="ref84">Schwarz and White, 2005</xref>; <xref ref-type="bibr" rid="ref72">Oh and Oh, 2011</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>). However, recent research has demonstrated that this cyclic nature of scientific modeling does not necessarily describe students&#x2019; real procedures, as they tend to omit or jump between phases (<xref ref-type="bibr" rid="ref48">Knuuttila and Boon, 2011</xref>; <xref ref-type="bibr" rid="ref64">Meister and Upmeier zu Belzen, 2020</xref>; <xref ref-type="bibr" rid="ref30">G&#x00F6;hner and Krell, 2022</xref>). To investigate the modeling process depicted, <xref ref-type="bibr" rid="ref50">Krell et al. (2019)</xref> developed a coding manual that allows to analyze student activities during scientific modeling.</p>
</sec>
<sec id="sec5">
<label>1.4.</label>
<title>Simulation of complex systems</title>
<p>Exploring complex phenomena, like ecosystems in biology, is an essential part of science education. However, ecosystems are characterized by an effectively infinite number of variables that students cannot capture easily. A common tool to investigate the understanding processes of complex systems is simulations (<xref ref-type="bibr" rid="ref21">Eilam, 2012</xref>; <xref ref-type="bibr" rid="ref34">Grotzer et al., 2015</xref>; <xref ref-type="bibr" rid="ref36">Hmelo-Silver et al., 2015</xref>; <xref ref-type="bibr" rid="ref103">Yoon et al., 2018</xref>; <xref ref-type="bibr" rid="ref88">Streiling et al., 2021</xref>). Simulations give access to complex systems, for example, by making the structure and dynamics observable. Due to this, simulations are employed in various studies, although there are major differences regarding the options for manipulation and the research aim. Often, these system simulations are used within interventional settings in order to improve systems thinking (<xref ref-type="bibr" rid="ref44">Jordan et al., 2013</xref>; <xref ref-type="bibr" rid="ref35">Hmelo-Silver et al., 2017</xref>; <xref ref-type="bibr" rid="ref102">Yoon et al., 2017</xref>; <xref ref-type="bibr" rid="ref100">Wiebe et al., 2019</xref>; <xref ref-type="bibr" rid="ref22">Eilam and Omar, 2022</xref>; <xref ref-type="bibr" rid="ref77">Rachmatullah and Wiebe, 2022</xref>; <xref ref-type="bibr" rid="ref90">Torkar and Korfiatis, 2022</xref>). A slightly different approach is using simulations to foster systems thinking from a modeling perspective (<xref ref-type="bibr" rid="ref18">Damelin et al., 2017</xref>; <xref ref-type="bibr" rid="ref8">Bielik et al., 2022</xref>; <xref ref-type="bibr" rid="ref10">Bowers et al., 2022</xref>). However, digital tools such as simulations can also be used as representations of complex systems within the context of non-interventional studies as integral parts of assessments (<xref ref-type="bibr" rid="ref80">Sauv&#x00E9; et al., 2007</xref>).</p>
</sec>
</sec>
<sec id="sec6">
<label>2.</label>
<title>Research questions and aim of the study</title>
<p>Given the current biodiversity crisis, understanding complex phenomena is of paramount importance. Systems thinking and scientific modeling are two different but likely interdependent perspectives on understanding complex phenomena. In the context of this study, we sought to examine the relationship between these concurrent perspectives more closely. Therefore, we address the following research questions:</p>
<list list-type="order">
<list-item><p>Which systems thinking skills do students commonly use while analyzing a simulation of a dynamic ecological system?</p></list-item>
<list-item><p>Which activities of scientific modeling do students commonly use while analyzing a simulation of a dynamic ecological system?</p></list-item>
<list-item><p>How do students&#x2019; systems thinking and scientific modeling activities interrelate in terms of co-occurrence during their analysis of a simulation of a dynamic ecological system?</p></list-item>
</list>
</sec>
<sec id="sec7" sec-type="methods">
<label>3.</label>
<title>Method</title>
<sec id="sec8">
<label>3.1.</label>
<title>Simulation of a dynamic ecological system</title>
<p>Following these considerations, to engage students with a dynamic ecological phenomenon we have developed an interactive data-driven web application in R (<xref ref-type="bibr" rid="ref76">R Core Team, 2022</xref>) utilizing the packages Shiny (<xref ref-type="bibr" rid="ref15">Chang et al., 2022</xref>) and ggplot2 (<xref ref-type="bibr" rid="ref99">Wickham, 2016</xref>). The web application illustrates the dynamics of an ecological system comprising six different species, including pasture grass, red clover, European rabbit, graylag goose, Canada goose, and red kite through an adjustable line graph showing the relative population sizes and thus their changes over time (see <xref rid="fig1" ref-type="fig">Figure 1</xref>). Decisive for the dynamic of the simulation are multiple predator&#x2013;prey relationships, whereas the individual population sizes are estimated based on coupled differential equations by the R package deSolve (<xref ref-type="bibr" rid="ref86">Soetaert et al., 2010</xref>). Initially, the simulation includes populations of pasture grass, red clover, European rabbit, graylag goose, and red kite, with the Canada goose being introduced later as an alien species. In consequence of the invasion, the dynamics of the ecological system are clearly disturbed and demonstrate emergent behavior in the extinction of the red kite. The participants&#x2019; task was to infer the structure of the underlying food web based on the observed population dynamics. They were explicitly requested to develop a graphical model that explains the observed population trends by stating underlying relationships and how the species behave in relation to each other. Beyond that, the students did not receive any additional information about the species or instructions on how to proceed. The development of a graphical model was intended to ensure that the students keep their own modeling process in mind instead of reasoning on what they immediately see (<xref ref-type="bibr" rid="ref78">Rellensmann et al., 2017</xref>). Within the web application, the participants could select the displayed time period and species at any time and as often as they wanted. Additionally, they could switch between one line graph for all species and individual graphs for each species (see <xref rid="fig1" ref-type="fig">Figure 1</xref>) to make it easier for the participants to examine individual relationships if desired. The correct solution to the students&#x2019; task is represented in the food web in <xref rid="fig2" ref-type="fig">Figure 2</xref>, which served as basis for programming the simulation.</p>
<fig position="float" id="fig1">
<label>Figure 1</label>
<caption>
<p>The simulation analyzed by the students.</p>
</caption>
<graphic xlink:href="feduc-08-1187237-g001.tif"/>
</fig>
<fig position="float" id="fig2">
<label>Figure 2</label>
<caption>
<p>Food web underlying the simulation. It is not shown to the students.</p>
</caption>
<graphic xlink:href="feduc-08-1187237-g002.tif"/>
</fig>
</sec>
<sec id="sec9">
<label>3.2.</label>
<title>Data collection</title>
<p>To gain insights into the thinking process of the participants, we asked them to think aloud while exploring the given phenomenon (<xref ref-type="bibr" rid="ref24">Ericsson and Simon, 1993</xref>; <xref ref-type="bibr" rid="ref104">Zhang and Zhang, 2019</xref>). Students were tested individually, which avoids social influences and takes into account that the given scientific modeling category system (<xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>) has been proved for individual problem solving, unlike group proceedings (<xref ref-type="bibr" rid="ref65">M&#x00E9;trailler et al., 2008</xref>). The think-aloud protocol was explained to the participants and practiced on a simple and non-specialized topic before the actual survey started (<xref ref-type="bibr" rid="ref73">Oh and Wildemuth, 2016</xref>). The students were reminded to think aloud and graphically develop their model during the survey, if necessary. For later evaluation, students&#x2019; complete work was recorded (audio, screen recording). In addition, students were requested to develop their models on a tablet computer utilizing a digital whiteboard that enabled them to freely draw and illustrate their ideas (<xref ref-type="bibr" rid="ref66">Microsoft Corporation, 2020</xref>). Since there was no time limitation, students could end the survey at any point they wanted, whether or not they judged the task to be completed. The mean processing time was <italic>M</italic> =&#x2009;43.3 (<italic>SD</italic> =&#x2009;17.7) minutes. Before further analysis, the collected data were processed through transcription of the audio files. Transcripts were supplemented by screenshots of the recorded graphical modeling process, where the audio transcript alone appeared to be ambiguous.</p>
</sec>
<sec id="sec10">
<label>3.3.</label>
<title>Sample</title>
<p>Nine German upper secondary level school students participated in our study (6 female and 3 male participants). Students&#x2019; age ranged from 15 to 18 (<italic>M</italic> =&#x2009;16.7) years. One-third of the students attended an advanced biology course, while the others took a basic biology course. Unfortunately, we do not have detailed records of individual students&#x2019; current and completed biology teaching topics. However, all students were enrolled in the qualification phase for the <italic>Abitur</italic>, the highest secondary school diploma and higher education entry certificate in Germany. Ecology is one of three main subject areas in biology courses during this qualification phase. Courses at both levels include concepts regarding anthropogenic impacts on ecosystems and their consequences as well as concepts dealing with biodiversity and population dynamics (<xref ref-type="bibr" rid="ref52">Kultusministerkonferenz, 2004</xref>). In addition, ecology is also part of the educational standards for the intermediate secondary level. Therefore, all students should in any case be familiar with basic concepts in the realm of ecology, for example regarding ecosystem organization, dynamic processes within ecosystems, and predator&#x2013;prey relationships (<xref ref-type="bibr" rid="ref53">Kultusministerkonferenz, 2005</xref>).</p>
</sec>
<sec id="sec11">
<label>3.4.</label>
<title>Coding scheme</title>
<p>To evaluate the students&#x2019; statements, we performed a qualitative content analysis (<xref ref-type="bibr" rid="ref82">Schreier, 2012</xref>; <xref ref-type="bibr" rid="ref61">Mayring, 2014</xref>) using MAXQDA 2022 (<xref ref-type="bibr" rid="ref94">VERBI Software, 2022</xref>), which is a well-established method for analyzing qualitative data whose value has already been proven in the context of modeling processes and systems thinking approaches (<xref ref-type="bibr" rid="ref29">Gogolin and Kr&#x00FC;ger, 2018</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>; <xref ref-type="bibr" rid="ref98">Wellmanns and Schmiemann, 2022</xref>; <xref ref-type="bibr" rid="ref58">Mambrey et al., 2022a</xref>). For the qualitative content analysis, we developed coding schemes based on the model of systems thinking by <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref> and the activities of scientific modeling described by <xref ref-type="bibr" rid="ref50">Krell et al. (2019)</xref>. In order to evaluate students&#x2019; systems thinking in detail, we differentiated the levels of students&#x2019; systems thinking described by <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref>. We added multiple subcategories using a deductive approach to account for nuances in students&#x2019; systems thinking (<xref ref-type="bibr" rid="ref82">Schreier, 2012</xref>): We broke the level of direct relations down by distinguishing unidirectional relations and bidirectional relations. The level of linear relations was divided into linear relations bridging one element and linear relations bridging multiple elements. Similarly, we distinguished complex relations bridging one element from complex relations bridging multiple elements (see <xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>). An overview of the coding scheme is given in <xref rid="tab1" ref-type="table">Table 1</xref>. For the coding of students&#x2019; scientific modeling activities, we adjusted the descriptions of the coding framework of <xref ref-type="bibr" rid="ref50">Krell et al. (2019)</xref> to match our ecological context (<xref ref-type="bibr" rid="ref82">Schreier, 2012</xref>). In contrast to our procedure for systems thinking, we used an inductive approach to extend the coding framework for scientific modeling by new categories. This inductive extension was necessary for the analysis of scientific modeling activities as the existing framework of <xref ref-type="bibr" rid="ref50">Krell et al. (2019)</xref>, unlike the systems thinking framework of <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref>, required adaption for the presented multilayered phenomenon in an ecological context. We disclose the final coding frame for students&#x2019; scientific modeling in the results section, as it is involved in answering the first research question. New categories are listed in <xref rid="tab2" ref-type="table">Table 2</xref> and marked with an asterisk.</p>
<table-wrap position="float" id="tab1">
<label>Table 1</label>
<caption>
<p>Overview of students&#x2019; levels of systems thinking in the context of a dynamic ecological system.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="middle" rowspan="2">Level</th>
<th align="center" valign="middle" colspan="3">Systems Thinking Skill</th>
</tr>
<tr>
<th align="center" valign="middle">Identifying System Organization</th>
<th align="center" valign="middle">Analyzing System Behavior</th>
<th align="center" valign="middle">Modeling System Evolution</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="middle" colspan="4">Reasoning Based on Simple Relations</td>
</tr>
<tr>
<td align="left" valign="top">Unidirectional Relation</td>
<td align="center" valign="top">130</td>
<td align="center" valign="top">66</td>
<td align="center" valign="top">6</td>
</tr>
<tr>
<td align="left" valign="top">Bidirectional Relation</td>
<td align="center" valign="top">40</td>
<td align="center" valign="top">31</td>
<td align="center" valign="top">0</td>
</tr>
<tr>
<td align="left" valign="top" colspan="4">Reasoning Based on Linear Relations</td>
</tr>
<tr>
<td align="left" valign="top">Linear Relation Bridging One Element</td>
<td align="center" valign="top">0</td>
<td align="center" valign="top">6</td>
<td align="center" valign="top">1</td>
</tr>
<tr>
<td align="left" valign="top">Linear Relation Bridging Multiple Elements</td>
<td align="center" valign="top">0</td>
<td align="center" valign="top">0</td>
<td align="center" valign="top">0</td>
</tr>
<tr>
<td align="left" valign="top" colspan="4">Reasoning Based on Complex Relations</td>
</tr>
<tr>
<td align="left" valign="top">Complex Relation Bridging One Element</td>
<td align="center" valign="top">87</td>
<td align="center" valign="top">30</td>
<td align="center" valign="top">7</td>
</tr>
<tr>
<td align="left" valign="top">Complex Relation Bridging Multiple Elements</td>
<td align="center" valign="top">4</td>
<td align="center" valign="top">5</td>
<td align="center" valign="top">1</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p>The levels, based on the framework of <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref>, are listed in differentiated subcategories. Numbers represent the frequency of occurrence. Examples of student statements are available as <xref rid="SM1" ref-type="supplementary-material">Supplementary material</xref>.</p>
</table-wrap-foot>
</table-wrap>
<table-wrap position="float" id="tab2">
<label>Table 2</label>
<caption>
<p>Overview of students&#x2019; activities in the process of scientific modeling in the context of a dynamic ecological system.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top">Phase</th>
<th align="center" valign="top">ID</th>
<th align="left" valign="top">Activity</th>
<th align="left" valign="top">Category with Explanation</th>
<th align="center" valign="top">Frequency</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="top" rowspan="8">Exploration</td>
<td align="center" valign="top">1</td>
<td align="left" valign="top">Perception of Phenomena</td>
<td align="left" valign="top">Student describes observed behavior of the ecological system as spontaneous or incomprehensible.</td>
<td align="center" valign="top">53</td>
</tr>
<tr>
<td align="center" valign="top">2</td>
<td align="left" valign="top">Arbitrary Exploration of the System</td>
<td align="left" valign="top">Student arbitrarily chooses a time interval and/or species and/or display mode.</td>
<td align="center" valign="top">78</td>
</tr>
<tr>
<td align="center" valign="top">3a</td>
<td align="left" valign="top">Description of Observations</td>
<td align="left" valign="top">Student describes observed behavior of the ecological system without recognizing any pattern.</td>
<td align="center" valign="top">145</td>
</tr>
<tr>
<td align="center" valign="top">3b<sup>&#x002A;</sup></td>
<td align="left" valign="top">Notes on Observations</td>
<td align="left" valign="top">Student takes notes on the observed behavior of the ecological system (see 3a).</td>
<td align="center" valign="top">17</td>
</tr>
<tr>
<td align="center" valign="top">4</td>
<td align="left" valign="top">Purposive Exploration of the System</td>
<td align="left" valign="top">Student selects a time interval and/or species and/or display mode to detect a pattern in the behavior of the ecological system.</td>
<td align="center" valign="top">153</td>
</tr>
<tr>
<td align="center" valign="top">5a</td>
<td align="left" valign="top">Recognition of Pattern</td>
<td align="left" valign="top">Student recognizes or confirms a pattern (see 4).</td>
<td align="center" valign="top">130</td>
</tr>
<tr>
<td align="center" valign="top">5b<sup>&#x002A;</sup></td>
<td align="left" valign="top">Negation of Pattern</td>
<td align="left" valign="top">Student negates presence or detection of a pattern (see 4).</td>
<td align="center" valign="top">21</td>
</tr>
<tr>
<td align="center" valign="top">5c<sup>&#x002A;</sup></td>
<td align="left" valign="top">Notes on Pattern</td>
<td align="left" valign="top">Student takes notes on the recognized, confirmed, or negated pattern.</td>
<td align="center" valign="top">27</td>
</tr>
<tr>
<td align="left" valign="top">&#x2013;</td>
<td align="center" valign="top">6a</td>
<td align="left" valign="top">Activation of Analogies and Experiences</td>
<td align="left" valign="top">Student verbalizes ideas about the organization and/or behavior of the ecological system and/or thinks about these ideas aloud, based in each case on analogies and/or experiences.</td>
<td align="center" valign="top">64</td>
</tr>
<tr>
<td align="left" valign="top">&#x2013;</td>
<td align="center" valign="top">6b<sup>&#x002A;</sup></td>
<td align="left" valign="top">Mental Modeling</td>
<td align="left" valign="top">Student verbalizes ideas about the organization and/or behavior of the ecological system and/or thinks about these ideas aloud, based in each case on observations of the ecological system.</td>
<td align="center" valign="top">103</td>
</tr>
<tr>
<td align="left" valign="top" rowspan="8">Development</td>
<td align="center" valign="top">7</td>
<td align="left" valign="top">Analogy and Experience-Based Model Development</td>
<td align="left" valign="top">Student develops graphical model of the ecological system based on analogies and/or experiences (see 6a).</td>
<td align="center" valign="top">8</td>
</tr>
<tr>
<td align="center" valign="top">8</td>
<td align="left" valign="top">Design-Based Model Development</td>
<td align="left" valign="top">Student develops model to improve internal logic, functionality, or aesthetics.</td>
<td align="center" valign="top">15</td>
</tr>
<tr>
<td align="center" valign="top">9</td>
<td align="left" valign="top">Observation-Based Model Development</td>
<td align="left" valign="top">Student develops graphical model retrospectively based on observations of the ecological system.</td>
<td align="center" valign="top">85</td>
</tr>
<tr>
<td align="center" valign="top">10</td>
<td align="left" valign="top">Observation-Based Model Rejection</td>
<td align="left" valign="top">Student rejects model retrospectively based on observations of the ecological system.</td>
<td align="center" valign="top">0</td>
</tr>
<tr>
<td align="center" valign="top">11</td>
<td align="left" valign="top">Design-Based Model Evaluation</td>
<td align="left" valign="top">Student evaluates model regarding internal logic, functionality, or aesthetics.</td>
<td align="center" valign="top">1</td>
</tr>
<tr>
<td align="center" valign="top">12</td>
<td align="left" valign="top">Observation-Based Model Evaluation</td>
<td align="left" valign="top">Student compares model with observations of the ecological system.</td>
<td align="center" valign="top">12</td>
</tr>
<tr>
<td align="center" valign="top">13a</td>
<td align="left" valign="top">Confirmation of Model Consistency</td>
<td align="left" valign="top">Student confirms model consistency (see 11) and/or consistency between model and observations (see 12).</td>
<td align="center" valign="top">29</td>
</tr>
<tr>
<td align="center" valign="top">13b<sup>&#x002A;</sup></td>
<td align="left" valign="top">Denial of Model Consistency</td>
<td align="left" valign="top">Student identifies model inconsistency (see 11) and/or inconsistency between model and observations (see 12).</td>
<td align="center" valign="top">4</td>
</tr>
<tr>
<td align="left" valign="top" rowspan="7">Prediction</td>
<td align="center" valign="top">14</td>
<td align="left" valign="top">Generation of Predictions</td>
<td align="left" valign="top">Student deduces hypothesis based on the model about the organization and/or behavior of the ecological system.</td>
<td align="center" valign="top">13</td>
</tr>
<tr>
<td align="center" valign="top">15</td>
<td align="left" valign="top">Purposive Manipulation of the System</td>
<td align="left" valign="top">Student selects a time interval and/or species and/or display mode based on a hypothesis (see 14).</td>
<td align="center" valign="top">18</td>
</tr>
<tr>
<td align="center" valign="top">16</td>
<td align="left" valign="top">Confirmation of Predictions</td>
<td align="left" valign="top">Student confirms hypothesis (see 14) by observing a pattern within the behavior of the ecological system (usually through 15).</td>
<td align="center" valign="top">10</td>
</tr>
<tr>
<td align="center" valign="top">17</td>
<td align="left" valign="top">Falsification of Predictions</td>
<td align="left" valign="top">Student falsifies hypothesis (see 14) by observing a pattern within the behavior of the ecological system (usually through 15).</td>
<td align="center" valign="top">5</td>
</tr>
<tr>
<td align="center" valign="top">18a<sup>&#x002A;</sup></td>
<td align="left" valign="top">Model Modification Based on Confirmed Prediction</td>
<td align="left" valign="top">Student modifies model based on a confirmed hypothesis (see 16).</td>
<td align="center" valign="top">5</td>
</tr>
<tr>
<td align="center" valign="top">18b</td>
<td align="left" valign="top">Model Modification Based on Falsified Prediction</td>
<td align="left" valign="top">Student modifies model based on a falsified hypothesis (see 17).</td>
<td align="center" valign="top">2</td>
</tr>
<tr>
<td align="center" valign="top">19</td>
<td align="left" valign="top">Prediction-Based Model Rejection</td>
<td align="left" valign="top">Student rejects model based on a falsified hypothesis (see 18).</td>
<td align="center" valign="top">0</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p>The coding scheme is based on the existing framework of <xref ref-type="bibr" rid="ref50">Krell et al. (2019)</xref>. New categories are marked with an asterisk. Examples of student statements are available as <xref rid="SM1" ref-type="supplementary-material">Supplementary material</xref>.</p>
</table-wrap-foot>
</table-wrap>
</sec>
<sec id="sec12">
<label>3.5.</label>
<title>Data analysis</title>
<p>Before coding the students&#x2019; statements, two raters extensively discussed the existing frameworks of <xref ref-type="bibr" rid="ref50">Krell et al. (2019)</xref> and <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref> and adapted them to this study and the ecological context. They then independently coded the data of a test participant and compared their coding result afterwards. Non-matching codes were analyzed, and the coding instructions were refined, as was the inductive differentiation of the category system in scientific modeling. An analysis of inter-rater reliability after re-coding of the test participant&#x2019;s statements revealed an almost perfect result for our coding schemes, for both systems thinking and scientific modeling (&#x03BA;&#x2009;&#x003E;&#x2009;0.8) (<xref ref-type="bibr" rid="ref55">Landis and Koch, 1977</xref>). As a result, the coding manual was classified as final and ready to use to code the participants&#x2019; statements (<xref ref-type="bibr" rid="ref51">Kuckartz and R&#x00E4;diker, 2019</xref>). In the next step, the two raters independently coded all statements of the participating students. To ensure that the complete thinking process is considered, we used event-based coding (<xref ref-type="bibr" rid="ref16">Ciesielska et al., 2018</xref>), whereas content correctness was no criterion (e.g., <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>). Consistency of the coding was checked by measuring inter-rater reliabilities for each participant and for systems thinking and scientific modeling separately using Kappa (<xref ref-type="bibr" rid="ref12">Brennan and Prediger, 1981</xref>). In this regard, a coding was scored as matching if the segments of both codings were assigned to the same category and substantially overlapped. Overall, the analysis of the inter-rater reliabilities showed good results: For systems thinking, we achieved values of Kappa between 0.84 and 0.96 (<italic>M</italic> =&#x2009;0.88) for eight students, which indicates an almost perfect agreement (<xref ref-type="bibr" rid="ref55">Landis and Koch, 1977</xref>). For one student, neither rater found any statement related to systems thinking. For scientific modeling, the estimated values of Kappa are in the range of 0.73 and 0.86 (<italic>M</italic> =&#x2009;0.79) for the nine students, which may be interpreted as substantial (&#x03BA;&#x2009;&#x003E;&#x2009;0.6) to almost perfect (&#x03BA;&#x2009;&#x003E;&#x2009;0.8) agreement (<xref ref-type="bibr" rid="ref55">Landis and Koch, 1977</xref>).</p>
<p>To address our first two research questions, we analyzed the observed activities of scientific modeling and systems thinking using absolute and relative frequencies. In order to examine our third research question, which refers to the interrelationship between both perspectives, we created a contingency table. Using MAXQDA&#x2019;s Code Relations Browser (<xref ref-type="bibr" rid="ref94">VERBI Software, 2022</xref>), we estimated the intersection of the applied codes of systems thinking with those of scientific modeling to reveal patterns of co-occurrence (<xref ref-type="bibr" rid="ref51">Kuckartz and R&#x00E4;diker, 2019</xref>).</p>
</sec>
</sec>
<sec id="sec13" sec-type="results">
<label>4.</label>
<title>Results</title>
<p>Overall, we coded 1,412 students&#x2019; statements, which included 414 statements regarding systems thinking and 998 statements regarding scientific modeling. The number of segments per student varied widely, ranging from 103 to 292 (<italic>M</italic> =&#x2009;173). Statements regarding systems thinking were most frequently (63.0%; absolute frequency of 261) assigned to the skill of identifying the system organization, which means, they describe the type of relationship between two or more species. For instance, one student stated: <italic>&#x201C;I would say now that European rabbits eat pasture grass.</italic>&#x201D; (P. 4, Pos. 183). Also common were statements about analyzing the system behavior at 33.3% (absolute frequency of 138). Statements in this category describe a cause for a change in at least one population size usually by reference to another (changing) population size. An example of such a statement is: <italic>&#x201C;The Canada goose gained quite strongly in population in the following year, which is probably due to the high food supply of pasture grass.&#x201D;</italic> (P. 5, Pos. 93). The third skill of systems thinking, modeling the system evolution, in our context entails developing a forecast of one or more population size changes, such as: &#x201C;<italic>My assumption is that if the</italic> [red] <italic>kite goes extinct, there must be more and more</italic> [European] <italic>rabbits because then there are no more predators.</italic>&#x201D; (P. 10, Pos. 287). Statements regarding modeling the system evolution were rare (3.6%; absolute frequency of 15). The evaluation of the named linkage (see <xref rid="tab1" ref-type="table">Table 1</xref>) reveals frequent mentions of simple relations, including unidirectional and bidirectional relations. For instance, one student pointed out the bidirectional predator&#x2013;prey relation between European rabbits and red clover: &#x201C;<italic>If the number of European rabbits is very low, the red clover can of course grow well and then it still rises here. There are more and more European rabbits because they find more to eat and at a certain point, the climax is reached by red clover</italic> [...] <italic>and then it</italic> [red clover] <italic>falls</italic>. <italic>And then at some point the European rabbit decreases as well.&#x201D;</italic> (P. 10, Pos. 45). Furthermore, students frequently used complex relations, typically bridging one element: &#x201C;<italic>Well, pasture grass and red clover is the food, especially for the European rabbit</italic>.&#x201D; (P. 2, Pos. 12). Whereby complex relations bridging multiple elements were less frequent. Students rarely referred to linear relations bridging one element, like: &#x201C;<italic>Then the European rabbit eats the red clover and if there is a lot of red kite, then there is little of European rabbits, which means that the population</italic> [of red clover] <italic>increases. And conversely, when there are few European rabbits, there is a lot of red clover.&#x201D;</italic> (P. 5, Pos. 112), whereas linear relations bridging multiple elements were never mentioned. For additional examples of students&#x2019; statements on systems thinking see the <xref rid="SM1" ref-type="supplementary-material">Supplementary material</xref>.</p>
<p>The surveyed activities of scientific modeling were classified as follows: 62.5% exploration (absolute frequency 624), 6.4% activation of analogies and experiences (absolute frequency 64), 10.3% mental modeling (absolute frequency 103), 15.4% development (absolute frequency 154), and 5.3% prediction (absolute frequency 53). In addition to the deductively derived categories (<xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>), six categories were added by induction. Five of these categories follow a logical addition (<xref ref-type="bibr" rid="ref82">Schreier, 2012</xref>). For example, negating statements about pattern, like: &#x201C;[...] <italic>however, bears no resemblance to either of the other two</italic> [graphs of red clover and pasture grass], <italic>i.e., the European rabbit&#x201D;</italic> (P. 8, Pos. 147), were added in the newly included category 5b: Student negates presence or detection of a pattern. Statements describing a model&#x2019;s inconsistency and/or inconsistency between model and observation, such as: &#x201C;<italic>That means that</italic> [the drawn arrow] <italic>makes no sense to me here. The red kite decreases very strongly, very suddenly, and the graylag goose decreases only very slowly.</italic>&#x201D; (P. 3, Pos 192), were covered by the new category 13b. Similarly, student statements about modifying the model based on confirmed hypothesis, like: <italic>&#x201C;This allows us to conclude that the Canada goose feeds a bit on the red clover.</italic> [Student writes down] <italic>Canada goose eats red clover</italic>.&#x201D; (P. 3, Pos 167&#x2013;169), were added as affirmation in category 18a. During the survey, students repeatedly took written notes in their representation regarding descriptions and patterns of the ecological system. For example, one student pointed out while observing the system behavior: &#x201C;<italic>I&#x2019;m writing this down now. Pasture grass increases at 500, then decreases again.&#x201D;</italic> (P. 3, Pos. 55&#x2013;56). To differentiate these activities from the existing categories, which solely pertain to verbal expressions, we added the categories 3b: Student takes notes on the observed behavior of the ecological system, see previous student statement, and 5c: Student takes notes on the recognized, confirmed, or negated pattern. Another new activity that emerged within the process of scientific modeling is described in category 6b: Mental modeling, as the students in our study did not always document their ideas about the model mechanism in written form, opting to express them verbally instead. For instance, a student verbalizes the assumed correlation between the population dynamics of the Canada goose and its food sources: &#x201C;<italic>I would have thought that if I added these two feed together, I could conclude that by having a lower feed supply, I would also have fewer populations</italic> [of Canada geese].&#x201D; (P. 6, Pos 160). Such verbal expressions were even observed when the students were reminded to create a graphical representation. <xref rid="tab2" ref-type="table">Table 2</xref> shows the full coding scheme, including observed frequencies. Exemplary statements from the present study for each category of scientific modeling are provided in the <xref rid="SM1" ref-type="supplementary-material">Supplementary material</xref>.</p>
<p>To answer research question three on how students&#x2019; systems thinking and scientific modeling activities interrelate when they examine the dynamic ecological system, we determined co-occurrences of coded segments of both perspectives (see <xref rid="tab3" ref-type="table">Table 3</xref>). This analysis revealed several noticeable interrelations. During system exploration, students demonstrated systems thinking almost exclusively in the area of system behavior and during the interaction with data patterns (modeling activities 5a-c). When students addressed notions referring to the system organization, it was in the context of deducing hypothesis-based graph interactions (modeling activity 4). In the phases of model development, we frequently observed systems thinking. Especially during the broadly observed phases of developing and refining the model, students dealt with the system organization and behavior. In both scientific modeling phases, system exploration and model development, no modeling of the system evolution took place. This changes for the development of predictions, where all three skills of systems thinking were observable. The system modeling, however, is almost entirely tied to the activity of using the model to deduce hypotheses. In addition, activities 6a, referring to the verbalization of analogies and experiences, and 6b, covering the development and usage of mental models, have a special status. During the activation of analogies and experiences (modeling activity 6a), students extensively refer to notions related to the system organization and occasionally deal with concepts regarding the system behavior as well. The mental modeling (modeling activity 6b) was accompanied by systems thinking skills of all kinds. While the system behavior and especially the system organization predominate, modeling system evolution was also observed at least occasionally.</p>
<table-wrap position="float" id="tab3">
<label>Table 3</label>
<caption>
<p>Contingency table of the interrelationship between scientific modeling and systems thinking.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th rowspan="2"/>
<th rowspan="2"/>
<th rowspan="2"/>
<th align="center" valign="middle" colspan="6">Identifying System Organization</th>
<th align="center" valign="middle" colspan="6">Analyzing System Behavior</th>
<th align="center" valign="middle" colspan="6">Modeling System Evolution</th>
</tr>
<tr>
<th align="center" valign="middle">Unidirectional Relation</th>
<th align="center" valign="middle">Bidirectional Relation</th>
<th align="center" valign="middle">Linear Relation Bridging One Element</th>
<th align="center" valign="middle">Linear Relation Bridging Multiple Elements</th>
<th align="center" valign="middle">Complex Relation Bridging One Element</th>
<th align="center" valign="middle">Complex Relation Bridging Multiple Elements</th>
<th align="center" valign="middle">Unidirectional Relation</th>
<th align="center" valign="middle">Bidirectional Relation</th>
<th align="center" valign="middle">Linear Relation Bridging One Element</th>
<th align="center" valign="middle">Linear Relation Bridging Multiple Elements</th>
<th align="center" valign="middle">Complex Relation Bridging One Element</th>
<th align="center" valign="middle">Complex Relation Bridging Multiple Elements</th>
<th align="center" valign="middle">Unidirectional Relation</th>
<th align="center" valign="middle">Bidirectional Relation</th>
<th align="center" valign="middle">Linear Relation Bridging One Element</th>
<th align="center" valign="middle">Linear Relation Bridging Multiple Elements</th>
<th align="center" valign="middle">Complex Relation Bridging One Element</th>
<th align="center" valign="middle">Complex Relation Bridging Multiple Elements</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center" valign="middle" rowspan="8">Exploration</td>
<td align="left" valign="middle">1</td>
<td align="left" valign="middle">Perception of Phenomena</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="middle">2</td>
<td align="left" valign="middle">Arbitrary&#x2009;Exploration of the System</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="middle">3a</td>
<td align="left" valign="middle">Description of Observations</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="middle">3b&#x002A;</td>
<td align="left" valign="middle">Notes on Observations</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="middle">4</td>
<td align="left" valign="middle">Purposive Exploration of the System</td>
<td/>
<td align="center" valign="middle">1</td>
<td/>
<td/>
<td align="center" valign="middle">4</td>
<td/>
<td align="center" valign="middle">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="middle">5a</td>
<td align="left" valign="middle">Recognition of&#x2009;Pattern</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td align="center" valign="middle">16</td>
<td align="center" valign="middle">9</td>
<td align="center" valign="middle">1</td>
<td/>
<td align="center" valign="middle">3</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="middle">5b&#x002A;</td>
<td align="left" valign="middle">Negation of Pattern</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td align="center" valign="middle">1</td>
<td align="center" valign="middle">2</td>
<td/>
<td/>
<td align="center" valign="middle">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="middle">5c&#x002A;</td>
<td align="left" valign="middle">Notes on Pattern</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td align="center" valign="middle">5</td>
<td align="center" valign="middle">3</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td/>
<td align="left" valign="middle">6a</td>
<td align="left" valign="middle">Activation of Analogies and Experiences</td>
<td align="center" valign="middle">15</td>
<td align="center" valign="middle">2</td>
<td/>
<td/>
<td align="center" valign="middle">9</td>
<td/>
<td align="center" valign="middle">4</td>
<td/>
<td/>
<td/>
<td align="center" valign="middle">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td/>
<td align="left" valign="middle">6b&#x002A;</td>
<td align="left" valign="middle">Mental Modeling</td>
<td align="center" valign="middle">37</td>
<td align="center" valign="middle">10</td>
<td/>
<td/>
<td align="center" valign="middle">18</td>
<td align="center" valign="middle">2</td>
<td align="center" valign="middle">16</td>
<td align="center" valign="middle">10</td>
<td align="center" valign="middle">1</td>
<td/>
<td align="center" valign="middle">15</td>
<td align="center" valign="middle">2</td>
<td/>
<td/>
<td/>
<td/>
<td align="center" valign="middle">4</td>
<td/>
</tr>
<tr>
<td align="center" valign="middle" rowspan="8">Development</td>
<td align="left" valign="middle">7</td>
<td align="left" valign="middle">Analogy and Experience-Based Model Development</td>
<td/>
<td align="center" valign="middle">1</td>
<td/>
<td/>
<td align="center" valign="middle">3</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="middle">8</td>
<td align="left" valign="middle">Design-Based Model Development</td>
<td align="center" valign="middle">2</td>
<td/>
<td/>
<td/>
<td align="center" valign="top">1</td>
<td/>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">9</td>
<td align="left" valign="top">Observation-Based Model Development</td>
<td align="center" valign="top">60</td>
<td align="center" valign="top">20</td>
<td/>
<td/>
<td align="center" valign="top">38</td>
<td align="center" valign="top">2</td>
<td align="center" valign="top">20</td>
<td align="center" valign="top">9</td>
<td align="center" valign="top">1</td>
<td/>
<td align="center" valign="top">7</td>
<td align="center" valign="top">3</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">10</td>
<td align="left" valign="top">Observation-Based Model&#x2009;Rejection</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">11</td>
<td align="left" valign="top">Design-Based Model Evaluation</td>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">12</td>
<td align="left" valign="top">Observation-Based Model Evaluation</td>
<td align="center" valign="top">2</td>
<td/>
<td/>
<td/>
<td align="center" valign="top">7</td>
<td/>
<td/>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">13a</td>
<td align="left" valign="top">Confirmation of Model Consistency</td>
<td align="center" valign="top">5</td>
<td align="center" valign="top">5</td>
<td/>
<td/>
<td align="center" valign="top">6</td>
<td/>
<td align="center" valign="top">6</td>
<td align="center" valign="top">3</td>
<td align="center" valign="top">1</td>
<td/>
<td align="center" valign="top">4</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">13b&#x002A;</td>
<td align="left" valign="top">Denial of Model Consistency</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td align="center" valign="top">2</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="center" valign="middle" rowspan="7">Prediction</td>
<td align="left" valign="top">14</td>
<td align="left" valign="top">Generation of Predictions</td>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td align="center" valign="top">6</td>
<td/>
<td align="center" valign="top">1</td>
<td/>
<td align="center" valign="top">5</td>
<td align="center" valign="top">1</td>
</tr>
<tr>
<td align="left" valign="top">15</td>
<td align="left" valign="top">Purposive&#x2009;Manipulation&#x2009;of the System</td>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">16</td>
<td align="left" valign="top">Confirmation of Predictions</td>
<td align="center" valign="top">1</td>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td align="center" valign="top">1</td>
<td/>
<td align="center" valign="top">2</td>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td align="center" valign="top">6</td>
<td/>
<td align="center" valign="top"/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">17</td>
<td align="left" valign="top">Falsification of Predictions</td>
<td align="center" valign="top">2</td>
<td align="center" valign="top"/>
<td/>
<td/>
<td align="center" valign="top"/>
<td/>
<td align="center" valign="top"/>
<td align="center" valign="top"/>
<td/>
<td/>
<td align="center" valign="top"/>
<td/>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">18a&#x002A;</td>
<td align="left" valign="top">Model Modification&#x2009;Based on Confirmed Prediction</td>
<td align="center" valign="top">4</td>
<td/>
<td/>
<td/>
<td align="center" valign="top">2</td>
<td/>
<td/>
<td/>
<td align="center" valign="top">2</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">18b</td>
<td align="left" valign="top">Model&#x2009;Modification&#x2009;Based on Falsified Prediction</td>
<td/>
<td align="center" valign="top">1</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td align="left" valign="top">19</td>
<td align="left" valign="top">Prediction-Based Model Rejection</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p>The frequency of the overlaps is given in absolute numbers. The coding scheme for scientific modeling is based on the existing framework of <xref ref-type="bibr" rid="ref50">Krell et al. (2019)</xref>. The systems thinking skills and levels are based on the framework of <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref>.</p>
</table-wrap-foot>
</table-wrap>
</sec>
<sec id="sec14" sec-type="discussions">
<label>5.</label>
<title>Discussion</title>
<p>In this study, we confronted students with a dynamic ecological phenomenon in order to examine their systems thinking and scientific modeling as well as the interrelations between these perspectives. The results enable us to provide a detailed account of the students&#x2019; comprehension process of a complex and dynamic system.</p>
<sec id="sec15">
<label>5.1.</label>
<title>Systems thinking</title>
<p>We were able to assess students&#x2019; systems thinking using a coding scheme based on the framework of <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref>. However, when examining the frequency of occurrence, it was observed that activities of identifying the system organization and analyzing the system behavior were much more frequent than modeling the system evolution. This pattern may be attributed to a difference in the cognitive perspective, as identifying the system organization and analyzing the system behavior requires the application of broad, globally applicable skills, whereas modeling the system evolution demands mental application specific to the task at hand. Students must use their gained knowledge about the system to derive predictions about system development and future states (<xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>), which also requires imagination and creative thinking (<xref ref-type="bibr" rid="ref58">Mambrey et al., 2022a</xref>). Moreover, and due to the openness of the survey, the task may lack the imperative character necessary to evoke prediction generation. In our opinion, it could therefore be highly rewarding to fill this gap by adding tasks in future research that demand prediction generation, even when this means structuring the process through individual instructions or prompts. In order to analyze students&#x2019; systems thinking in detail, we also examined the linked structure of relations they took into account. First, and in line with previous results (<xref ref-type="bibr" rid="ref89">Strogatz, 2001</xref>; <xref ref-type="bibr" rid="ref31">Gotwals and Songer, 2010</xref>; <xref ref-type="bibr" rid="ref39">Hokayem, 2016</xref>; <xref ref-type="bibr" rid="ref63">Mehren et al., 2018</xref>; <xref ref-type="bibr" rid="ref60">Mambrey et al., 2022b</xref>), we observed that students&#x2019; mentions of low linked relationships greatly outweighed the references to high linked relations connecting multiple elements. When examining the specific types of relations, there was a particularly high occurrence of unidirectional, bidirectional, and complex relations bridging one element. Whereas complex relations bridging multiple elements or linear relations were rarely mentioned. We suppose this pattern arises from a moderate linking performance in combination with a sound understanding of isolated predator&#x2013;prey relationships among the participants (<xref ref-type="bibr" rid="ref33">Grotzer and Basca, 2003</xref>; <xref ref-type="bibr" rid="ref87">Sommer and L&#x00FC;cken, 2010</xref>; <xref ref-type="bibr" rid="ref58">Mambrey et al., 2022a</xref>). Indeed, students often referred to feeding relationships commonly covered in class, such as a predator&#x2013;prey and competitive relationship, with and without naming the competing food source (<xref ref-type="bibr" rid="ref70">NGSS Lead States, 2013</xref>). This may even be a good strategy to identify the system organization. However, it is not possible to understand the emergent extinction of the red kite by analyzing individual predator&#x2013;prey or competitive relationships. Even though the presented phenomenon demands an understanding of emergence, we have not explicitly demanded this and a food web, as typical representation of the trophic relationships within an ecosystem, does not directly reflect this emergent behavior. Therefore, future systems thinking research could focus on systems demonstrating emergent behaviors or systems with an ever-changing system organization in order to evoke more complex systems thinking.</p>
</sec>
<sec id="sec16">
<label>5.2.</label>
<title>Scientific modeling</title>
<p>We successfully utilized the scientific modeling framework developed by <xref ref-type="bibr" rid="ref50">Krell et al. (2019)</xref> to analyze students&#x2019; scientific modeling activities in the context of a dynamic ecological system: Our analysis revealed that students were extensively engaged in the exploration of the system, which is consistent with previous research (<xref ref-type="bibr" rid="ref30">G&#x00F6;hner and Krell, 2022</xref>). Frequently, we observed students taking notes during exploration, particularly when summarizing or describing an observed behavior (modeling activity 3) and recognizing or confirming data patterns (modeling activity 5). Therefore, we inductively added categories in each phase to account for this behavior (<xref ref-type="bibr" rid="ref82">Schreier, 2012</xref>). We suppose that taking notes may be functional or even necessary in some situations due to the system&#x2019;s complexity, which includes numerous not directly recognizable relations. Some students probably used this strategy as a cognitive aid to visually represent knowledge about specific relations (<xref ref-type="bibr" rid="ref75">Peper and Mayer, 1986</xref>). Generally speaking, prior knowledge in the form of analogies and experiences (modeling activity 6a) is frequently observed during scientific modeling (<xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>; <xref ref-type="bibr" rid="ref64">Meister and Upmeier zu Belzen, 2020</xref>). Besides exploring the system or reflecting on prior knowledge (modeling activity 6a), students comprehensively dealt with graphical model development. They frequently incorporated findings from their observations of the system (modeling activity 9) and evaluated the model consistency (modeling activities 11, 12, 13). By extending the coding scheme to differentiate students who found consistency (modeling activity 13a) from those detecting inconsistency between their graphical model and the systems behavior, we noted an interesting pattern: At this point of model development, students almost never detect inconsistency. This could be the case, as they frequently perform mental modeling instead of recording their ideas about the model in a graphic format. Indeed, this process of mental modeling is a key phase of scientific modeling (<xref ref-type="bibr" rid="ref17">Clement, 1989</xref>; <xref ref-type="bibr" rid="ref69">Nersessian, 2008</xref>; <xref ref-type="bibr" rid="ref27">Gilbert and Justi, 2016</xref>) and has been observed in previous modeling studies (<xref ref-type="bibr" rid="ref64">Meister and Upmeier zu Belzen, 2020</xref>). We added a category to our coding scheme to account for this behavior (modeling activity 6b), though the frequency of occurrence of this activity certainly raises the question of the underlying substructure of mental modeling for future research. To investigate this question, analyzing students&#x2019; individual modeling processes will help determine the position and weight of mental modeling within the overall process of scientific modeling (e.g., <xref ref-type="bibr" rid="ref30">G&#x00F6;hner and Krell, 2022</xref>). In particular, there is evidence that the mental model and the emerging graphical representation of the model may not be fully equivalent (<xref ref-type="bibr" rid="ref14">Chandrasekharan and Nersessian, 2015</xref>).</p>
<p>Where we found limited evidence, however, was in the phase of predictive model usage. This could be the case because students often do not fully understand the use of models for prediction, as noted by <xref ref-type="bibr" rid="ref29">Gogolin and Kr&#x00FC;ger (2018)</xref>. In addition, previous research indicates that model application can be challenging (<xref ref-type="bibr" rid="ref13">Campbell et al., 2013</xref>; <xref ref-type="bibr" rid="ref74">Passmore et al., 2014</xref>; <xref ref-type="bibr" rid="ref49">Krell and Kr&#x00FC;ger, 2016</xref>; <xref ref-type="bibr" rid="ref64">Meister and Upmeier zu Belzen, 2020</xref>; <xref ref-type="bibr" rid="ref30">G&#x00F6;hner and Krell, 2022</xref>). The lack of predictive model usage may also be an expression of students&#x2019; struggling to generate hypotheses from prior knowledge, their model in progress or previous observations of the system behavior. This would fit the results and the model of <xref ref-type="bibr" rid="ref47">Klahr and Dunbar (1988)</xref>, who assume that exploration without hypothesis is a strategy to search for some kind of pattern within the data and is applied if the generation of hypotheses fails. Surprisingly, when students derived hypotheses, they modified their model not only in response to falsified hypotheses but also in response to confirmed hypotheses (modeling activity 18b). This may be due to the complexity of the system, as new knowledge about individual relationships within the system may require adjustments to the model or allow for model enhancements without necessitating a complete overhaul of the previous understanding. In addition, confirming findings may be suitable not only to the recent hypothesis but also to previous ideas which appear more promising or worthy of investigation (<xref ref-type="bibr" rid="ref47">Klahr and Dunbar, 1988</xref>).</p>
</sec>
<sec id="sec17">
<label>5.3.</label>
<title>Interrelation between scientific modeling and systems thinking</title>
<p>From the combined analysis of systems thinking and scientific modeling, we can identify the following relationships in our study: During the exploration of the ecological system, we rarely observed systems thinking. In the event of pattern recognition, we detected systems thinking almost exclusively with the objective of analyzing the system behavior. Otherwise, systems thinking occurred frequently during model development. We suppose that these findings perfectly illustrate a central difference in the perspectives of systems thinking and scientific modeling. While the former includes the description and explanation of complex phenomena as well as the prediction thereof, the latter additionally involves incidental perception as the early phase of exploration. Another difference in perspective is that systems thinking often describes several skills that relate to specific system properties (<xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>; <xref ref-type="bibr" rid="ref67">Momsen et al., 2022</xref>), whereas scientific modeling primarily focuses on the process and activities of modeling the initial natural phenomenon or system. However, identifying the system organization and analyzing the system behavior are mandatory preconditions for successfully developing a model that explains the phenomenon or system (<xref ref-type="bibr" rid="ref74">Passmore et al., 2014</xref>; <xref ref-type="bibr" rid="ref32">Gouvea and Passmore, 2017</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>). Our results match this assumption, as the students frequently utilize both skills of identifying the system organization and analyzing the system behavior during model development. Another result of the independent coding of systems thinking and scientific modeling was the third skill of systems thinking, modeling the system evolution, being closely linked to the scientific modeling activity of hypothesis generation. This even makes sense, as the former entails statements developing a forecast of one or more population size changes, while the latter covers statements deducing predictions based on an existing model (<xref ref-type="bibr" rid="ref17">Clement, 1989</xref>; <xref ref-type="bibr" rid="ref45">Justi and Gilbert, 2002</xref>; <xref ref-type="bibr" rid="ref26">Giere et al., 2006</xref>; <xref ref-type="bibr" rid="ref27">Gilbert and Justi, 2016</xref>; <xref ref-type="bibr" rid="ref50">Krell et al., 2019</xref>; <xref ref-type="bibr" rid="ref93">Upmeier zu Belzen et al., 2019</xref>). The juxtaposition makes clear that the definition of modeling the system evolution and hypothesis generation in the realm of scientific modeling are closely related. It is therefore not surprising that beyond this, modeling the system evolution as one skill of systems thinking only appeared in conjunction with the phase of mental modeling. This seems even plausible, as the modeling process starts in mental space and can involve cycles of development, evaluation, and modification before any graphic development (<xref ref-type="bibr" rid="ref64">Meister and Upmeier zu Belzen, 2020</xref>). And indeed, students generally utilize all systems thinking skills in the process of mental modeling. Considering the ratios throughout the mental modeling phase, the participants are more often involved in analyzing the system behavior than in identifying the system organization, while identifying the system organization appears slightly more frequently during the graphical production phase. Perhaps this may be explained by a finding of <xref ref-type="bibr" rid="ref58">Mambrey et al. (2022a)</xref>, who demonstrated that a food web as representation is strongly associated with identifying the system organization but rather no reference for students when analyzing the system behavior or modeling the system evolution. This may explain our observations, as most students drew food webs as models to represent the ecological system. As this type of representation does not provide quantitative information about population growth or decline (<xref ref-type="bibr" rid="ref2">Begon and Townsend, 2021</xref>), it may, in fact, not evoke thinking about the system behavior. In contrast, the line graph we used in our task focuses on the population dynamics and should thus be a stronger trigger for analyzing the system behavior. Our data clearly support that assumption.</p>
</sec>
</sec>
<sec id="sec18">
<label>6.</label>
<title>Limitations and implications</title>
<p>Our study is limited in some areas that we would like to discuss briefly in the following: to gain a comprehensive understanding of students&#x2019; systems thinking and scientific modeling, we provided each participant individual time for analyzing the ecological system simulation. Hence and due to the task complexity, it was not possible to test multiple systems. Therefore, the results may be limited regarding potential context effects (<xref ref-type="bibr" rid="ref33">Grotzer and Basca, 2003</xref>; <xref ref-type="bibr" rid="ref95">Verhoeff et al., 2013</xref>; <xref ref-type="bibr" rid="ref59">Mambrey et al., 2020</xref>). Furthermore, our analysis focused on the general expression of systems thinking and scientific modeling. In consequence, we did not assess students&#x2019; content knowledge explicitly, even though modeling activity 7 does provide evidence of the incorporation of analogies and experiences that directly relate to the process of model development. In addition, our coding scheme as well as the underlying framework of <xref ref-type="bibr" rid="ref59">Mambrey et al. (2020)</xref> do not address systems thinking from a perspective of system properties. As individual system properties, including dynamic changes and susceptibility to disruptive factors like invasive species, play an important role in our simulation, this perspective may be interesting for future research. Furthermore, characteristics of our simulation may also impact students&#x2019; thinking. For example, the simulated population dynamics may have drawn students&#x2019; attention on thinking about the system behavior in particular. We found that students rarely mentioned emergent effects resulting from complex interrelationships, as well as predictions about future system states or behaviors. One reason for this may be that the task and the simulation do not encourage this explicitly. Hence, future research should emphasize on systems with greater changes in their dynamics and complex emergent effects. Additionally, it would be worthwhile to include tasks that stimulate the prediction of these changes in system development.</p>
<p>Although we did not analyze the students&#x2019; individual procedures yet, we wish to discuss possible implications for engaging students in systems thinking and scientific modeling. Our results imply that a task involving students in the exploration of a dynamic system to deduce the system structure seems suitable to evoke systems thinking and modeling insofar as the students develop models. However, the generation of predictions and, consequently, the modeling of the system evolution as well fell short of our expectations. We suppose that the regular oscillatory nature in population dynamics is too foreseeable, even if the introduction of the invasive Canada goose was a functional disturbance. In this regard, it may be beneficial to utilize even less predictable system dynamics, for example, by modeling alternative stable states or multiple regime shifts in ecosystems (<xref ref-type="bibr" rid="ref81">Scheffer and Carpenter, 2003</xref>), to focus on the cyclic nature of modeling through predicting outcomes and modifying the model. However, it remains open whether all students used the same approach or if they showed distinctive styles of scientific modeling and systems thinking (e.g., <xref ref-type="bibr" rid="ref64">Meister and Upmeier zu Belzen, 2020</xref>; <xref ref-type="bibr" rid="ref30">G&#x00F6;hner and Krell, 2022</xref>). In order to answer this research question, it might be beneficial to analyze the individual chronological sequences of students&#x2019; proceedings. Additional insights could be revealed by analyzing students&#x2019; drawings, as individual students also created representations other than food webs. In this context, we are especially interested in whether different types of representations correlate with distinct patterns of systems thinking and how students account for the system&#x2019;s dynamic behavior in their representations.</p>
</sec>
<sec id="sec19" sec-type="data-availability">
<title>Data availability statement</title>
<p>The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.</p>
</sec>
<sec id="sec20">
<title>Ethics statement</title>
<p>Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was provided by the participants&#x2019; legal guardian/next of kin.</p>
</sec>
<sec id="sec21">
<title>Author contributions</title>
<p>AL, JT, and PS: conception and design of the study and manuscript revision. AL and JT: development of the test instrument and manuscript writing. AL: data collection and data analysis. PS: supervision. All authors contributed to the article and approved the submitted version.</p>
</sec>
<sec id="sec22" sec-type="funding-information">
<title>Funding</title>
<p>This project is part of the &#x201C;Qualit&#x00E4;tsoffensive Lehrerbildung,&#x201D; a joint initiative of the Federal Government and the L&#x00E4;nder which aims to improve the quality of teacher training. The program is funded by the German Federal Ministry of Education and Research (BMBF) under Grant Number 01JA1910. The Interdisciplinary Center for Educational Research (IZfB) of the University of Duisburg-Essen provided funding support for linguistic editing services. We perceived support from the Open Access Publication Fund of the University of Duisburg-Essen.</p>
</sec>
<sec id="conf1" sec-type="COI-statement">
<title>Conflict of interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec id="sec100" sec-type="disclaimer">
<title>Publisher&#x2019;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<sec id="sec24">
<title>Author disclaimer</title>
<p>The opinions and statements presented in this article are the responsibility of the authors alone and may not reflect the views of their affiliated institutions, the funding agencies, the publisher, editors or reviewers.</p>
</sec>
</body>
<back>
<ack>
<p>We would like to express our gratitude to all students, who participated in this study. Additionally, we extend our appreciation to our student research assistant, Annika Stratmann, for her invaluable contributions in transcription and co-coding of the think-aloud protocols.</p>
</ack>
<sec id="sec25" sec-type="supplementary-material">
<title>Supplementary material</title>
<p>The Supplementary material for this article can be found online at: <ext-link xlink:href="https://www.frontiersin.org/articles/10.3389/feduc.2023.1187237/full#supplementary-material" ext-link-type="uri">https://www.frontiersin.org/articles/10.3389/feduc.2023.1187237/full#supplementary-material</ext-link></p>
<supplementary-material xlink:href="Table_1.pdf" id="SM1" mimetype="application/pdf" xmlns:xlink="http://www.w3.org/1999/xlink"/>
</sec>
<ref-list>
<title>References</title>
<ref id="ref1"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Andrade</surname> <given-names>B. O.</given-names></name> <name><surname>Koch</surname> <given-names>C.</given-names></name> <name><surname>Boldrini</surname> <given-names>I. I.</given-names></name> <name><surname>V&#x00E9;lez-Martin</surname> <given-names>E.</given-names></name> <name><surname>Hasenack</surname> <given-names>H.</given-names></name> <name><surname>Hermann</surname> <given-names>J.-M.</given-names></name> <etal/></person-group>. (<year>2015</year>). <article-title>Grassland degradation and restoration: a conceptual framework of stages and thresholds illustrated by southern Brazilian grasslands</article-title>. <source>Nat. Conserv.</source> <volume>13</volume>, <fpage>95</fpage>&#x2013;<lpage>104</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.ncon.2015.08.002</pub-id></citation></ref>
<ref id="ref2"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Begon</surname> <given-names>M.</given-names></name> <name><surname>Townsend</surname> <given-names>C. R.</given-names></name></person-group> (<year>2021</year>). <source>Ecology: from individuals to ecosystems</source>. <publisher-loc>Hoboken</publisher-loc>: <publisher-name>Wiley</publisher-name>.</citation></ref>
<ref id="ref3"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Ben Zvi Assaraf</surname> <given-names>O</given-names></name></person-group>., and <person-group person-group-type="author"><name><surname>Knippels</surname> <given-names>M.-C. P. J</given-names></name></person-group>., eds. (<year>2022</year>). <source>Fostering understanding of complex systems in biology education: pedagogies, guidelines and insights from classroom-based research</source>. <publisher-loc>Cham</publisher-loc>: <publisher-name>Springer.</publisher-name></citation></ref>
<ref id="ref4"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ben Zvi Assaraf</surname> <given-names>O.</given-names></name> <name><surname>Orion</surname> <given-names>N.</given-names></name></person-group> (<year>2005</year>). <article-title>Development of system thinking skills in the context of earth system education</article-title>. <source>J. Res. Sci. Teach.</source> <volume>42</volume>, <fpage>518</fpage>&#x2013;<lpage>560</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.20061</pub-id></citation></ref>
<ref id="ref5"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ben Zvi Assaraf</surname> <given-names>O.</given-names></name> <name><surname>Orion</surname> <given-names>N.</given-names></name></person-group> (<year>2010</year>). <article-title>System thinking skills at the elementary school level</article-title>. <source>J. Res. Sci. Teach.</source> <volume>47</volume>, <fpage>540</fpage>&#x2013;<lpage>563</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.20351</pub-id></citation></ref>
<ref id="ref6"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bielik</surname> <given-names>T.</given-names></name> <name><surname>Delen</surname> <given-names>I.</given-names></name> <name><surname>Krell</surname> <given-names>M.</given-names></name> <name><surname>Assaraf</surname> <given-names>B. Z.</given-names></name></person-group> (<year>2023</year>). <article-title>Characterising the literature on the teaching and learning of system thinking and complexity in STEM education: A bibliometric analysis and research synthesis</article-title>. <source>J. STEM Educ. Res.</source> doi: <pub-id pub-id-type="doi">10.1007/s41979-023-00087-9</pub-id></citation></ref>
<ref id="ref7"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bielik</surname> <given-names>T.</given-names></name> <name><surname>Fonio</surname> <given-names>E.</given-names></name> <name><surname>Feinerman</surname> <given-names>O.</given-names></name> <name><surname>Duncan</surname> <given-names>R. G.</given-names></name> <name><surname>Levy</surname> <given-names>S. T.</given-names></name></person-group> (<year>2021</year>). <article-title>Working together: integrating computational modeling approaches to investigate complex phenomena</article-title>. <source>J. Sci. Educ. Technol.</source> <volume>30</volume>, <fpage>40</fpage>&#x2013;<lpage>57</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10956-020-09869-x</pub-id></citation></ref>
<ref id="ref8"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bielik</surname> <given-names>T.</given-names></name> <name><surname>Stephens</surname> <given-names>L.</given-names></name> <name><surname>McIntyre</surname> <given-names>C.</given-names></name> <name><surname>Damelin</surname> <given-names>D.</given-names></name> <name><surname>Krajcik</surname> <given-names>J. S.</given-names></name></person-group> (<year>2022</year>). <article-title>Supporting student system modelling practice through curriculum and technology design</article-title>. <source>J. Sci. Educ. Technol.</source> <volume>31</volume>, <fpage>217</fpage>&#x2013;<lpage>231</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10956-021-09943-y</pub-id></citation></ref>
<ref id="ref9"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Boersma</surname> <given-names>K. T.</given-names></name> <name><surname>Waarlo</surname> <given-names>A. J.</given-names></name> <name><surname>Klaassen</surname> <given-names>K.</given-names></name></person-group> (<year>2011</year>). <article-title>The feasibility of systems thinking in biology education</article-title>. <source>J. Biol. Educ.</source> <volume>45</volume>, <fpage>190</fpage>&#x2013;<lpage>197</lpage>. doi: <pub-id pub-id-type="doi">10.1080/00219266.2011.627139</pub-id></citation></ref>
<ref id="ref10"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bowers</surname> <given-names>J.</given-names></name> <name><surname>Damelin</surname> <given-names>D.</given-names></name> <name><surname>Eidin</surname> <given-names>E.</given-names></name> <name><surname>McIntyre</surname> <given-names>C.</given-names></name></person-group> (<year>2022</year>). <article-title>Keeping cool with SageModeler: engaging students in systems thinking and computational thinking through modeling</article-title>. <source>Sci. Teach.</source> <volume>89</volume>, <fpage>18</fpage>&#x2013;<lpage>25</lpage>.</citation></ref>
<ref id="ref11"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Brandst&#x00E4;dter</surname> <given-names>K.</given-names></name> <name><surname>Harms</surname> <given-names>U.</given-names></name> <name><surname>Gro&#x00DF;schedl</surname> <given-names>J.</given-names></name></person-group> (<year>2012</year>). <article-title>Assessing system thinking through different concept-mapping practices</article-title>. <source>Int. J. Sci. Educ.</source> <volume>34</volume>, <fpage>2147</fpage>&#x2013;<lpage>2170</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500693.2012.716549</pub-id></citation></ref>
<ref id="ref12"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Brennan</surname> <given-names>R. L.</given-names></name> <name><surname>Prediger</surname> <given-names>D. J.</given-names></name></person-group> (<year>1981</year>). <article-title>Coefficient kappa: some uses, misuses, and alternatives</article-title>. <source>Educ. Psychol. Meas.</source> <volume>41</volume>, <fpage>687</fpage>&#x2013;<lpage>699</lpage>. doi: <pub-id pub-id-type="doi">10.1177/001316448104100307</pub-id></citation></ref>
<ref id="ref13"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Campbell</surname> <given-names>T.</given-names></name> <name><surname>Oh</surname> <given-names>P. S.</given-names></name> <name><surname>Neilson</surname> <given-names>D.</given-names></name></person-group> (<year>2013</year>). &#x201C;<article-title>Reification of five types of modeling pedagogies with model-based inquiry (MBI) modules for high school science classrooms</article-title>&#x201D; in <source>Approaches and strategies in next generation science learning</source>. eds. <person-group person-group-type="editor"><name><surname>Khine</surname> <given-names>M. S.</given-names></name> <name><surname>Saleh</surname> <given-names>I. M.</given-names></name></person-group> (<publisher-loc>Hershey, PA</publisher-loc>: <publisher-name>IGI Global</publisher-name>), <fpage>106</fpage>&#x2013;<lpage>126</lpage>.</citation></ref>
<ref id="ref14"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chandrasekharan</surname> <given-names>S.</given-names></name> <name><surname>Nersessian</surname> <given-names>N. J.</given-names></name></person-group> (<year>2015</year>). <article-title>Building cognition: the construction of computational representations for scientific discovery</article-title>. <source>Cogn. Sci.</source> <volume>39</volume>, <fpage>1727</fpage>&#x2013;<lpage>1763</lpage>. doi: <pub-id pub-id-type="doi">10.1111/cogs.12203</pub-id>, PMID: <pub-id pub-id-type="pmid">25444018</pub-id></citation></ref>
<ref id="ref15"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Chang</surname> <given-names>W.</given-names></name> <name><surname>Cheng</surname> <given-names>J.</given-names></name> <name><surname>Allaire</surname> <given-names>J. J.</given-names></name> <name><surname>Sievert</surname> <given-names>C.</given-names></name> <name><surname>Schloerke</surname> <given-names>B.</given-names></name> <name><surname>Xie</surname> <given-names>Y.</given-names></name> <etal/></person-group>. (<year>2022</year>). <source>Shiny: Web application framework for R [computer program]</source>.</citation></ref>
<ref id="ref16"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Ciesielska</surname> <given-names>M.</given-names></name> <name><surname>Bostr&#x00F6;m</surname> <given-names>K. W.</given-names></name> <name><surname>&#x00D6;hlander</surname> <given-names>M.</given-names></name></person-group> (<year>2018</year>). &#x201C;<article-title>Observation methods</article-title>&#x201D; in <source>Qualitative methodologies in organization studies: Volume II: Methods and possibilities</source>. eds. <person-group person-group-type="editor"><name><surname>Ciesielska</surname> <given-names>M.</given-names></name> <name><surname>Jemielniak</surname> <given-names>D.</given-names></name></person-group> (<publisher-loc>Cham</publisher-loc>: <publisher-name>Palgrave Macmillan</publisher-name>), <fpage>33</fpage>&#x2013;<lpage>52</lpage>. doi: <pub-id pub-id-type="doi">10.1007/978-3-319-65217-7</pub-id></citation></ref>
<ref id="ref17"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Clement</surname> <given-names>J.</given-names></name></person-group> (<year>1989</year>). &#x201C;<article-title>Learning via model construction and criticism</article-title>&#x201D; in <source>Handbook of creativity</source>. eds. <person-group person-group-type="editor"><name><surname>Glover</surname> <given-names>J. A.</given-names></name> <name><surname>Ronning</surname> <given-names>R. R.</given-names></name> <name><surname>Reynolds</surname> <given-names>C. R.</given-names></name></person-group> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>341</fpage>&#x2013;<lpage>381</lpage>.</citation></ref>
<ref id="ref18"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Damelin</surname> <given-names>D.</given-names></name> <name><surname>Krajcik</surname> <given-names>J. S.</given-names></name> <name><surname>McIntyre</surname> <given-names>C.</given-names></name> <name><surname>Bielik</surname> <given-names>T.</given-names></name></person-group> (<year>2017</year>). <article-title>Students making system models: an accessible approach</article-title>. <source>Sci. Scope</source> <volume>40</volume>, <fpage>78</fpage>&#x2013;<lpage>82</lpage>. doi: <pub-id pub-id-type="doi">10.2505/4/ss17_040_05_78</pub-id></citation></ref>
<ref id="ref19"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Dor-Haim</surname> <given-names>S.</given-names></name> <name><surname>Assaraf</surname> <given-names>B. Z.</given-names></name></person-group> (<year>2022</year>). &#x201C;<article-title>Long term ecological research as a learning environment: evaluating its impact in developing the understanding of ecological systems thinking &#x2013; A case study</article-title>&#x201D; in <source>Fostering understanding of complex systems in biology education: pedagogies, guidelines and insights from classroom-based research</source>. eds. <person-group person-group-type="editor"><name><surname>Assaraf</surname> <given-names>O. B. Z.</given-names></name> <name><surname>Knippels</surname> <given-names>M.-C. P. J.</given-names></name></person-group> (<publisher-loc>Cham, Switzerland</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>17</fpage>&#x2013;<lpage>40</lpage>.</citation></ref>
<ref id="ref20"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Early</surname> <given-names>R.</given-names></name> <name><surname>Bradley</surname> <given-names>B. A.</given-names></name> <name><surname>Dukes</surname> <given-names>J. S.</given-names></name> <name><surname>Lawler</surname> <given-names>J. J.</given-names></name> <name><surname>Olden</surname> <given-names>J. D.</given-names></name> <name><surname>Blumenthal</surname> <given-names>D. M.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>Global threats from invasive alien species in the twenty-first century and national response capacities</article-title>. <source>Nat. Commun.</source> <volume>7</volume>:<fpage>12485</fpage>. doi: <pub-id pub-id-type="doi">10.1038/ncomms12485</pub-id>, PMID: <pub-id pub-id-type="pmid">27549569</pub-id></citation></ref>
<ref id="ref21"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Eilam</surname> <given-names>B.</given-names></name></person-group> (<year>2012</year>). <article-title>System thinking and feeding relations: learning with a live ecosystem model</article-title>. <source>Instr. Sci.</source> <volume>40</volume>, <fpage>213</fpage>&#x2013;<lpage>239</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11251-011-9175-4</pub-id></citation></ref>
<ref id="ref22"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Eilam</surname> <given-names>B.</given-names></name> <name><surname>Omar</surname> <given-names>S. Y.</given-names></name></person-group> (<year>2022</year>). &#x201C;<article-title>Science teachers&#x2019; construction of knowledge about simulations and population size via performing inquiry with simulations of growing vs. descending levels of complexity</article-title>&#x201D; in <source>Fostering understanding of complex systems in biology education: pedagogies, guidelines and insights from classroom-based research</source>. eds. <person-group person-group-type="editor"><name><surname>Assaraf</surname> <given-names>O. B. Z.</given-names></name> <name><surname>Knippels</surname> <given-names>M.-C. P. J.</given-names></name></person-group> (<publisher-loc>Cham</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>205</fpage>&#x2013;<lpage>226</lpage>. doi: <pub-id pub-id-type="doi">10.1007/978-3-030-98144-0_10</pub-id></citation></ref>
<ref id="ref23"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Eliam</surname> <given-names>B.</given-names></name> <name><surname>Reisfeld</surname> <given-names>D.</given-names></name></person-group> (<year>2017</year>). <article-title>A curriculum unit for promoting complex system thinking: the case of combined system dynamics and agent based models for population growth</article-title>. <source>J. Adv. Educ. Res.</source> <volume>2</volume>, <fpage>39</fpage>&#x2013;<lpage>60</lpage>. doi: <pub-id pub-id-type="doi">10.22606/jaer.2017.22001</pub-id></citation></ref>
<ref id="ref24"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Ericsson</surname> <given-names>K. A.</given-names></name> <name><surname>Simon</surname> <given-names>H. A.</given-names></name></person-group> (<year>1993</year>). <source>Protocol analysis: Verbal reports as data</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press.</publisher-name></citation></ref>
<ref id="ref25"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Evagorou</surname> <given-names>M.</given-names></name> <name><surname>Korfiatis</surname> <given-names>K.</given-names></name> <name><surname>Nicolaou</surname> <given-names>C.</given-names></name> <name><surname>Constantinou</surname> <given-names>C.</given-names></name></person-group> (<year>2009</year>). <article-title>An investigation of the potential of interactive simulations for developing system thinking skills in elementary school: a case study with fifth-graders and sixth-graders</article-title>. <source>Int. J. Sci. Educ.</source> <volume>31</volume>, <fpage>655</fpage>&#x2013;<lpage>674</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500690701749313</pub-id></citation></ref>
<ref id="ref26"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Giere</surname> <given-names>R. N.</given-names></name> <name><surname>Bickle</surname> <given-names>J.</given-names></name> <name><surname>Mauldin</surname> <given-names>R. F.</given-names></name></person-group> (<year>2006</year>). <source>Understanding scientific reasoning</source>. <publisher-loc>Belmont, CA</publisher-loc>: <publisher-name>Wadsworth</publisher-name>.</citation></ref>
<ref id="ref27"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Gilbert</surname> <given-names>J. K.</given-names></name> <name><surname>Justi</surname> <given-names>R.</given-names></name></person-group> (<year>2016</year>). <source>Modelling-based teaching in science education</source>. <publisher-loc>Cham</publisher-loc>: <publisher-name>Springer.</publisher-name></citation></ref>
<ref id="ref28"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gilissen</surname> <given-names>M. G. R.</given-names></name> <name><surname>Knippels</surname> <given-names>M.-C. P. J.</given-names></name> <name><surname>Verhoeff</surname> <given-names>R. P.</given-names></name> <name><surname>van Joolingen</surname> <given-names>W. R.</given-names></name></person-group> (<year>2020</year>). <article-title>Teachers&#x2019; and educators&#x2019; perspectives on systems thinking and its implementation in Dutch biology education</article-title>. <source>J. Biol. Educ.</source> <volume>54</volume>, <fpage>485</fpage>&#x2013;<lpage>496</lpage>. doi: <pub-id pub-id-type="doi">10.1080/00219266.2019.1609564</pub-id></citation></ref>
<ref id="ref29"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gogolin</surname> <given-names>S.</given-names></name> <name><surname>Kr&#x00FC;ger</surname> <given-names>D.</given-names></name></person-group> (<year>2018</year>). <article-title>Students&#x2019; understanding of the nature and purpose of models</article-title>. <source>J. Res. Sci. Teach.</source> <volume>55</volume>, <fpage>1313</fpage>&#x2013;<lpage>1338</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.21453</pub-id></citation></ref>
<ref id="ref30"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>G&#x00F6;hner</surname> <given-names>M.</given-names></name> <name><surname>Krell</surname> <given-names>M.</given-names></name></person-group> (<year>2022</year>). <article-title>Preservice science teachers&#x2019; strategies in scientific reasoning: the case of modeling</article-title>. <source>Res. Sci. Educ.</source> <volume>52</volume>, <fpage>395</fpage>&#x2013;<lpage>414</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11165-020-09945-7</pub-id></citation></ref>
<ref id="ref31"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gotwals</surname> <given-names>A. W.</given-names></name> <name><surname>Songer</surname> <given-names>N. B.</given-names></name></person-group> (<year>2010</year>). <article-title>Reasoning up and down a food chain: using an assessment framework to investigate students&#x2019; middle knowledge</article-title>. <source>Sci. Educ.</source> <volume>94</volume>, <fpage>259</fpage>&#x2013;<lpage>281</lpage>. doi: <pub-id pub-id-type="doi">10.1002/sce.20368</pub-id></citation></ref>
<ref id="ref32"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gouvea</surname> <given-names>J.</given-names></name> <name><surname>Passmore</surname> <given-names>C.</given-names></name></person-group> (<year>2017</year>). <article-title>&#x2018;Models of&#x2019; versus &#x2018;models for&#x2019;: toward an agent-based conception of modeling in the science classroom</article-title>. <source>Sci. Educ. (Dordr.)</source> <volume>26</volume>, <fpage>49</fpage>&#x2013;<lpage>63</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11191-017-9884-4</pub-id></citation></ref>
<ref id="ref33"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Grotzer</surname> <given-names>T. A.</given-names></name> <name><surname>Basca</surname> <given-names>B. B.</given-names></name></person-group> (<year>2003</year>). <article-title>How does grasping the underlying causal structures of ecosystems impact students&#x2019; understanding?</article-title> <source>J. Biol. Educ.</source> <volume>38</volume>, <fpage>16</fpage>&#x2013;<lpage>29</lpage>. doi: <pub-id pub-id-type="doi">10.1080/00219266.2003.9655891</pub-id></citation></ref>
<ref id="ref34"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Grotzer</surname> <given-names>T. A.</given-names></name> <name><surname>Powell</surname> <given-names>M. M.</given-names></name> <name><surname>Derbiszewska</surname> <given-names>K. M.</given-names></name> <name><surname>Courter</surname> <given-names>C. J.</given-names></name> <name><surname>Kamarainen</surname> <given-names>A. M.</given-names></name> <name><surname>Metcalf</surname> <given-names>S. J.</given-names></name> <etal/></person-group>. (<year>2015</year>). <article-title>Turning transfer inside out: the affordances of virtual worlds and mobile devices in real world contexts for teaching about causality across time and distance in ecosystems</article-title>. <source>Technol. Knowl. Learn.</source> <volume>20</volume>, <fpage>43</fpage>&#x2013;<lpage>69</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10758-014-9241-5</pub-id></citation></ref>
<ref id="ref35"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hmelo-Silver</surname> <given-names>C. E.</given-names></name> <name><surname>Jordan</surname> <given-names>R. C.</given-names></name> <name><surname>Eberbach</surname> <given-names>C.</given-names></name> <name><surname>Sinha</surname> <given-names>S.</given-names></name></person-group> (<year>2017</year>). <article-title>Systems learning with a conceptual representation: A quasi-experimental study</article-title>. <source>Instr. Sci.</source> <volume>45</volume>, <fpage>53</fpage>&#x2013;<lpage>72</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11251-016-9392-y</pub-id></citation></ref>
<ref id="ref36"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hmelo-Silver</surname> <given-names>C. E.</given-names></name> <name><surname>Liu</surname> <given-names>L.</given-names></name> <name><surname>Gray</surname> <given-names>S.</given-names></name> <name><surname>Jordan</surname> <given-names>R. C.</given-names></name></person-group> (<year>2015</year>). <article-title>Using representational tools to learn about complex systems: A tale of two classrooms</article-title>. <source>J. Res. Sci. Teach.</source> <volume>52</volume>, <fpage>6</fpage>&#x2013;<lpage>35</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.21187</pub-id></citation></ref>
<ref id="ref37"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hmelo-Silver</surname> <given-names>C. E.</given-names></name> <name><surname>Marathe</surname> <given-names>S.</given-names></name> <name><surname>Liu</surname> <given-names>L.</given-names></name></person-group> (<year>2007</year>). <article-title>Fish swim, rocks sit, and lungs breathe: expert-novice understanding of complex systems</article-title>. <source>J. Learn. Sci.</source> <volume>16</volume>, <fpage>307</fpage>&#x2013;<lpage>331</lpage>. doi: <pub-id pub-id-type="doi">10.1080/10508400701413401</pub-id></citation></ref>
<ref id="ref38"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hogan</surname> <given-names>K.</given-names></name></person-group> (<year>2000</year>). <article-title>Assessing students&#x2019; systems reasoning in ecology</article-title>. <source>J. Biol. Educ.</source> <volume>35</volume>, <fpage>22</fpage>&#x2013;<lpage>28</lpage>. doi: <pub-id pub-id-type="doi">10.1080/00219266.2000.9655731</pub-id></citation></ref>
<ref id="ref39"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hokayem</surname> <given-names>H.</given-names></name></person-group> (<year>2016</year>). <article-title>Patterns of reasoning about ecological systemic reasoning for early elementary students</article-title>. <source>Sci. Educ. Int.</source> <volume>27</volume>, <fpage>117</fpage>&#x2013;<lpage>135</lpage>.</citation></ref>
<ref id="ref40"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hokayem</surname> <given-names>H.</given-names></name> <name><surname>Gotwals</surname> <given-names>A. W.</given-names></name></person-group> (<year>2016</year>). <article-title>Early elementary students&#x2019; understanding of complex ecosystems: A learning progression approach</article-title>. <source>J. Res. Sci. Teach.</source> <volume>53</volume>, <fpage>1524</fpage>&#x2013;<lpage>1545</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.21336</pub-id></citation></ref>
<ref id="ref41"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll1">IPBES</collab></person-group>. (<year>2019</year>). <source>Global assessment report on biodiversity and ecosystem services of the intergovernmental science-policy platform on biodiversity and ecosystem services</source>. <publisher-loc>Bonn</publisher-loc>: <publisher-name>IPBES</publisher-name>.</citation></ref>
<ref id="ref42"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Jackson</surname> <given-names>M. C.</given-names></name> <name><surname>Mansell</surname> <given-names>G. J.</given-names></name> <name><surname>Flood</surname> <given-names>R. L.</given-names></name> <name><surname>Blackham</surname> <given-names>R. B.</given-names></name> <name><surname>Probert</surname> <given-names>S. V. E.</given-names></name></person-group>, Eds. (<year>1991</year>). <source>Systems thinking in Europe</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Springer.</publisher-name></citation></ref>
<ref id="ref43"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jin</surname> <given-names>H.</given-names></name> <name><surname>Shin</surname> <given-names>H. J.</given-names></name> <name><surname>Hokayem</surname> <given-names>H.</given-names></name> <name><surname>Qureshi</surname> <given-names>F.</given-names></name> <name><surname>Jenkins</surname> <given-names>T.</given-names></name></person-group> (<year>2019</year>). <article-title>Secondary students&#x2019; understanding of ecosystems: A learning progression approach</article-title>. <source>Int. J. Sci. Math. Educ.</source> <volume>17</volume>, <fpage>217</fpage>&#x2013;<lpage>235</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10763-017-9864-9</pub-id></citation></ref>
<ref id="ref44"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jordan</surname> <given-names>R. C.</given-names></name> <name><surname>Hmelo-Silver</surname> <given-names>C. E.</given-names></name> <name><surname>Liu</surname> <given-names>L.</given-names></name> <name><surname>Gray</surname> <given-names>S. A.</given-names></name></person-group> (<year>2013</year>). <article-title>Fostering reasoning about complex systems: using the aquarium to teach systems thinking</article-title>. <source>Appl. Environ. Educ. Commun.</source> <volume>12</volume>, <fpage>55</fpage>&#x2013;<lpage>64</lpage>. doi: <pub-id pub-id-type="doi">10.1080/1533015X.2013.797860</pub-id></citation></ref>
<ref id="ref45"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Justi</surname> <given-names>R.</given-names></name> <name><surname>Gilbert</surname> <given-names>J. K.</given-names></name></person-group> (<year>2002</year>). <article-title>Modelling, teachers&#x2019; views on the nature of modelling, and implications for the education of modellers</article-title>. <source>Int. J. Sci. Educ.</source> <volume>24</volume>, <fpage>369</fpage>&#x2013;<lpage>387</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500690110110142</pub-id></citation></ref>
<ref id="ref46"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kiesewetter</surname> <given-names>A.</given-names></name> <name><surname>Schmiemann</surname> <given-names>P.</given-names></name></person-group> (<year>2022</year>). <article-title>Understanding homeostatic regulation: the role of relationships and conditions in feedback loop reasoning</article-title>. <source>CBE Life Sci. Educ.</source> <volume>21</volume>:<fpage>ar56</fpage>. doi: <pub-id pub-id-type="doi">10.1187/cbe.21-04-0092</pub-id>, PMID: <pub-id pub-id-type="pmid">35998161</pub-id></citation></ref>
<ref id="ref47"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Klahr</surname> <given-names>D.</given-names></name> <name><surname>Dunbar</surname> <given-names>K.</given-names></name></person-group> (<year>1988</year>). <article-title>Dual space search during scientific reasoning</article-title>. <source>Cogn. Sci.</source> <volume>12</volume>, <fpage>1</fpage>&#x2013;<lpage>48</lpage>. doi: <pub-id pub-id-type="doi">10.1207/s15516709cog1201_1</pub-id></citation></ref>
<ref id="ref48"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Knuuttila</surname> <given-names>T.</given-names></name> <name><surname>Boon</surname> <given-names>M.</given-names></name></person-group> (<year>2011</year>). <article-title>How do models give us knowledge? The case of Carnot&#x2019;s ideal heat engine</article-title>. <source>Eur. J. Philos. Sci.</source> <volume>1</volume>, <fpage>309</fpage>&#x2013;<lpage>334</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s13194-011-0029-3</pub-id></citation></ref>
<ref id="ref49"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Krell</surname> <given-names>M.</given-names></name> <name><surname>Kr&#x00FC;ger</surname> <given-names>D.</given-names></name></person-group> (<year>2016</year>). <article-title>Testing models: A key aspect to promote teaching activities related to models and modelling in biology lessons?</article-title> <source>J. Biol. Educ.</source> <volume>50</volume>, <fpage>160</fpage>&#x2013;<lpage>173</lpage>. doi: <pub-id pub-id-type="doi">10.1080/00219266.2015.1028570</pub-id></citation></ref>
<ref id="ref50"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Krell</surname> <given-names>M.</given-names></name> <name><surname>Walzer</surname> <given-names>C.</given-names></name> <name><surname>Hergert</surname> <given-names>S.</given-names></name> <name><surname>Kr&#x00FC;ger</surname> <given-names>D.</given-names></name></person-group> (<year>2019</year>). <article-title>Development and application of a category system to describe pre-service science teachers&#x2019; activities in the process of scientific modelling</article-title>. <source>Res. Sci. Educ.</source> <volume>49</volume>, <fpage>1319</fpage>&#x2013;<lpage>1345</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11165-017-9657-8</pub-id></citation></ref>
<ref id="ref51"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Kuckartz</surname> <given-names>U.</given-names></name> <name><surname>R&#x00E4;diker</surname> <given-names>S.</given-names></name></person-group> (<year>2019</year>). <source>Analyzing qualitative data with MAXQDA: Text, audio, and video</source>. <publisher-loc>Cham</publisher-loc>: <publisher-name>Springer.</publisher-name></citation></ref>
<ref id="ref52"><citation citation-type="other"><person-group person-group-type="author"><collab id="coll2">Kultusministerkonferenz</collab></person-group>. (<year>2004</year>). <source>Einheitliche Pr&#x00FC;fungsanforderungen in der Abiturpr&#x00FC;fung Biologie [common examination requirements in the final exam in the abitur, the highest secondary school diploma, in biology]</source>. Available at: <comment>(</comment><ext-link xlink:href="https://www.kmk.org/fileadmin/veroeffentlichungen_beschluesse/1989/1989_12_01-EPA-Biologie.pdf" ext-link-type="uri">https://www.kmk.org/fileadmin/veroeffentlichungen_beschluesse/1989/1989_12_01-EPA-Biologie.pdf</ext-link>)</citation></ref>
<ref id="ref53"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll3">Kultusministerkonferenz</collab></person-group>. (<year>2005</year>). <source>Bildungsstandards im Fach Biologie f&#x00FC;r den Mittleren Schulabschluss [educational standards in biology for the secondary school diploma]</source>. <publisher-loc>M&#x00FC;nchen</publisher-loc>: <publisher-name>Luchterhand</publisher-name>.</citation></ref>
<ref id="ref54"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ladrera</surname> <given-names>R.</given-names></name> <name><surname>Robredo</surname> <given-names>B.</given-names></name> <name><surname>Ortega-Lasuen</surname> <given-names>U.</given-names></name> <name><surname>D&#x00ED;ez</surname> <given-names>J. R.</given-names></name> <name><surname>Ruiz-Gonz&#x00E1;lez</surname> <given-names>A.</given-names></name></person-group> (<year>2020</year>). <article-title>Unprepared to deal with invasion: pre-service teachers&#x2019; perception, knowledge and attitudes toward invasive species</article-title>. <source>Sustainability</source> <volume>12</volume>:<fpage>10543</fpage>. doi: <pub-id pub-id-type="doi">10.3390/su122410543</pub-id></citation></ref>
<ref id="ref55"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Landis</surname> <given-names>J. R.</given-names></name> <name><surname>Koch</surname> <given-names>G. G.</given-names></name></person-group> (<year>1977</year>). <article-title>The measurement of observer agreement for categorical data</article-title>. <source>Biometrics</source> <volume>33</volume>, <fpage>159</fpage>&#x2013;<lpage>174</lpage>. doi: <pub-id pub-id-type="doi">10.2307/2529310</pub-id></citation></ref>
<ref id="ref56"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>T. D.</given-names></name> <name><surname>Gail Jones</surname> <given-names>M.</given-names></name> <name><surname>Chesnutt</surname> <given-names>K.</given-names></name></person-group> (<year>2019</year>). <article-title>Teaching systems thinking in the context of the water cycle</article-title>. <source>Res. Sci. Educ.</source> <volume>49</volume>, <fpage>137</fpage>&#x2013;<lpage>172</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11165-017-9613-7</pub-id></citation></ref>
<ref id="ref57"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liu</surname> <given-names>L.</given-names></name> <name><surname>Hmelo-Silver</surname> <given-names>C. E.</given-names></name></person-group> (<year>2009</year>). <article-title>Promoting complex systems learning through the use of conceptual representations in hypermedia</article-title>. <source>J. Res. Sci. Teach.</source> <volume>46</volume>, <fpage>1023</fpage>&#x2013;<lpage>1040</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.20297</pub-id></citation></ref>
<ref id="ref58"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mambrey</surname> <given-names>S.</given-names></name> <name><surname>Schreiber</surname> <given-names>N.</given-names></name> <name><surname>Schmiemann</surname> <given-names>P.</given-names></name></person-group> (<year>2022a</year>). <article-title>Young students&#x2019; reasoning about ecosystems: the role of systems thinking, knowledge, conceptions, and representation</article-title>. <source>Res. Sci. Educ.</source> <volume>52</volume>, <fpage>79</fpage>&#x2013;<lpage>98</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11165-020-09917-x</pub-id></citation></ref>
<ref id="ref59"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mambrey</surname> <given-names>S.</given-names></name> <name><surname>Timm</surname> <given-names>J.</given-names></name> <name><surname>Landskron</surname> <given-names>J. J.</given-names></name> <name><surname>Schmiemann</surname> <given-names>P.</given-names></name></person-group> (<year>2020</year>). <article-title>The impact of system specifics on systems thinking</article-title>. <source>J. Res. Sci. Teach.</source> <volume>57</volume>, <fpage>1632</fpage>&#x2013;<lpage>1651</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.21649</pub-id></citation></ref>
<ref id="ref60"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Mambrey</surname> <given-names>S.</given-names></name> <name><surname>Wellmanns</surname> <given-names>A.</given-names></name> <name><surname>Timm</surname> <given-names>J.</given-names></name> <name><surname>Schmiemann</surname> <given-names>P.</given-names></name></person-group> (<year>2022b</year>). &#x201C;<article-title>Systems thinking in ecological and physiological systems and the role of representations</article-title>&#x201D; in <source>Fostering understanding of complex systems in biology education: Pedagogies, guidelines and insights from classroom-based research</source>. eds. <person-group person-group-type="editor"><name><surname>Assaraf</surname> <given-names>O. B. Z.</given-names></name> <name><surname>Knippels</surname> <given-names>M.-C. P. J.</given-names></name></person-group> (<publisher-loc>Cham</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>105</fpage>&#x2013;<lpage>121</lpage>.</citation></ref>
<ref id="ref61"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Mayring</surname> <given-names>P.</given-names></name></person-group> (<year>2014</year>). <source>Qualitative content analysis: theoretical foundation, basic procedures and software solution</source>. <publisher-loc>Klagenfurt</publisher-loc>.</citation></ref>
<ref id="ref62"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>McKinney</surname> <given-names>M. L.</given-names></name> <name><surname>Lockwood</surname> <given-names>J. L.</given-names></name></person-group> (<year>1999</year>). <article-title>Biotic homogenization: a few winners replacing many losers in the next mass extinction</article-title>. <source>Trends Ecol. Evol.</source> <volume>14</volume>, <fpage>450</fpage>&#x2013;<lpage>453</lpage>. doi: <pub-id pub-id-type="doi">10.1016/s0169-5347(99)01679-1</pub-id>, PMID: <pub-id pub-id-type="pmid">10511724</pub-id></citation></ref>
<ref id="ref63"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mehren</surname> <given-names>R.</given-names></name> <name><surname>Rempfler</surname> <given-names>A.</given-names></name> <name><surname>Buchholz</surname> <given-names>J.</given-names></name> <name><surname>Hartig</surname> <given-names>J.</given-names></name> <name><surname>Ulrich-Riedhammer</surname> <given-names>E. M.</given-names></name></person-group> (<year>2018</year>). <article-title>System competence modelling: theoretical foundation and empirical validation of a model involving natural, social and human-environment systems</article-title>. <source>J. Res. Sci. Teach.</source> <volume>55</volume>, <fpage>685</fpage>&#x2013;<lpage>711</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.21436</pub-id></citation></ref>
<ref id="ref64"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Meister</surname> <given-names>J.</given-names></name> <name><surname>Upmeier zu Belzen</surname> <given-names>A.</given-names></name></person-group> (<year>2020</year>). <article-title>Investigating students&#x2019; modelling styles in the process of scientific-mathematical modelling</article-title>. <source>Sci. Educ. Rev. Lett.</source> <volume>2019</volume>, <fpage>8</fpage>&#x2013;<lpage>14</lpage>. doi: <pub-id pub-id-type="doi">10.18452/21039</pub-id></citation></ref>
<ref id="ref65"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>M&#x00E9;trailler</surname> <given-names>Y. A.</given-names></name> <name><surname>Reijnen</surname> <given-names>E.</given-names></name> <name><surname>Kneser</surname> <given-names>C.</given-names></name> <name><surname>Opwis</surname> <given-names>K.</given-names></name></person-group> (<year>2008</year>). <article-title>Scientific problem solving in a virtual laboratory: A comparison between individuals and pairs</article-title>. <source>Swiss J. Psychol.</source> <volume>67</volume>, <fpage>71</fpage>&#x2013;<lpage>83</lpage>. doi: <pub-id pub-id-type="doi">10.1024/1421-0185.67.2.71</pub-id></citation></ref>
<ref id="ref66"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll4">Microsoft Corporation</collab></person-group> (<year>2020</year>). <source>Microsoft whiteboard [computer program]: Version 22.11028.0.9070</source>. <publisher-loc>Redmond, WA</publisher-loc>: <publisher-name>Microsoft Corporation</publisher-name>.</citation></ref>
<ref id="ref67"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Momsen</surname> <given-names>J.</given-names></name> <name><surname>Speth</surname> <given-names>E. B.</given-names></name> <name><surname>Wyse</surname> <given-names>S.</given-names></name> <name><surname>Long</surname> <given-names>T.</given-names></name></person-group> (<year>2022</year>). <article-title>Using systems and systems thinking to unify biology education. CBE: life</article-title>. <source>Sci. Educ.</source> <volume>21</volume>:<fpage>es3</fpage>. doi: <pub-id pub-id-type="doi">10.1187/cbe.21-05-0118</pub-id>, PMID: <pub-id pub-id-type="pmid">35499820</pub-id></citation></ref>
<ref id="ref68"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll5">National Research Council</collab></person-group>. (<year>2012</year>). <source>A framework for K-12 science education: Practices, crosscutting concepts, and core ideas</source>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>National Academies Press</publisher-name>.</citation></ref>
<ref id="ref69"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Nersessian</surname> <given-names>N. J.</given-names></name></person-group> (<year>2008</year>). <source>Creating scientific concepts</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name></citation></ref>
<ref id="ref70"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll6">NGSS Lead States</collab></person-group>. (<year>2013</year>). <source>Next generation science standards: For states by states</source>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>National Academies Press</publisher-name>.</citation></ref>
<ref id="ref71"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll7">OECD</collab></person-group>. (<year>2022</year>). <source>Are students ready to take on environmental challenges?</source> <publisher-loc>Paris</publisher-loc>: <publisher-name>OECD Publishing.</publisher-name></citation></ref>
<ref id="ref72"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Oh</surname> <given-names>P. S.</given-names></name> <name><surname>Oh</surname> <given-names>S. J.</given-names></name></person-group> (<year>2011</year>). <article-title>What teachers of science need to know about models: an overview</article-title>. <source>Int. J. Sci. Educ.</source> <volume>33</volume>, <fpage>1109</fpage>&#x2013;<lpage>1130</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500693.2010.502191</pub-id></citation></ref>
<ref id="ref73"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Oh</surname> <given-names>S.</given-names></name> <name><surname>Wildemuth</surname> <given-names>B. M.</given-names></name></person-group> (<year>2016</year>). &#x201C;<article-title>Think-aloud protocols</article-title>&#x201D; in <source>Applications of social research methods to questions in information and library science</source>. ed. <person-group person-group-type="editor"><name><surname>Wildemuth</surname> <given-names>B. M.</given-names></name></person-group> (<publisher-loc>Santa Barbara, CA</publisher-loc>: <publisher-name>Libraries Unlimited</publisher-name>), <fpage>198</fpage>&#x2013;<lpage>208</lpage>.</citation></ref>
<ref id="ref74"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Passmore</surname> <given-names>C.</given-names></name> <name><surname>Gouvea</surname> <given-names>J. S.</given-names></name> <name><surname>Giere</surname> <given-names>R.</given-names></name></person-group> (<year>2014</year>). &#x201C;<article-title>Models in science and in learning science: focusing scientific practice on sense-making</article-title>&#x201D; in <source>International handbook of research in history, philosophy and science teaching</source>. ed. <person-group person-group-type="editor"><name><surname>Matthews</surname> <given-names>M. R.</given-names></name></person-group> (<publisher-loc>Dordrecht</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>1171</fpage>&#x2013;<lpage>1202</lpage>.</citation></ref>
<ref id="ref75"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Peper</surname> <given-names>R. J.</given-names></name> <name><surname>Mayer</surname> <given-names>R. E.</given-names></name></person-group> (<year>1986</year>). <article-title>Generative effects of note-taking during science lectures</article-title>. <source>J. Educ. Psychol.</source> <volume>78</volume>, <fpage>34</fpage>&#x2013;<lpage>38</lpage>. doi: <pub-id pub-id-type="doi">10.1037/0022-0663.78.1.34</pub-id></citation></ref>
<ref id="ref76"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll8">R Core Team</collab></person-group>. (<year>2022</year>). <source>R: A language and environment for statistical computing [computer program]</source>. <publisher-loc>Vienna</publisher-loc>: <publisher-name>R Foundation for Statistical Computing</publisher-name>.</citation></ref>
<ref id="ref77"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rachmatullah</surname> <given-names>A.</given-names></name> <name><surname>Wiebe</surname> <given-names>E. N.</given-names></name></person-group> (<year>2022</year>). <article-title>Building a computational model of food webs: impacts on middle school students&#x2019; computational and systems thinking skills</article-title>. <source>J. Res. Sci. Teach.</source> <volume>59</volume>, <fpage>585</fpage>&#x2013;<lpage>618</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.21738</pub-id></citation></ref>
<ref id="ref78"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rellensmann</surname> <given-names>J.</given-names></name> <name><surname>Schukajlow</surname> <given-names>S.</given-names></name> <name><surname>Leopold</surname> <given-names>C.</given-names></name></person-group> (<year>2017</year>). <article-title>Make a drawing: effects of strategic knowledge, drawing accuracy, and type of drawing on students&#x2019; mathematical modelling performance</article-title>. <source>Educ. Stud. Math.</source> <volume>95</volume>, <fpage>53</fpage>&#x2013;<lpage>78</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10649-016-9736-1</pub-id></citation></ref>
<ref id="ref79"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Riess</surname> <given-names>W.</given-names></name> <name><surname>Mischo</surname> <given-names>C.</given-names></name></person-group> (<year>2010</year>). <article-title>Promoting systems thinking through biology lessons</article-title>. <source>Int. J. Sci. Educ.</source> <volume>32</volume>, <fpage>705</fpage>&#x2013;<lpage>725</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500690902769946</pub-id></citation></ref>
<ref id="ref80"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sauv&#x00E9;</surname> <given-names>L.</given-names></name> <name><surname>Renaud</surname> <given-names>L.</given-names></name> <name><surname>Kaufmann</surname> <given-names>D.</given-names></name> <name><surname>Marquis</surname> <given-names>J.-S.</given-names></name></person-group> (<year>2007</year>). <article-title>Distinguishing between games and simulations: a systematic review</article-title>. <source>Educ. Technol. Soc.</source> <volume>10</volume>, <fpage>247</fpage>&#x2013;<lpage>256</lpage>.</citation></ref>
<ref id="ref81"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Scheffer</surname> <given-names>M.</given-names></name> <name><surname>Carpenter</surname> <given-names>S. R.</given-names></name></person-group> (<year>2003</year>). <article-title>Catastrophic regime shifts in ecosystems: linking theory to observation</article-title>. <source>Trends Ecol. Evol.</source> <volume>18</volume>, <fpage>648</fpage>&#x2013;<lpage>656</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.tree.2003.09.002</pub-id></citation></ref>
<ref id="ref82"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Schreier</surname> <given-names>M.</given-names></name></person-group> (<year>2012</year>). <source>Qualitative content analysis in practice</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Sage</publisher-name>.</citation></ref>
<ref id="ref83"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schwarz</surname> <given-names>C. V.</given-names></name> <name><surname>Ke</surname> <given-names>L.</given-names></name> <name><surname>Salgado</surname> <given-names>M.</given-names></name> <name><surname>Manz</surname> <given-names>E.</given-names></name></person-group> (<year>2022</year>). <article-title>Beyond assessing knowledge about models and modeling: moving toward expansive, meaningful, and equitable modeling practice</article-title>. <source>J. Res. Sci. Teach.</source> <volume>59</volume>, <fpage>1086</fpage>&#x2013;<lpage>1096</lpage>. doi: <pub-id pub-id-type="doi">10.1002/tea.21770</pub-id></citation></ref>
<ref id="ref84"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schwarz</surname> <given-names>C. V.</given-names></name> <name><surname>White</surname> <given-names>B. Y.</given-names></name></person-group> (<year>2005</year>). <article-title>Metamodeling knowledge: developing students&#x2019; understanding of scientific modeling</article-title>. <source>Cogn. Instr.</source> <volume>23</volume>, <fpage>165</fpage>&#x2013;<lpage>205</lpage>. doi: <pub-id pub-id-type="doi">10.1207/s1532690xci2302_1</pub-id></citation></ref>
<ref id="ref85"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Snapir</surname> <given-names>Z.</given-names></name> <name><surname>Eberbach</surname> <given-names>C.</given-names></name> <name><surname>Ben Zvi Assaraf</surname> <given-names>O.</given-names></name> <name><surname>Hmelo-Silver</surname> <given-names>C. E.</given-names></name> <name><surname>Tripto</surname> <given-names>J.</given-names></name></person-group> (<year>2017</year>). <article-title>Characterising the development of the understanding of human body systems in high-school biology students: a longitudinal study</article-title>. <source>Int. J. Sci. Educ.</source> <volume>39</volume>, <fpage>2092</fpage>&#x2013;<lpage>2127</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500693.2017.1364445</pub-id></citation></ref>
<ref id="ref86"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Soetaert</surname> <given-names>K.</given-names></name> <name><surname>Petzoldt</surname> <given-names>T.</given-names></name> <name><surname>Setzer</surname> <given-names>R. W.</given-names></name></person-group> (<year>2010</year>). <article-title>Solving differential equations in R: package deSolve</article-title>. <source>J. Stat. Soft.</source> <volume>33</volume>, <fpage>1</fpage>&#x2013;<lpage>25</lpage>. doi: <pub-id pub-id-type="doi">10.18637/jss.v033.i09</pub-id></citation></ref>
<ref id="ref87"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sommer</surname> <given-names>C.</given-names></name> <name><surname>L&#x00FC;cken</surname> <given-names>M.</given-names></name></person-group> (<year>2010</year>). <article-title>System competence: are elementary students able to deal with a biological system?</article-title> <source>Nord. Stud. Sci. Educ.</source> <volume>6</volume>, <fpage>125</fpage>&#x2013;<lpage>143</lpage>. doi: <pub-id pub-id-type="doi">10.5617/nordina.255</pub-id></citation></ref>
<ref id="ref88"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Streiling</surname> <given-names>S.</given-names></name> <name><surname>H&#x00F6;rsch</surname> <given-names>C.</given-names></name> <name><surname>Rie&#x00DF;</surname> <given-names>W.</given-names></name></person-group> (<year>2021</year>). <article-title>Effects of teacher training in systems thinking on biology students: an intervention study</article-title>. <source>Sustainability</source> <volume>13</volume>:<fpage>7631</fpage>. doi: <pub-id pub-id-type="doi">10.3390/su13147631</pub-id></citation></ref>
<ref id="ref89"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Strogatz</surname> <given-names>S. H.</given-names></name></person-group> (<year>2001</year>). <article-title>Exploring complex networks</article-title>. <source>Nature</source> <volume>410</volume>, <fpage>268</fpage>&#x2013;<lpage>276</lpage>. doi: <pub-id pub-id-type="doi">10.1038/35065725</pub-id></citation></ref>
<ref id="ref90"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Torkar</surname> <given-names>G.</given-names></name> <name><surname>Korfiatis</surname> <given-names>K.</given-names></name></person-group> (<year>2022</year>). &#x201C;<article-title>Pre-service teachers&#x2019; conceptual schemata and system reasoning about the carbon cycle and climate change: an exploratory study of a learning framework for understanding complex systems</article-title>&#x201D; in <source>Fostering understanding of complex systems in biology education: Pedagogies, guidelines and insights from classroom-based research</source>. eds. <person-group person-group-type="editor"><name><surname>Assaraf</surname> <given-names>O. B. Z.</given-names></name> <name><surname>Knippels</surname> <given-names>M.-C. P. J.</given-names></name></person-group> (<publisher-loc>Cham</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>151</fpage>&#x2013;<lpage>169</lpage>.</citation></ref>
<ref id="ref91"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tripto</surname> <given-names>J.</given-names></name> <name><surname>Ben Zvi Assaraf</surname> <given-names>O.</given-names></name> <name><surname>Amit</surname> <given-names>M.</given-names></name></person-group> (<year>2013</year>). <article-title>Mapping what they know: concept maps as an effective tool for assessing students&#x2019; systems thinking</article-title>. <source>Am. J. Oper. Res.</source> <volume>3</volume>, <fpage>245</fpage>&#x2013;<lpage>258</lpage>. doi: <pub-id pub-id-type="doi">10.4236/ajor.2013.31A022</pub-id></citation></ref>
<ref id="ref92"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tripto</surname> <given-names>J.</given-names></name> <name><surname>Ben Zvi Assaraf</surname> <given-names>O.</given-names></name> <name><surname>Amit</surname> <given-names>M.</given-names></name></person-group> (<year>2018</year>). <article-title>Recurring patterns in the development of high school biology students&#x2019; system thinking over time</article-title>. <source>Instr. Sci.</source> <volume>46</volume>, <fpage>639</fpage>&#x2013;<lpage>680</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11251-018-9447-3</pub-id></citation></ref>
<ref id="ref93"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Upmeier Zu Belzen</surname> <given-names>A.</given-names></name> <name><surname>Van Driel</surname> <given-names>J.</given-names></name> <name><surname>Kr&#x00FC;ger</surname> <given-names>D</given-names></name></person-group>. (<year>2019</year>). <article-title>Introducing a framework for modeling competence</article-title>, In <source>Towards a competence-based view on models and modeling in science education</source>, Ed. <person-group person-group-type="author"><name><surname>Belzen</surname> <given-names>A. Upmeier Zu</given-names></name> <name><surname>Kr&#x00FC;ger</surname> <given-names>D.</given-names></name> <name><surname>Van Driel</surname> <given-names>J.</given-names></name></person-group> (<publisher-loc>Cham</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>3</fpage>&#x2013;<lpage>19</lpage>.</citation></ref>
<ref id="ref94"><citation citation-type="book"><person-group person-group-type="author"><collab id="coll9">VERBI Software</collab></person-group>. (<year>2022</year>). <source>MAXQDA 2022 [computer program]</source>. <publisher-loc>Berlin</publisher-loc>: <publisher-name>VERBI Software</publisher-name>.</citation></ref>
<ref id="ref95"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Verhoeff</surname> <given-names>R. P.</given-names></name> <name><surname>Boersma</surname> <given-names>K. T.</given-names></name> <name><surname>Waarlo</surname> <given-names>A. J.</given-names></name></person-group> (<year>2013</year>). &#x201C;<article-title>Multiple representations in modeling strategies for the development of systems thinking in biology education</article-title>&#x201D; in <source>Multiple representations in biological education</source>. eds. <person-group person-group-type="editor"><name><surname>Treagust</surname> <given-names>D. F.</given-names></name> <name><surname>Tsui</surname> <given-names>C.-Y.</given-names></name></person-group> (<publisher-loc>Dordrecht</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>331</fpage>&#x2013;<lpage>348</lpage>.</citation></ref>
<ref id="ref96"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Verhoeff</surname> <given-names>R. P.</given-names></name> <name><surname>Knippels</surname> <given-names>M.-C. P. J.</given-names></name> <name><surname>Gilissen</surname> <given-names>M. G. R.</given-names></name> <name><surname>Boersma</surname> <given-names>K. T.</given-names></name></person-group> (<year>2018</year>). <article-title>The theoretical nature of systems thinking: perspectives on systems thinking in biology education</article-title>. <source>Front. Educ.</source> <volume>3</volume>:<fpage>40</fpage>. doi: <pub-id pub-id-type="doi">10.3389/feduc.2018.00040</pub-id></citation></ref>
<ref id="ref97"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Verhoeff</surname> <given-names>R. P.</given-names></name> <name><surname>Waarlo</surname> <given-names>A. J.</given-names></name> <name><surname>Boersma</surname> <given-names>K. T.</given-names></name></person-group> (<year>2008</year>). <article-title>Systems modelling and the development of coherent understanding of cell biology</article-title>. <source>Int. J. Sci. Educ.</source> <volume>30</volume>, <fpage>543</fpage>&#x2013;<lpage>568</lpage>. doi: <pub-id pub-id-type="doi">10.1080/09500690701237780</pub-id></citation></ref>
<ref id="ref98"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wellmanns</surname> <given-names>A.</given-names></name> <name><surname>Schmiemann</surname> <given-names>P.</given-names></name></person-group> (<year>2022</year>). <article-title>Feedback loop reasoning in physiological contexts</article-title>. <source>J. Biol. Educ.</source> <volume>56</volume>, <fpage>465</fpage>&#x2013;<lpage>485</lpage>. doi: <pub-id pub-id-type="doi">10.1080/00219266.2020.1858929</pub-id></citation></ref>
<ref id="ref99"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Wickham</surname> <given-names>H.</given-names></name></person-group> (<year>2016</year>). <source>ggplot2: Elegant graphics for data analysis</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Springer</publisher-name></citation></ref>
<ref id="ref100"><citation citation-type="confproc"><person-group person-group-type="author"><name><surname>Wiebe</surname> <given-names>E.</given-names></name> <name><surname>London</surname> <given-names>J.</given-names></name> <name><surname>Aksit</surname> <given-names>O.</given-names></name> <name><surname>Mott</surname> <given-names>B. W.</given-names></name> <name><surname>Boyer</surname> <given-names>K. E.</given-names></name> <name><surname>Lester</surname> <given-names>J. C.</given-names></name></person-group> (<year>2019</year>). <article-title>Development of a lean computational thinking abilities assessment for middle grades students</article-title>, In <conf-name>Proceedings of the 50th ACM technical symposium on computer science education</conf-name>, Eds. <person-group person-group-type="editor"><name><surname>Hawthorne</surname> <given-names>E. K.</given-names></name> <name><surname>P&#x00E9;rez-Qui&#x00F1;ones</surname> <given-names>M. A.</given-names></name> <name><surname>Heckman</surname> <given-names>S.</given-names></name> <name><surname>Zhang</surname> <given-names>J.</given-names></name></person-group> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>ACM</publisher-name>), <fpage>456</fpage>&#x2013;<lpage>461</lpage>.</citation></ref>
<ref id="ref101"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wilson</surname> <given-names>K. J.</given-names></name> <name><surname>Long</surname> <given-names>T. M.</given-names></name> <name><surname>Momsen</surname> <given-names>J. L.</given-names></name> <name><surname>Bray Speth</surname> <given-names>E.</given-names></name></person-group> (<year>2020</year>). <article-title>Modeling in the classroom: making relationships and systems visible. CBE: life</article-title>. <source>Sci. Educ.</source> <volume>19</volume>:<fpage>fe1</fpage>. doi: <pub-id pub-id-type="doi">10.1187/cbe.19-11-0255</pub-id>, PMID: <pub-id pub-id-type="pmid">31971875</pub-id></citation></ref>
<ref id="ref102"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yoon</surname> <given-names>S. A.</given-names></name> <name><surname>Anderson</surname> <given-names>E.</given-names></name> <name><surname>Koehler-Yom</surname> <given-names>J.</given-names></name> <name><surname>Evans</surname> <given-names>C.</given-names></name> <name><surname>Park</surname> <given-names>M.</given-names></name> <name><surname>Sheldon</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>Teaching about complex systems is no simple matter: building effective professional development for computer-supported complex systems instruction</article-title>. <source>Instr. Sci.</source> <volume>45</volume>, <fpage>99</fpage>&#x2013;<lpage>121</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s11251-016-9388-7</pub-id></citation></ref>
<ref id="ref103"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yoon</surname> <given-names>S. A.</given-names></name> <name><surname>Goh</surname> <given-names>S.-E.</given-names></name> <name><surname>Park</surname> <given-names>M.</given-names></name></person-group> (<year>2018</year>). <article-title>Teaching and learning about complex systems in K&#x2013;12 science education: A review of empirical studies 1995-2015</article-title>. <source>Rev. Educ. Res.</source> <volume>88</volume>, <fpage>285</fpage>&#x2013;<lpage>325</lpage>. doi: <pub-id pub-id-type="doi">10.3102/0034654317746090</pub-id></citation></ref>
<ref id="ref104"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Zhang</surname> <given-names>L. J.</given-names></name> <name><surname>Zhang</surname> <given-names>D.</given-names></name></person-group> (<year>2019</year>). &#x201C;<article-title>Think-aloud protocols</article-title>&#x201D; in <source>The Routledge handbook of research methods in applied linguistics</source>. eds. <person-group person-group-type="editor"><name><surname>McKinley</surname> <given-names>J.</given-names></name> <name><surname>Rose</surname> <given-names>H.</given-names></name></person-group> (<publisher-loc>London</publisher-loc>: <publisher-name>Routledge</publisher-name>), <fpage>302</fpage>&#x2013;<lpage>311</lpage>.</citation></ref>
<ref id="ref105"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zimmerman</surname> <given-names>C.</given-names></name> <name><surname>Cuddington</surname> <given-names>K.</given-names></name></person-group> (<year>2007</year>). <article-title>Ambiguous, circular and polysemous: Students' definitions of the &#x201C;balance of nature&#x201D; metaphor</article-title>. <source>Public Underst. Sci.</source> <volume>16</volume>, <fpage>393</fpage>&#x2013;<lpage>406</lpage>. doi: <pub-id pub-id-type="doi">10.1177/0963662505063022</pub-id></citation></ref></ref-list>
</back>
</article>