Promoting systems thinking and pro-environmental policy support through serious games

1 Introduction

The “Critical Zone”, or CZ, refers to the region of the planet critical to life. The CZ is the Earth’s outer skin, spanning from the top of the vegetation canopy down into the subsurface, to the bottom of the fresh groundwater zone of the planet (National Research Council, 2001). Research on the CZ typically focuses on the interconnections among different earth systems (i.e., the hydrosphere, lithosphere, biosphere, and atmosphere) that make all terrestrial life possible. Understanding the critical zone represents a way to understand interconnected environmental systems. For example, the hydrosphere flows through the lithosphere, biosphere, and atmosphere, and human actions can alter the flow of water through these systems. As such, human influences on coupled physical environments manifest themselves in the critical zone. Learning about the human-environment interactions in the CZ could potentially facilitate systems thinking.

The purpose of the present research is to test the capacity of a digital serious game, the “CZ investigator”, to increase different audiences’ (e.g., defined by their prior demonstrated interest in science) systems thinking and support for policies that would regulate human impacts on environmental systems. We specifically test whether the “CZ investigator” provided inroads to the increasing appreciation of the Food-Energy-Water (FEW) nexus and willingness to regulate human impacts on the FEW nexus, a particular instantiation of human impact on environmental systems important to human life. The benefit of learning about the CZ in a manner that helps facilitate holistic and relational thinking would become evident in an improved understanding of the FEW nexus. Serious games can potentially better facilitate this type of learning than more standard web communications about the same topics.

1.1 Systems thinking

Systems thinking is defined as thinking holistically and perceiving dynamic and complex causal relationships among components of systems (Arnold and Wade, 2015). Systems thinking is associated with ecological world views and beliefs and concerns about, for example, climate change (Davis and Stroink, 2016; Lezak and Thibodeau, 2016; Ballew et al., 2019). Systems thinking also has the potential to aid support for policies that address complex social and environmental problems where cause and effect relations are not necessarily directly evident when occurring through many interlocking complex links across space and time (Lane, 2016). Consistent with this assertion, system thinkers tend to support climate change policies (Lezak and Thibodeau, 2016). These associations suggest that systems thinking can advance knowledge about environmental issues and understanding of complex environmental systems that impact and are impacted by humans. Therefore, systems thinking can facilitate support for efforts to address these complex problems (Lezak and Thibodeau, 2016; Clayton, 2017; Thibodeau et al., 2017; Ballew et al., 2019).

Systems thinking is core to understanding the CZ. For about two decades, interdisciplinary teams of natural scientists have been researching the critical zone. They have studied how the physical characteristics of the CZ impact coupled biological, geological, chemical, and physical processes. For example, they study how features of the CZ impact and link water purification, soil nourishment, changes to landscapes, and atmospheric gas (Brantley et al., 2005). Very few studies take a social science perspective on the CZ (Herlin et al., 2021). In this age of the Anthropocene, it is vital to understand how human actions are impacting the CZ and the ecosystem services the CZ facilitates. Learning about the CZ could improve systems thinking about the environment. Moreover, learning how human actions can influence the CZ could have subsequent benefits for supporting environmental policies that regulate human actions.

The FEW nexus is a particular representation of human-environment interactions that could benefit from learning about the CZ. The ability to address food, energy, and water security is aided by knowledge about links among them, such as knowing that water consumption affects food production and energy generation. Individuals who know technical and scientific issues related to the FEW nexus are more aware of and concerned about the policies targeting each of the three domains separately and in combination, more so than those with less knowledge (Bullock and Bowman, 2018). Similarly, awareness of relations between food, energy, and water is associated with greater endorsement of support for environmental policies (Portney et al., 2018). Further, improving stakeholders’ understanding of the dependencies among food, energy, and water has the potential to improve the planning and management of resources (Purwanto et al., 2021). Understanding how human behavior that alters the CZ, such as transforming forests into farms, influences the flow and storage of water in the CZ, could help people understand why food and water are tightly linked. More generally, learning about the CZ might improve systems thinking about food, energy, and water. Such improved knowledge might aid people’s understanding of the benefits of policies that influence peoples’ use and protection of the environment.

1.2 Environmental Education and Serious Games

Given the potential benefits of systems thinking for supporting efforts to protect the environment, it is unfortunate that students have few opportunities to train their systems thinking ability (Cox et al., 2019). Moreover, although natural scientists understand the very close relations among food, energy, and water systems, the average person has only moderate awareness of their interdependence (Portney et al., 2018). The lack of understanding of the interplay of human and environmental systems could, subsequently, also affect public opinions about how societies can lessen the environmental harms of human activity.

The development of systems thinking has been a goal in environmental education. By adopting a systems approach to environmental education (e.g., understanding how Earth’s subsystems function and interact with each other), students can gain important insights into environmental realities and better understand that there are ways to improve sustainability and lessen environmental harms caused by human activities (Vasconcelos and Orion, 2021). The importance of systems thinking has also been discussed in geography education (Cox et al., 2019; Raath and Hay, 2019). Moreover, efforts to incorporate systems thinking into environmental and specifically climate change messaging have been developed for the general public outside of formal educational settings (Thibodeau et al., 2017; Swim et al., 2018).

Serious games have the potential to teach effectively about environmental systems, like those that make up the CZ. Serious games utilize the engaging and entertaining characteristics of video games for educational purposes (Harteveld, 2011). Examples in the environmental domain include serious games teaching climate change (Wu and Lee, 2015), agroecology (Jouan et al., 2020), and the FEW nexus (Sušnik et al., 2021). Serious games are well suited to teach people about environmental systems because of their ability to simulate reality (Harteveld, 2011) and offer interactivity, immediacy, and visual feedback (Michael and Chen, 2005; Fox et al., 2020; Wolf, 2020). These qualities can aid higher-order learning activities (analyzing, evaluating, and creating (Bloom, 1956) that enable learners to interpret, relate, argue, criticize, investigate, and construct new knowledge (Collins, 2014; Yen and Halili, 2015). While research on serious games has demonstrated their ability to improve support for environmental policies, planning, and management (e.g., Fox et al., 2020), they have not specifically assessed their impact on systems thinking.

1.3 Present Research

Our primary goal was to test whether an interactive digital serious game could provide a superior method of teaching people about environmental systems and increasing support for environmental policies relative to a static presentation of information about the CZ on a website. Systems thinking was assessed in terms of improved understanding of the FEW nexus and support for environmental policies that would affect the FEW nexus.

We constructed a digital serious game, the “CZ investigator” and, for comparison purposes, a static informational website. In both formats, the users assumed the role of a journalist investigating how the CZ could inform the environmental impacts of transforming a forest into a farm. The materials visually illustrated the components of the CZ and their dynamic qualities by teaching about how the hydrosphere intersected with the rest of the CZ, such as how the flow and storage of water differed depending upon the qualities of the biosphere. The material integrated several aspects of systems thinking (Arnold and Wade, 2015): 1) dynamic, nonlinear relations among elements of systems (e.g., temporal changes in these relations as a result of forces that affect the CZ); and 2) recognizing that systems exist at different scales (e.g., local, and regional views of the CZ).

In the game version users interacted with a virtual natural environment, testing how, for example changing characteristics of the CZ (e.g., cutting down trees) influence water flow. Although the farm and water implicitly connect to the FEW nexus, the game did not mention energy and did not directly draw connections among food, energy, and water. We proposed that the greater engagement with environmental systems in the CZ provided by the serious game relative to the static information format of the website would facilitate systems thinking that would generalize to perceiving stronger links within the FEW nexus. For example, the design of the learning experience could facilitate the understanding of intersecting systems and therefore, the learners would be more likely to infer that frequently washing cars leads to groundwater table depletion which would subsequently impact farmers who depend upon the groundwater to grow crops. Moreover, understanding these stronger links would affect support for policies that influence the frequency of washing cars.

Thus, we propose the following hypotheses:

Hypothesis 1. The serious game would improve systems thinking about the FEW nexus and support for policies that regulated human impact on elements of the FEW nexus, more so than the static website.

Hypothesis 2. Improved systems thinking about the FEW nexus from the game (vs. the website) format would mediate the relation between intervention format and policy support.Of secondary interest was whether the learning experience with the serious game would explain the superior impact of the game on the website. In addition to providing opportunities for higher-order learning, serious games can facilitate learning by eliciting a better learning experience from people than other forms of education (Wouters et al., 2009). The benefits include improved perception of learning effectiveness, motivation, engagement, self-efficacy, presence, ease of use, and challenge (e.g., Dankbaar et al., 2017; Licorish et al., 2017; Koroleva & Novak, 2020). Furthermore, games are known to have the ability to transform educational materials into fun and engaging activities (e.g., Michael & Chen, 2005; Harteveld, 2011; Arnab et al., 2013; Boyle et al., 2016) suggesting that a game more than a website, would promote more positive (happiness) than negative emotions (boredom and anxiety). Furthermore, relative to the website, the serious game allows for more exploration and manipulation of the environment, enabling learners to test their ideas. As such, we expected the learners to report higher levels of curiosity, awe, and science interest. As such, we offer the following hypotheses:

Hypothesis 3. Participants will report more positive learning experiences (ease and enjoyment, sense of presence, effectiveness for learning, reflective learning, challenge, happiness, awe, science interest) and less negative emotions (boredom and anxiety) with the serious game than the website.

Hypothesis 4. The effect of learning format on systems thinking and policy support (i.e., H1) would be mediated by the learning experience.We also explored whether the advantages of the serious game would be more evident for particular users. Many researchers have raised the importance of considering individual differences among players (e.g., Lopes & Bidarra, 2011; Vandewaetere et al., 2013). As argued by Charles et al. (2005), people learn in different ways, at different paces, and based on different styles and strategies. Furthermore, the players’ range of skills and capabilities usually vary (e.g., Hocine et al., 2014; Hendrix et al., 2018). Greater skills may come from better spatial abilities or developed from having more experience playing games. Spatial ability is defined by Poltrock & Brown (1984) as the capacity to produce, transform, and interpret mental images. Research has shown that the spatial visualization abilities of users affect their performance in digital educational experiences (Sajjadi et al., 2021). Furthermore, having extensive experience with playing games can help players quickly master the controls of the game and exhibit competence in playing.Another potentially important difference among users may be the extent of players’ past science education. If the game requires more time and energy investment than a website, it might require those with more past science education to put forth the effort it takes to learn from it. Yet, those with more science education have demonstrated that they are willing to learn about science in different venues and, as such, the two formats might be equally effective. In contrast, a serious game might require a more concrete, engaging, and emotionally enjoyable experience to aid learning among those with less past science education experiences. Thus, we explored whether previous science education influenced the likelihood that the game would produce more systems thinking and policy support than a website and whether the difference might be accounted for by differences in learning experiences and the emotions generated by the game.These considerations about game skills and past science education lead us to ask the following exploratory question:

Research Question 1. Are the Hypotheses 1 to 4 moderated by game skill or past learning experiences?Taken together, our hypotheses can be graphically illustrated by the model in Figure 1. The serious game (vs. the website) is predicted to increase systems thinking (Path B) and policy support (Path C, stated together in H1). The effect of the format on policy support is predicted to be mediated by the effect on systems thinking (paths B and F in H2). The serious game (vs. the website) is also predicted to result in more positive and less negative learning experiences (H3 path A) and these experiences are predicted to explain the impact of format on systems thinking (Paths A and D) and policy support (Paths A and E, both mediations stated together in H4). The effects of format on all outcomes may be moderated by game skills and past science experiences (Moderated paths), and as such, there may also be evidence of conditional mediation; i.e., the research question).

FIGURE 1

FIGURE 1. Conceptual model illustrating hypotheses and research question.

2 Methodology

2.1 Participants

An a priori power analysis (power = .80, α = .05) with an effect size (partial eta² = .048) derived from a pilot study determined the goal of 158 participants. A total of 357 participants were recruited from Amazon Mechanical Turk¹. After exclusions noted below, the final sample size was 152 participants. Each participant, regardless of exclusion, was paid $10.

They were informed that they were required to have access to a PC with a Microsoft Windows operating system (OS) with the following system specifications: an Intel Core i7-8705G processor or higher, an Nvidia GTX 1050 graphics card or higher, 16 GB of RAM or higher, and at least 4 GB of disk space. This inclusion criterion was assessed at the start of a survey. Fifty-three percent provided insufficient data for analyses because they either 1) did not proceed far enough into the survey to be assigned to a study condition (N = 93), were assigned to the web condition but did proceed through to the end (N = 3), 2) were assigned to the game condition but did not click through to download the game (N = 55), or 3) completed either the web or game intervention but did not complete the survey that followed (N = 40). Of the remaining participants (N = 166), 14 were excluded for low-quality data because of either indicating at the end of the study that their data were unusable (N = 4), not answering the question as to whether or not their data should be used (N = 1), taking over 15 h to complete the study (N = 2), taking less than 1/3 of the median completion time (N = 4), or more than three standard deviations of the mean completion time (N = 3) for their condition of the study.

After the above exclusions, there were 80 participants in the web condition of the study (43 women and 35 men) and 72 participants in the game condition (32 women and 38 men). It should be noted that we over recruited women to balance gender across the study, particularly within the game condition. The need to over recruit suggested that women were not interested in participating in the study after they read the description of the study, not interested in playing the game, or did not want to download the game on their computer to participate. Also, since this study was performed online and remotely, lack of access to a PC with the system specifications could be an alternative explanation for women’s lower initial participation rate.

Most participants were White/Caucasian (N = 106; 70%), followed by Asian (N = 16, 11%), Black/African American (N = 14, 9.2%), Hispanic/Latino (N = 12; 7.9%), Mixed Race (N = 3; 2%) and then Native American (N = 1; 0.7%). Nearly all participants indicated English as their primary language (N = 150, 98.7%). Most participants had at least some degree of college education (some college but no degree, N = 38, 25.7%; Associate degree in college, N = 19, 12.5%), Bachelor’s degree in college—4-years degree, N = 59, 38.8%; Master’s degree, N = 14, 9.2%; Doctoral degree N = 2, 1.3%; professional degree–JD, MD, N = 1, 0.7%. One participant (0.7%) indicated not obtaining a high school degree, and 17 (11%) indicated having a high school diploma or equivalent. Participants were mid-level on social-economic status (M = 3.46, Mdn = 3.00, SD = 1.16) based upon their placement of themselves on a stack of seven coins that they were told represented where people stand in the United States (money, jobs, respected jobs).

2.2 Procedure

The study was approved as meeting the requirement of ethical treatment of human subjects by the University’s Institutional Review Board (IRB) before the commencement of data collection. After indicating that they were using a PC with a Microsoft Windows operating system and providing informed consent, the survey randomly assigned each participant to learn about the “Critical Zone” by either advancing to the website or receiving instructions on how to download and play the game on their PC. At the end of either condition, participants were instructed to return to the main survey, where first, they completed an open-ended question describing the consequences they thought would emerge, consistent with what a journalist would do. Then they completed the outcome, individual difference, and demographic measures.

2.3 Instruments

2.3.1 Stimulus

Participants in both format conditions were informed that they were to play the role of a journalist and determine both good and bad impacts of changing an actual forested location in the community into a farm. From a design perspective, there are two primary goals in the game. First, it intended to educate the public about the CZ and the concept of the interconnectivity of environmental and biological systems to those who do not have access to this information or the physical place to experience it. Second, most of the CZ is hidden underground and out of sight. As a result, it is challenging to comprehend how different CZ components interact and influence what is visible above ground. Therefore, the game aimed to make it easier to see the CZ’s subsurface portions and how they affect the above-ground portions that we see every day.

2.3.1.1 The serious game

The CZ Investigator (Sajjadi et al., 2020) is a narrative-driven game built for a Microsoft Windows desktop application (i.e., high scalability (Klippel et al., 2020)). The game’s narrative is one of its key design aspects that facilitates the contextualization of the learning activities and engages the players with them (Dickey, 2011). The game starts with a short textual introduction informing the player that they are a newspaper journalist and need to meet with their editor about a story the newspaper is developing.

The game fades to the first scene (i.e., briefing), where through interactions with the newspaper editor, represented as an embodied conversational agent (ECA) (Sajjadi et al., 2019), the player is informed about the main objectives of the game and what they need to accomplish (Figure 2). The newspaper editor briefs the player about the newspaper’s plan to write a story about a logging company that will deforest part of the Shale Hills Critical Zone in central Pennsylvania. The article will focus on how this action will affect the CZ and, consequently, the FEW nexus. During the briefing, the editor explains the basic concept of the critical zone (using an abstract 2D graphical representation) and assigns a mission to the player to go to the Shale Hills (central Pennsylvania) CZO and conduct a field investigation and evidence collection.

FIGURE 2

FIGURE 2. The office of the newspaper editor. Interaction with the newspaper editor (A), briefing about the CZ (B).

To fully understand how natural and human processes can affect the CZ and the FEW nexus, one must understand their effect on all four components of the CZ mentioned in the introduction. Notwithstanding, this version of the CZ Investigator game focuses only on the hydrosphere (i.e., water). Therefore, the game’s main objective is for the player to explore the effect of natural and human processes on the flow and storage of water in Shale Hills CZ. Once briefed, the game fades to the second scene (i.e., Shale Hills), where the player would find themselves in the presence of a third character, a CZ expert named Brian. The CZ expert accompanied the player throughout their investigation by providing scientific guidance about the different components of the CZ and their interconnections.

In the second scene, the player arrives at Shale Hills. At the start of this scene, there is a short training exercise familiarizing the player with the game’s mechanics, how to navigate the environment, use their virtual tablet for communication, and manipulate environmental objects. Throughout the game, the CZ expert stays in touch with the player by virtual emails and audio instructions while conducting their investigation.

The virtual environment of this scene is a semi-accurate replica of a real environment in Shale Hills. This environment was made using a Digital Elevation Model (DEM) based on high-resolution Lidar data with an average of 10 points/m² with 2–4 cm vertical accuracy. Therefore, the natural environment of the game can be described as a “model of reality” (Harteveld, 2011) and was populated with a variety of vegetation species found in the Shale Hills area, with a reasonable level of accuracy (Sajjadi et al., 2020). To complete their investigation, the player needs to accomplish five objectives related to the hydrosphere component of the CZ. Before each objective, an email from the CZ expert is sent to the player’s tablet containing an overview description of what needs to be done for that objective.

The first two objectives of the game require the player to examine the flow and storage of water in several measurement wells and a weir. Using the x-ray functionality of their virtual tablet (Figure 3), the player can examine the current level of water the wells are holding. The player can also simulate rain (a natural process) to see how it affects the storage of the water at different wells depending on their geographical location and proximity to water source (i.e., The closer the wells are to the valley, the more water they can store, the faster they will fill up in the presence of rain, and the slower they will empty over time).

FIGURE 3

FIGURE 3. Exploring the effect of rain on the measurement wells using the x-ray feature of the tablet. (A) without rain, and (B) with rain.

These actions, accompanied by the scientific instructions from the CZ expert, helps the player to reflect on their concrete experience with how water is stored. Like the wells, the player can explore the environment, locate the weir (Figure 4), simulate rain, and concretely experience how the flow and storage of water are affected by this natural process.

FIGURE 4

FIGURE 4. Exploring the effect of rain on the storage of water in the weir. (A) without rain, and (B) with rain.

The next set of objectives requires the player to explore and understand how the rainwater infiltrates the soil and the subsurface layers of the Earth. To adequately visualize the subsurface layers and what happens when water reaches them, parts of the terrain are designed to cut out a “wedge”, allowing the player to see components of the CZ. Three wedges are used to illustrate different layers of soil, subsoil, shale, and bedrock that comprise the subsurface part of the Critical Zone.

The player interacts with these wedges by cutting them out of the ground and examining their layers to investigate how rainwater moves through them. Two of these wedges (i.e., objectives three and four) are at different geographical locations in the environment [e.g., hill (Figure 5) versus valley (Figure 6)], which affects the height and material of their subsurface layers. Once the player simulates rainfall, the textures of the subsurface layers gradually changes color to indicate how much each layer is saturated with water as it rains. Like the previous objectives, these exploratory tasks are accompanied by scientific explanations from the CZ expert.

FIGURE 5

FIGURE 5. Exploring the effect of rain on a wedge on top of a hill. (A) without rain, and (B) with rain.

FIGURE 6

FIGURE 6. Exploring the effect of rain on a wedge in the valley. (A) without rain, and (B) with rain.

The fifth and last objective requires the player to investigate the effect of human intervention (i.e., deforestation) on the CZ’s hydrosphere component. In this objective, the player navigates to a wedge and then deforests all trees within a one-acre radius around it. While performing this action, the player experiences how deforestation affects the texture of the soil sub-layer because the root density in that layer is reduced. Afterward, the player explores the effect of rain on how water passes through the wedge in a deforested area (Figure 7). The game is designed to simulate a severe outcome, such as flooding, because deforestation will cause less water absorption by trees, leading to an increase in surface water. While consequences of deforestation would likely take considerable time to happen, the game enables the player to experience the long-term effects of deforestation immediately. Like previous objectives, the scientific explanations behind what the player is experiencing are provided by the CZ expert through the users’ earpieces.

FIGURE 7

FIGURE 7. Exploring the effect of deforestation on the infiltration of the rainwater into the Earth. (A) without rain, and (B) with rain.

The game is considered finished once the player completes all the objectives. However, a player has the freedom to revisit and interact with any of the objectives for as long as they want. The player’s activities, combined with the CZ expert explanations, take them through the concrete experience, reflective observation, abstract conceptualization, and active experimentation phases of Kolb’s experiential learning cycle (Kolb, 2014).

2.3.1.2 The website

To explore the effect of the serious game on users, we compared it to using a website², a more common method for the public to learn about science and the environment. The content from the first scene of the game was given to the users of the website with texts rather than told to them by the editor. The content from the second scene of the game was transformed into informational content teaching the same objectives in text format (rather than learned via exploration). Still images in the form of screenshots from the game accompanied the textual content on the website. In addition, the self-paced progression through the narrative on the website was similar to the game. Unlike the game, however, the content was limited to text descriptions and screenshots and no interaction allowed the users to explore the environment or test the effects of different water conditions on the CZ. Thus, the user experience is quite passive compared to the game.

2.3.2 Measures³

2.3.2.1 Reflection on experience

All experience measures were assessed on seven-point scales ranging from -3 (strongly disagree) to 3 (strongly agree). These measures were: 1) A five-item measure of how easy and enjoyable they found the experience to be (α = .83), derived from (Maor and Fraser, 2005); 2) One item from (Lee et al., 2010) that measured sense of presence [“There was a sense of presence (being there)”], 3) a five-item measure of the perceived effectiveness of the learning material (α = .86) derived from (Lee et al., 2010) (modified from (Benbunan-Fich and Hiltz, 2003; Marks et al., 2005; Martens et al., 2007)). Items assessed desire to learn more about the Critical Zone, understanding of concepts, ability to summarize information, meaningfulness, and ability to apply what one had learned 4) A three-item measure of reflective learning (e.g., “I was able to think deeply about my own ideas”, derived from (Maor and Fraser, 2005); α = .86); and 5) A four-item measure of challenge (e.g., The program made me think; α = .82).

2.3.2.2 Emotions

Using a five-point scale ranging from 0 (none of this emotion) to 4 (a great deal of this emotion) to indicate their reflections on five types of emotions, with three words to represent each emotion, participants indicated whether they currently felt curious (curious, intrigued, inquisitive, α = .85), bored (bored, indifferent, not caring, α = .76), in awe (in awe, amazed, full of wonder, α = .89), tense (tense, upset, worried, α = .84), and happy (joyful, cheerful, happy, α = .91).

Using five seven-point semantic differential scales ranging from -3 to 3, participants indicated the extent to which they found the critical zone (α = .84) and environmental science interesting (α = .87, fascinating to mundane, appealing to unappealing, exciting to unexciting, means nothing to means a lot, boring to interesting). The two measures were strongly correlated, r (151) = .78, p < .001, so they were averaged together to form one measure of interest in the science.

2.3.2.3 Systems thinking

Participants were presented with four scenarios that started with an initial behavior followed by four effects of the individual behavior on food, energy, and water with effects that built off each other. For instance, the first behavior was someone “living in one of the Midwest states of the United States, gets their car washed twice a month at a car wash in their community”. Using a slider scale ranging from 0 (not at all) to 100 (completely), participants estimated the extent to which this behavior took water out of local groundwater (step 1), whether taking water out of the groundwater would affect farmers’ ability to irrigate crops (step 2), whether less ability to irrigate crops would reduce crop yields, including corn, sugarcane, or sweet sorghum (step 3), and, finally, whether reduced yields in these crops would reduce the production of ethanol from these crops (step 4). The second behavior was purchasing non-organic rather than organic apples, and the end consequence was fewer fish to eat because of fish dying in dead zones in streams. The third behavior was volunteering to participate in stream restoration, and the end consequence was less sediment in streams improving the stability and reliability of hydropower. The last behavior was using an air conditioner to reduce temperatures in one’s home by two degrees, with the final consequence being heated water that hurt wildlife and hunting and fishing. Responses were averaged across the four steps and four scenarios (α = .93). Independent assessment of the measure illustrates that it is associated with general systems thinking and policy support as described next.

2.3.2.4 Policy support

Using slider scales ranging from 0 (not at all) to 100 (completely), participants rated their support for different policies that would protect the critical zone and were connected to managing the first step in each of the four FEW scenarios (water use regulation limiting the number of people using car washes when groundwater levels are low, a small tax on non-organic fruits and vegetables with the tax used to clean waterways from pesticides and fertilizers, taxpayer money going to stream restoration, and taxpayer-funded rebates to encourage purchasing of energy-efficient air conditioners, α = .87).

2.3.2.5 Individual difference

Gaming experience. On a five-point scale ranging from 0 “not at all” to 4 “a great deal”, participants indicated the extent to which they used video games, first-person shooter video games, and virtual reality apps and technology (α = .73; M = 2.02, SD = 1.08).

Sense of direction. On a seven-point scale ranging from strongly, disagree (-3) to (strongly agree (3), participants completed 14 items from the Santa Barbara scale (Hegarty et al., 2002) that measured their spatial ability in terms of sensitivity to directions (α = .77; M = .44, SD = .88).

Science education. On a five-point scale ranging from “Does not describe me” (0) to “Describes me extremely well” (5), participants completed five items that indicated how much they learned about natural sciences (e.g., biology, physics, geography, ecology) in four different locations (high school; informal science learning centers (e.g., zoos, museums, national parks); while obtaining an advanced degree (i.e., post-high school) in another area; and while earning an advanced degree (i.e., a post-high school specializing in natural sciences). Additionally, the previous question assessing gaming experiences included an item that asked how frequently they used internet sites to learn about science information. Because this item fits conceptually better with learning about science, it was included in the measure of science education. Responses were averaged to form one measure of science education experiences (α = .78; M = 1.61; SD = .87).

3 Results

3.1 Overview

Mean responses to reflections on the game and outcome measures and the association between the effects of information format (i.e., website vs. game) and individual differences can be found in Table 1. Separate regressions tested the effects of each individual difference measure and the interaction between the individual differences and format. In Step 1, the format and individual differences (gender or one of the other mean-centered individual differences - sense of direction, gamer experience, and science education) were entered. In Step 2, the interaction between format and one of the individual difference measures was entered. Science education was the only individual difference measure that interacted with the format. Thus, for simplicity, we only present interaction results with science education. All main effects and interactions with science education can be found in Table 1. Significant interaction effects were followed-up by assessing simple slopes (nonstandardized) for the effect of format on outcome measures for those with little science education (M—1 SD) and a lot of science education (M + 1 SD). As explained in more detail below, we used Hayes Process models to test hypothesized mediation models (Hayes, 2013).

TABLE 1

TABLE 1. Descriptive and association among game format and individual differences on learning experience, emotions, systems thinking, and policy support. Note: N = 152 per cell, except for correlations with gender where n = 148; * p < .05; ** p < .01. Format ^ϕ: 0 = web; 1 = Game; Gender^Ϯ: 0 = women, 1 = men; Emotions and science interested rating^ϰ: −3 (strongly disagree) to 3 (strongly agree); Policy support^†: from 0 (not at all) to 100 (completely).

3.2 Systems thinking and policy support

Participants generally saw connections across the FEW systems and supported the policy that influenced the initial step in the chain of events. Interactions revealed that the hypothesized effect of format on systems thinking and policy support (Hypothesis 1, Research Question 1) was supported for those with less past science education but not for those with more past science education (see Table 1 for interactions and Figure 3 in supplementary materials). Specifically, the game format (relative to the web format) increased perceived connections for those with less past science education experiences, b = 8.09, SE = 3.86, t (148) = 2.09, p = .04, but not for those with more past science education experiences (mean + 1 SD), b = -4.31, SE = 3.89, t (148) = -1.11, p = .26, and increased policy support for those with less past science education experiences, b = 15.82, SE = 5.03, t (148) = 3.14, p = .002, but not for those with a lot of past science education experiences (mean +1 SD), b = -1.83, SE = 5.07, t (148) = -.36, p = .72.

Because of interactions between format and science education on systems thinking and policy support (see below), we used Hayes Process model eight to test conditional mediation. Thus, the hypothesized indirect effects from format to policy support via systems thinking (Hypothesis 2, Research Question 1) were compared for those with less vs. more past science education. Mediation was supported for those with less past science education and not for those with more past science education. Specifically, as illustrated in Figure 8, for those with less science education, the serious game format resulted in greater systems thinking, greater systems thinking was associated with more policy support, and the indirect effect was significant.

FIGURE 8

FIGURE 8. Explaining effects of format policy support via systems thinking.

3.3 Learning experience

3.3.1 Reflections on the experience

Participants reported positive experiences learning about the CZ because it was easy, gave them a sense of presence, encouraged reflection on the game, and was challenging. Positive reflections were generally stronger for those with more gaming experience and broader science education experiences.

Consistent with Hypothesis 3, participant’s reflections on their experiences were more positive with the serious game than the website. The game (M = 2.19, SD = 0.70) was easier to use than the website (M = 1.47, SD = 0.94), t (151) = 5.25, p = .02, and participants felt greater presence with the game (M = 2.01, SD = 1.07) than the website (M = 0.83, SD = 1.63), t (151) = 5.26, p < .001.

Positive reflections on experiences were more evident for those with less past science education than those with more past science education (see Table 1 for interactions and Figures 1, 2 in supplementary materials). The superior effect of the serious game over the website format on ease and enjoyment was stronger for those with less past science education, b = 1.03, SE = 0.18, t (148) = 5.63, p < .01, than those with more science education, b = 0.44, SE = 0.19, t (148) = 2.39, p = .02. Although the interaction was marginally significant, the same pattern was found for the feeling of presence. The effect of game format on presence tended to be stronger among those with less past science education, b = 1.62, SE = 0.31, t (148) = 5.23, p < .01, than for those with more past science education, b = 0.85, SE = .31, t (148) = 2.74, p = .01. Last, self-reported perceived effectiveness was stronger in the game versus the website format for those with less science education experience: least education, b = 0.48, SE = 0.17, t (148) = 2.78, p < .01, and not different for those with the most education, b = -0.15, SE = 0.17, t (148) = -0.89, p = .37).

3.3.2 Emotions and science interest

Participants reported relatively strong tendencies to feel curious, express interest in science, and weak tendencies to feel bored. Participants reported more awe with the game (M = 1.57, SD = 1.09) than the website format (M = 1.14, SD = 1.08), t (150) = 2.42, p = .02. Participants with more past gaming experience reported stronger feelings of awe. More science education was positively associated with feeling happy during the game and greater interest in science. However, emotions and science interest were not increased by learning about the CZ via the serious game more than the website format, and, unlike reflections on their experiences, there were no interactions with past science education.

3.2.3 Mediation

Because game format influenced feelings of presence and awe, we tested whether they mediated the effect of format on systems thinking and on policy support, per Hypothesis 4. Using Hayes process model four to test parallel mediation, we found support for presence more so than awe in mediating the effect of format on systems thinking (see Figure 9). However, neither mediated the effect of format on policy support. As noted above, format predicted presence and awe. Presence subsequently predicted systems thinking but the relation between awe and systems thinking was not significant (p = .08). Correspondingly, the indirect effect from format to systems thinking via presence was significant but the indirect effect from format to systems thinking via awe was not significant. Suggesting overlap between presence and awe, the effect from awe to systems thinking, is significant if presence is not included in the model, b = 3.69, SE = 1.24, t (149) = 2.86, p = .004, and the indirect effect is also significant, 1.58, 95% CI [.24 to 3.54]. In contrast, when explaining the effect from format to policy support, neither presence, b = 1.91, SE = 1.41, t (148) = 1.35, p = .18, nor awe, b = 1.92, SE = 1.81, t (148) = 1.06, p = .29, predicted policy support, and neither corresponding indirect effects were significant, 2.27, 95% CI [-1.72 to 6.19] and .82, 95% CI [-.44 to 6.84].

FIGURE 9

FIGURE 9. Explaining effects of format on systems thinking via presence and awe.

Because of interactions between format and science education on ease and enjoyment and perceived effectiveness, systems thinking, and policy support, we used Hayes Process model eight to test conditional mediation. Thus, the hypothesized indirect effects from format to policy support via learning experiences (Hypothesis 4) were compared for those with less vs. more past science education (Research Question 1). As illustrated in Figures 9 and 10, conditional mediation indicated that self-reported perceived effectiveness was a better explanation than ease and enjoyment for explaining the effect of format on systems thinking and policy support. Specifically, replicating what was noted above, the game format (more than the website format) increased ease and enjoyment and perceived effectiveness more for those with less past science education than those with more past science education. Perceived effectiveness, not ease and enjoyment, were associated with systems thinking and policy support. As a result, the indirect effects were significant via perceived effectiveness, not ease and enjoyment, among those with less past science education. None of the indirect effects were significant for those with more past science education.

FIGURE 10

FIGURE 10. Explaining effects of format on systems thinking via ease and enjoyment and perceived effectiveness of the experience.

4 Discussion, limitations, and future work

Our results demonstrate the potential of a digital serious game to facilitate environmental systems thinking and policy support. Although participants in both the serious game and website conditions displayed systems thinking and policy support, the game was a more effective educational tool. Participants with less past science education who played the serious game (relative to those in the website condition) reported more connections across various elements of the FEW nexus and increased support for policies that influenced the spread of effects across the FEW nexus. These effects of the serious game were not found for those with more past science education. Additionally, mediation analyses suggested that a reason why the game format was successful at increasing policy support for those with less science education is that it improved systems thinking. Thus, our results supported Hypothesis 1 and 2, but only for those with less science education (Research Question 1). Notably, improvement for those with less past science education resulted in them seeing connections and supporting policies to the same extent as those with more previous science education, suggesting that serious games can potentially play a role in lessening science-related educational disparities.

Per Hypothesis 3, the serious game improved some aspects of the learning experience. First, those who played the serious game reported greater presence and awe than those who viewed the website. We believe that the effect on presence and awe is because the better synthesis of the natural environment in the game produced a more realistic experience than the web format. We modeled the game’s natural environment using data from the actual site to improve the realism of the experience. Exploration and investigation in this environment would be an immersive experience on their own. When combined with performing actions pertinent and well-integrated into the environment, the learners were placed in a realistic and almost tangible environment, thereby generating awe and presence. Second, among those with less science education, the benefits of the game on reflections about their learning were stronger for those with less science education: those with less past science experience reported greater ease and enjoyment and more effective learning than did those who viewed the website. These effects of information format were weaker or not significant, respectively, for those with more past science education.

Mediation analyses, per Hypothesis 4 and Research Question 1, indicated that the benefit of the serious game on learning experiences contributed to systems thinking and also on policy support for those with less science education. First, the serious game’s ability to increase presence more so than the website helped explain the effect of the game format on systems thinking. It is informative that awe had the same effect, but only when presence was not included in the model, suggesting there is some overlap between the two reported experiences. Second, the game improved their self-perceived learning which then subsequently improved their systems thinking and policy support. It is informative that enjoyment and ease of learning did not mediate the effect of format on systems thinking and policy support. Thus, the benefit of the game on systems thinking and policy support for those with less science education was not because it was fun to play, per se, but because participants believed it was better able to inform and educate them about the CZ. This finding can inform future game design as it suggests that meaningful responses to serious games are as important, if not more so than traditional notions of enjoyment. As such, serious game designers may want to pretest game elements and scenarios to ensure they are perceived as educational and meaningful.

4.1 Limitations and future research

We encountered difficulty when recruiting women to participate in the study. We asked participants to report their gender at the end of the study, so we cannot tell at what point in the study women declined to participate (i.e., when they were told the purpose of the study when they were reviewing the study material, or after they learned about the CZ). Thus, the women who went through the study may have been different from those who did not sign up or declined to participate. Therefore, while the study may not generalize to men and women who declined to participate, it may be especially less representative of women than men.

However, differences between women’s and men’s participation might be captured by the individual differences we included in the study. Women are less likely to play video games than men (Borgonovi, 2016). Thus, women may have declined to participate or finish the study because they believed they lacked the skill sets. Yet, lacking skill sets did not moderate the effects of the game format. If women who may have self-selected out because they believed they lacked skill sets had participated, our results suggest we still would not have found differences between women and men. Women are less likely to express interest in natural sciences than men (Ceci and Williams, 2007). If we had found gender differences, it might have been accounted for by this difference and, if so, the game may benefit women more than men. Yet this possibility suggests that to obtain the benefits from serious games teaching environmental science, attention may be needed to encourage women to opt into opportunities to learn about science in this manner.

Although we can be confident of the causal effect of the information format on our outcomes, the mediation analyses are limited because the path from our mediators to outcome variables are correlations. Thus, we cannot as confidently indicate that FEW systems thinking caused policy support.

Future research might benefit from more immersive means of learning about the CZ, perhaps through virtual reality (VR) with a head-mounted display as suggested by (Zhao et al., 2020) or augmented reality to learn about the hidden nature of the CZ while in a natural environment. Though we found some effects of the game experience on learning experiences, contrary to Hypothesis 3, the game did not improve self-reported reflective learning, challenge, curiosity, and happiness and did not diminish boredom and anxiety. More immersive experience might better improve these outcomes, thus, also potentially increasing the impact of learning about the CZ on systems thinking and policy support. Yet, it is also notable that, participants in both conditions reported being curious, interested in science, and not bored, suggesting that learning about the CZ may have been sufficiently novel such that the delivery format did not matter as much. As such, the CZ may be a helpful topic to introduce in public environmental outreach campaigns as something new and interesting that could attract public attention to the role of human activity in shaping the environment.

Lastly, the current version of the CZ Investigator game is limited to the hydrosphere. Future game experiences should include other components of the CZ, which will change the dynamic between how the CZ affects the FEW nexus and how learners understand the complex interrelations between all the involved components and systems. We anticipate numerous research opportunities to emerge from such extensions.

5 Conclusion

Using a serious game to teach about complex environmental systems like the CZ can potentially expand the public’s ability to think in terms of distinct but related systems (i.e., the FEW nexus) and endorse policies that have a positive impact on these systems. These effects are mainly for those less likely to be interested in science, as suggested by the greater benefits of the game for those with less science education. From the participants’ perspective, the benefit of the serious game is that it produces more feelings of presence and awe, and for those with less science education is more enjoyable, easier, and improves the effectiveness of learning the science, more so than a website. Moreover, the greater feelings of presence and, for those with less science education, learning effectiveness helps explain the effect of the serious game on systems thinking.

The greater benefits of the game for those with less science education and its benefit via the effectiveness of learning are worth highlighting. Our game design is not just appealing to domain experts or those who are already savvy about water-related environmental challenges (as was examined in previous research, den Haan et al., 2020; Sermet et al., 2020). People who already possess an adequate level of science education are intrinsically driven to learn about such topics, as suggested by their higher reported interest in science, and can probably learn it on their own or using more ubiquitous but less interactive media such as a website. It is for the people who do not have such a strong scientific background that we design such games and try to engage with topics that they may not otherwise encounter.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by The Pennsylvania State University—IRB. The patients/participants provided their written informed consent to participate in this study.

Author contributions

PS: Conceptualization, Methodology, Software, Formal Analysis, Investigation, Data Curation, Writing—Original Draft, Writing—Review and Editing; MMB: Conceptualization, Software; JGM: Conceptualization, Methodology, Resources, Writing—Original Draft, Funding acquisition; JGG: Writing—Original Draft; TSW: Conceptualization, Funding acquisition; AK: Conceptualization, Methodology, Investigation, Writing—Original Draft, Writing—Review and Editing, Supervision; JKS: Conceptualization, Methodology, Formal Analysis, Investigation, Resources, Data Curation, Writing—Original Draft, Writing—Review and Editing, Funding acquisition.

Funding

The study was funded by the Pennsylvania State University Institute for Energy and the Environment awarded to authors JKS , AK, JGM, and TSW and the National Science Foundation (Award number 2129402) awarded to authors TSW and JKS.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fenvs.2022.957204/full#supplementary-material

Footnotes

¹https://www.mturk.com/

²https://czexperience.weebly.com/

³We included measures of learning basic information about the CZ (i.e., knowledge). The measures indicate participant scored well and there were no effects of format on knowledge. However, the reliability of the measures was low, so we do not report them further (α = .39 and .49, respectively).

References

Arnab, S., Brown, K., Clarke, S., Dunwell, I., Lim, T., Suttie, N., et al. (2013). The development approach of a pedagogically-driven serious game to support Relationship and Sex Education (RSE) within a classroom setting. Comput. Educ. 69, 15–30. doi:10.1016/j.compedu.2013.06.013

CrossRef Full Text | Google Scholar

Arnold, R. D., and Wade, J. P. (2015). A definition of systems thinking: A systems approach. Procedia Comput. Sci. 44, 669–678. doi:10.1016/j.procs.2015.03.050

CrossRef Full Text | Google Scholar

Ballew, M. T., Goldberg, M. H., Rosenthal, S. A., Gustafson, A., and Leiserowitz, A. (2019). Systems thinking as a pathway to global warming beliefs and attitudes through an ecological worldview. Proc. Natl. Acad. Sci. U. S. A. 116, 8214–8219. doi:10.1073/pnas.1819310116

PubMed Abstract | CrossRef Full Text | Google Scholar

Benbunan-Fich, R., and Hiltz, S. R. (2003). Mediators of the effectiveness of online courses. IEEE Trans. Prof. Commun. 46, 298–312. doi:10.1109/tpc.2003.819639

CrossRef Full Text | Google Scholar

Bloom, B. S. (1956). Taxonomy of educational objectivesCognitive domain. N. Y. McKay 120, 24.

Google Scholar

Borgonovi, F. (2016). Video gaming and gender differences in digital and printed reading performance among 15-year-olds students in 26 countries. J. Adolesc. 48, 45–61. doi:10.1016/j.adolescence.2016.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Boyle, E. A., Hainey, T., Connolly, T. M., Gray, G., Earp, J., Ott, M., et al. (2016). An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Comput. Educ. 94, 178–192. doi:10.1016/j.compedu.2015.11.003

CrossRef Full Text | Google Scholar

Brantley, S., White, T., and White, A. (2005). Frontiers in exploration of the critical zone.

Google Scholar

Bullock, J. B., and Bowman, A. O. (2018). Exploring citizens’ support for policy tools at the food, energy, water nexus. Environ. Prog. Sustain. Energy 37, 148–154. doi:10.1002/ep.12727

CrossRef Full Text | Google Scholar

Ceci, S. J., and Williams, W. M. (2007). Why aren’t more women in science?: Top researchers debate the evidence. Washington, DC: American Psychological Association. doi:10.1037/11546-000

CrossRef Full Text | Google Scholar

Charles, D., Mcneill, M., McAlister, M., Black, M., Moore, A., Stringer, K., et al. (2005). Player-centred game design: Player modelling and adaptive digital games, 285–298. doi:10.1145/1125451.1125774

CrossRef Full Text | Google Scholar

Clayton, S. (2017). Social issues and personal life: Considering the environment. J. Soc. Issues 73, 667–681. doi:10.1111/josi.12237

CrossRef Full Text | Google Scholar

Collins, R. (2014). Skills for the 21st century: Teaching higher-order thinking. Curric. Leadersh. J. 12.

Google Scholar

Cox, M., Elen, J., and Steegen, A. (2019). Systems thinking in geography: Can high school students do it? Int. Res. Geogr. Environ. Educ. 28, 37–52. doi:10.1080/10382046.2017.1386413

CrossRef Full Text | Google Scholar

Dankbaar, M. E., Richters, O., Kalkman, C. J., Prins, G., Ten Cate, O. T., van Merrienboer, J. J., et al. (2017). Comparative effectiveness of a serious game and an e-module to support patient safety knowledge and awareness. BMC Med. Educ. 17, 30–10. doi:10.1186/s12909-016-0836-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Davis, A. C., and Stroink, M. L. (2016). The relationship between systems thinking and the new ecological paradigm. Syst. Res. Behav. Sci. 33, 575–586. doi:10.1002/sres.2371

CrossRef Full Text | Google Scholar

den Haan, R.-J., van der Voort, M. C., Baart, F., Berends, K., Van Den Berg, M., Straatsma, M., et al. (2020). The Virtual River Game: Gaming using models to collaboratively explore river management complexity. Environ. Model. Softw. 134, 104855. doi:10.1016/j.envsoft.2020.104855

CrossRef Full Text | Google Scholar

Dickey, M. D. (2011). Murder on Grimm Isle: The impact of game narrative design in an educational game-based learning environment. Br. J. Educ. Technol. 42, 456–469. doi:10.1111/j.1467-8535.2009.01032.x

CrossRef Full Text | Google Scholar

Fox, J., McKnight, J., Sun, Y., Maung, D., and Crawfis, R. (2020). Using a serious game to communicate risk and minimize psychological distance regarding environmental pollution. Telemat. Inf. 46, 101320. doi:10.1016/j.tele.2019.101320

CrossRef Full Text | Google Scholar

Harteveld, C. (2011). Triadic game design: Balancing reality, meaning and play. Springer Science & Business Media.

Google Scholar

Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York: Guilford press.

Google Scholar

Hegarty, M., Richardson, A. E., Montello, D. R., Lovelace, K., and Subbiah, I. (2002). Development of a self-report measure of environmental spatial ability. Intelligence 30, 425–447. doi:10.1016/s0160-2896(02)00116-2

CrossRef Full Text | Google Scholar

Hendrix, M., Bellamy-Wood, T., McKay, S., Bloom, V., and Dunwell, I. (2018). Implementing adaptive game difficulty balancing in serious games. IEEE Trans. Games 11, 320–327. doi:10.1109/tg.2018.2791019

CrossRef Full Text | Google Scholar

Herlin, C., Thi Phan, L., and Jou, S.-C. (2021). “Social science approaches to critical zone studies: A review,” in T1 - ecosystem services assessment covering diverse beneficiaries to improve ES related policy. Presented at the 3rd ESP asia conference (Nagasaki, Japan).

Google Scholar

Hocine, N., Gouaich, A., and Cerri, S. A. (2014). “Dynamic difficulty adaptation in serious games for motor rehabilitation,” in International conference on serious games (Springer), 115–128.

CrossRef Full Text | Google Scholar

Jouan, J., De Graeuwe, M., Carof, M., Baccar, R., Bareille, N., Bastian, S., et al. (2020). Learning interdisciplinarity and systems approaches in agroecology: Experience with the serious game SEGAE. Sustainability 12, 4351. doi:10.3390/su12114351

CrossRef Full Text | Google Scholar

Klippel, A., Zhao, J., Sajjadi, P., Wallgrün, J. O., Bagher, M. M., and Oprean, D. (2020). “Immersive place-based learning-an extended research framework,” in 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 449–454. Piscataway, NJ: IEEE.

CrossRef Full Text | Google Scholar

Kolb, D. A. (2014). Experiential learning: Experience as the source of learning and development. New Jersey: FT press.

Google Scholar

Koroleva, K., and Novak, J. (2020). How to engage with sustainability issues we rarely experience? A gamification model for collective awareness platforms in water-related sustainability. Sustainability 12, 712. doi:10.3390/su12020712

CrossRef Full Text | Google Scholar

Lane, D. C. (2016). Till the muddle in my mind have cleared awa’: Can we help shape policy using systems modelling? Syst. Res. Behav. Sci. 33, 633–650. doi:10.1002/sres.2422

CrossRef Full Text | Google Scholar

Lee, E. A.-L., Wong, K. W., and Fung, C. C. (2010). How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Comput. Educ. 55, 1424–1442. doi:10.1016/j.compedu.2010.06.006

CrossRef Full Text | Google Scholar

Lezak, S. B., and Thibodeau, P. H. (2016). Systems thinking and environmental concern. J. Environ. Psychol. 46, 143–153. doi:10.1016/j.jenvp.2016.04.005

CrossRef Full Text | Google Scholar

Licorish, S. A., George, J. L., Owen, H. E., and Daniel, B. (2017). “Go Kahoot!” enriching classroom engagement, motivation and learning experience with games,” in 25th international conference on computers in education New.Zealand: Asia-pacific society for computers in education.

Google Scholar

Lopes, R., and Bidarra, R. (2011). Adaptivity challenges in games and simulations: A survey. IEEE Trans. Comput. Intell. AI Games 3, 85–99. doi:10.1109/tciaig.2011.2152841

CrossRef Full Text | Google Scholar

Maor, D., and Fraser, B. J. (2005). An online questionnaire for evaluating students’ and teachers’ perceptions of constructivist multimedia learning environments. Res. Sci. Educ. 35, 221–244. doi:10.1007/s11165-005-2148-3

CrossRef Full Text | Google Scholar

Marks, R. B., Sibley, S. D., and Arbaugh, J. B. (2005). A structural equation model of predictors for effective online learning. J. Manag. Educ. Newbury. Park. 29, 531–563. doi:10.1177/1052562904271199

CrossRef Full Text | Google Scholar

Martens, R., Bastiaens, T., and Kirschner, P. A. (2007). New learning design in distance education: The impact on student perception and motivation. Distance Educ. 28, 81–93. doi:10.1080/01587910701305327

CrossRef Full Text | Google Scholar

Michael, D. R., and Chen, S. L. (2005). Serious games: Games that educate, train, and inform. Boston, MA: Muska & Lipman/Premier-Trade.

Google Scholar

Poltrock, S. E., and Brown, P. (1984). Individual differences in visual imagery and spatial ability. Intelligence 8, 93–138. doi:10.1016/0160-2896(84)90019-9

CrossRef Full Text | Google Scholar

Portney, K. E., Hannibal, B., Goldsmith, C., McGee, P., Liu, X., and Vedlitz, A. (2018). Awareness of the food–energy–water nexus and public policy support in the United States: Public attitudes among the American people. Environ. Behav. 50, 375–400. doi:10.1177/0013916517706531

CrossRef Full Text | Google Scholar

Purwanto, A., Sušnik, J., Suryadi, F. X., and de Fraiture, C. (2021). Water-energy-food nexus: Critical review, practical applications, and prospects for future research. Sustainability 13, 1919. doi:10.3390/su13041919

CrossRef Full Text | Google Scholar

Raath, S., and Hay, A. (2019). Preservice geography students’ exposure to systems thinking and cooperative learning in environmental education. J. Geogr. 118, 66–76. doi:10.1080/00221341.2018.1516231

CrossRef Full Text | Google Scholar

Sajjadi, P., Zhao, J., Wallgrün, J. O., La Femina, P., and Klippel, A. (2021). “Influence of HMD type and spatial ability on experiences and learning in place-based education,” in 2021 7th International Conference of the Immersive Learning Research Network (iLRN), 1–8. Piscataway, NJ: IEEE.

CrossRef Full Text | Google Scholar

Sajjadi, P., Bagher, M. M., Cui, Z., Myrick, J. G., Swim, J. K., and White, T. S. (2020). “Design of a Serious Game to Inform the Public About the Critical Zone,” in 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH), 1–8. Piscataway, NJ: IEEE.

CrossRef Full Text | Google Scholar

Sajjadi, P., Hoffmann, L., Cimiano, P., and Kopp, S. (2019). A personality-based emotional model for embodied conversational agents: Effects on perceived social presence and game experience of users. Entertain. Comput. 32, 100313.

CrossRef Full Text | Google Scholar

Sermet, Y., Demir, I., and Muste, M. (2020). A serious gaming framework for decision support on hydrological hazards. Sci. Total Environ. 728, 138895. doi:10.1016/j.scitotenv.2020.138895

PubMed Abstract | CrossRef Full Text | Google Scholar

Sušnik, J., Masia, S., Indriksone, D., Brēmere, I., and Vamvakeridou-Lydroudia, L. (2021). System dynamics modelling to explore the impacts of policies on the water-energy-food-land-climate nexus in Latvia. Sci. Total Environ. 775, 145827. doi:10.1016/j.scitotenv.2021.145827

PubMed Abstract | CrossRef Full Text | Google Scholar

Swim, J. K., Geiger, N., Sweetland, J., and Fraser, J. (2018). “Social construction of scientifically grounded climate change discussions,” in Psychology and climate change: From denial and depression to adaptation and resilience. Editors S. Clayton, and C. Manning (San Diego, CA, US: Academic Press), 65–93.

CrossRef Full Text | Google Scholar

Thibodeau, P. H., Frantz, C. M., and Berretta, M. (2017). The earth is our home: Systemic metaphors to redefine our relationship with nature. Clim. Change 142, 287–300. doi:10.1007/s10584-017-1926-z

CrossRef Full Text | Google Scholar

National Research Council (2001). Basic research opportunities in earth sciences. Washington, DC: The National Academies Press. doi:10.17226/9981

CrossRef Full Text | Google Scholar

Vandewaetere, M., Cornillie, F., Clarebout, G., and Desmet, P. (2013). Adaptivity in educational games: Including player and gameplay characteristics. Int. J. High. Educ. 2, 106–114. doi:10.5430/ijhe.v2n2p106

CrossRef Full Text | Google Scholar

Vasconcelos, C., and Orion, N. (2021). Earth science education as a key component of education for sustainability. Sustainability 13, 1316. doi:10.3390/su13031316

CrossRef Full Text | Google Scholar

Wolf, T. (2020). “Green gamification: How gamified information presentation affects pro-environmental behavior,” in GamiFIN, 82–91.

Google Scholar

Wouters, P., Van der Spek, E. D., and Van Oostendorp, H. (2009). “Current practices in serious game research: A review from a learning outcomes perspective,” in Games-based learning advancements for multi-sensory human computer interfaces: Techniques and effective practices (Pennsylvania, United States: IGI Global), 232–250.

CrossRef Full Text | Google Scholar

Wu, J. S., and Lee, J. J. (2015). Climate change games as tools for education and engagement. Nat. Clim. Chang. 5, 413–418. doi:10.1038/nclimate2566

CrossRef Full Text | Google Scholar

Yen, T. S., and Halili, S. H. (2015). Effective teaching of higher order thinking (HOT) in education. Online J. Distance Educ. e-learn. 3, 41–47.

Google Scholar

Zhao, J., LaFemina, P., Carr, J., Sajjadi, P., Wallgrün, J. O., and Klippel, A. (2020). “Learning in the field: Comparison of desktop, immersive virtual reality, and actual field trips for place-based STEM education,” in 2020 IEEE conference on virtual reality and 3D user interfaces (VR). , 893–902. Piscataway, NJ: IEEE.

CrossRef Full Text | Google Scholar